From the optic tectum to the primary visual cortex: migration through evolution of the saliency map for exogenous attentional guidance

Central visual pathways affected by degenerative retinal disease before and after gene therapy

Genetic diseases affecting the retina can result in partial or complete loss of visual function. Leber's congenital amaurosis (LCA) is a rare blinding disease, usually inherited in an autosomally recessive manner, with no cure. Retinal gene therapy has been shown to improve vision in LCA patients caused by mutations in the RPE65 gene (LCA2). However, little is known about how activity in central visual pathways is affected by the disease or by subsequent gene therapy. Functional MRI (fMRI) was used to assess retinal signal transmission in cortical and subcortical visual structures before and 1 year after retinal intervention. The fMRI paradigm consisted of 15-s blocks of flickering (8 Hz) black and white checkerboards interleaved with 15 s of blank (black) screen. Visual activation in the brain was assessed using the general linear model, with multiple comparisons corrected using the false discovery rate method. Response to visual stimulation through untreated eyes of LCA2 patients showed heightened fMRI responses in the superior colliculus and diminished activities in the lateral geniculate nucleus (LGN) compared to controls, indicating a shift in the patients' visual processing towards the retinotectal pathway. Following gene therapy, stimuli presented to the treated eye elicited significantly stronger fMRI responses in the LGN and primary visual cortex, indicating some re-engagement of the geniculostriate pathway (GS) pathway. Across patients, the post-treatment LGN fMRI responses correlated significantly with performance on a clinical test measuring light sensitivity. Our results demonstrate that the low vision observed in LCA2 patients involves a shift in visual processing toward the retinotectal pathway, and that gene therapy partially reinstates visual transmission through the GS pathway. This selective boosting of retinal output through the GS pathway and its correlation to improved visual performance, following several years of degenerative retinal disease, is striking. However, while retinal gene therapy and other ocular interventions have given hope to RPE65 patients, it may take years before development of therapies tailored to treat the diseases in other low vision patients are available. Our demonstration of a shift toward the retinotectal pathway in these patients may spur the development of new tools and rehabilitation strategies to help maximize the use of residual visual abilities and augment experience-dependent plasticity.

Seeing without a Scene: Neurological Observations on the Origin and Function of the Dorsal Visual Stream

In all vertebrates, visual signals from each visual field project to the opposite midbrain tectum (called the superior colliculus in mammals). The tectum/colliculus computes visual salience to select targets for context-contingent visually guided behavior: a frog will orient toward a small, moving stimulus (insect prey) but away from a large, looming stimulus (a predator). In mammals, visual signals competing for behavioral salience are also transmitted to the visual cortex, where they are integrated with collicular signals and then projected via the dorsal visual stream to the parietal and frontal cortices. To control visually guided behavior, visual signals must be encoded in body-centered (egocentric) coordinates, and so visual signals must be integrated with information encoding eye position in the orbit-where the individual is looking. Eye position information is derived from copies of eye movement signals transmitted from the colliculus to the frontal and parietal cortices. In the intraparietal cortex of the dorsal stream, eye movement signals from the colliculus are used to predict the sensory consequences of action. These eye position signals are integrated with retinotopic visual signals to generate scaffolding for a visual scene that contains goal-relevant objects that are seen to have spatial relationships with each other and with the observer. Patients with degeneration of the superior colliculus, although they can see, behave as though they are blind. Bilateral damage to the intraparietal cortex of the dorsal stream causes the visual scene to disappear, leaving awareness of only one object that is lost in space. This tutorial considers what we have learned from patients with damage to the colliculus, or to the intraparietal cortex, about how the phylogenetically older midbrain and the newer mammalian dorsal cortical visual stream jointly coordinate the experience of a spatially and temporally coherent visual scene.

Visual shape discrimination in goldfish, modelled with the neural circuitry of optic tectum and torus longitudinalis

There is no satisfactory neurally-based theory as to how vertebrates that lack a neocortex discriminate even simple geometric shapes. In fishes, an intact optic tectum is necessary for such discriminations, but physiological studies of it have found nothing like the hierarchically arranged feature detecting neurons of mammalian visual cortex. Here, a neural model attempts a solution by basing shape discrimination upon the responses of only those elementary detectors (e.g. of size) that are within a focus of attention, formed by a winner-take-all arrangement of retinotopically mapped units representing tectal pyramidal cells. While this relatively primitive mechanism could recognize an object irrespective of position in space, it fails to distinguish patterns that differ only in their features' spatial relationships. The model's solution - imitating goldfish that naturally attend to the top of shapes - is to shift attention to the edges of a shape by spatially offsetting inputs to the pyramidal neurons, effected by the torus longitudinalis and its prolific synapses on pyramidal dendrites. The model's shape discrimination was compared to an extensive behavioral study using shapes with points and projections. In one test series fish were sensitive to the relative number of points on the tops of shapes. In another, fish were trained to discriminate points on the sides. By using different offset connections and only one elementary feature detector for small dark spots, the model successfully emulated the two sets of goldfish data, as judged by significant correlations between model response and fish discrimination.

Intracortical recordings reveal vision-to-action cortical gradients driving human exogenous attention

Exogenous attention, the process that makes external salient stimuli pop-out of a visual scene, is essential for survival. How attention-capturing events modulate human brain processing remains unclear. Here we show how the psychological construct of exogenous attention gradually emerges over large-scale gradients in the human cortex, by analyzing activity from 1,403 intracortical contacts implanted in 28 individuals, while they performed an exogenous attention task. The timing, location and task-relevance of attentional events defined a spatiotemporal gradient of three neural clusters, which mapped onto cortical gradients and presented a hierarchy of timescales. Visual attributes modulated neural activity at one end of the gradient, while at the other end it reflected the upcoming response timing, with attentional effects occurring at the intersection of visual and response signals. These findings challenge multi-step models of attention, and suggest that frontoparietal networks, which process sequential stimuli as separate events sharing the same location, drive exogenous attention phenomena such as inhibition of return.

Spatial and single-nucleus transcriptomics decoding the molecular landscape and cellular organization of avian optic tectum

The avian optic tectum (OT) has been studied for its diverse functions, yet a comprehensive molecular landscape at the cellular level has been lacking. In this study, we applied spatial transcriptome sequencing and single-nucleus RNA sequencing (snRNA-seq) to explore the cellular organization and molecular characteristics of the avian OT from two species: Columba livia and Taeniopygia guttata. We identified precise layer structures and provided comprehensive layer-specific signatures of avian OT. Furthermore, we elucidated diverse functions in different layers, with the stratum griseum periventriculare (SGP) potentially playing a key role in advanced functions of OT, like fear response and associative learning. We characterized detailed neuronal subtypes and identified a population of FOXG1+ excitatory neurons, resembling those found in the mouse neocortex, potentially involved in neocortex-related functions and expansion of avian OT. These findings could contribute to our understanding of the architecture of OT, shedding light on visual perception and multifunctional association.

Involvement of superior colliculus in complex figure detection of mice

Object detection is an essential function of the visual system. Although the visual cortex plays an important role in object detection, the superior colliculus can support detection when the visual cortex is ablated or silenced. Moreover, it has been shown that superficial layers of mouse SC (sSC) encode visual features of complex objects, and that this code is not inherited from the primary visual cortex. This suggests that mouse sSC may provide a significant contribution to complex object vision. Here, we use optogenetics to show that mouse sSC is involved in figure detection based on differences in figure contrast, orientation, and phase. Additionally, our neural recordings show that in mouse sSC, image elements that belong to a figure elicit stronger activity than those same elements when they are part of the background. The discriminability of this neural code is higher for correct trials than for incorrect trials. Our results provide new insight into the behavioral relevance of the visual processing that takes place in sSC.

Pathways from the Superior Colliculus to the Basal Ganglia

The present work aims to review the structural organization of the mammalian superior colliculus (SC), the putative pathways connecting the SC and the basal ganglia, and their role in organizing complex behavioral output. First, we review how the complex intrinsic connections between the SC's laminae projections allow for the construction of spatially aligned, visual-multisensory maps of the surrounding environment. Moreover, we present a summary of the sensory-motor inputs of the SC, including a description of the integration of multi-sensory inputs relevant to behavioral control. We further examine the major descending outputs toward the brainstem and spinal cord. As the central piece of this review, we provide a thorough analysis covering the putative interactions between the SC and the basal ganglia. To this end, we explore the diverse thalamic routes by which information from the SC may reach the striatum, including the pathways through the lateral posterior, parafascicular, and rostral intralaminar thalamic nuclei. We also examine the interactions between the SC and subthalamic nucleus, representing an additional pathway for the tectal modulation of the basal ganglia. Moreover, we discuss how information from the SC might also be relayed to the basal ganglia through midbrain tectonigral and tectotegmental projections directed at the substantia nigra compacta and ventrotegmental area, respectively, influencing the dopaminergic outflow to the dorsal and ventral striatum. We highlight the vast interplay between the SC and the basal ganglia and raise several missing points that warrant being addressed in future studies.

Eye movement evidence for the V1 Saliency Hypothesis and the Central-peripheral Dichotomy theory in an anomalous visual search task

Typically, searching for a target among uniformly tilted non-targets is easier when this target is perpendicular, rather than parallel, to the non-targets. The V1 Saliency Hypothesis (V1SH) - that V1 creates a saliency map to guide attention exogenously - predicts exactly the opposite in a special case: each target or non-target is a pair of equally-sized disks, a homo-pair of two disks of the same color, black or white, or a hetero-pair of two disks of the opposite color; the inter-disk displacement defines its orientation. This prediction - parallel advantage - was supported by the finding that parallel targets require shorter reaction times (RTs) to report targets' locations. Furthermore, it is stronger for targets further from the center of search images, as predicted by the Central-peripheral Dichotomy (CPD) theory entailing that saliency effects are stronger in peripheral than in central vision. However, the parallel advantage could arise from a shorter time required to recognize - rather than to shift attention to - the parallel target. By gaze tracking, the present study confirms that the parallel advantage is solely due to the RTs for the gaze to reach the target. Furthermore, when the gaze is sufficiently far from the target during search, saccade to a parallel, rather than perpendicular, target is more likely, demonstrating the Central-peripheral Dichotomy more directly. Parallel advantage is stronger among observers encouraged to let their search be guided by spontaneous gaze shifts, which are presumably guided by bottom-up saliency rather than top-down factors.

Crowding expands and is less sensitive to target-flanker differences during a shift of visual attention

Target-flanker similarity and critical spacing control visual crowding when attention is pre-allocated, but these have not been studied when attention shifts. Flanked target Gabors appeared 8° left and right of central fixation throughout each 1.5 s trial. Subjects reported target Gabor tilt. In Expt. 1, target blinks increased accuracy, and flanker blinks decreased it, but only when attention shifted left or right from a central RSVP cue, hardly before it, indicating an exogenous/endogenous synergy. Whether parallel or orthogonal, flankers of the same wavelength as the target crowded substantially. Parallel half-wavelength flankers also crowded, but orthogonal half-wavelength ones did not. In Expt. 2, crowding when attention shifts was the same for targets and flankers within Bouma's bound (2.5° apart) as outside it (5.0° apart.) In Expt. 3, Bouma's bound was restored when attention was focused continuously on the target. We conclude that crowding temporarily expands and becomes less discriminative when attention shifts.

When less is more: coy display behaviours and the temporal dynamics of animal courtship

Sexual selection research has been dominated by the notion that mate choice selects for the most vigorous displays that best reflect the quality of the courter. However, courtship displays are often temporally structured, containing different elements with varying degrees of intensity and conspicuousness. For example, highly intense movements are often coupled with more subtle components such as static postures or hiding displays. Here, we refer to such subtle display traits as 'coy', as they involve the withholding of information about maximal display capabilities. We examine the role of intensity variation within temporally dynamic displays, and discuss three hypotheses for the evolution of coy courtship behaviours. We first review the threat reduction hypothesis, which points to sexual coercion and sexual autonomy as important facets of sexual selection. We then suggest that variation in display magnitude exploits pre-existing perceptual biases for temporal contrast. Lastly, we propose that information withholding may leverage receivers' predispositions for filling gaps in information-the 'curiosity bias'. Overall, our goal is to draw attention to temporal variation in display magnitude, and to advocate possible scenarios for the evolution of courtship traits that regularly occur below performance maxima. Throughout, we highlight novel directions for empirical and theoretical investigations.

Stimulus edges induce orientation tuning in superior colliculus

Orientation columns exist in the primary visual cortex (V1) of cat and primates but not mouse. Intriguingly, some recent studies reported the presence of orientation and direction columns in the mouse superficial superior colliculus (sSC), while others reported a lack of columnar organization therein. Using in vivo calcium imaging of sSC in the awake mouse brain, we found that the presence of columns is highly stimulus dependent. Specifically, we observed orientation and direction columns formed by sSC neurons retinotopically mapped to the edge of grating stimuli. For both excitatory and inhibitory neurons in sSC, orientation selectivity can be induced by the edge with their preferred orientation perpendicular to the edge orientation. Furthermore, we found that this edge-induced orientation selectivity is associated with saliency encoding. These findings indicate that the tuning properties of sSC neurons are not fixed by circuit architecture but rather dependent on the spatiotemporal properties of the stimulus.

Understanding brain functional architecture through robotics

Robotics is increasingly seen as a useful test bed for computational models of the brain functional architecture underlying animal behavior. We provide an overview of past and current work, focusing on probabilistic and dynamical models, including approaches premised on the free energy principle, situating this endeavor in relation to evidence that the brain constitutes a layered control system. We argue that future neurorobotic models should integrate multiple neurobiological constraints and be hybrid in nature.

Peripheral and central sensation: multisensory orienting and recognition across species

Attentional bottlenecks force animals to deeply process only a selected fraction of sensory inputs. This motivates a unifying central-peripheral dichotomy (CPD), which separates multisensory processing into functionally defined central and peripheral senses. Peripheral senses (e.g., human audition and peripheral vision) select a fraction of the sensory inputs by orienting animals' attention; central senses (e.g., human foveal vision) allow animals to recognize the selected inputs. Originally used to understand human vision, CPD can be applied to multisensory processes across species. I first describe key characteristics of central and peripheral senses, such as the degree of top-down feedback and density of sensory receptors, and then show CPD as a framework to link ecological, behavioral, neurophysiological, and anatomical data and produce falsifiable predictions.

Saturation of visual responses explains size tuning in rat collicular neurons

The receptive field of many visual neurons is composed of a central responsive area, the classical receptive field, and a non-classical receptive field, also called the "suppressive surround." A visual stimulus placed in the suppressive surround does not induce any response but modulates visual responses to stimuli within the classical receptive field, usually by suppressing them. Therefore, visual responses become smaller when stimuli exceed the classical receptive field size. The stimulus size inducing the maximal response is called the preferred stimulus size. In cortex, there is good correspondence between the sizes of the classical receptive field and the preferred stimulus. In contrast, in the rodent superior colliculus, the preferred size is often several fold smaller than the classical receptive field size. Here, we show that in the rat superior colliculus, the preferred stimulus size changes as a square root of the contrast inverse and the classical receptive field size is independent of contrast. In addition, responses to annulus were largely independent of the inner hole size. To explain these data, three models were tested: the divisive modulation of the gain by the suppressive surround (the "normalization" model), the difference of the Gaussians, and a divisive model that incorporates saturation to light flux. Despite the same number of free parameters, the model incorporating saturation to light performed the best. Thus, our data indicate that in rats, the saturation to light can be a dominant phenomenon even at relatively low illumination levels defining visual responses in the collicular neurons.

Topographically Localized Modulation of Tectal Cell Spatial Tuning by Complex Natural Scenes

The tuning properties of neurons in the visual system can be contextually modulated by the statistics of the area surrounding their receptive field (RF), particularly when the surround contains natural features. However, stimuli presented in specific egocentric locations may have greater behavioral relevance, raising the possibility that the extent of contextual modulation may vary with position in visual space. To explore this possibility, we utilized the small size and optical transparency of the larval zebrafish to describe the form and spatial arrangement of contextually modulated cells throughout an entire tectal hemisphere. We found that the spatial tuning of tectal neurons to a prey-like stimulus sharpens when the stimulus is presented against a background with the statistics of complex natural scenes, relative to a featureless background. These neurons are confined to a spatially restricted region of the tectum and have receptive fields centered within a region of visual space in which the presence of prey preferentially triggers hunting behavior. Our results suggest that contextual modulation of tectal neurons by complex backgrounds may facilitate prey-localization in cluttered visual environments.

Input-dependent segregation of visual and somatosensory circuits in the mouse superior colliculus

Whereas sensory perception relies on specialized sensory pathways, it is unclear whether these pathways originate as modality-specific circuits. We demonstrated that somatosensory and visual circuits are not by default segregated but require the earliest retinal activity to do so. In the embryo, somatosensory and visual circuits are intermingled in the superior colliculus, leading to cortical multimodal responses to whisker pad stimulation. At birth, these circuits segregate, and responses switch to unimodal. Blocking stage I retinal waves prolongs the multimodal configuration into postnatal life, with the superior colliculus retaining a mixed somato-visual molecular identity and defects arising in the spatial organization of the visual system. Hence, the superior colliculus mediates the timely segregation of sensory modalities in an input-dependent manner, channeling specific sensory cues to their appropriate sensory pathway.

The Superior Colliculus: Cell Types, Connectivity, and Behavior

The superior colliculus (SC), one of the most well-characterized midbrain sensorimotor structures where visual, auditory, and somatosensory information are integrated to initiate motor commands, is highly conserved across vertebrate evolution. Moreover, cell-type-specific SC neurons integrate afferent signals within local networks to generate defined output related to innate and cognitive behaviors. This review focuses on the recent progress in understanding of phenotypic diversity amongst SC neurons and their intrinsic circuits and long-projection targets. We further describe relevant neural circuits and specific cell types in relation to behavioral outputs and cognitive functions. The systematic delineation of SC organization, cell types, and neural connections is further put into context across species as these depend upon laminar architecture. Moreover, we focus on SC neural circuitry involving saccadic eye movement, and cognitive and innate behaviors. Overall, the review provides insight into SC functioning and represents a basis for further understanding of the pathology associated with SC dysfunction.

Scaffolding layered control architectures through constraint closure: insights into brain evolution and development

The functional organization of the mammalian brain can be considered to form a layered control architecture, but how this complex system has emerged through evolution and is constructed during development remains a puzzle. Here we consider brain organization through the framework of constraint closure, viewed as a general characteristic of living systems, that they are composed of multiple sub-systems that constrain each other at different timescales. We do so by developing a new formalism for constraint closure, inspired by a previous model showing how within-lifetime dynamics can constrain between-lifetime dynamics, and we demonstrate how this interaction can be generalized to multi-layered systems. Through this model, we consider brain organization in the context of two major examples of constraint closure-physiological regulation and visual orienting. Our analysis draws attention to the capacity of layered brain architectures to scaffold themselves across multiple timescales, including the ability of cortical processes to constrain the evolution of sub-cortical processes, and of the latter to constrain the space in which cortical systems self-organize and refine themselves. This article is part of the theme issue 'Systems neuroscience through the lens of evolutionary theory'.

Parallel Advantage: Further Evidence for Bottom-up Saliency Computation by Human Primary Visual Cortex

Finding a target among uniformly oriented non-targets is typically faster when this target is perpendicular, rather than parallel, to the non-targets. The V1 Saliency Hypothesis (V1SH), that neurons in the primary visual cortex (V1) signal saliency for exogenous attentional attraction, predicts exactly the opposite in a special case: each target or non-target comprises two equally sized disks displaced from each other by 1.2 disk diameters center-to-center along a line defining its orientation. A target has two white or two black disks. Each non-target has one white disk and one black disk, and thus, unlike the target, activates V1 neurons less when its orientation is parallel rather than perpendicular to the neurons' preferred orientations. When the target is parallel, rather than perpendicular, to the uniformly oriented non-targets, the target's evoked V1 response escapes V1's iso-orientation surround suppression, making the target more salient. I present behavioral observations confirming this prediction.

Comparative anatomy of dissected optic lobes, optic ventricles, midbrain tectum, collicular ventricles, and aqueduct: evolutionary modifications as potential explanation for non-tumoral aqueductal anomalies in humans

PURPOSE

An extensive literature has postulated multiple etiologies for aqueductal stenosis. No publications were found, discussing that evolutionary modifications might explain aqueductal anomalies. This study's objectives were to review the evolutionary modifications of vertebrates' tectum structures that might explain human aqueduct anomalies. Undertaking vertebrate comparative study is currently not feasible in view of limitations in obtaining vertebrate material. Thus, vertebrate material collected, injected, dissected, and radiographed in the early 1970s was analyzed, focusing on the aqueduct and components of the midbrain tectum.

METHODS

Photographs of brain dissections and radiographs of the cerebral ventricles and arteries of adult shark, frog, iguana, rabbit, cat, dog, and primate specimens, containing a barium-gelatin radiopaque compound, were analyzed focusing on the aqueduct, the optic ventricles, the quadrigeminal plate, and collicular ventricles. The anatomic information provided by the dissections and radiographs is not reproducible by any other radiopaque contrast currently available.

RESULTS

Dissected and radiographed cerebral ventricular and arterial systems of the vertebrates demonstrated midbrain tectum changes, including relative size modifications of the mammalian components of the tectum, simultaneously with the enlargement of the occipital lobe. There is a transformation of pre-mammalian optic ventricles to what appear to be collicular ventricles in mammals, as the aqueduct and collicular ventricle form a continuous cavity.

CONCLUSIONS

The mammalian tectum undergoes an evolutionary cephalization process consisting of relative size changes of the midbrain tectum structures. This is associated with enlargement of the occipital lobe, as part of overall neocortical expansion. Potentially, aqueductal anomalies could be explained by evolutionary modifications.

Neural circuitry for stimulus selection in the zebrafish visual system

When navigating the environment, animals need to prioritize responses to the most relevant stimuli. Although a theoretical framework for selective visual attention exists, its circuit implementation has remained obscure. Here we investigated how larval zebrafish select between simultaneously presented visual stimuli. We found that a mix of winner-take-all (WTA) and averaging strategies best simulates behavioral responses. We identified two circuits whose activity patterns predict the relative saliencies of competing visual objects. Stimuli presented to only one eye are selected by WTA computation in the inner retina. Binocularly presented stimuli, on the other hand, are processed by reciprocal, bilateral connections between the nucleus isthmi (NI) and the tectum. This interhemispheric computation leads to WTA or averaging responses. Optogenetic stimulation and laser ablation of NI neurons disrupt stimulus selection and behavioral action selection. Thus, depending on the relative locations of competing stimuli, a combination of retinotectal and isthmotectal circuits enables selective visual attention.

Archerfish vision: Visual challenges faced by a predator with a unique hunting technique

Archerfish are well-known for their ballistic hunting behaviour, in which they shoot down aerial prey with a well-aimed jet of water. This unique hunting strategy poses several challenges for visual systems. Archerfish face significant distortion to the appearance of targets due to refraction at the air/water interface, they search for prey against a complex background of foliage, they change prey targeting behaviour as conditions change, and they must make high speed decisions to avoid competition. By studying how archerfish have overcome these challenges, we have been able to understand more about fundamental problems faced by visual systems and the mechanisms used to solve them. In some cases, such as when searching for targets, the visual capabilities of archerfish are functionally similar to those of humans, despite significant differences in neuroanatomy. In other cases, the particular challenge faced by archerfish magnifies fundamental problems generally faced by visual systems, such as recognizing objects given strong viewpoint dependent changes to appearance. The efficiency of archerfish retrieving fallen prey to avoid kleptoparasitism, demonstrates that their visual processing excels in both speed and accuracy. In this review, we attempt to provide an overview of the many facets of visually driven behaviour of archerfish, and how they have been studied. In addition to their hunting technique, archerfish are ideal for visual processing experiments as they can be quickly trained to perform a range of non-ecologically relevant tasks. Their behavioural flexibility moreover, introduces the opportunity to study how experience-dependence and choice affects visual processing.

A Neural Representation of Naturalistic Motion-Guided Behavior in the Zebrafish Brain

All animals must transform ambiguous sensory data into successful behavior. This requires sensory representations that accurately reflect the statistics of natural stimuli and behavior. Multiple studies show that visual motion processing is tuned for accuracy under naturalistic conditions, but the sensorimotor circuits extracting these cues and implementing motion-guided behavior remain unclear. Here we show that the larval zebrafish retina extracts a diversity of naturalistic motion cues, and the retinorecipient pretectum organizes these cues around the elements of behavior. We find that higher-order motion stimuli, gliders, induce optomotor behavior matching expectations from natural scene analyses. We then image activity of retinal ganglion cell terminals and pretectal neurons. The retina exhibits direction-selective responses across glider stimuli, and anatomically clustered pretectal neurons respond with magnitudes matching behavior. Peripheral computations thus reflect natural input statistics, whereas central brain activity precisely codes information needed for behavior. This general principle could organize sensorimotor transformations across animal species.

Behavioral and neuronal study of inhibition of return in barn owls

Inhibition of return (IOR) is the reduction of detection speed and/or detection accuracy of a target in a recently attended location. This phenomenon, which has been discovered and studied thoroughly in humans, is believed to reflect a brain mechanism for controlling the allocation of spatial attention in a manner that enhances efficient search. Findings showing that IOR is robust, apparent at a very early age and seemingly dependent on midbrain activity suggest that IOR is a universal attentional mechanism in vertebrates. However, studies in non-mammalian species are scarce. To explore this hypothesis comparatively, we tested for IOR in barn owls (Tyto alba) using the classical Posner cueing paradigm. Two barn owls were trained to initiate a trial by fixating on the center of a computer screen and then turning their gaze to the location of a target. A short, non-informative cue appeared before the target, either at a location predicting the target (valid) or a location not predicting the target (invalid). In one barn owl, the response times (RT) to the valid targets compared to the invalid targets shifted from facilitation (lower RTs) to inhibition (higher RTs) when increasing the time lag between the cue and the target. The second owl mostly failed to maintain fixation and responded to the cue before the target onset. However, when including in the analysis only the trials in which the owl maintained fixation, an inhibition in the valid trials could be detected. To search for the neural correlates of IOR, we recorded multiunit responses in the optic tectum (OT) of four head-fixed owls passively viewing a cueing paradigm as in the behavioral experiments. At short cue to target lags (<100 ms), neural responses to the target in the receptive field (RF) were usually enhanced if the cue appeared earlier inside the RF (valid) and were suppressed if the cue appeared earlier outside the RF (invalid). This was reversed at longer lags: neural responses were suppressed in the valid conditions and were unaffected in the invalid conditions. The findings support the notion that IOR is a basic mechanism in the evolution of vertebrate behavior and suggest that the effect appears as a result of the interaction between lateral and forward inhibition in the tectal circuitry.

On the organization of receptive fields of retinal spot detectors projecting to the fish tectum: Analogies with the local edge detectors in frogs and mammals

Responses of ON- and OFF-ganglion cells (GCs) were recorded extracellularly from their axon terminals in the medial sublamina of tectal retino-recipient layer of immobilized cyprinid fish (goldfish and carp). These units were recorded deeper than direction selective (DS) ones and at the same depth where responses of orientation selective (OS) GCs were recorded. Prominent responses of these units are evoked by small contrast spots flickering within or moving across their visual field. They are not selective either to the direction of motion or to the orientation of stimuli and are not characterized by any spontaneous spike activity. We refer to these fish GCs as spot detectors (SDs) by analogy with the frog SD. Receptive fields (RFs) of SDs are organized concentrically: the excitatory center (about 4.5°) is surrounded by opponent periphery. Study of interactions in the RF has shown that inhibitory influences are generated already inside the central RF area. This fact suggests that RFs of SDs cannot be defined as homogeneous sensory zone driven by a linear mechanism of response generation. Physiological properties of fish SDs are compared with the properties of frog SDs and analogous mammalian retinal GCs-local edge detectors (LEDs). The potential role of the SDs in visually guided fish behavior is discussed.

What is a preattentive feature?

The concept of a preattentive feature has been central to vision and attention research for about half a century. A preattentive feature is a feature that guides attention in visual search and that cannot be decomposed into simpler features. While that definition seems straightforward, there is no simple diagnostic test that infallibly identifies a preattentive feature. This paper briefly reviews the criteria that have been proposed and illustrates some of the difficulties of definition.

Visual Cortex Gains Independence from Peripheral Drive before Eye Opening

Visual spatial perception in the mammalian brain occurs through two parallel pathways: one reaches the primary visual cortex (V1) through the thalamus and another the superior colliculus (SC) via direct projections from the retina. The origin, development, and relative function of these two evolutionarily distinct pathways remain obscure. We examined the early functional development of both pathways by simultaneously imaging pre- and post-synaptic spontaneous neuronal activity. We observed that the quality of retinal activity transfer to the thalamus and superior colliculus does not change across the first two postnatal weeks. However, beginning in the second postnatal week, retinal activity does not drive V1 as strongly as earlier wave activity, suggesting that intrinsic cortical activity competes with signals from the sensory periphery as the cortex matures. Together, these findings bring new insight into the function of the SC and V1 and the role of peripheral activity in driving both circuits across development.

A new framework for understanding vision from the perspective of the primary visual cortex

Visual attention selects only a tiny fraction of visual input information for further processing. Selection starts in the primary visual cortex (V1), which creates a bottom-up saliency map to guide the fovea to selected visual locations via gaze shifts. This motivates a new framework that views vision as consisting of encoding, selection, and decoding stages, placing selection on center stage. It suggests a massive loss of non-selected information from V1 downstream along the visual pathway. Hence, feedback from downstream visual cortical areas to V1 for better decoding (recognition), through analysis-by-synthesis, should query for additional information and be mainly directed at the foveal region. Accordingly, non-foveal vision is not only poorer in spatial resolution, but also more susceptible to many illusions.

The neural instantiation of a priority map

The term priority map is commonly used to describe a map of the visual scene, in which objects and locations are represented by their attentional priority, which itself is a combination of low-level salience and top-down control. The aim of this review is to examine how such a map may be represented at the neuronal level. We propose that there is not a single, common map in the brain, but that a number of cortical areas work together to generate the resultant behavior. Specifically, we suggest that the lateral intraparietal area (LIP) of posterior parietal cortex provides a simple representation of attentional priority, which remaps across saccades, so that there is an apparent allocentric map in a region with retinocentric encoding scheme. We propose that the frontal eye field (FEF) of prefrontal cortex receives the responses from LIP, but can suppress them to control the flow of eye movement behavior, and that the intermediate layers of the superior colliculus (SCi) reflect the final saccade goal. Together, these areas function to guide eye movements and may play a similar role in allocating covert visual attention.

What pops out for you pops out for fish: Four common visual features

Visual search is the ability to detect a target of interest against a background of distracting objects. For many animals, performing this task fast and accurately is crucial for survival. Typically, visual-search performance is measured by the time it takes the observer to detect a target against a backdrop of distractors. The efficiency of a visual search depends fundamentally on the features of the target, the distractors, and the interaction between them. Substantial efforts have been devoted to investigating the influence of different visual features on visual-search performance in humans. In particular, it has been demonstrated that color, size, orientation, and motion are efficient visual features to guide attention in humans. However, little is known about which features are efficient and which are not in other vertebrates. Given earlier observations that moving targets elicit pop-out and parallel search in the archerfish during visual-search tasks, here we investigate and confirm that all four of these visual features also facilitate efficient search in the archerfish in a manner comparable to humans. In conjunction with results reported for other species, these finding suggest universality in the way visual search is carried out by animals despite very different brain anatomies and living environments.

Bottom-up saliency and top-down learning in the primary visual cortex of monkeys

Early sensory cortex is better known for representing sensory inputs but less for the effect of its responses on behavior. Here we explore the behavioral correlates of neuronal responses in primary visual cortex (V1) in a task to detect a uniquely oriented bar-the orientation singleton-in a background of uniformly oriented bars. This singleton is salient or inconspicuous when the orientation contrast between the singleton and background bars is sufficiently large or small, respectively. Using implanted microelectrodes, we measured V1 activities while monkeys were trained to quickly saccade to the singleton. A neuron's responses to the singleton within its receptive field had an early and a late component, both increased with the orientation contrast. The early component started from the outset of neuronal responses; it remained unchanged before and after training on the singleton detection. The late component started ∼40 ms after the early one; it emerged and evolved with practicing the detection task. Training increased the behavioral accuracy and speed of singleton detection and increased the amount of information in the late response component about a singleton's presence or absence. Furthermore, for a given singleton, faster detection performance was associated with higher V1 responses; training increased this behavioral-neural correlate in the early V1 responses but decreased it in the late V1 responses. Therefore, V1's early responses are directly linked with behavior and represent the bottom-up saliency signals. Learning strengthens this link, likely serving as the basis for making the detection task more reflexive and less top-down driven.

Functional modulation of primary visual cortex by the superior colliculus in the mouse

The largest targets of retinal input in mammals are the dorsal lateral geniculate nucleus (dLGN), a relay to the primary visual cortex (V1), and the superior colliculus. V1 innervates and influences the superior colliculus. Here, we find that, in turn, superior colliculus modulates responses in mouse V1. Optogenetically inhibiting the superior colliculus reduces responses in V1 to optimally sized stimuli. Superior colliculus could influence V1 via its strong projection to the lateral posterior nucleus (LP/Pulvinar) or its weaker projection to the dLGN. Inhibiting superior colliculus strongly reduces activity in LP. Pharmacologically silencing LP itself, however, does not remove collicular modulation of V1. The modulation is instead due to a collicular gain modulation of the dLGN. Surround suppression operating in V1 explains the different effects for differently sized stimuli. Computations of visual saliency in the superior colliculus can thus influence tuning in the visual cortex via a tectogeniculate pathway.

The role of the superior colliculus (SC) in modulating V1 cortical activity is not clear. Here, the authors demonstrate the functional role of SC in modulating V1 responses through an excitatory pathway via the dorsal lateral geniculate nucleus.

Neural Circuits That Mediate Selective Attention: A Comparative Perspective

Selective attention is central to cognition. Dramatic advances have been made in understanding the neural circuits that mediate selective attention. Forebrain networks, most elaborated in primates, control all forms of attention based on task demands and the physical salience of stimuli. These networks contain circuits that distribute top-down signals to sensory processing areas and enhance information processing in those areas. A midbrain network, most elaborated in birds, controls spatial attention. It contains circuits that continuously compute the highest priority stimulus location and route sensory information from the selected location to forebrain networks that make cognitive decisions. The identification of these circuits, their functions and mechanisms represent a major advance in our understanding of how the vertebrate brain mediates selective attention.

Bidirectional encoding of motion contrast in the mouse superior colliculus

Detection of salient objects in the visual scene is a vital aspect of an animal's interactions with its environment. Here, we show that neurons in the mouse superior colliculus (SC) encode visual saliency by detecting motion contrast between stimulus center and surround. Excitatory neurons in the most superficial lamina of the SC are contextually modulated, monotonically increasing their response from suppression by the same-direction surround to maximal potentiation by an oppositely-moving surround. The degree of this potentiation declines with depth in the SC. Inhibitory neurons are suppressed by any surround at all depths. These response modulations in both neuronal populations are much more prominent to direction contrast than to phase, temporal frequency, or static orientation contrast, suggesting feature-specific saliency encoding in the mouse SC. Together, our findings provide evidence supporting locally generated feature representations in the SC, and lay the foundations towards a mechanistic and evolutionary understanding of their emergence.

What a predator can teach us about visual processing: a lesson from the archerfish

The archerfish is a predator with highly unusual visually guided behavior. It is most famous for its ability to hunt by shooting water jets at static or dynamic insect prey, up to two meters above the water's surface. In the lab, the archerfish can learn to distinguish and shoot at artificial targets presented on a computer screen, thus enabling well-controlled experiments. In recent years, these capacities have turned the archerfish into a model animal for studying a variety of visual functions, from visual saliency and visual search, through fast visually guided prediction, and all the way to higher level visual processing such as face recognition. Here we review these recent developments and show how they fall into two emerging lines of research on this animal model. The first is ethologically motivated and emphasizes how the natural environment and habitat of the archerfish interact with its visual processing during predation. The second is driven by parallels to the primate brain and aims to determine whether the latter's characteristic visual information processing capacities can also be found in the qualitatively different fish brain, thereby underscoring the functional universality of certain visual processes. We discuss the differences between these two lines of research and possible future directions.

Coding Schemes in the Archerfish Optic Tectum

Many studies have yielded valuable knowledge on the early visual system but it is biased since the studies have focused on terrestrial mammals alone. Here, to better account for visual systems in different environments and animal classes, we studied the structure of early visual processing in the archerfish which harnesses its extreme visual ability to hunt by shooting water jets at prey hanging on vegetation above the water. Thus, the archerfish provides a unique opportunity to study visual processing in a vertebrate which is an expert vision-guided predator with a very different brain structure than mammals. The receptive field structures in the archerfish (both sexes) optic tectum, the main visual processing region in the fish brain, were measured and linear non-linear cascades were used to analyze their properties. The findings indicate that the spatial receptive field structures lie on a continuum between circular and elliptical shapes. In addition, the cells' functional properties display a richness of response characteristics, since many cells could be captured by more than a single linear filter. Finally, the non-linear response functions that link linear filters and neuronal responses were found to be similar to the non-linear functions of models that describe terrestrial mammalian single cell activity. Overall our results help to better understand the early visual processing system across vertebrates.

Monocular channels have a functional role in endogenous orienting

The literature has long emphasized the role of higher cortical structures in endogenous orienting. Based on evolutionary explanation and previous data, we explored the possibility that lower monocular channels may also have a functional role in endogenous orienting of attention. Sensitive behavioral manipulation was used to probe the contribution of monocularly segregated regions in a simple cue - target detection task. A central spatially informative cue, and its ensuing target, were presented to the same or different eyes at varying cue-target intervals. Results indicated that the onset of endogenous orienting was apparent earlier when the cue and target were presented to the same eye. The data provides converging evidence for the notion that endogenous facilitation is modulated by monocular portions of the visual stream. This, in turn, suggests that higher cortical mechanisms are not exclusively responsible for endogenous orienting, and that a dynamic interaction between higher and lower neural levels, might be involved.

Superior colliculus encodes visual saliency before the primary visual cortex

Models of visual attention postulate the existence of a bottom-up saliency map that is formed early in the visual processing stream. Although studies have reported evidence of a saliency map in various cortical brain areas, determining the contribution of phylogenetically older pathways is crucial to understanding its origin. Here, we compared saliency coding from neurons in two early gateways into the visual system: the primary visual cortex (V1) and the evolutionarily older superior colliculus (SC). We found that, while the response latency to visual stimulus onset was earlier for V1 neurons than superior colliculus superficial visual-layer neurons (SCs), the saliency representation emerged earlier in SCs than in V1. Because the dominant input to the SCs arises from V1, these relative timings are consistent with the hypothesis that SCs neurons pool the inputs from multiple V1 neurons to form a feature-agnostic saliency map, which may then be relayed to other brain areas.

Endogenous orienting in the archer fish

The literature has long emphasized the neocortex's role in volitional processes. In this work, we examined endogenous orienting in an evolutionarily older species, the archer fish, which lacks neocortex-like cells. We used Posner's classic endogenous cuing task, in which a centrally presented, spatially informative cue is followed by a target. The fish responded to the target by shooting a stream of water at it. Interestingly, the fish demonstrated a human-like "volitional" facilitation effect: their reaction times to targets that appeared on the side indicated by the precue were faster than their reaction times to targets on the opposite side. The fish also exhibited inhibition of return, an aftermath of orienting that commonly emerges only in reflexive orienting tasks in human participants. We believe that this pattern demonstrates the acquisition of an arbitrary connection between spatial orienting and a nonspatial feature of a centrally presented stimulus in nonprimate species. In the literature on human attention, orienting in response to such contingencies has been strongly associated with volitional control. We discuss the implications of these results for the evolution of orienting, and for the study of volitional processes in all species, including humans.

Connecting the Brain to Itself through an Emulation

Pilot clinical trials of human patients implanted with devices that can chronically record and stimulate ensembles of hundreds to thousands of individual neurons offer the possibility of expanding the substrate of cognition. Parallel trains of firing rate activity can be delivered in real-time to an array of intermediate external modules that in turn can trigger parallel trains of stimulation back into the brain. These modules may be built in software, VLSI firmware, or biological tissue as in vitro culture preparations or in vivo ectopic construct organoids. Arrays of modules can be constructed as early stage whole brain emulators, following canonical intra- and inter-regional circuits. By using machine learning algorithms and classic tasks known to activate quasi-orthogonal functional connectivity patterns, bedside testing can rapidly identify ensemble tuning properties and in turn cycle through a sequence of external module architectures to explore which can causatively alter perception and behavior. Whole brain emulation both (1) serves to augment human neural function, compensating for disease and injury as an auxiliary parallel system, and (2) has its independent operation bootstrapped by a human-in-the-loop to identify optimal micro- and macro-architectures, update synaptic weights, and entrain behaviors. In this manner, closed-loop brain-computer interface pilot clinical trials can advance strong artificial intelligence development and forge new therapies to restore independence in children and adults with neurological conditions.

Deafferentation of the Superior Colliculus Abolishes Spatial Summation of Redundant Visual Signals

Two visual signals appearing simultaneously are detected more rapidly than either signal appearing alone. Part of this redundant target effect (RTE) can be attributed to neural summation that has been proposed to occur in the superior colliculus (SC). We report direct evidence in two neurological patients for neural summation in the SC, and that it is mediated by afferent visual information transmitted through its brachium. The RTE was abolished in one patient with a hemorrhage involving the right posterior thalamus that damaged part of the SC and that disrupted its brachium; and in another patient in whom the SC appeared intact but deafferented due to traumatic avulsion of its brachium. In addition reaction time for unilateral targets in the contralesional field was slowed in both patients, providing the first evidence that visual afferents to the SC contribute to the efficiency of target detection.

Wireless electrophysiology of the brain of freely swimming goldfish

BACKGROUND

Fish are a diverse group of vertebrates with very different brain structures. The study of the neurobiology of fish can thus lead to many important insights on information processing in the brain in a variety of environments.

NEW METHOD

We developed a novel wireless technique to record extracellular neural signal activity in freely behaving fish. The system is based on a data logger and enables continues recording of up to 2.5h.

RESULTS

For proof of concept we recorded from the optic tectum of goldfish. We found correlations between the activity of the optic tectum and the light intensity in the room, as expected.

COMPARISON WITH EXISTING METHODS

Due to the technical difficulties involved in making electrophysiological recordings of behaving aquatic animals, there is no method for recording electrical neural activity from the brain of freely swimming fish.

CONCLUSIONS

This powerful tool should facilitate studies of information processing in behaving fish and other behaving aquatic animals.