Saccades during visual exploration align hippocampal 3-8 Hz rhythms in human and non-human primates

Spatial memory and learning: investigating the role of dynamic visual acuity

Introduction

The vestibular system's contribution to spatial learning and memory abilities may be clarified using the virtual Morris Water Maze Task (vMWMT). This is important because of the connections between the vestibular system and the hippocampus area. However, there is ongoing debate over the role of the vestibular system in developing spatial abilities. This study aimed to evaluate the relationship between Dynamic Visual Acuity (DVA) across three planes and spatial abilities.

Methods

This cross-sectional study was conducted with 50 healthy adults aged 18 to 55 with normal stress levels and mental health and no neurological, audiological, or vestibular complaints. The Trail-Making Test (TMT) Forms A and B for the assessment of executive functions, the DVA test battery for the evaluation of visual motor functions, and the Virtual Morris Water Maze Test (vMWMT) for the assessment of spatial learning and spatial memory were performed. All participants also underwent the Benton Face Recognition Test (BFRT) and Digit Symbol Substitution Tests (DSST) to assess their relation with spatial memory.

Results

DVA values in horizontal (H-DVA), vertical (V-DVA), and sagittal (S-DVA) planes ranged from (-0.26) to 0.36 logMAR, (-0.20) to 0.36 logMAR, and (-0.28) to 0.33 logMAR, respectively. The latency of three planes of DVA was affected by vMWMT (Horizontal, Vertical, and Sagittal; Estimate: 22.733, 18.787, 13.341, respectively p < 0.001). Moreover, a moderately significant correlation was also found, with a value of 0.571 between the Virtual MWM test and BFRT and a value of 0.539 between the DSST (p < 0.001).

Conclusion

Spatial abilities in healthy adults were significantly influenced by dynamic visual functions across horizontal, vertical, and sagittal planes. These findings are expected to trigger essential discussions about the mechanisms that connect the vestibular-visual system to the hippocampus. The original vMWMT protocol is likely to serve as a model for future studies utilizing this technology.

Circuit dynamics of superficial and deep CA1 pyramidal cells and inhibitory cells in freely moving macaques

Diverse neuron classes in hippocampal CA1 have been identified through the heterogeneity of their cellular/molecular composition. How these classes relate to hippocampal function and the network dynamics that support cognition in primates remains unclear. Here, we report inhibitory functional cell groups in CA1 of freely moving macaques whose diverse response profiles to network states and each other suggest distinct and specific roles in the functional microcircuit of CA1. In addition, pyramidal cells that were grouped by their superficial or deep layer position differed in firing rate, burstiness, and sharp-wave ripple-associated firing. They also showed strata-specific spike-timing interactions with inhibitory cell groups, suggestive of segregated neural populations. Furthermore, ensemble recordings revealed that cell assemblies were preferentially organized according to these strata. These results suggest that hippocampal CA1 in freely moving macaques bears a sublayer-specific circuit organization that may shape its role in cognition.

Theta activity and cognitive functioning: Integrating evidence from resting-state and task-related developmental electroencephalography (EEG) research

The theta band is one of the most prominent frequency bands in the electroencephalography (EEG) power spectrum and presents an interesting paradox: while elevated theta power during resting state is linked to lower cognitive abilities in children and adolescents, increased theta power during cognitive tasks is associated with higher cognitive performance. Why does theta power, measured during resting state versus cognitive tasks, show differential correlations with cognitive functioning? This review provides an integrated account of the functional correlates of theta across different contexts. We first present evidence that higher theta power during resting state is correlated with lower executive functioning, attentional abilities, language skills, and IQ. Next, we review research showing that theta power increases during memory, attention, and cognitive control, and that higher theta power during these processes is correlated with better performance. Finally, we discuss potential explanations for the differential correlations between resting/task-related theta and cognitive functioning, and offer suggestions for future research in this area.

Circuit dynamics of superficial and deep CA1 pyramidal cells and inhibitory cells in freely-moving macaques

Diverse neuron classes in hippocampal CA1 have been identified through the heterogeneity of their cellular/molecular composition. How these classes relate to hippocampal function and the network dynamics that support cognition in primates remains unclear. Here we report inhibitory functional cell groups in CA1 of freely-moving macaques whose diverse response profiles to network states and each other suggest distinct and specific roles in the functional microcircuit of CA1. In addition, pyramidal cells that were segregated into superficial and deep layers differed in firing rate, burstiness, and sharp-wave ripple-associated firing. They also showed strata-specific spike-timing interactions with inhibitory cell groups, suggestive of segregated neural populations. Furthermore, ensemble recordings revealed that cell assemblies were preferentially organized according to these strata. These results suggest sublayer-specific circuit organization in hippocampal CA1 of the freely-moving macaques that may underlie its role in cognition.

Primacy of vision shapes behavioral strategies and neural substrates of spatial navigation in marmoset hippocampus

The role of the hippocampus in spatial navigation has been primarily studied in nocturnal mammals, such as rats, that lack many adaptations for daylight vision. Here we demonstrate that during 3D navigation, the common marmoset, a new world primate adapted to daylight, predominantly uses rapid head-gaze shifts for visual exploration while remaining stationary. During active locomotion marmosets stabilize the head, in contrast to rats that use low-velocity head movements to scan the environment as they locomote. Pyramidal neurons in the marmoset hippocampus CA3/CA1 regions predominantly show mixed selectivity for 3D spatial view, head direction, and place. Exclusive place selectivity is scarce. Inhibitory interneurons are predominantly mixed selective for angular head velocity and translation speed. Finally, we found theta phase resetting of local field potential oscillations triggered by head-gaze shifts. Our findings indicate that marmosets adapted to their daylight ecological niche by modifying exploration/navigation strategies and their corresponding hippocampal specializations.

Hierarchical consciousness: the Nested Observer Windows model

Foremost in our experience is the intuition that we possess a unified conscious experience. However, many observations run counter to this intuition: we experience paralyzing indecision when faced with two appealing behavioral choices, we simultaneously hold contradictory beliefs, and the content of our thought is often characterized by an internal debate. Here, we propose the Nested Observer Windows (NOW) Model, a framework for hierarchical consciousness wherein information processed across many spatiotemporal scales of the brain feeds into subjective experience. The model likens the mind to a hierarchy of nested mosaic tiles-where an image is composed of mosaic tiles, and each of these tiles is itself an image composed of mosaic tiles. Unitary consciousness exists at the apex of this nested hierarchy where perceptual constructs become fully integrated and complex behaviors are initiated via abstract commands. We define an observer window as a spatially and temporally constrained system within which information is integrated, e.g. in functional brain regions and neurons. Three principles from the signal analysis of electrical activity describe the nested hierarchy and generate testable predictions. First, nested observer windows disseminate information across spatiotemporal scales with cross-frequency coupling. Second, observer windows are characterized by a high degree of internal synchrony (with zero phase lag). Third, observer windows at the same spatiotemporal level share information with each other through coherence (with non-zero phase lag). The theoretical framework of the NOW Model accounts for a wide range of subjective experiences and a novel approach for integrating prominent theories of consciousness.

Rhythms in Speech

Speech can be defined as the human ability to communicate through a sequence of vocal sounds. Consequently, speech requires an emitter (the speaker) capable of generating the acoustic signal and a receiver (the listener) able to successfully decode the sounds produced by the emitter (i.e., the acoustic signal). Time plays a central role at both ends of this interaction. On the one hand, speech production requires precise and rapid coordination, typically within the order of milliseconds, of the upper vocal tract articulators (i.e., tongue, jaw, lips, and velum), their composite movements, and the activation of the vocal folds. On the other hand, the generated acoustic signal unfolds in time, carrying information at different timescales. This information must be parsed and integrated by the receiver for the correct transmission of meaning. This chapter describes the temporal patterns that characterize the speech signal and reviews research that explores the neural mechanisms underlying the generation of these patterns and the role they play in speech comprehension.

Orchestration of saccadic eye-movements by brain rhythms in macaque Frontal Eye Field

Visual perception has been suggested to operate on temporal 'chunks' of sensory input, rather than on a continuous stream of visual information. Saccadic eye movements impose a natural rhythm on the sensory input, as periods of steady fixation between these rapid eye movements provide distinct temporal segments of information. Ideally, the timing of saccades should be precisely locked to the brain's rhythms of information processing. Here, we investigated such locking of saccades to rhythmic neural activity in rhesus monkeys performing a visual foraging task. We found that saccades are phase-locked to local field potential oscillations (especially, 9-22 Hz) in the Frontal Eye Field, with the phase of oscillations predictive of the saccade onset as early as 100 ms prior to these movements. Our data also indicate a functional role of this phase-locking in determining the direction of saccades. These findings show a tight-and likely important-link between oscillatory brain activity and rhythmic behavior that imposes a rhythmic temporal structure on sensory input, such as saccadic eye movements.

An oscillatory mechanism for multi-level storage in short-term memory

Oscillatory activity is commonly observed during the maintenance of information in short-term memory, but its role remains unclear. Non-oscillatory models of short-term memory storage are able to encode stimulus identity through their spatial patterns of activity, but are typically limited to either an all-or-none representation of stimulus amplitude or exhibit a biologically implausible exact-tuning condition. Here we demonstrate a simple mechanism by which oscillatory input enables a circuit to generate persistent or sequential activity that encodes information not only in the spatial pattern of activity, but also in the amplitude of activity. This is accomplished through a phase-locking phenomenon that permits many different amplitudes of persistent activity to be stored without requiring exact tuning of model parameters. Altogether, this work proposes a class of models for the storage of information in working memory, a potential role for brain oscillations, and a dynamical mechanism for maintaining multi-stable neural representations.

Cortico-ocular Coupling in the Service of Episodic Memory Formation

Encoding of visual information is a necessary requirement for most types of episodic memories. In search for a neural signature of memory formation, amplitude modulation of neural activity has been repeatedly shown to correlate with and suggested to be functionally involved in successful memory encoding. We here report a complementary view on why and how brain activity relates to memory, indicating a functional role of cortico-ocular interactions for episodic memory formation. Recording simultaneous magnetoencephalography and eye tracking in 35 human participants, we demonstrate that gaze variability and amplitude modulations of alpha/beta oscillations (10-20Hz) in visual cortex covary and predict subsequent memory performance between and within participants. Amplitude variation during pre-stimulus baseline was associated with gaze direction variability, echoing the co-variation observed during scene encoding. We conclude that encoding of visual information engages unison coupling between oculomotor and visual areas in the service of memory formation.

Effects of fearful face presentation time and observer's eye movement on the gaze cue effect

BACKGROUND

There are many conflicting findings on the gaze cueing effect (GCE) of emotional facial expressions. This study aimed to investigate whether an averted gaze, accompanied by a fearful expression of different durations, could enhance attentional orientation, as measured by a participant's eye movements.

METHODS

Twelve participants (3 females) completed the gaze cue task, reacting to a target location after observing changes in the gaze and expression of a face illustrated on a computer screen. Meanwhile, participants' eye movements were monitored by electrooculography. The GCE was calculated by reaction time as an indicator of attention shift.

RESULTS

The analysis of the overall data did not find a significant effect of fearful facial expressions on the GCE. However, analysis of trial data that excluded a participant's eye movement data showed that brief (0, 100 ms) presentation of the fearful facial expression enhanced the GCE compared to that during a neutral facial expression, although when the presentation time of the fearful expression was increased to 200 or 400 ms, the GCE of the fearful expression was at the same level as when model showed a neutral expression.

CONCLUSIONS

The results suggest that the attention-enhancing effect of gaze cues induced by rapidly presented fearful expressions occurs only when the effect of eye movement trials is excluded. This effect may be mediated by reflexively neural circuits in the amygdala that process threatening stimuli. However, as the expression duration increased, the fearful expression's attention-enhancing effect decreased. We suggest that future studies on the emotion modulation of GCE should consider the negative effects of participants' saccades and blinks on the experimental results.

Gated transformations from egocentric to allocentric reference frames involving retrosplenial cortex, entorhinal cortex, and hippocampus

This paper reviews the recent experimental finding that neurons in behaving rodents show egocentric coding of the environment in a number of structures associated with the hippocampus. Many animals generating behavior on the basis of sensory input must deal with the transformation of coordinates from the egocentric position of sensory input relative to the animal, into an allocentric framework concerning the position of multiple goals and objects relative to each other in the environment. Neurons in retrosplenial cortex show egocentric coding of the position of boundaries in relation to an animal. These neuronal responses are discussed in relation to existing models of the transformation from egocentric to allocentric coordinates using gain fields and a new model proposing transformations of phase coding that differ from current models. The same type of transformations could allow hierarchical representations of complex scenes. The responses in rodents are also discussed in comparison to work on coordinate transformations in humans and non-human primates.

Saccadic modulation of neural excitability in auditory areas of the neocortex

In natural "active" vision, humans and other primates use eye movements (saccades) to sample bits of information from visual scenes. In the visual cortex, non-retinal signals linked to saccades shift visual cortical neurons into a high excitability state as each saccade ends. The extent of this saccadic modulation outside of the visual system is unknown. Here, we show that during natural viewing, saccades modulate excitability in numerous auditory cortical areas with a temporal pattern complementary to that seen in visual areas. Control somatosensory cortical recordings indicate that the temporal pattern is unique to auditory areas. Bidirectional functional connectivity patterns suggest that these effects may arise from regions involved in saccade generation. We propose that by using saccadic signals to yoke excitability states in auditory areas to those in visual areas, the brain can improve information processing in complex natural settings.

Gaze onsets during naturalistic infant-caregiver interaction associate with 'sender' but not 'receiver' neural responses, and do not lead to changes in inter-brain synchrony

Temporal coordination during infant-caregiver social interaction is thought to be crucial for supporting early language acquisition and cognitive development. Despite a growing prevalence of theories suggesting that increased inter-brain synchrony associates with many key aspects of social interactions such as mutual gaze, little is known about how this arises during development. Here, we investigated the role of mutual gaze onsets as a potential driver of inter-brain synchrony. We extracted dual EEG activity around naturally occurring gaze onsets during infant-caregiver social interactions in N = 55 dyads (mean age 12 months). We differentiated between two types of gaze onset, depending on each partners' role. 'Sender' gaze onsets were defined at a time when either the adult or the infant made a gaze shift towards their partner at a time when their partner was either already looking at them (mutual) or not looking at them (non-mutual). 'Receiver' gaze onsets were defined at a time when their partner made a gaze shift towards them at a time when either the adult or the infant was already looking at their partner (mutual) or not (non-mutual). Contrary to our hypothesis we found that, during a naturalistic interaction, both mutual and non-mutual gaze onsets were associated with changes in the sender, but not the receiver's brain activity and were not associated with increases in inter-brain synchrony above baseline. Further, we found that mutual, compared to non-mutual gaze onsets were not associated with increased inter brain synchrony. Overall, our results suggest that the effects of mutual gaze are strongest at the intra-brain level, in the 'sender' but not the 'receiver' of the mutual gaze.

Episodic memory formation in unrestricted viewing

The brain systems of episodic memory and oculomotor control are tightly linked, suggesting a crucial role of eye movements in memory. But little is known about the neural mechanisms of memory formation across eye movements in unrestricted viewing behavior. Here, we leverage simultaneous eye tracking and EEG recording to examine episodic memory formation in free viewing. Participants memorized multi-element events while their EEG and eye movements were concurrently recorded. Each event comprised elements from three categories (face, object, place), with two exemplars from each category, in different locations on the screen. A subsequent associative memory test assessed participants' memory for the between-category associations that specified each event. We used a deconvolution approach to overcome the problem of overlapping EEG responses to sequential saccades in free viewing. Brain activity was time-locked to the fixation onsets, and we examined EEG power in the theta and alpha frequency bands, the putative oscillatory correlates of episodic encoding mechanisms. Three modulations of fixation-related EEG predicted high subsequent memory performance: (1) theta increase at fixations after between-category gaze transitions, (2) theta and alpha increase at fixations after within-element gaze transitions, (3) alpha decrease at fixations after between-exemplar gaze transitions. Thus, event encoding with unrestricted viewing behavior was characterized by three neural mechanisms, manifested in fixation-locked theta and alpha EEG activity that rapidly turned on and off during the unfolding eye movement sequences. These three distinct neural mechanisms may be the essential building blocks that subserve the buildup of coherent episodic memories during unrestricted viewing behavior.

Neural Correlates of Spatial Navigation in Primate Hippocampus

The hippocampus has been extensively implicated in spatial navigation in rodents and more recently in bats. Numerous studies have revealed that various kinds of spatial information are encoded across hippocampal regions. In contrast, investigations of spatial behavioral correlates in the primate hippocampus are scarce and have been mostly limited to head-restrained subjects during virtual navigation. However, recent advances made in freely-moving primates suggest marked differences in spatial representations from rodents, albeit some similarities. Here, we review empirical studies examining the neural correlates of spatial navigation in the primate (including human) hippocampus at the levels of local field potentials and single units. The lower frequency theta oscillations are often intermittent. Single neuron responses are highly mixed and task-dependent. We also discuss neuronal selectivity in the eye and head coordinates. Finally, we propose that future studies should focus on investigating both intrinsic and extrinsic population activity and examining spatial coding properties in large-scale hippocampal-neocortical networks across tasks.

Retrosplenial and Hippocampal Synchrony during Retrieval of Old Memories in Macaques

Memory for events from the distant past relies on multiple brain regions, but little is known about the underlying neural dynamics that give rise to such abilities. We recorded neural activity in the hippocampus and retrosplenial cortex of two female rhesus macaques as they visually selected targets in year-old and newly acquired object-scene associations. Whereas hippocampal activity was unchanging with memory age, the retrosplenial cortex responded with greater magnitude alpha oscillations (10-15 Hz) and greater phase locking to memory-guided eye movements during retrieval of old events. A similar old-memory enhancement was observed in the anterior cingulate cortex but in a beta2/gamma band (28-35 Hz). In contrast, remote retrieval was associated with decreased gamma-band synchrony between the hippocampus and each neocortical area. The increasing retrosplenial alpha oscillation and decreasing hippocampocortical synchrony with memory age may signify a shift in frank memory allocation or, alternatively, changes in selection among distributed memory representations in the primate brain.SIGNIFICANCE STATEMENT Memory depends on multiple brain regions, whose involvement is thought to change with time. Here, we recorded neuronal population activity from the hippocampus and retrosplenial cortex as nonhuman primates searched for objects embedded in scenes. These memoranda were either newly presented or a year old. Remembering old material drove stronger oscillations in the retrosplenial cortex and led to a greater locking of neural activity to search movements. Remembering new material revealed stronger oscillatory synchrony between the hippocampus and retrosplenial cortex. These results suggest that with age, memories may come to rely more exclusively on neocortical oscillations for retrieval and search guidance and less on long-range coupling with the hippocampus.

A consensus statement on detection of hippocampal sharp wave ripples and differentiation from other fast oscillations

Decades of rodent research have established the role of hippocampal sharp wave ripples (SPW-Rs) in consolidating and guiding experience. More recently, intracranial recordings in humans have suggested their role in episodic and semantic memory. Yet, common standards for recording, detection, and reporting do not exist. Here, we outline the methodological challenges involved in detecting ripple events and offer practical recommendations to improve separation from other high-frequency oscillations. We argue that shared experimental, detection, and reporting standards will provide a solid foundation for future translational discovery.

Modulation of Neuronal Activity and Saccades at Theta Rhythm During Visual Search in Non-human Primates

Active exploratory behaviors have often been associated with theta oscillations in rodents, while theta oscillations during active exploration in non-human primates are still not well understood. We recorded neural activities in the frontal eye field (FEF) and V4 simultaneously when monkeys performed a free-gaze visual search task. Saccades were strongly phase-locked to theta oscillations of V4 and FEF local field potentials, and the phase-locking was dependent on saccade direction. The spiking probability of V4 and FEF units was significantly modulated by the theta phase in addition to the time-locked modulation associated with the evoked response. V4 and FEF units showed significantly stronger responses following saccades initiated at their preferred phases. Granger causality and ridge regression analysis showed modulatory effects of theta oscillations on saccade timing. Together, our study suggests phase-locking of saccades to the theta modulation of neural activity in visual and oculomotor cortical areas, in addition to the theta phase locking caused by saccade-triggered responses.

A corollary discharge mediates saccade-related inhibition of single units in mnemonic structures of the human brain

Despite the critical link between visual exploration and memory, little is known about how neuronal activity in the human mesial temporal lobe (MTL) is modulated by saccades. Here, we characterize saccade-associated neuronal modulations, unit-by-unit, and contrast them to image onset and to occipital lobe neurons. We reveal evidence for a corollary discharge (CD)-like modulatory signal that accompanies saccades, inhibiting/exciting a unique population of broad-/narrow-spiking units, respectively, before and during saccades and with directional selectivity. These findings comport well with the timing, directional nature, and inhibitory circuit implementation of a CD. Additionally, by linking neuronal activity to event-related potentials (ERPs), which are directionally modulated following saccades, we recontextualize the ERP associated with saccades as a proxy for both the strength of inhibition and saccade direction, providing a mechanistic underpinning for the more commonly recorded saccade-related ERP in the human brain.

Saccade rate is associated with recall of items in working memory

Working memory has been shown to rely on theta oscillations' phase synchronicity for item encoding and recall. At the same time, saccadic eye movements during visual exploration have been observed to trigger theta-phase resets, raising the question of whether the neuronal substrates of mnemonic processing rely on motor-evoked responses. To quantify the relationship between saccades and working memory load, we recorded eye tracking and behavioral data from human participants simultaneously performing an n-back Sternberg auditory task and a hue-based catch detection task. In addition to task-specific interference in performance, we also found that saccade rate was modulated by working memory load in the Sternberg task's preresponse stage. Our results support the possibility of interplay between saccades and hippocampal theta during working memory retrieval of items.

Aging changes the interactions between the oculomotor and memory systems

The use of multi-modal approaches, particularly in conjunction with multivariate analytic techniques, can enrich models of cognition, brain function, and how they change with age. Recently, multivariate approaches have been applied to the study of eye movements in a manner akin to that of neural activity (i.e., pattern similarity). Here, we review the literature regarding multi-modal and/or multivariate approaches, with specific reference to the use of eyetracking to characterize age-related changes in memory. By applying multi-modal and multivariate approaches to the study of aging, research has shown that aging is characterized by moment-to-moment alterations in the amount and pattern of visual exploration, and by extension, alterations in the activity and function of the hippocampus and broader medial temporal lobe (MTL). These methodological advances suggest that age-related declines in the integrity of the memory system has consequences for oculomotor behavior in the moment, in a reciprocal fashion. Age-related changes in hippocampal and MTL structure and function may lead to an increase in, and change in the patterns of, visual exploration in an effort to upregulate the encoding of information. However, such visual exploration patterns may be non-optimal and actually reduce the amount and/or type of incoming information that is bound into a lasting memory representation. This research indicates that age-related cognitive impairments are considerably broader in scope than previously realized.

Saccade-related neural communication in the human medial temporal lobe is modulated by the social relevance of stimuli

Humans predominantly explore their environment by moving their eyes. To optimally communicate and process visual information, neural activity needs to be coordinated with the execution of eye movements. We investigated the coordination between visual exploration and interareal neural communication by analyzing local field potentials and single neuron activity in patients with epilepsy. We demonstrated that during the free viewing of images, neural communication between the human amygdala and hippocampus is coordinated with the execution of eye movements. The strength and direction of neural communication and hippocampal saccade-related phase alignment were strongest for fixations that landed on human faces. Our results argue that the state of the human medial temporal lobe network is selectively coordinated with motor behavior. Interareal neural communication was facilitated for social stimuli as indexed by the category of the attended information.

Theta rhythmicity governs human behavior and hippocampal signals during memory-dependent tasks

Memory formation and reinstatement are thought to lock to the hippocampal theta rhythm, predicting that encoding and retrieval processes appear rhythmic themselves. Here, we show that rhythmicity can be observed in behavioral responses from memory tasks, where participants indicate, using button presses, the timing of encoding and recall of cue-object associative memories. We find no evidence for rhythmicity in button presses for visual tasks using the same stimuli, or for questions about already retrieved objects. The oscillations for correctly remembered trials center in the slow theta frequency range (1-5 Hz). Using intracranial EEG recordings, we show that the memory task induces temporally extended phase consistency in hippocampal local field potentials at slow theta frequencies, but significantly more for remembered than forgotten trials, providing a potential mechanistic underpinning for the theta oscillations found in behavioral responses.

Neural activity in the human anterior thalamus during natural vision

In natural vision humans and other primates explore environment by active sensing, using saccadic eye movements to relocate the fovea and sample different bits of information multiple times per second. Saccades induce a phase reset of ongoing neuronal oscillations in primary and higher-order visual cortices and in the medial temporal lobe. As a result, neuron ensembles are shifted to a common state at the time visual input propagates through the system (i.e., just after fixation). The extent of the brain’s circuitry that is modulated by saccades is not yet known. Here, we evaluate the possibility that saccadic phase reset impacts the anterior nuclei of the thalamus (ANT). Using recordings in the human thalamus of three surgical patients during natural vision, we found that saccades and visual stimulus onset both modulate neural activity, but with distinct field potential morphologies. Specifically, we found that fixation-locked field potentials had a component that preceded saccade onset. It was followed by an early negativity around 50 ms after fixation onset which is significantly faster than any response to visual stimulus presentation. The timing of these events suggests that the ANT is predictively modulated before the saccadic eye movement. We also found oscillatory phase concentration, peaking at 3–4 Hz, coincident with suppression of Broadband High-frequency Activity (BHA; 80–180 Hz), both locked to fixation onset supporting the idea that neural oscillations in these nuclei are reorganized to a low excitability state right after fixation onset. These findings show that during real-world natural visual exploration neural dynamics in the human ANT is influenced by visual and oculomotor events, which supports the idea that ANT, apart from their contribution to episodic memory, also play a role in natural vision.

Memory-Guided Saccades in Psychosis: Effects of Medication and Stimulus Location

The memory-guided saccade task requires the remembrance of a peripheral target location, whilst inhibiting the urge to make a saccade ahead of an auditory cue. The literature has explored the endophenotypic deficits associated with differences in target laterality, but less is known about target amplitude. The data presented came from Crawford et al. (1995), employing a memory-guided saccade task among neuroleptically medicated and non-medicated patients with schizophrenia (n = 31, n = 12), neuroleptically medicated and non-medicated bipolar affective disorder (n = 12, n = 17), and neurotypical controls (n = 30). The current analyses explore the relationships between memory-guided saccades toward targets with different eccentricities (7.5° and 15°), the discernible behaviour exhibited amongst diagnostic groups, and cohorts distinguished based on psychotic symptomatology. Saccade gain control and final eye position were reduced among medicated-schizophrenia patients. These metrics were reduced further among targets with greater amplitudes (15°), indicating greater deficit. The medicated cohort exhibited reduced gain control and final eye positions in both amplitudes compared to the non-medicated cohort, with deficits markedly observed for the furthest targets. No group differences in symptomatology (positive and negative) were reported, however, a greater deficit was observed toward the larger amplitude. This suggests that within the memory-guided saccade paradigm, diagnostic classification is more prominent in characterising disparities in saccade performance than symptomatology.

Neural Correlates of Subsequent Memory-Related Gaze Reinstatement

Mounting evidence linking gaze reinstatement-the recapitulation of encoding-related gaze patterns during retrieval-to behavioral measures of memory suggests that eye movements play an important role in mnemonic processing. Yet, the nature of the gaze scanpath, including its informational content and neural correlates, has remained in question. In this study, we examined eye movement and neural data from a recognition memory task to further elucidate the behavioral and neural bases of functional gaze reinstatement. Consistent with previous work, gaze reinstatement during retrieval of freely viewed scene images was greater than chance and predictive of recognition memory performance. Gaze reinstatement was also associated with viewing of informationally salient image regions at encoding, suggesting that scanpaths may encode and contain high-level scene content. At the brain level, gaze reinstatement was predicted by encoding-related activity in the occipital pole and BG, neural regions associated with visual processing and oculomotor control. Finally, cross-voxel brain pattern similarity analysis revealed overlapping subsequent memory and subsequent gaze reinstatement modulation effects in the parahippocampal place area and hippocampus, in addition to the occipital pole and BG. Together, these findings suggest that encoding-related activity in brain regions associated with scene processing, oculomotor control, and memory supports the formation, and subsequent recapitulation, of functional scanpaths. More broadly, these findings lend support to scanpath theory's assertion that eye movements both encode, and are themselves embedded in, mnemonic representations.

Robot navigation as hierarchical active inference

Localization and mapping has been a long standing area of research, both in neuroscience, to understand how mammals navigate their environment, as well as in robotics, to enable autonomous mobile robots. In this paper, we treat navigation as inferring actions that minimize (expected) variational free energy under a hierarchical generative model. We find that familiar concepts like perception, path integration, localization and mapping naturally emerge from this active inference formulation. Moreover, we show that this model is consistent with models of hippocampal functions, and can be implemented in silico on a real-world robot. Our experiments illustrate that a robot equipped with our hierarchical model is able to generate topologically consistent maps, and correct navigation behaviour is inferred when a goal location is provided to the system.

Restricting Visual Exploration Directly Impedes Neural Activity, Functional Connectivity, and Memory

We move our eyes to explore the visual world, extract information, and create memories. The number of gaze fixations-the stops that the eyes make-has been shown to correlate with activity in the hippocampus, a region critical for memory, and with later recognition memory. Here, we combined eyetracking with fMRI to provide direct evidence for the relationships between gaze fixations, neural activity, and memory during scene viewing. Compared to free viewing, fixating a single location reduced: 1) subsequent memory, 2) neural activity along the ventral visual stream into the hippocampus, 3) neural similarity between effects of subsequent memory and visual exploration, and 4) functional connectivity among the hippocampus, parahippocampal place area, and other cortical regions. Gaze fixations were uniquely related to hippocampal activity, even after controlling for neural effects due to subsequent memory. Therefore, this study provides key causal evidence supporting the notion that the oculomotor and memory systems are intrinsically related at both the behavioral and neural level. Individual gaze fixations may provide the basic unit of information on which memory binding processes operate.

The Role of Motor and Environmental Visual Rhythms in Structuring Auditory Cortical Excitability

Previous studies indicate that motor sampling patterns modulate neuronal excitability in sensory brain regions by entraining brain rhythms, a process termed motor-initiated entrainment. In addition, rhythms of the external environment are also capable of entraining brain rhythms. Our first goal was to investigate the properties of motor-initiated entrainment in the auditory system using a prominent visual motor sampling pattern in primates, saccades. Second, we wanted to determine whether/how motor-initiated entrainment interacts with visual environmental entrainment. We examined laminar profiles of neuronal ensemble activity in primary auditory cortex and found that whereas motor-initiated entrainment has a suppressive effect, visual environmental entrainment has an enhancive effect. We also found that these processes are temporally coupled, and their temporal relationship ensures that their effect on excitability is complementary rather than interfering. Altogether, our results demonstrate that motor and sensory systems continuously interact in orchestrating the brain's context for the optimal sampling of our multisensory environment.

A New Unifying Account of the Roles of Neuronal Entrainment

Rhythms are a fundamental and defining feature of neuronal activity in animals including humans. This rhythmic brain activity interacts in complex ways with rhythms in the internal and external environment through the phenomenon of 'neuronal entrainment', which is attracting increasing attention due to its suggested role in a multitude of sensory and cognitive processes. Some senses, such as touch and vision, sample the environment rhythmically, while others, like audition, are faced with mostly rhythmic inputs. Entrainment couples rhythmic brain activity to external and internal rhythmic events, serving fine-grained routing and modulation of external and internal signals across multiple spatial and temporal hierarchies. This interaction between a brain and its environment can be experimentally investigated and even modified by rhythmic sensory stimuli or invasive and non-invasive neuromodulation techniques. We provide a comprehensive overview of the topic and propose a theoretical framework of how neuronal entrainment dynamically structures information from incoming neuronal, bodily and environmental sources. We discuss the different types of neuronal entrainment, the conceptual advances in the field, and converging evidence for general principles.

A roadmap for development of neuro-oscillations as translational biomarkers for treatment development in neuropsychopharmacology

New treatment development for psychiatric disorders depends critically upon the development of physiological measures that can accurately translate between preclinical animal models and clinical human studies. Such measures can be used both as stratification biomarkers to define pathophysiologically homogeneous patient populations and as target engagement biomarkers to verify similarity of effects across preclinical and clinical intervention. Traditional "time-domain" event-related potentials (ERP) have been used translationally to date but are limited by the significant differences in timing and distribution across rodent, monkey and human studies. By contrast, neuro-oscillatory responses, analyzed within the "time-frequency" domain, are relatively preserved across species permitting more precise translational comparisons. Moreover, neuro-oscillatory responses are increasingly being mapped to local circuit mechanisms and may be useful for investigating effects of both pharmacological and neuromodulatory interventions on excitatory/inhibitory balance. The present paper provides a roadmap for development of neuro-oscillatory responses as translational biomarkers in neuropsychiatric treatment development.

Anatomy and Function of the Primate Entorhinal Cortex

The entorhinal cortex (EC) is a critical element of the hippocampal formation located within the medial temporal lobe (MTL) in primates. The EC has historically received attention for being the primary mediator of cortical information going into and coming from the hippocampus proper. In this review, we highlight the significance of the EC as a major player in memory processing, along with other associated structures in the primate MTL. The complex, convergent topographies of cortical and subcortical input to the EC, combined with short-range intrinsic connectivity and the selective targeting of EC efferents to the hippocampus, provide evidence for subregional specialization and integration of information beyond what would be expected if this structure were a simple conduit of information for the hippocampus. Lesion studies of the EC provide evidence implicating this region as critical for memory and the flexible use of complex relational associations between experienced events. The physiology of this structure's constituent principal cells mirrors the complexity of its anatomy. EC neurons respond preferentially to aspects of memory-dependent paradigms including object, place, and time. EC neurons also show striking spatial representations as primates explore visual space, similar to those identified in rodents navigating physical space. In this review, we highlight the great strides that have been made toward furthering our understanding of the primate EC, and we identify paths forward for future experiments to provide additional insight into the role of this structure in learning and memory.

Differential Generation of Saccade, Fixation, and Image-Onset Event-Related Potentials in the Human Mesial Temporal Lobe

Event-related potentials (ERPs) are a commonly used electrophysiological signature for studying mesial temporal lobe (MTL) function during visual memory tasks. The ERPs associated with the onset of visual stimuli (image-onset) and eye movements (saccades and fixations) provide insights into the mechanisms of their generation. We hypothesized that since eye movements and image-onset provide MTL structures with salient visual information, perhaps they both engage similar neural mechanisms. To explore this question, we used intracranial electroencephalographic data from the MTLs of 11 patients with medically refractory epilepsy who participated in a visual search task. We characterized the electrophysiological responses of MTL structures to saccades, fixations, and image-onset. We demonstrated that the image-onset response is an evoked/additive response with a low-frequency power increase. In contrast, ERPs following eye movements appeared to arise from phase resetting of higher frequencies than the image-onset ERP. Intriguingly, this reset was associated with saccade onset and not termination (fixation), suggesting it is likely the MTL response to a corollary discharge, rather than a response to visual stimulation. We discuss the distinct mechanistic underpinnings of these responses which shed light on the underlying neural circuitry involved in visual memory processing.

Hippocampal theta coordinates memory processing during visual exploration

The hippocampus supports memory encoding and retrieval, which may occur at distinct phases of the theta cycle. These processes dynamically interact over rapid timescales, especially when sensory information conflicts with memory. The ability to link hippocampal dynamics to memory-guided behaviors has been limited by experiments that lack the temporal resolution to segregate encoding and retrieval. Here, we simultaneously tracked eye movements and hippocampal field potentials while neurosurgical patients performed a spatial memory task. Phase-locking at the peak of theta preceded fixations to retrieved locations, indicating that the hippocampus coordinates memory-guided eye movements. In contrast, phase-locking at the trough of theta followed fixations to novel object-locations and predicted intact memory of the original location. Theta-gamma phase amplitude coupling increased during fixations to conflicting visual content, but predicted memory updating. Hippocampal theta thus supports learning through two interleaved processes: strengthening encoding of novel information and guiding exploration based on prior experience.

Modeling the influence of the hippocampal memory system on the oculomotor system

Visual exploration is related to activity in the hippocampus (HC) and/or extended medial temporal lobe system (MTL), is influenced by stored memories, and is altered in amnesic cases. An extensive set of polysynaptic connections exists both within and between the HC and oculomotor systems such that investigating how HC responses ultimately influence neural activity in the oculomotor system, and the timing by which such neural modulation could occur, is not trivial. We leveraged TheVirtualBrain, a software platform for large-scale network simulations, to model the functional dynamics that govern the interactions between the two systems in the macaque cortex. Evoked responses following the stimulation of the MTL and some, but not all, subfields of the HC resulted in observable responses in oculomotor regions, including the frontal eye fields, within the time of a gaze fixation. Modeled lesions to some MTL regions slowed the dissipation of HC signal to oculomotor regions, whereas HC lesions generally did not affect the rapid MTL activity propagation to oculomotor regions. These findings provide a framework for investigating how information represented by the HC/MTL may influence the oculomotor system during a fixation and predict how HC lesions may affect visual exploration.

The intersection between the oculomotor and hippocampal memory systems: empirical developments and clinical implications

Decades of cognitive neuroscience research has shown that where we look is intimately connected to what we remember. In this article, we review findings from human and nonhuman animals, using behavioral, neuropsychological, neuroimaging, and computational modeling methods, to show that the oculomotor and hippocampal memory systems interact in a reciprocal manner, on a moment-to-moment basis, mediated by a vast structural and functional network. Visual exploration serves to efficiently gather information from the environment for the purpose of creating new memories, updating existing memories, and reconstructing the rich, vivid details from memory. Conversely, memory increases the efficiency of visual exploration. We call for models of oculomotor control to consider the influence of the hippocampal memory system on the cognitive control of eye movements, and for models of hippocampal and broader medial temporal lobe function to consider the influence of the oculomotor system on the development and expression of memory. We describe eye movement-based applications for the detection of neurodegeneration and delivery of therapeutic interventions for mental health disorders for which the hippocampus is implicated and memory dysfunctions are at the forefront.

Modulation of local field potentials and neuronal activity in primate hippocampus during saccades

Primates use saccades to gather information about objects and their relative spatial arrangement, a process essential for visual perception and memory. It has been proposed that signals linked to saccades reset the phase of local field potential (LFP) oscillations in the hippocampus, providing a temporal window for visual signals to activate neurons in this region and influence memory formation. We investigated this issue by measuring hippocampal LFPs and spikes in two macaques performing different tasks with unconstrained eye movements. We found that LFP phase clustering (PC) in the alpha/beta (8-16 Hz) frequencies followed foveation onsets, while PC in frequencies lower than 8 Hz followed spontaneous saccades, even on a homogeneous background. Saccades to a solid grey background were not followed by increases in local neuronal firing, whereas saccades toward appearing visual stimuli were. Finally, saccade parameters correlated with LFPs phase and amplitude: saccade direction correlated with delta (≤4 Hz) phase, and saccade amplitude with theta (4-8 Hz) power. Our results suggest that signals linked to saccades reach the hippocampus, producing synchronization of delta/theta LFPs without a general activation of local neurons. Moreover, some visual inputs co-occurring with saccades produce LFP synchronization in the alpha/beta bands and elevated neuronal firing. Our findings support the hypothesis that saccade-related signals enact sensory input-dependent plasticity and therefore memory formation in the primate hippocampus.

The Role of Neuronal Oscillations in Visual Active Sensing

Visual perception is most often studied as a "passive" process in which an observer fixates steadily at point in space so that stimuli can be delivered to the system with spatial precision. Analysis of neuronal signals related to vision is generally keyed to stimulus onset, stimulus movement, etc.; i.e., events external to the observer. In natural "active" vision, however, information is systematically acquired by using eye movements including rapid (saccadic) eye movements, as well as smooth ocular pursuit of moving objects and slower drifts. Here we consider the use of alternating saccades and fixations to gather information from a visual scene. The underlying motor sampling plan contains highly reliable information regarding "where" and "when" the eyes will land, this information can be used predictively to modify firing properties of neurons precisely at the time when this "contextual" information is most useful - when a volley of retinal input enters the system at the onset of each fixation. Analyses focusing on neural events leading to and resulting from shifts in fixation, as well as visual events external to the observer, can provide a more complete and mechanistic understanding of visual information processing. Studies thus far suggest that active vision may be a fundamentally different from that process we usually study with more traditional passive viewing paradigms. In this Perspective we note that active saccadic sampling behavior imposes robust temporal patterning on the activity of neuron ensembles and large-scale neural dynamics throughout the brain's visual pathways whose mechanistic effects on information processing are not yet fully understood. The spatio-temporal sequence of eye movements elicits a succession of temporally predictable quasi-rhythmic sensory inputs, whose encoding is enhanced by entrainment of low frequency oscillations to the rate of eye movements. Review of the pertinent findings underscores the fact that temporal coordination between motor and visual cortices is critical for understanding neural dynamics of active vision and posits that phase entrainment of neuronal oscillations plays a mechanistic role in this process.

The Common Rhythm of Action and Perception

Research in the last decade has undermined the idea of perception as a continuous process, providing strong empirical support for its rhythmic modulation. More recently, it has been revealed that the ongoing motor processes influence the rhythmic sampling of sensory information. In this review, we will focus on a growing body of evidence suggesting that oscillation-based mechanisms may structure the dynamic interplay between the motor and sensory system and provide a unified temporal frame for their effective coordination. We will describe neurophysiological data, primarily collected in animals, showing phase-locking of neuronal oscillations to the onset of (eye) movements. These data are complemented by novel evidence in humans, which demonstrate the behavioral relevance of these oscillatory modulations and their domain-general nature. Finally, we will discuss the possible implications of these modulations for action-perception coupling mechanisms.

Dynamic Causal Modelling of Active Vision

In this paper, we draw from recent theoretical work on active perception, which suggests that the brain makes use of an internal (i.e., generative) model to make inferences about the causes of sensations. This view treats visual sensations as consequent on action (i.e., saccades) and implies that visual percepts must be actively constructed via a sequence of eye movements. Oculomotor control calls on a distributed set of brain sources that includes the dorsal and ventral frontoparietal (attention) networks. We argue that connections from the frontal eye fields to ventral parietal sources represent the mapping from “where”, fixation location to information derived from “what” representations in the ventral visual stream. During scene construction, this mapping must be learned, putatively through changes in the effective connectivity of these synapses. Here, we test the hypothesis that the coupling between the dorsal frontal cortex and the right temporoparietal cortex is modulated during saccadic interrogation of a simple visual scene. Using dynamic causal modeling for magnetoencephalography with (male and female) human participants, we assess the evidence for changes in effective connectivity by comparing models that allow for this modulation with models that do not. We find strong evidence for modulation of connections between the two attention networks; namely, a disinhibition of the ventral network by its dorsal counterpart.

SIGNIFICANCE STATEMENT This work draws from recent theoretical accounts of active vision and provides empirical evidence for changes in synaptic efficacy consistent with these computational models. In brief, we used magnetoencephalography in combination with eye-tracking to assess the neural correlates of a form of short-term memory during a dot cancellation task. Using dynamic causal modeling to quantify changes in effective connectivity, we found evidence that the coupling between the dorsal and ventral attention networks changed during the saccadic interrogation of a simple visual scene. Intuitively, this is consistent with the idea that these neuronal connections may encode beliefs about “what I would see if I looked there”, and that this mapping is optimized as new data are obtained with each fixation.

The Hippocampus Promotes Effective Saccadic Information Gathering in Humans

It is well established that the hippocampus is critical for memory. Recent evidence suggests that one function of hippocampal memory processing is to optimize how people actively explore the world. Here we demonstrate that the link between the hippocampus and exploration extends even to the moment-to-moment use of eye movements during visuospatial memory encoding. In Experiment 1, we examined relationships between study-phase eye movements in healthy individuals and subsequent performance on a spatial reconstruction test. In addition to quantitative measures of viewing behaviors (e.g., how many fixations or saccades were deployed during study), we used the information-theoretic measure of entropy to assess the amount of randomness or disorganization in participants' scanning behaviors. We found that the use of scanpaths during study that were lower in entropy (e.g., more organized, less random) predicted more accurate spatial reconstruction both within and between participants. Scanpath entropy was a better predictor of reconstruction accuracy than were the quantitative measures of viewing. In Experiment 2, we found that individuals with hippocampal amnesia tended to engage in viewing patterns that were higher in entropy (less organized) relative to healthy comparisons. These findings reveal a critical role of the hippocampus in guiding eye movement exploration to optimize visuospatial relational memory.

Age-related changes in the relationship between visual exploration and hippocampal activity

Deciphering the mechanisms underlying age-related memory declines remains an important goal in cognitive neuroscience. Recently, we observed that visual sampling behavior predicted activity within the hippocampus, a region critical for memory. In younger adults, increases in the number of gaze fixations were associated with increases in hippocampal activity (Liu et al., 2017). This finding suggests a close coupling between the oculomotor and memory system. However, the extent to which this coupling is altered with aging has not been investigated. In this study, we gave older adults the same face processing task used in Liu et al. (2017) and compared their visual exploration behavior and neural activation in the hippocampus and the fusiform face area (FFA) to those of younger adults. Compared to younger adults, older adults showed an increase in visual exploration as indexed by the number of gaze fixations. However, the relationship between visual exploration and neural responses in the hippocampus and FFA was weaker than that of younger adults. Older adults also showed weaker responses to novel faces and a smaller repetition suppression effect in the hippocampus and FFA compared to younger adults. All together, this study provides novel evidence that the capacity to bind visually sampled information, in real-time, into coherent representations along the ventral visual stream and the medial temporal lobe declines with aging.

How the Brain's Navigation System Shapes Our Visual Experience

We explore the environment not only by navigating, but also by viewing our surroundings with our eyes. Here we review growing evidence that the mammalian hippocampal formation, extensively studied in the context of navigation and memory, mediates a representation of visual space that is stably anchored to the external world. This visual representation puts the hippocampal formation in a central position to guide viewing behavior and to modulate visual processing beyond the medial temporal lobe (MTL). We suggest that vision and navigation share several key computational challenges that are solved by overlapping and potentially common neural systems, making vision an optimal domain to explore whether and how the MTL supports cognitive operations beyond navigation.

A Closer Look at the Hippocampus and Memory

Current interpretations of hippocampal memory function are blind to the fact that viewing behaviors are pervasive and complicate the relationships among perception, behavior, memory, and brain activity. For example, hippocampal activity and associative memory demands increase with stimulus complexity. Stimulus complexity also strongly modulates viewing. Associative processing and viewing thus are often confounded, rendering interpretation of hippocampal activity ambiguous. Similar considerations challenge many accounts of hippocampal function. To explain relationships between memory and viewing, we propose that the hippocampus supports the online memory demands necessary to guide visual exploration. The hippocampus thus orchestrates memory-guided exploration that unfolds over time to build coherent memories. This new perspective on hippocampal function harmonizes with the fact that memory formation and exploratory viewing are tightly intertwined.