Evidence for an attentional priority map in inferotemporal cortex

Salient distractor processing: inhibition following attentional capture

Salient objects often capture attention in a purely exogenous way, followed by inhibition of their locations after a period. Yet, the neural circuits underlying the exogenous attention remain underspecified. Seidel Malkinson et al. explore this by uncovering large-scale cortical gradients associated with exogenous attention within the human cortex.

Distinct visual processing networks for foveal and peripheral visual fields

Foveal and peripheral vision are two distinct modes of visual processing essential for navigating the world. However, it remains unclear if they engage different neural mechanisms and circuits within the visual attentional system. Here, we trained macaques to perform a free-gaze visual search task using natural face and object stimuli and recorded a large number of 14588 visually responsive neurons from a broadly distributed network of brain regions involved in visual attentional processing. Foveal and peripheral units had substantially different proportions across brain regions and exhibited systematic differences in encoding visual information and visual attention. The spike-LFP coherence of foveal units was more extensively modulated by both attention and visual selectivity, thus indicating differential engagement of the attention and visual coding network compared to peripheral units. Furthermore, we delineated the interaction and coordination between foveal and peripheral processing for spatial attention and saccade selection. Finally, the search became more efficient with increasing target-induced desynchronization, and foveal and peripheral units exhibited different correlations between neural responses and search behavior. Together, the systematic differences between foveal and peripheral processing provide valuable insights into how the brain processes and integrates visual information from different regions of the visual field.

Significance Statement

This study investigates the systematic differences between foveal and peripheral vision, two crucial components of visual processing essential for navigating our surroundings. By simultaneously recording from a large number of neurons in the visual attentional neural network, we revealed substantial variations in the proportion and functional characteristics of foveal and peripheral units across different brain regions. We uncovered differential modulation of functional connectivity by attention and visual selectivity, elucidated the intricate interplay between foveal and peripheral processing in spatial attention and saccade selection, and linked neural responses to search behavior. Overall, our study contributes to a deeper understanding of how the brain processes and integrates visual information for active visual behaviors.

Common and distinct neural mechanisms of attention

Despite a constant deluge of sensory stimulation, only a fraction of it is used to guide behavior. This selective processing is generally referred to as attention, and much research has focused on the neural mechanisms controlling it. Recently, research has broadened to include more ways by which different species selectively process sensory information, whether due to the sensory input itself or to different behavioral and brain states. This work has produced a complex and disjointed body of evidence across different species and forms of attention. However, it has also provided opportunities to better understand the breadth of attentional mechanisms. Here, we summarize the evidence that suggests that different forms of selective processing are supported by mechanisms both common and distinct.

Orienting role of the putative human posterior infero-temporal area in visual attention

The dorsal attention network (DAN) is a network of brain regions essential for attentional orienting, which includes the lateral intraparietal area (LIP) and frontal eye field (FEF). Recently, the putative human dorsal posterior infero-temporal area (phPITd) has been identified as a new node of the DAN. However, its functional relationship with other areas of the DAN and its specific role in visual attention remained unclear. In this study, we analyzed a large publicly available neuroimaging dataset to investigate the intrinsic functional connectivities (FCs) of the phPITd with other brain areas. The results showed that the intrinsic FCs of the phPITd with the areas of the visual network and the DAN were significantly stronger than those with the ventral attention network (VAN) areas and areas of other networks. We further conducted individual difference analyses with a sample size of 295 participants and a series of attentional tasks to investigate which attentional components each phPITd-based DAN edge predicts. Our findings revealed that the intrinsic FC of the left phPITd with the LIPv could predict individual ability in attentional orienting, but not in alerting, executive control, and distractor suppression. Our results not only provide direct evidence of the phPITd's functional relationship with the LIPv, but also offer a comprehensive understanding of its specific role in visual attention.

Neural mechanisms of resource allocation in working memory

To mitigate capacity limits of working memory, people allocate resources according to an item's relevance. However, the neural mechanisms supporting such a critical operation remain unknown. Here, we developed computational neuroimaging methods to decode and demix neural responses associated with multiple items in working memory with different priorities. In striate and extrastriate cortex, the gain of neural responses tracked the priority of memoranda. Higher-priority memoranda were decoded with smaller error and lower uncertainty. Moreover, these neural differences predicted behavioral differences in memory prioritization. Remarkably, trialwise variability in the magnitude of delay activity in frontal cortex predicted differences in decoded precision between low and high-priority items in visual cortex. These results suggest a model in which feedback signals broadcast from frontal cortex sculpt the gain of memory representations in visual cortex according to behavioral relevance, thus, identifying a neural mechanism for resource allocation.

Neural evidence for attentional capture by salient distractors

Salient objects often capture our attention, serving as distractors and hindering our current goals. It remains unclear when and how salient distractors interact with our goals, and our knowledge on the neural mechanisms responsible for attentional capture is limited to a few brain regions recorded from non-human primates. Here we conducted a multivariate analysis on human intracranial signals covering most brain regions and successfully dissociated distractor-specific representations from target-arousal signals in the high-frequency (60-100 Hz) activity. We found that salient distractors were processed rapidly around 220 ms, while target-tuning attention was attenuated simultaneously, supporting initial capture by distractors. Notably, neuronal activity specific to the distractor representation was strongest in the superior and middle temporal gyrus, amygdala and anterior cingulate cortex, while there were smaller contributions from the parietal and frontal cortices. These results provide neural evidence for attentional capture by salient distractors engaging a much larger network than previously appreciated.

Intracortical recordings reveal vision-to-action cortical gradients driving human exogenous attention

Exogenous attention, the process that makes external salient stimuli pop-out of a visual scene, is essential for survival. How attention-capturing events modulate human brain processing remains unclear. Here we show how the psychological construct of exogenous attention gradually emerges over large-scale gradients in the human cortex, by analyzing activity from 1,403 intracortical contacts implanted in 28 individuals, while they performed an exogenous attention task. The timing, location and task-relevance of attentional events defined a spatiotemporal gradient of three neural clusters, which mapped onto cortical gradients and presented a hierarchy of timescales. Visual attributes modulated neural activity at one end of the gradient, while at the other end it reflected the upcoming response timing, with attentional effects occurring at the intersection of visual and response signals. These findings challenge multi-step models of attention, and suggest that frontoparietal networks, which process sequential stimuli as separate events sharing the same location, drive exogenous attention phenomena such as inhibition of return.

Feedforward attentional selection in sensory cortex

Salient objects grab attention because they stand out from their surroundings. Whether this phenomenon is accomplished by bottom-up sensory processing or requires top-down guidance is debated. We tested these alternative hypotheses by measuring how early and in which cortical layer(s) neural spiking distinguished a target from a distractor. We measured synaptic and spiking activity across cortical columns in mid-level area V4 of male macaque monkeys performing visual search for a color singleton. A neural signature of attentional capture was observed in the earliest response in the input layer 4. The magnitude of this response predicted response time and accuracy. Errant behavior followed errant selection. Because this response preceded top-down influences and arose in the cortical layer not targeted by top-down connections, these findings demonstrate that feedforward activation of sensory cortex can underlie attentional priority.

Causal Manipulation of Gaze-Following in the Macaque Temporal Cortex

Gaze-following, the ability to shift one's own attention to places or objects others are looking at, is essential for social interactions. Single unit recordings from the monkey cortex and neuroimaging work on the human and monkey brain suggest that a distinct region in the temporal cortex, the gaze-following patch (GFP), underpins this ability. Since previous studies of the GFP have relied on correlational techniques, it remains unclear whether gaze-following related activity in the GFP indicates a causal role rather than being just a reverberation of behaviorally relevant information produced elsewhere. To answer this question, we applied focal electrical and pharmacological perturbation to the GFP. Both approaches, when applied to the GFP, disrupted gaze-following if the monkeys had been instructed to follow gaze, along with the ability to suppress it if vetoed by the context. Hence the GFP is necessary for gaze-following as well as its cognitive control.

A frontoparietal network for volitional control of gaze following

Gaze following is a major element of non-verbal communication and important for successful social interactions. Human gaze following is a fast and almost reflex-like behaviour, yet it can be volitionally controlled and suppressed to some extent if inappropriate or unnecessary, given the social context. In order to identify the neural basis of the cognitive control of gaze following, we carried out an event-related fMRI experiment, in which human subjects' eye movements were tracked while they were exposed to gaze cues in two distinct contexts: A baseline gaze following condition in which subjects were instructed to use gaze cues to shift their attention to a gazed-at spatial target and a control condition in which the subjects were required to ignore the gaze cue and instead to shift their attention to a distinct spatial target to be selected based on a colour mapping rule, requiring the suppression of gaze following. We could identify a suppression-related blood-oxygen-level-dependent (BOLD) response in a frontoparietal network comprising dorsolateral prefrontal cortex (dlPFC), orbitofrontal cortex (OFC), the anterior insula, precuneus, and posterior parietal cortex (PPC). These findings suggest that overexcitation of frontoparietal circuits in turn suppressing the gaze following patch might be a potential cause of gaze following deficits in clinical populations.

Intracranial study in humans: Neural spectral changes during watching comedy movie of Charlie Chaplin

Humor plays a prominent role in our lives. Thus, understanding the cognitive and neural mechanisms of humor is particularly important. Previous studies that investigated neural substrates of humor used functional MRI and to a lesser extent EEG. In the present study, we conducted intracranial recording in human patients, enabling us to obtain the signal with high temporal precision from within specific brain locations. Our analysis focused on the temporal lobe and the surrounding areas, the temporal lobe was most densely covered in our recording. Thirteen patients watched a fragment of a Charlie Chaplin movie. An independent group of healthy participants rated the same movie fragment, helping us to identify the most funny and the least funny frames of the movie. We compared neural activity occurring during the most funny and least funny frames across frequencies in the range of 1-170 Hz. The most funny compared to least funny parts of the movie were associated with activity modulation in the broadband high-gamma (70-170 Hz; mostly activation) and to a lesser extent gamma band (40-69Hz; activation) and low frequencies (1-12 Hz, delta, theta, alpha bands; mostly deactivation). With regard to regional specificity, we found three types of brain areas: (I) temporal pole, middle and inferior temporal gyrus (both anterior and posterior) in which there was both activation in the high-gamma/gamma bands and deactivation in low frequencies; (II) ventral part of the temporal lobe such as the fusiform gyrus, in which there was mostly deactivation the low frequencies; (III) posterior temporal cortex and its environment, such as the middle occipital and the temporo-parietal junction, in which there was activation in the high-gamma/gamma band. Overall, our results suggest that humor appreciation might be achieved by neural activity across the frequency spectrum.

Solving the binding problem: Assemblies form when neurons enhance their firing rate-they don't need to oscillate or synchronize

When we look at an image, its features are represented in our visual system in a highly distributed manner, calling for a mechanism that binds them into coherent object representations. There have been different proposals for the neuronal mechanisms that can mediate binding. One hypothesis is that binding is achieved by oscillations that synchronize neurons representing features of the same perceptual object. This view allows separate communication channels between different brain areas. Another hypothesis is that binding of features that are represented in different brain regions occurs when the neurons in these areas that respond to the same object simultaneously enhance their firing rate, which would correspond to directing object-based attention to these features. This review summarizes evidence in favor of and against these two hypotheses, examining the neuronal correlates of binding and assessing the time course of perceptual grouping. I conclude that enhanced neuronal firing rates bind features into coherent object representations, whereas oscillations and synchrony are unrelated to binding.

Spatiotemporal dynamics of selective attention and visual conflict monitoring using a Stroop task

Selective attention and conflict monitoring are daily human phenomena, yet the spatial and temporal neurological underpinnings of these processes are not fully understood. Current literature suggests these executive functions occur via diverse and highly interconnected neural networks, including top-down, bottom-up, and conflict-control loops. To investigate the spatiotemporal activity of these processes, we collected neuromagnetic data using magnetoencephalography (MEG) in 28 healthy adults (age 19-36), while they performed a computerized Stroop task based on color naming. We focused on low-frequency oscillations in the context of top-down control and hypothesized that conflict monitoring-related activity would first be observed in the left anterior cingulate cortex, followed by the left dorsolateral prefrontal cortex, and subsequently in the parietal and temporal lobes. Significant activity between 600-1000 ms post-stimulus onset was found for incongruent vs. congruent/neutral contrasts. Interestingly, spatiotemporal analysis did not provide evidence for a top-down pattern of activation, instead revealing a simultaneous pattern of activation in the frontal and temporal lobes. Most notable is the involvement of the left posterior inferior temporal cortex (pITC) and the left temporoparietal junction (TPJ), which have not conventionally been considered active players in attentional control. These results may be largely driven by alpha and beta oscillations from our sample population. Our findings challenge early theoretical models of top-down processing in the context of cognitive control from an attention perspective and also suggest a need to investigate attentional centers in the temporal lobe. Furthermore, the study highlights the valuable temporal data provided by MEG, which has been missing from previous studies.

Attention control in the primate brain

Attention is fundamental to all cognition. In the primate brain, it is implemented by a large-scale network that consists of areas spanning across all major lobes, also including subcortical regions. Classical attention accounts assume that control over the selection process in this network is exerted by 'top-down' mechanisms in the fronto-parietal cortex that influence sensory representations via feedback signals. More recent studies have expanded this view of attentional control. In this review, we will start from a traditional top-down account of attention control, and then discuss more recent findings on feature-based attention, thalamic influences, temporal network dynamics, and behavioral dynamics that collectively lead to substantial modifications. We outline how the different emerging accounts can be reconciled and integrated into a unified theory.

What to expect where and when: how statistical learning drives visual selection

While the visual environment contains massive amounts of information, we should not and cannot pay attention to all events. Instead, we need to direct attention to those events that have proven to be important in the past and suppress those that were distracting and irrelevant. Experiences molded through a learning process enable us to extract and adapt to the statistical regularities in the world. While previous studies have shown that visual statistical learning (VSL) is critical for representing higher order units of perception, here we review the role of VSL in attentional selection. Evidence suggests that through VSL, attentional priority settings are optimally adjusted to regularities in the environment, without intention and without conscious awareness.

The role of temporal cortex in the control of attention

Attention is an indispensable component of active vision. Contrary to the widely accepted notion that temporal cortex processing primarily focusses on passive object recognition, a series of very recent studies emphasize the role of temporal cortex structures, specifically the superior temporal sulcus (STS) and inferotemporal (IT) cortex, in guiding attention and implementing cognitive programs relevant for behavioral tasks. The goal of this theoretical paper is to advance the hypothesis that the temporal cortex attention network (TAN) entails necessary components to actively participate in attentional control in a flexible task-dependent manner. First, we will briefly discuss the general architecture of the temporal cortex with a focus on the STS and IT cortex of monkeys and their modulation with attention. Then we will review evidence from behavioral and neurophysiological studies that support their guidance of attention in the presence of cognitive control signals. Next, we propose a mechanistic framework for executive control of attention in the temporal cortex. Finally, we summarize the role of temporal cortex in implementing cognitive programs and discuss how they contribute to the dynamic nature of visual attention to ensure flexible behavior.

Does feature intertrial priming guide attention? The jury is still out

Our search performance is strongly influenced by our past experience. In the lab, this influence has been demonstrated by investigating a variety of phenomena, including intertrial priming, statistical learning, and reward history, and collectively referred to as selection history. The resulting findings have led researchers to claim that selection history guides attention, thereby challenging the prevailing dichotomy, according to which top-down and bottom-up factors alone determine attentional priority. Here, we re-examine this claim with regard to one selection-history phenomenon, feature intertrial priming (aka priming of pop-out). We evaluate the evidence that specifically pertains to the role of feature intertrial priming in attentional guidance, rather than in later selective processes occurring after the target is found. We distinguish between the main experimental rationales, while considering the extent to which feature intertrial priming, as studied through different protocols, shares characteristics of top-down attention. We show that there is strong evidence that feature intertrial priming guides attention when the experimental protocol departs from the canonical paradigm and encourages observers to maintain the critical feature in visual working memory or to form expectations about the upcoming target. By contrast, the current evidence regarding the standard feature intertrial priming phenomenon is inconclusive. We propose directions for future research and suggest that applying the methodology used here in order to re-evaluate of the role of other selection history phenomena in attentional guidance should clarify the mechanisms underlying the strong impact of past experience on visual search performance.

Effects of binaural and monaural beat stimulation on attention and EEG

Nowadays a popular technique to improve mood and cognition is auditory beat stimulation (ABS), which is thought to induce a frequency-following response of brainwaves. The main types of ABS are monaural beats (MB) and binaural beats (BB). BB involves the presentation of a specific frequency to one ear and another frequency to the other ear which may induce neural entrainment. A difference between the frequencies of 40 Hz is assumed to improve cognition. The present study examined the effect of 40 Hz binaural beats (BB) and monaural beats (MB) on attention and electroencephalography (EEG). A total of 25 first-year psychology students (11 males, 14 females) performed a Flanker task while EEG was recorded during the 5 min-presentation of pink noise (PN), MB and BB. With respect to attention, as measured by the Flanker task, the number of false responses in the BB condition was smaller than that in the PN condition while the number of false responses in the MB condition was larger as compared to the PN condition. As there was no association of BB with a consistent increase in absolute 40 or 45 Hz power compared to PN or MB, EEG recordings could not confirm the hypothesized neural entrainment in the brain. Overall, the current findings show that listening to 40 Hz BB improves attention but do not show the occurrence of neural entrainment. Future research is recommended to include a larger sample, to use a broader cognitive test battery and to present auditory beats with a longer duration.

Common functional localizers to enhance NHP & cross-species neuroscience imaging research

Functional localizers are invaluable as they can help define regions of interest, provide cross-study comparisons, and most importantly, allow for the aggregation and meta-analyses of data across studies and laboratories. To achieve these goals within the non-human primate (NHP) imaging community, there is a pressing need for the use of standardized and validated localizers that can be readily implemented across different groups. The goal of this paper is to provide an overview of the value of localizer protocols to imaging research and we describe a number of commonly used or novel localizers within NHPs, and keys to implement them across studies. As has been shown with the aggregation of resting-state imaging data in the original PRIME-DE submissions, we believe that the field is ready to apply the same initiative for task-based functional localizers in NHP imaging. By coming together to collect large datasets across research group, implementing the same functional localizers, and sharing the localizers and data via PRIME-DE, it is now possible to fully test their robustness, selectivity and specificity. To do this, we reviewed a number of common localizers and we created a repository of well-established localizer that are easily accessible and implemented through the PRIME-RE platform.

Combining brain perturbation and neuroimaging in non-human primates

Brain perturbation studies allow detailed causal inferences of behavioral and neural processes. Because the combination of brain perturbation methods and neural measurement techniques is inherently challenging, research in humans has predominantly focused on non-invasive, indirect brain perturbations, or neurological lesion studies. Non-human primates have been indispensable as a neurobiological system that is highly similar to humans while simultaneously being more experimentally tractable, allowing visualization of the functional and structural impact of systematic brain perturbation. This review considers the state of the art in non-human primate brain perturbation with a focus on approaches that can be combined with neuroimaging. We consider both non-reversible (lesions) and reversible or temporary perturbations such as electrical, pharmacological, optical, optogenetic, chemogenetic, pathway-selective, and ultrasound based interference methods. Method-specific considerations from the research and development community are offered to facilitate research in this field and support further innovations. We conclude by identifying novel avenues for further research and innovation and by highlighting the clinical translational potential of the methods.

Transcranial Magnetic Stimulation Over the Right Posterior Superior Temporal Sulcus Promotes the Feature Discrimination Processing

Attention is the dynamic process of allocating limited resources to the information that is most relevant to our goals. Accumulating studies have demonstrated the crucial role of frontal and parietal areas in attention. However, the effect of posterior superior temporal sulcus (pSTS) in attention is still unclear. To address this question, in this study, we measured transcranial magnetic stimulation (TMS)-induced event-related potentials (ERPs) to determine the extent of involvement of the right pSTS in attentional processing. We hypothesized that TMS would enhance the activation of the right pSTS during feature discrimination processing. We recruited 21 healthy subjects who performed the dual-feature delayed matching task while undergoing single-pulse sham or real TMS to the right pSTS 300 ms before the second stimulus onset. The results showed that the response time was reduced by real TMS of the pSTS as compared to sham stimulation. N270 amplitude was reduced during conflict processing, and the time-varying network analysis revealed increased connectivity between the frontal lobe and temporo-parietal and occipital regions. Thus, single-pulse TMS of the right pSTS enhances feature discrimination processing and task performance by reducing N270 amplitude and increasing connections between the frontal pole and temporo-parietal and occipital regions. These findings provide evidence that the right pSTS facilitates feature discrimination by accelerating the formation of a dynamic network.

Midbrain activity shapes high-level visual properties in the primate temporal cortex

Recent fMRI experiments identified an attention-related region in the macaque temporal cortex, here called the floor of the superior temporal sulcus (fSTS), as the primary cortical target of superior colliculus (SC) activity. However, it remains unclear which aspects of attention are processed by fSTS neurons and how or why these might depend on SC activity. Here, we show that SC inactivation decreases attentional modulations in fSTS neurons by increasing their activity for ignored stimuli in addition to decreasing their activity for attended stimuli. Neurons in the fSTS also exhibit event-related activity during attention tasks linked to detection performance, and this link is eliminated during SC inactivation. Finally, fSTS neurons respond selectively to particular visual objects, and this selectivity is reduced markedly during SC inactivation. These diverse, high-level properties of fSTS neurons all involve visual signals that carry behavioral relevance. Their dependence on SC activity could reflect a circuit that prioritizes cortical processing of events detected subcortically.

The human endogenous attentional control network includes a ventro-temporal cortical node

Endogenous attention is the cognitive function that selects the relevant pieces of sensory information to achieve goals and it is known to be controlled by dorsal fronto-parietal brain areas. Here we expand this notion by identifying a control attention area located in the temporal lobe. By combining a demanding behavioral paradigm with functional neuroimaging and diffusion tractography, we show that like fronto-parietal attentional areas, the human posterior inferotemporal cortex exhibits significant attentional modulatory activity. This area is functionally distinct from surrounding cortical areas, and is directly connected to parietal and frontal attentional regions. These results show that attentional control spans three cortical lobes and overarches large distances through fiber pathways that run orthogonally to the dominant anterior-posterior axes of sensory processing, thus suggesting a different organizing principle for cognitive control.

Task-Irrelevant Visual Forms Facilitate Covert and Overt Spatial Selection

Covert and overt spatial selection behaviors are guided by both visual saliency maps derived from early visual features as well as priority maps reflecting high-level cognitive factors. However, whether mid-level perceptual processes associated with visual form recognition contribute to covert and overt spatial selection behaviors remains unclear. We hypothesized that if peripheral visual forms contribute to spatial selection behaviors, then they should do so even when the visual forms are task-irrelevant. We tested this hypothesis in male and female human subjects as well as in male macaque monkeys performing a visual detection task. In this task, subjects reported the detection of a suprathreshold target spot presented on top of one of two peripheral images, and they did so with either a speeded manual button press (humans) or a speeded saccadic eye movement response (humans and monkeys). Crucially, the two images, one with a visual form and the other with a partially phase-scrambled visual form, were completely irrelevant to the task. In both manual (covert) and oculomotor (overt) response modalities, and in both humans and monkeys, response times were faster when the target was congruent with a visual form than when it was incongruent. Importantly, incongruent targets were associated with almost all errors, suggesting that forms automatically captured selection behaviors. These findings demonstrate that mid-level perceptual processes associated with visual form recognition contribute to covert and overt spatial selection. This indicates that neural circuits associated with target selection, such as the superior colliculus, may have privileged access to visual form information.SIGNIFICANCE STATEMENT Spatial selection of visual information either with (overt) or without (covert) foveating eye movements is critical to primate behavior. However, it is still not clear whether spatial maps in sensorimotor regions known to guide overt and covert spatial selection are influenced by peripheral visual forms. We probed the ability of humans and monkeys to perform overt and covert target selection in the presence of spatially congruent or incongruent visual forms. Even when completely task-irrelevant, images of visual objects had a dramatic effect on target selection, acting much like spatial cues used in spatial attention tasks. Our results demonstrate that traditional brain circuits for orienting behaviors, such as the superior colliculus, likely have privileged access to visual object representations.

Social interaction networks in the primate brain

Primate brains have evolved to understand and engage with their social world. Much about the structure of this world can be gleaned from social interactions. Circuits for the analysis of and participation in social interactions have now been mapped. Increased knowledge about their functional specializations and relative spatial locations promises to greatly improve the understanding of the functional organization of the primate social brain. Detailed electrophysiology, as in the case of the face-processing network, of local operations and functional interactions between areas is necessary to uncover neural mechanisms and computation principles of social cognition. New naturalistic behavioral paradigms, behavioral tracking, and new analytical approaches for parallel non-stationary data will be important components toward a neuroscientific theory of primates' interactive minds.

Decoding of the other's focus of attention by a temporal cortex module

Faces attract the observer's attention toward objects and locations of interest for the other, thereby allowing the two agents to establish joint attention. Previous work has delineated a network of cortical "patches" in the macaque cortex, processing faces, eventually also extracting information on the other's gaze direction. Yet, the neural mechanism that links information on gaze direction, guiding the observer's attention to the relevant object, has remained elusive. Here we present electrophysiological evidence for the existence of a distinct "gaze-following patch" (GFP) with neurons that establish this linkage in a highly flexible manner. The other's gaze and the object, singled out by the gaze, are linked only if this linkage is pertinent within the prevailing social context. The properties of these neurons establish the GFP as a key switch in controlling social interactions based on the other's gaze.