Attention Periodically Binds Visual Features As Single Events Depending on Neural Oscillations Phase-Locked to Action

Theoretical and Technical Issues Concerning the Measurement of Alpha Frequency and the Application of Signal Detection Theory: Comment on Buergers and Noppeney (2022)

Classical and recent evidence has suggested that alpha oscillations play a critical role in temporally discriminating or binding successively presented items. Challenging this view, Buergers and Noppeney [Buergers, S., & Noppeney, U. The role of alpha oscillations in temporal binding within and across the senses. Nature Human Behaviour, 6, 732-742, 2022] found that by combining EEG, psychophysics, and signal detection theory, neither prestimulus nor resting-state alpha frequency influences perceptual sensitivity and bias in the temporal binding task. We propose the following four points that should be considered when interpreting the role of alpha oscillations, and especially their frequency, on perceptual temporal binding: (1) Multiple alpha components can be contaminated in conventional EEG analysis; (2) the effect of alpha frequency on perception will interact with alpha power; (3) prestimulus and resting-state alpha frequency can be different from poststimulus alpha frequency, which is the frequency during temporal binding and should be more directly related to temporal binding; and (4) when applying signal detection theory under the assumption of equal variance, the assumption is often incomplete and can be problematic (e.g., the magnitude relationships between individuals in parametric sensitivity may change when converted into nonparametric sensitivity). Future directions, including solutions to each of the issues, are discussed.

The self and conscious experience

The primary determinant of the self (S) is the conscious experience (CE) we have of it. Therefore, it does not come as a surprise that empirical research on S mainly resorts to the CE (or lack of CE) that subjects have of their S. What comes as a surprise is that empirical research on S does not tackle the problem of how CE contributes to building S. Empirical research investigates how S either biases the cognitive processing of stimuli or is altered through a wide range of means (meditation, hypnosis, etc.). In either case, even for different reasons, considerations of how CE contributes to building S are left unspecified in empirical research. This article analyzes these reasons and proposes a theoretical model of how CE contributes to building S. According to the proposed model, the phenomenal aspect of consciousness is produced by the modulation-engendered by attentional activity-of the energy level of the neural substrate (that is, the organ of attention) that underpins attentional activity. The phenomenal aspect of consciousness supplies the agent with a sense of S and informs the agent on how its S is affected by the agent's own operations. The phenomenal aspect of consciousness performs its functions through its five main dimensions: qualitative, quantitative, hedonic, temporal, and spatial. Each dimension of the phenomenal aspect of consciousness can be explained by a specific aspect of the modulation of the energy level of the organ of attention. Among other advantages, the model explains the various forms of S as outcomes resulting from the operations of a single mechanism and provides a unifying framework for empirical research on the neural underpinnings of S.

Scale-invariant changes in corticospinal excitability reflect multiplexed oscillations in the motor output

In the absence of disease, humans produce smooth and accurate movement trajectories. Despite such 'macroscopic' aspect, the 'microscopic' structure of movements reveals recurrent (quasi-rhythmic) discontinuities. To date, it is unclear how the sensorimotor system contributes to the macroscopic and microscopic architecture of movement. Here, we investigated how corticospinal excitability changes in relation to microscopic fluctuations that are naturally embedded within larger macroscopic variations in motor output. Participants performed a visuomotor tracking task. In addition to the 0.25 Hz modulation that is required for task fulfilment (macroscopic scale), the motor output shows tiny but systematic fluctuations at ∼2 and 8 Hz (microscopic scales). We show that motor-evoked potentials (MEPs) elicited by transcranial magnetic stimulation (TMS) during task performance are consistently modulated at all (time) scales. Surprisingly, MEP modulation covers a similar range at both micro- and macroscopic scales, even though the motor output differs by several orders of magnitude. Thus, corticospinal excitability finely maps the multiscale temporal patterning of the motor output, but it does so according to a principle of scale invariance. These results suggest that corticospinal excitability indexes a relatively abstract level of movement encoding that may reflect the hierarchical organisation of sensorimotor processes. KEY POINTS: Motor behaviour is organised on multiple (time)scales. Small but systematic ('microscopic') fluctuations are engrained in larger and slower ('macroscopic') variations in motor output, which are instrumental in deploying the desired motor plan. Corticospinal excitability is modulated in relation to motor fluctuations on both macroscopic and microscopic (time)scales. Corticospinal excitability obeys a principle of scale invariance, that is, it is modulated similarly at all (time)scales, possibly reflecting hierarchical mechanisms that optimise motor encoding.

The microstructure of intra- and interpersonal coordination

Movements are naturally composed of submovements, i.e. recurrent speed pulses (2-3 Hz), possibly reflecting intermittent feedback-based motor adjustments. In visuomotor (unimanual) synchronization tasks, partners alternate submovements over time, indicating mutual coregulation. However, it is unclear whether submovement coordination is organized differently between and within individuals. Indeed, different types of information may be variably exploited for intrapersonal and interpersonal coordination. Participants performed a series of bimanual tasks alone or in pairs, with or without visual feedback (solo task only). We analysed the relative timing of submovements between their own hands or between their own hands and those of their partner. Distinct coordinative structures emerged at the submovement level depending on the relevance of visual feedback. Specifically, the relative timing of submovements (between partners/effectors) shifts from alternation to simultaneity and a mixture of both when coordination is achieved using vision (interpersonal), proprioception/efference-copy only (intrapersonal, without vision) or all information sources (intrapersonal, with vision), respectively. These results suggest that submovement coordination represents a behavioural proxy for the adaptive weighting of different sources of information within action-perception loops. In sum, the microstructure of movement reveals common principles governing the dynamics of sensorimotor control to achieve both intra- and interpersonal coordination.

SoftMatch: Comparing Scanpaths Using Combinatorial Spatio-Temporal Sequences with Fractal Curves

Recent studies matching eye gaze patterns with those of others contain research that is heavily reliant on string editing methods borrowed from early work in bioinformatics. Previous studies have shown string editing methods to be susceptible to false negative results when matching mutated genes or unordered regions of interest in scanpaths. Even as new methods have emerged for matching amino acids using novel combinatorial techniques, scanpath matching is still limited by a traditional collinear approach. This approach reduces the ability to discriminate between free viewing scanpaths of two people looking at the same stimulus due to the heavy weight placed on linearity. To overcome this limitation, we here introduce a new method called SoftMatch to compare pairs of scanpaths. SoftMatch diverges from traditional scanpath matching in two different ways: firstly, by preserving locality using fractal curves to reduce dimensionality from 2D Cartesian (x,y) coordinates into 1D (h) Hilbert distances, and secondly by taking a combinatorial approach to fixation matching using discrete Fréchet distance measurements between segments of scanpath fixation sequences. These matching "sequences of fixations over time" are a loose acronym for SoftMatch. Results indicate high degrees of statistical and substantive significance when scoring matches between scanpaths made during free-form viewing of unfamiliar stimuli. Applications of this method can be used to better understand bottom up perceptual processes extending to scanpath outlier detection, expertise analysis, pathological screening, and salience prediction.

Attention rhythmically samples multi-feature objects in working memory

Attention allows us to selectively enhance processing of specific locations or features in our external environment while filtering out irrelevant information. It is currently hypothesized that this is achieved through boosting of relevant sensory signals which biases the competition between neural representations. Recent neurophysiological and behavioral studies revealed that attention is a fundamentally rhythmic process, tightly linked to neural oscillations in frontoparietal networks. Instead of continuously highlighting a single object or location, attention rhythmically alternates between multiple relevant representations at a frequency of 3–8 Hz. However, attention cannot only be directed towards the external world but also towards internal visual working memory (VWM) representations, e.g. when selecting one of several search templates to find corresponding objects in the external world. Two recent studies demonstrate that single-feature objects in VWM are attended in a similar rhythmic fashion as perceived objects. Here we add to the literature by showing that non-spatial retro-cues initiate comparable theta-rhythmic sampling of multi-feature objects in VWM. Our findings add to the converging body of evidence that external and internal visual representations are accessed by similar rhythmic attentional mechanisms and present a potential solution to the binding problem in working memory.

Reclaiming saliency: Rhythmic precision-modulated action and perception

Computational models of visual attention in artificial intelligence and robotics have been inspired by the concept of a saliency map. These models account for the mutual information between the (current) visual information and its estimated causes. However, they fail to consider the circular causality between perception and action. In other words, they do not consider where to sample next, given current beliefs. Here, we reclaim salience as an active inference process that relies on two basic principles: uncertainty minimization and rhythmic scheduling. For this, we make a distinction between attention and salience. Briefly, we associate attention with precision control, i.e., the confidence with which beliefs can be updated given sampled sensory data, and salience with uncertainty minimization that underwrites the selection of future sensory data. Using this, we propose a new account of attention based on rhythmic precision-modulation and discuss its potential in robotics, providing numerical experiments that showcase its advantages for state and noise estimation, system identification and action selection for informative path planning.

Interpersonal synchronization of movement intermittency

Most animal species group together and coordinate their behavior in quite sophisticated manners for mating, hunting, or defense purposes. In humans, coordination at a macroscopic level (the pacing of movements) is evident both in daily life (e.g., walking) and skilled (e.g., music and dance) behaviors. By examining the fine structure of movement, we here show that interpersonal coordination is established also at a microscopic – submovement – level. Natural movements appear as marked by recurrent (2–3 Hz) speed breaks, i.e., submovements, that are traditionally considered the result of intermittency in (visuo)motor feedback-based control. In a series of interpersonal coordination tasks, we show that submovements produced by interacting partners are not independent but alternate tightly over time, reflecting online mutual adaptation. These findings unveil a potential core mechanism for behavioral coordination that is based on between-persons synchronization of the intrinsic dynamics of action-perception cycles.

•

Movements show intermittent speed pulses occurring at 2–3 Hz, called submovements

•

Submovements are actively coordinated in counter-phase by interacting partners

•

Submovements coordination depends on spatial alignment but not movement congruency

•

Behavioral coordination occurs both at macro- and microscopic movement scales

Theta-Rhythmic Oscillation of Working Memory Performance

Representations held in working memory are crucial in guiding human attention in a goal-directed fashion. Currently, it is debated whether only a single representation or several of these representations can be active and bias behavior at any given moment. In the present study, 25 university students performed a behavioral dense-sampling experiment to produce an estimate of the temporal-activation patterns of two simultaneously held visual templates. We report two key novel results. First, performance related to both representations was not continuous but fluctuated rhythmically at 6 Hz. This corresponds to neural oscillations in the theta band, the functional importance of which in working memory is well established. Second, our findings suggest that two concurrently held representations may be prioritized in alternation, not simultaneously. Our data extend recent research on rhythmic sampling of external information by demonstrating an analogous mechanism in the cyclic activation of internal working memory representations.

Prediction of cortical theta oscillations in humans for phase-locked visual stimulation

BACKGROUND

The timing of an event within an oscillatory phase is considered to be one of the key strategies used by the brain to code and process neural information. Whereas existing methods of studying this phenomenon are chiefly based on retrospective analysis of electroencephalography (EEG) data, we now present a method to study it prospectively. New method: We present a system that allows for the delivery of visual stimuli at a specific phase of the cortical theta oscillation by fitting a sine to raw surface EEG data to estimate and predict the phase. One noteworthy feature of the method is that it can minimize potentially confounding effects of previous trials by using only a short sequence of past data.

RESULTS

In a trial with 10 human participants we achieved a significant phase locking with an inter-trial phase coherence of 0.39. We demonstrated successful phase locking on synthetic signals with a signal-to-noise ratio of less than - 20 dB. Comparison with existing method(s): We compared the new method to an autoregressive method published in the literature and found the new method was superior in mean phase offset, circular standard deviation, and prediction latency.

CONCLUSIONS

By fitting sine waves to raw EEG traces, we locked visual stimuli to arbitrary phases within the theta oscillatory cycle of healthy humans.

The neural bases of spatial attention and perceptual rhythms

Attentional processes allow the brain to overcome its processing capacities limitations by enhancing relevant visual information and suppressing irrelevant information. Thus attention plays a critical role, shaping our perception of the world. Several models have been proposed to describe the neuronal bases of attention and its mechanistic underlyings. Recent electrophysiological evidence show that attentional processes rely on oscillatory brain activities that correlate with rhythmic changes in cognitive performance. In the present review, we first take a historical perspective on how attention is viewed, from the initial spotlight theory of attention to the recent dynamic view of attention selection and we review their supporting psychophysical evidence. Based on recent prefrontal electrophysiological evidence, we refine the most recent models of attention sampling by proposing a rhythmic and continuous model of attentional sampling. In particular, we show that attention involves a continuous exploration of space, shifting within and across visual hemifield at specific alpha and theta rhythms, independently of the current attentional load. In addition, we show that this prefrontal attentional spotlight implements conjointly selection and suppression mechanisms, and is captured by salient incoming items. Last, we argue that this attention spotlight implements a highly flexible alternation of attentional exploration and exploitation epochs, depending on ongoing task contingencies. In a last part, we review the local and network oscillatory mechanisms that correlate with rhythmic attentional sampling, describing multiple rhythmic generators and complex network interactions.

Visual detection is locked to the internal dynamics of cortico-motor control

Movements overtly sample sensory information, making sensory analysis an active-sensing process. In this study, we show that visual information sampling is not just locked to the (overt) movement dynamics but to the internal (covert) dynamics of cortico-motor control. We asked human participants to perform continuous isometric contraction while detecting unrelated and unpredictable near-threshold visual stimuli. The motor output (force) shows zero-lag coherence with brain activity (recorded via electroencephalography) in the beta-band, as previously reported. In contrast, cortical rhythms in the alpha-band systematically forerun the motor output by 200 milliseconds. Importantly, visual detection is facilitated when cortico-motor alpha (not beta) synchronization is enhanced immediately before stimulus onset, namely, at the optimal phase relationship for sensorimotor communication. These findings demonstrate an ongoing coupling between visual sampling and motor control, suggesting the operation of an internal and alpha-cycling visuomotor loop.

A New Unifying Account of the Roles of Neuronal Entrainment

Rhythms are a fundamental and defining feature of neuronal activity in animals including humans. This rhythmic brain activity interacts in complex ways with rhythms in the internal and external environment through the phenomenon of 'neuronal entrainment', which is attracting increasing attention due to its suggested role in a multitude of sensory and cognitive processes. Some senses, such as touch and vision, sample the environment rhythmically, while others, like audition, are faced with mostly rhythmic inputs. Entrainment couples rhythmic brain activity to external and internal rhythmic events, serving fine-grained routing and modulation of external and internal signals across multiple spatial and temporal hierarchies. This interaction between a brain and its environment can be experimentally investigated and even modified by rhythmic sensory stimuli or invasive and non-invasive neuromodulation techniques. We provide a comprehensive overview of the topic and propose a theoretical framework of how neuronal entrainment dynamically structures information from incoming neuronal, bodily and environmental sources. We discuss the different types of neuronal entrainment, the conceptual advances in the field, and converging evidence for general principles.