EEG frequency tagging dissociates between neural processing of motion synchrony and human quality of multiple point-light dancers

Measuring attentional selection of object categories using hierarchical frequency tagging

In the present study, we used Hierarchical Frequency Tagging (Gordon et al., 2017) to investigate in electroencephalography how different levels of the neural processing hierarchy interact with category-selective attention during visual object recognition. We constructed stimulus sequences of cyclic wavelet scrambled face and house stimuli at two different frequencies (f1 = 0.8 Hz and f2 = 1 Hz). For each trial, two stimulus sequences of different frequencies were superimposed and additionally augmented by a sinusoidal contrast modulation with f3 = 12.5 Hz. This allowed us to simultaneously assess higher level processing using semantic wavelet-induced frequency-tagging (SWIFT) and processing in earlier visual levels using steady-state visually evoked potentials (SSVEPs), along with their intermodulation (IM) components. To investigate the category specificity of the SWIFT signal, we manipulated the category congruence between target and distractor by superimposing two sequences containing stimuli from the same or different object categories. Participants attended to one stimulus (target) and ignored the other (distractor). Our results showed successful tagging of different levels of the cortical hierarchy. Using linear mixed-effects modeling, we detected different attentional modulation effects on lower versus higher processing levels. SWIFT and IM components were substantially increased for target versus distractor stimuli, reflecting attentional selection of the target stimuli. In addition, distractor stimuli from the same category as targets elicited stronger SWIFT signals than distractor stimuli from a different category indicating category-selective attention. In contrast, for IM components, this category-selective attention effect was largely absent, indicating that IM components probably reflect more stimulus-specific processing.

Evidence for a role of synchrony but not common fate in the perception of biological group movements

Extensive research has shown that observers are able to efficiently extract summary information from groups of people. However, little is known about the cues that determine whether multiple people are represented as a social group or as independent individuals. Initial research on this topic has primarily focused on the role of static cues. Here, we instead investigate the role of dynamic cues. In two experiments with male and female human participants, we use EEG frequency tagging to investigate the influence of two fundamental Gestalt principles - synchrony and common fate - on the grouping of biological movements. In Experiment 1, we find that brain responses coupled to four point-light figures walking together are enhanced when they move in sync vs. out of sync, but only when they are presented upright. In contrast, we found no effect of movement direction (i.e., common fate). In Experiment 2, we rule out that synchrony takes precedence over common fate by replicating the null effect of movement direction while keeping synchrony constant. These results suggest that synchrony plays an important role in the processing of biological group movements. In contrast, the role of common fate is less clear and will require further research.

Bayesian analysis of phase data in EEG and MEG

Electroencephalography and magnetoencephalography recordings are non-invasive and temporally precise, making them invaluable tools in the investigation of neural responses in humans. However, these recordings are noisy, both because the neuronal electrodynamics involved produces a muffled signal and because the neuronal processes of interest compete with numerous other processes, from blinking to day-dreaming. One fruitful response to this noisiness has been to use stimuli with a specific frequency and to look for the signal of interest in the response at that frequency. Typically this signal involves measuring the coherence of response phase: here, a Bayesian approach to measuring phase coherence is described. This Bayesian approach is illustrated using two examples from neurolinguistics and its properties are explored using simulated data. We suggest that the Bayesian approach is more descriptive than traditional statistical approaches because it provides an explicit, interpretable generative model of how the data arises. It is also more data-efficient: it detects stimulus-related differences for smaller participant numbers than the standard approach.

Intermodulation responses show integration of interacting bodies in a new whole

People are often seen among other people, relating to and interacting with one another. Recent studies suggest that socially relevant spatial relations between bodies, such as the face-to-face positioning, or facingness, change the visual representation of those bodies, relative to when the same items appear unrelated (e.g., back-to-back) or in isolation. The current study addresses the hypothesis that face-to-face bodies give rise to a new whole, an integrated representation of individual bodies in a new perceptual unit. Using frequency-tagging EEG, we targeted, as a measure of integration, an EEG correlate of the non-linear combination of the neural responses to each of two individual bodies presented either face-to-face as if interacting, or back-to-back. During EEG recording, participants (N = 32) viewed two bodies, either face-to-face or back-to-back, flickering at two different frequencies (F1 and F2), yielding two distinctive responses in the EEG signal. Spectral analysis examined the responses at the intermodulation frequencies (nF1±mF2), signaling integration of individual responses. An anterior intermodulation response was observed for face-to-face bodies, but not for back-to-back bodies, nor for face-to-face chairs and machines. These results show that interacting bodies are integrated into a representation that is more than the sum of its parts. This effect, specific to body dyads, may mark an early step in the transformation towards an integrated representation of a social event, from the visual representation of individual participants in that event.

The Neuroscience of Dance: A Conceptual Framework and Systematic Review

Ancient and culturally universal, dance pervades many areas of life and has multiple benefits. In this article, we provide a conceptual framework and systematic review, as a guide for researching the neuroscience of dance. We identified relevant articles following PRISMA guidelines, and summarised and evaluated all original results. We identified avenues for future research in: the interactive and collective aspects of dance; groove; dance performance; dance observation; and dance therapy. Furthermore, the interactive and collective aspects of dance constitute a vital part of the field but have received almost no attention from a neuroscientific perspective so far. Dance and music engage overlapping brain networks, including common regions involved in perception, action, and emotion. In music and dance, rhythm, melody, and harmony are processed in an active, sustained pleasure cycle giving rise to action, emotion, and learning, led by activity in specific hedonic brain networks. The neuroscience of dance is an exciting field, which may yield information concerning links between psychological processes and behaviour, human flourishing, and the concept of eudaimonia.

Using EEG movement tagging to isolate brain responses coupled to biological movements

Detecting biological motion is essential for adaptive social behavior. Previous research has revealed the brain processes underlying this ability. However, brain activity during biological motion perception captures a multitude of processes. As a result, it is often unclear which processes reflect movement processing and which processes reflect secondary processes that build on movement processing. To address this issue, we developed a new approach to measure brain responses directly coupled to observed movements. Specifically, we showed 30 male and female adults a point-light walker moving at a pace of 2.4 Hz and used EEG frequency tagging to measure the brain response coupled to that pace ('movement tagging'). The results revealed a reliable response at the walking frequency that was reduced by two manipulations known to disrupt biological motion perception: phase scrambling and inversion. Interestingly, we also identified a brain response at half the walking frequency (i.e., 1.2 Hz), corresponding to the rate at which the individual dots completed a cycle. In contrast to the 2.4 Hz response, the response at 1.2 Hz was increased for scrambled (vs. unscrambled) walkers. These results show that frequency tagging can be used to capture the visual processing of biological movements and can dissociate between global (2.4 Hz) and local (1.2 Hz) processes involved in biological motion perception, at different frequencies of the brain signal.

Neural correlates of integration processes during dynamic face perception

Integrating the spatiotemporal information acquired from the highly dynamic world around us is essential to navigate, reason, and decide properly. Although this is particularly important in a face-to-face conversation, very little research to date has specifically examined the neural correlates of temporal integration in dynamic face perception. Here we present statistically robust observations regarding the brain activations measured via electroencephalography (EEG) that are specific to the temporal integration. To that end, we generate videos of neutral faces of individuals and non-face objects, modulate the contrast of the even and odd frames at two specific frequencies (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f_1$$\end{document}f1 and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f_2$$\end{document}f2) in an interlaced manner, and measure the steady-state visual evoked potential as participants view the videos. Then, we analyze the intermodulation components (IMs: (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$nf_1\pm mf_2$$\end{document}nf1±mf2), a linear combination of the fundamentals with integer multipliers) that consequently reflect the nonlinear processing and indicate temporal integration by design. We show that electrodes around the medial temporal, inferior, and medial frontal areas respond strongly and selectively when viewing dynamic faces, which manifests the essential processes underlying our ability to perceive and understand our social world. The generation of IMs is only possible if even and odd frames are processed in succession and integrated temporally, therefore, the strong IMs in our frequency spectrum analysis show that the time between frames (1/60 s) is sufficient for temporal integration.

EEG Frequency Tagging Reveals the Integration of Form and Motion Cues into the Perception of Group Movement

The human brain has dedicated mechanisms for processing other people’s movements. Previous research has revealed how these mechanisms contribute to perceiving the movements of individuals but has left open how we perceive groups of people moving together. Across three experiments, we test whether movement perception depends on the spatiotemporal relationships among the movements of multiple agents. In Experiment 1, we combine EEG frequency tagging with apparent human motion and show that posture and movement perception can be dissociated at harmonically related frequencies of stimulus presentation. We then show that movement but not posture processing is enhanced when observing multiple agents move in synchrony. Movement processing was strongest for fluently moving synchronous groups (Experiment 2) and was perturbed by inversion (Experiment 3). Our findings suggest that processing group movement relies on binding body postures into movements and individual movements into groups. Enhanced perceptual processing of movement synchrony may form the basis for higher order social phenomena such as group alignment and its social consequences.

Blending into the Crowd: Electrophysiological Evidence of Gestalt Perception of a Human Dyad

Human faces and bodies are environmental stimuli of special importance that the brain processes with selective attention and a highly specialized visual system. It has been shown recently that the human brain also has dedicated networks for perception of pluralities of human bodies in synchronous motion or in face-to-face interaction. Here we show that a plurality of human bodies that are merely in close spatial proximity are automatically integrated into a coherent perceptual unit. We used an EEG frequency tagging technique allowing the dissociation of the brain activity related to the component parts of an image from the activity related to the global image configuration. We presented to participants images of two silhouettes flickering at different frequencies (5.88 vs. 7.14 Hz). Clear response at these stimulation frequencies reflected response to each part of the dyad. An emerging intermodulation component (7.14 + 5.88 = 13.02 Hz), a nonlinear response regarded as an objective signature of holistic representation, was significantly enhanced in the (typical) upright relative to an (altered) inverted position. Moreover, the inversion effect was significant for the intermodulation component but not for the stimulation frequencies, suggesting a trade-off between the processing of the global dyad configuration and that of the structural properties of the dyad elements. Our results show that when presented with two humans merely in close proximity the perceptual visual system will bind them. Hence the perception of the human form might be of a fundamentally different nature when it is part of a plurality.

Effects of Temporal Expectations on the Perception of Motion Gestalts

Gestalt psychology has traditionally ignored the role of attention in perception, leading to the view that autonomous processes create perceptual configurations that are then attended. More recent research, however, has shown that spatial attention influences a form of Gestalt perception: the coherence of random-dot kinematograms (RDKs). Using ERPs, we investigated whether temporal expectations exert analogous attentional effects on the perception of coherence level in RDKs. Participants were presented fixed-length sequences of RDKs and reported the coherence level of a target RDK. The target was indicated immediately after its appearance by a postcue. Target expectancy increased as the sequence progressed until target presentation; afterward, remaining RDKs were perceived without target expectancy. Expectancy influenced the amplitudes of ERP components P1 and N2. Crucially, expectancy interacted with coherence level at N2, but not at P1. Specifically, P1 amplitudes decreased linearly as a function of RDK coherence irrespective of expectancy, whereas N2 exhibited a quadratic dependence on coherence: larger amplitudes for RDKs with intermediate coherence levels, and only when they were expected. These results suggest that expectancy at early processing stages is an unspecific, general readiness for perception. At later stages, expectancy becomes stimulus specific and nonlinearly related to Gestalt coherence.

The Validity of Steady-State Visual Evoked Potentials as Attention Tags and Input Signals: A Critical Perspective of Frequency Allocation and Number of Stimuli

Steady-state visual evoked potential (SSVEP) is a periodic response to a repetitive visual stimulus at a specific frequency. Currently, SSVEP is widely treated as an attention tag in cognitive activities and is used as an input signal for brain-computer interfaces (BCIs). However, whether SSVEP can be used as a reliable indicator has been a controversial issue. We focused on the independence of SSVEP from frequency allocation and number of stimuli. First, a cue-target paradigm was adopted to examine the interaction between SSVEPs evoked by two stimuli with different frequency allocations under different attention conditions. Second, we explored whether signal strength and the performance of SSVEP-based BCIs were affected by the number of stimuli. The results revealed that no significant interaction of SSVEP responses appeared between attended and unattended stimuli under various frequency allocations, regardless of their appearance in the fundamental or second-order harmonic. The amplitude of SSVEP suffered no significant gain or loss under different numbers of stimuli, but the performance of SSVEP-based BCIs varied along with duration of stimuli; that is, the recognition rate was not affected by the number of stimuli when the duration of stimuli was long enough, while the information transfer rate (ITR) presented the opposite trend. It can be concluded that SSVEP is a reliable tool for marking and monitoring multiple stimuli simultaneously in cognitive studies, but much caution should be taken when choosing a suitable duration and the number of stimuli, in order to achieve optimal utility of BCIs in the future.

Enhanced perceptual processing of self-generated motion: Evidence from steady-state visual evoked potentials

The sense of agency emerges when our voluntary actions produce anticipated or predictable outcomes in the external world. It remains unclear how the sense of control also influences our perception of the external world. The present study examined perceptual processing of self-generated motion versus non-self-generated motion using steady-state visual evoked potentials (SSVEPs). Participants continuously moved their finger on a touchpad to trigger the movements of two shapes (Experiment 1) or two groups of dots (Experiment 2) on a monitor. Degree of control was manipulated by varying the spatial relation between finger movement and stimulus trajectory across conditions. However, the velocity, onset time, and offset time of visual stimuli always corresponded to participants' finger movement. Stimuli flickered at a frequency of either 7.5 Hz or 10 Hz, thus SSVEPs of these frequencies and their harmonics provided a frequency-tagged measurement of perceptual processing. Participants triggered the motion of all stimuli simultaneously, but had greater levels of control over some stimuli than over others. Their task was to detect a brief colour change on the border(s) of one shape (Experiment 1) or of one group of dots (Experiment 2). Although control over shapes/dots was irrelevant to the visual detection task, we found stronger SSVEPs for stimuli that were under a high level of control, compared with the stimuli that were under a low level of control. Our results suggest that the spatial regularity between self-generated movements and visual input boosted the neural responses underlying perceptual processing. Our results support the preactivation account of sensory attenuation, suggesting that perceptual processing of self-generated events is enhanced rather than inhibited.

Measuring Integration Processes in Visual Symmetry with Frequency-Tagged EEG

Symmetry is a highly salient feature of the natural world which requires integration of visual features over space. The aim of the current work is to isolate dynamic neural correlates of symmetry-specific integration processes. We measured steady-state visual evoked potentials (SSVEP) as participants viewed symmetric patterns comprised of distinct spatial regions presented at two different frequencies (f₁ and f₂). We measured intermodulation components, shown to reflect non-linear processing at the neural level, indicating integration of spatially separated parts of the pattern. We generated a wallpaper pattern containing two reflection symmetry axes by tiling the plane with a two-fold reflection symmetric unit-pattern and split each unit-pattern diagonally into separate parts which could be presented at different frequencies. We compared SSVEPs measured for wallpapers and control patterns for which both images were equal in terms of translation and rotation symmetry but reflection symmetry could only emerge for the wallpaper pattern through integration of the image-pairs. We found that low-frequency intermodulation components differed between the wallpaper and control stimuli, indicating the presence of integration mechanisms specific to reflection symmetry. These results showed that spatial integration specific to symmetry perception can be isolated through a combination of stimulus design and the frequency tagging approach.

EEG frequency tagging reveals higher order intermodulation components as neural markers of learned holistic shape representations

Shape perception is intrinsically holistic: combinations of features give rise to configurations with emergent properties that are different from the sum of the parts. The current study investigated neural markers of holistic shape representations learned by means of categorization training. We used the EEG frequency tagging technique, where two parts of a shape stimulus were 'tagged' by modifying their contrast at different temporal frequencies. Signals from both parts are integrated and, as a result, emergent frequency components (so-called, intermodulation responses, IMs), caused by nonlinear interaction of two frequency signals, are observed in the EEG spectrum. First, participants were trained in 4 sessions to discriminate highly similar, unfamiliar shapes into two categories, defined based on the combination of features. After training, EEG was recorded while frequency-tagged shapes from either the trained or the untrained shape family were presented. For all IMs combined, no learning effects were detected, but post hoc analyses of higher-order IMs revealed stronger occipital and occipito-temporal IMs for both trained and untrained exemplars of the trained shape family as compared to the untrained shape family. In line with recent findings, we suggest that the higher-order IMs may reflect high-level visual computations, like holistic shape categorization, resulting from a cascade of non-linear operations. Higher order frequency responses are relatively low in power, hence results should be interpreted cautiously and future research is needed to confirm these effects. In general, these findings are, to our knowledge, the first to show IMs as a neural correlate of perceptual learning.