Combined MEG and EEG show reliable patterns of electromagnetic brain activity during natural viewing

Time-varying similarity of neural responses to musical tension is shaped by physical features and musical themes

The similarity of understanding is important for music experience and communication, but little is understood about the sources of this common knowledge. Although neural responses to the same piece of music are known to be similar across listeners, it remains unclear whether this neural response similarity is linked to musical understanding and the role of dynamic musical attributes in shaping it. Our study addresses this gap by investigating the relationship between neural response similarity, musical tension, and dynamic musical attributes. Using electroencephalography-based inter-subject correlation (EEG-ISC), we examined how the neural response similarity among listeners varies throughout the evaluation of musical tension in the first movement of Beethoven's Piano Sonata No. 8. Participants continuously rated the degree of alignment between musical events and their expectations, while neural activity was recorded using electroencephalography (EEG). The results showed that neural response similarity fluctuated in tandem with musical tension, with increased similarity observed during moments of heightened tension. This time-varying neural response similarity was influenced by two dynamic attributes contributing to musical tension: physical features and musical themes. Specifically, its fluctuation was driven by physical features, and the patterns of its variation were modulated by musical themes, with similar time-varying patterns observed across similar thematic materials. These findings offer valuable insight into the role of dynamic musical attributes in shaping neural response similarity, and reveal an important source and mechanism of shared musical understandings.

The convergence of naturalistic paradigms and cognitive neuroscience methods to investigate memory and its development

Studies that involve lab-based stimuli (e.g., words, pictures) are fundamental in the memory literature. At the same time, there is growing acknowledgment that memory processes assessed in the lab may not be analogous to how memory operates in the real world. Naturalistic paradigms can bridge this gap and over the decades a growing proportion of memory research has involved more naturalistic events. However, there is significant variation in the types of naturalistic studies used to study memory and its development, each with various advantages and limitations. Further, there are notable gaps in how often different types of naturalistic approaches have been combined with cognitive neuroscience methods (e.g., fMRI, EEG) to elucidate the neural processes and substrates involved in memory encoding and retrieval in the real world. Here we summarize and discuss what we identify as progressively more naturalistic methodologies used in the memory literature (movie, virtual reality, staged-events inside and outside of the lab, photo-taking, and naturally occurring event studies). Our goal is to describe each approach's benefits (e.g., naturalistic quality, feasibility), limitations (e.g., viability of neuroimaging method for event encoding versus event retrieval), and discuss possible future directions with each approach. We focus on child studies, when available, but also highlight past adult studies. Although there is a growing body of child memory research, naturalistic approaches combined with cognitive neuroscience methodologies in this domain remain sparse. Overall, this viewpoint article reviews how we can study memory through the lens of developmental cognitive neuroscience, while utilizing naturalistic and real-world events.

An evaluation of inter-brain EEG coupling methods in hyperscanning studies

EEG-based hyperscanning technology has been increasingly applied to analyze interpersonal interactions in social neuroscience in recent years. However, different methods are employed in various of studies without a complete investigation of the suitability of these methods. Our study aimed to systematically compare typical inter-brain EEG coupling methods, with simulated EEG data generated by real EEG data. In particular, two critical metrics of noise level and time delay were manipulated, and three different coupling models were tested. The results revealed that: (1) under certain conditions, various methods were leveraged by noise level and time delay, leading to different performances; (2) most algorithms achieved better experimental results and performance under high coupling degree; (3) with our simulation process, temporal and spectral models showed relatively good results, while data simulated with phase coupling model performed worse. This is the first systematic comparison of typical inter-brain EEG coupling methods, with simulated EEG data generated by real EEG data from different subjects. Existing methods mainly focused on intra-brain coupling. To our knowledge, there was only one previous study that compared five inter-brain EEG coupling methods (Burgess in Front Human Neurosci 7:881, 2013). However, the simulated data used in this study were generated time series with varied degrees of phase coupling without considering any EEG characteristics. For future research, appropriate methods need to be selected based on possible underlying mechanisms (temporal, spectral and phase coupling model hypothesis) of a specific study, as well as the expected coupling degree and conditions.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11571-022-09911-1.

Exploring the Neural Processes behind Narrative Engagement: An EEG Study

Past cognitive neuroscience studies using naturalistic stimuli have considered narratives holistically and focused on cognitive processes. In this study, we incorporated the narrative structure, the dramatic arc, as an object of investigation, to examine how engagement levels fluctuate across a narrative-aligned dramatic arc. We explored the possibility of predicting self-reported engagement ratings from neural activity and investigated the idiosyncratic effects of each phase of the dramatic arc on brain responses as well as the relationship between engagement and brain responses. We presented a movie excerpt following the six-phase narrative arc structure to female and male participants while collecting EEG signals. We then asked this group of participants to recall the excerpt, another group to segment the video based on the dramatic arc model, and a third to rate their engagement levels while watching the movie. The results showed that the self-reported engagement ratings followed the pattern of the narrative dramatic arc. Moreover, while EEG amplitude could not predict group-averaged engagement ratings, other features comprising dynamic intersubject correlation (dISC), including certain frequency bands, dynamic functional connectivity patterns and graph features were able to achieve this. Furthermore, neural activity in the last two phases of the dramatic arc significantly predicted engagement patterns. This study is the first to explore the cognitive processes behind the dramatic arc and its phases. By demonstrating how neural activity predicts self-reported engagement, which itself aligns with the narrative structure, this study provides insights on the interrelationships between narrative structure, neural responses, and viewer engagement.

As without, so within: how the brain's temporo-spatial alignment to the environment shapes consciousness

Consciousness is constituted by a structure that includes contents as foreground and the environment as background. This structural relation between the experiential foreground and background presupposes a relationship between the brain and the environment, often neglected in theories of consciousness. The temporo-spatial theory of consciousness addresses the brain-environment relation by a concept labelled 'temporo-spatial alignment'. Briefly, temporo-spatial alignment refers to the brain's neuronal activity's interaction with and adaption to interoceptive bodily and exteroceptive environmental stimuli, including their symmetry as key for consciousness. Combining theory and empirical data, this article attempts to demonstrate the yet unclear neuro-phenomenal mechanisms of temporo-spatial alignment. First, we suggest three neuronal layers of the brain's temporo-spatial alignment to the environment. These neuronal layers span across a continuum from longer to shorter timescales. (i) The background layer comprises longer and more powerful timescales mediating topographic-dynamic similarities between different subjects' brains. (ii) The intermediate layer includes a mixture of medium-scaled timescales allowing for stochastic matching between environmental inputs and neuronal activity through the brain's intrinsic neuronal timescales and temporal receptive windows. (iii) The foreground layer comprises shorter and less powerful timescales for neuronal entrainment of stimuli temporal onset through neuronal phase shifting and resetting. Second, we elaborate on how the three neuronal layers of temporo-spatial alignment correspond to their respective phenomenal layers of consciousness. (i) The inter-subjectively shared contextual background of consciousness. (ii) An intermediate layer that mediates the relationship between different contents of consciousness. (iii) A foreground layer that includes specific fast-changing contents of consciousness. Overall, temporo-spatial alignment may provide a mechanism whose different neuronal layers modulate corresponding phenomenal layers of consciousness. Temporo-spatial alignment can provide a bridging principle for linking physical-energetic (free energy), dynamic (symmetry), neuronal (three layers of distinct time-space scales) and phenomenal (form featured by background-intermediate-foreground) mechanisms of consciousness.

Higher levels of narrativity lead to similar patterns of posterior EEG activity across individuals

Introduction

The focus of cognitive and psychological approaches to narrative has not so much been on the elucidation of important aspects of narrative, but rather on using narratives as tools for the investigation of higher order cognitive processes elicited by narratives (e.g., understanding, empathy, etc.). In this study, we work toward a scalar model of narrativity, which can provide testable criteria for selecting and classifying communication forms in their level of narrativity. We investigated whether being exposed to videos with different levels of narrativity modulates shared neural responses, measured by inter-subject correlation, and engagement levels.

Methods

Thirty-two participants watched video advertisements with high-level and low-level of narrativity while their neural responses were measured through electroencephalogram. Additionally, participants' engagement levels were calculated based on the composite of their self-reported attention and immersion scores.

Results

Results demonstrated that both calculated inter-subject correlation and engagement scores for high-level video ads were significantly higher than those for low-level, suggesting that narrativity levels modulate inter-subject correlation and engagement.

Discussion

We believe that these findings are a step toward the elucidation of the viewers' way of processing and understanding a given communication artifact as a function of the narrative qualities expressed by the level of narrativity.

Functional near-infrared spectroscopy imaging of the prefrontal cortex during a naturalistic comedy movie

Naturalistic stimulation (i.e., movies and auditory narratives of some minutes' length) has been a powerful approach to bringing more real-life experiences into laboratory experiments. Data-driven, intersubject correlation (ISC) analysis permits examining to what extent activity in a specific brain region correlates across participants during exposure to a naturalistic stimulus, as well as testing whether neural activity correlates with behavioral measures. Notably, most of the previous research with naturalistic stimuli was conducted using functional fMRI (fMRI). Here, we tested whether a naturalistic approach and the ISC are feasible using functional near-infrared spectroscopy (fNIRS) - the imaging method particularly suited for populations of patients and children. Fifty-three healthy adult participants watched twice a 3-min segment of a Charlie Chaplin movie while we recorded the brain activity on the surface of their prefrontal cortex using fNIRS. In addition, an independent group of 18 participants used a continuous scoring procedure to rate the extent to which they felt that different parts of the movie fragment were funny. Our two findings were as follows. First, we found higher-than-zero ISC in fNIRS signals in the prefrontal cortex lobes, a result that was particularly high in the oxygenated channels during the first repetition of the movie. Second, we found a significant negative correlation between oxygenated brain signals and ratings of the movie's humorousness. In a series of control analyses we demonstrated that this latter correlation could not be explained by various non-humor-related movie sensory properties (e.g., auditory volume and image brightness). The key overall outcome of the present study is that fNIRS in combination with the naturalistic paradigms and the ISC might be a sensitive and powerful research method to explore cognitive processing. Our results also suggest a potential role of the prefrontal cortex in humor appreciation.

A studyforrest extension, MEG recordings while watching the audio-visual movie "Forrest Gump"

Naturalistic stimuli, such as movies, are being increasingly used to map brain function because of their high ecological validity. The pioneering studyforrest and other naturalistic neuroimaging projects have provided free access to multiple movie-watching functional magnetic resonance imaging (fMRI) datasets to prompt the community for naturalistic experimental paradigms. However, sluggish blood-oxygenation-level-dependent fMRI signals are incapable of resolving neuronal activity with the temporal resolution at which it unfolds. Instead, magnetoencephalography (MEG) measures changes in the magnetic field produced by neuronal activity and is able to capture rich dynamics of the brain at the millisecond level while watching naturalistic movies. Herein, we present the first public prolonged MEG dataset collected from 11 participants while watching the 2 h long audio-visual movie "Forrest Gump". Minimally preprocessed data was also provided to facilitate the use of the dataset. As a studyforrest extension, we envision that this dataset, together with fMRI data from the studyforrest project, will serve as a foundation for exploring the neural dynamics of various cognitive functions in real-world contexts.

Naturalistic Stimuli: A Paradigm for Multi-Scale Functional Characterization of the Human Brain

Movies, audio stories, and virtual reality are increasingly used as stimuli for functional brain imaging. Such naturalistic paradigms are in sharp contrast to the tradition of experimental reductionism in neuroscience research. Being complex, dynamic, and diverse, naturalistic stimuli set up a more ecologically relevant condition and induce highly reproducible brain responses across a wide range of spatiotemporal scales. Here, we review recent technical advances and scientific findings on imaging the brain under naturalistic stimuli. Then we elaborate on the premise of using naturalistic paradigms for multi-scale, multi-modal, and high-throughput functional characterization of the human brain. We further highlight the growing potential of using deep learning models to infer neural information processing from brain responses to naturalistic stimuli. Lastly, we advocate large-scale collaborations to combine brain imaging and recording data across experiments, subjects, and labs that use the same set of naturalistic stimuli.

Decreased intersubject synchrony in dynamic valence ratings of sad movie contents in dysphoric individuals

Emotional reactions to movies are typically similar between people. However, depressive symptoms decrease synchrony in brain responses. Less is known about the effect of depressive symptoms on intersubject synchrony in conscious stimulus-related processing. In this study, we presented amusing, sad and fearful movie clips to dysphoric individuals (those with elevated depressive symptoms) and control participants to dynamically rate the clips' valences (positive vs. negative). We analysed both the valence ratings' mean values and intersubject correlation (ISC). We used electrodermal activity (EDA) to complement the measurement in a separate session. There were no group differences in either the EDA or mean valence rating values for each movie type. As expected, the valence ratings' ISC was lower in the dysphoric than the control group, specifically for the sad movie clips. In addition, there was a negative relationship between the valence ratings' ISC and depressive symptoms for sad movie clips in the full sample. The results are discussed in the context of the negative attentional bias in depression. The findings extend previous brain activity results of ISC by showing that depressive symptoms also increase variance in conscious ratings of valence of stimuli in a mood-congruent manner.

Watching Movies Unfold, a Frame-by-Frame Analysis of the Associated Neural Dynamics

Our lives unfold as sequences of events. We experience these events as seamless, although they are composed of individual images captured in between the interruptions imposed by eye blinks and saccades. Events typically involve visual imagery from the real world (scenes), and the hippocampus is frequently engaged in this context. It is unclear, however, whether the hippocampus would be similarly responsive to unfolding events that involve abstract imagery. Addressing this issue could provide insights into the nature of its contribution to event processing, with relevance for theories of hippocampal function. Consequently, during magnetoencephalography (MEG), we had female and male humans watch highly matched unfolding movie events composed of either scene image frames that reflected the real world, or frames depicting abstract patterns. We examined the evoked neuronal responses to each image frame along the time course of the movie events. Only one difference between the two conditions was evident, and that was during the viewing of the first image frame of events, detectable across frontotemporal sensors. Further probing of this difference using source reconstruction revealed greater engagement of a set of brain regions across parietal, frontal, premotor, and cerebellar cortices, with the largest change in broadband (1–30 Hz) power in the hippocampus during scene-based movie events. Hippocampal engagement during the first image frame of scene-based events could reflect its role in registering a recognizable context perhaps based on templates or schemas. The hippocampus, therefore, may help to set the scene for events very early on.

MEG Intersubject Phase Locking of Stimulus-Driven Activity during Naturalistic Speech Listening Correlates with Musical Training

Musical training is associated with increased structural and functional connectivity between auditory sensory areas and higher-order brain networks involved in speech and motor processing. Whether such changed connectivity patterns facilitate the cortical propagation of speech information in musicians remains poorly understood. We here used magnetoencephalography (MEG) source imaging and a novel seed-based intersubject phase-locking approach to investigate the effects of musical training on the interregional synchronization of stimulus-driven neural responses during listening to naturalistic continuous speech presented in silence. MEG data were obtained from 20 young human subjects (both sexes) with different degrees of musical training. Our data show robust bilateral patterns of stimulus-driven interregional phase synchronization between auditory cortex and frontotemporal brain regions previously associated with speech processing. Stimulus-driven phase locking was maximal in the delta band, but was also observed in the theta and alpha bands. The individual duration of musical training was positively associated with the magnitude of stimulus-driven alpha-band phase locking between auditory cortex and parts of the dorsal and ventral auditory processing streams. These findings provide evidence for a positive relationship between musical training and the propagation of speech-related information between auditory sensory areas and higher-order processing networks, even when speech is presented in silence. We suggest that the increased synchronization of higher-order cortical regions to auditory cortex may contribute to the previously described musician advantage in processing speech in background noise.SIGNIFICANCE STATEMENT Musical training has been associated with widespread structural and functional brain plasticity. It has been suggested that these changes benefit the production and perception of music but can also translate to other domains of auditory processing, such as speech. We developed a new magnetoencephalography intersubject analysis approach to study the cortical synchronization of stimulus-driven neural responses during the perception of continuous natural speech and its relationship to individual musical training. Our results provide evidence that musical training is associated with higher synchronization of stimulus-driven activity between brain regions involved in early auditory sensory and higher-order processing. We suggest that the increased synchronized propagation of speech information may contribute to the previously described musician advantage in processing speech in background noise.

Characterizing Inscapes and resting-state in MEG: Effects in typical and atypical development

Examining the brain at rest is a powerful approach used to understand the intrinsic properties of typical and disordered human brain function, yet task-free paradigms are associated with greater head motion, particularly in young and/or clinical populations such as autism spectrum disorder (ASD) and attention-deficit/hyperactivity disorder (ADHD). Inscapes, a non-social and non-verbal movie paradigm, has been introduced to increase attention, thus mitigating head motion, while reducing the task-induced activations found during typical movie watching. Inscapes has not yet been validated for use in magnetoencephalography (MEG), and it has yet to be shown whether its effects are stable in clinical populations. Across typically developing (N = 32) children and adolescents and those with ASD (N = 46) and ADHD (N = 42), we demonstrate that head motion is reduced during Inscapes. Due to the task state evoked by movie paradigms, we also expectedly observed concomitant modulations in local neural activity (oscillatory power) and functional connectivity (phase and envelope coupling) in intrinsic resting-state networks and across the frequency spectra compared to a fixation cross resting-state. Increases in local activity were accompanied by decreases in low-frequency connectivity within and between resting-state networks, primarily the visual network, suggesting that task-state evoked by Inscapes moderates ongoing and spontaneous cortical inhibition that forms the idling intrinsic networks found during a fixation cross resting-state. Importantly, these effects were similar in ASD and ADHD, making Inscapes a well-suited advancement for investigations of resting brain function in young and clinical populations.

Inter-subject correlations during natural viewing: A filter-bank approach

A central question in neuroscience is how the brain processes real-world sensory input. For decades most classical studies focus on carefully controlled artificial stimuli. More recently researchers started to investigate brain activity under more realistic conditions. The main challenge in this setting is the analysis of the complex signals obtained with modern neuroimaging methods in response to natural stimuli. Inter-subject correlations (ISCs) have become a popular paradigm to study brain activation under natural stimulation. The underlying assumption of this analysis is that features of natural stimuli that are perceived and processed by all subjects exposed to the same stimulus result in similar activation patterns across subjects. Higher degrees of realism in stimulation, for instance audiovisual stimulation is more realistic than auditory stimulation, is usually associated with higher ISC values. We can confirm these findings in experiments in which we present a movie stimulus with varying degrees of realism. Extending previous findings we highlight the importance of artifact removal when evaluating ISCs and show that the impact of realism in natural stimulation on ISCs is frequency-dependent. A major challenge associated with this type of analysis is that it can be difficult to attribute the correlation strength to the physiological process of interest. In this study, we demonstrate that ISCs of neural activation as measured by electroencephalogram (EEG) recordings are influenced significantly by non-neural artifacts such as occulograms. Our findings highlight the potential of inter-subject correlations as a biomarker for immersion: If more realistic stimuli consistently lead to higher inter-subject correlations, then inter-subject correlations can serve as a quantitative marker for how engaging audiovisual stimuli are perceived.Clinical relevance- Future research will evaluate if correlation levels among subjects, who are exposed to natural stimuli are affected by neurological diseases such as Alzheimers, Parkinsons, and Schizophrenia among others.

Intersubject Spatial Pattern Correlations During Movie Viewing Are Stimulus-Driven and Nonuniform Across the Cortex

A fundamental step to predicting brain activity in healthy and diseased populations is characterizing the common spatio-temporal response to a shared experience. Multivoxel pattern analysis allows us to investigate information encoding through these patterns; however, we have yet to explore local, stimulus-driven, patterns of cortical activity during naturalistic stimulation. We sought to examine these patterns with minimum interpolation—excluding functional alignment—to characterize the most basic degree of shared response between subjects. We used an unbiased analytic approach, combined with rich, naturalistic, and nonsemantic stimulation to estimate shared spatial patterns in functional magnetic resonance imaging responses across a large group. We found that meso-scale spatial patterns were shared nonuniformly across the visual cortex and represent information distinct from the shared temporal response. Shared spatial patterns were stimulus-driven, modulated by pattern size, and more sensitive to the contrast of 3D versus 2D stimulus differences than the temporal signals. Although the grand functional structure of the brain is understood to be common, these results suggest that even at a meso-scale, we share common spatial structures with anatomical alignment alone. The strength of this similarity varies across the cortex, suggesting some spatial structures are innately organized, whereas others are shaped by factors such as learning and plasticity.

Heart Rate and Respiration Affect the Functional Connectivity of Default Mode Network in Resting-State Functional Magnetic Resonance Imaging

A growing number of brain imaging studies show functional connectivity (FC) between regions during emotional and cognitive tasks in humans. However, emotions are accompanied by changes in physiological parameters such as heart rate and respiration. These changes may affect blood oxygen level-dependent signals, as well as connectivity between brain areas. This study aimed to clarify the effects of physiological noise on the connectivity between areas related to the default mode network using resting-state functional magnetic resonance imaging (rs-fMRI). Healthy adult volunteers (age range: 19–51 years, mean age: 26.9 ± 9.1 years, 8 males and 8 females) underwent rs-fMRI for 10 min using a clinical 3T scanner (MAGNETOM Trio A Tim System, Siemens) with simultaneously recorded respiration and cardiac output. Physiological noise signals were subsequently removed from the acquired fMRI data using the DRIFTER toolbox. Image processing and analysis of the FC between areas related to the default mode network were performed using DPARSF. Network-Based Statistic (NBS) analysis of the functional connectome of the DMN and DMN-related area was used to perform three groups of comparison: without physiological noise correction, with cardiac noise correction, and with cardiac and respiratory noise correction. NBS analysis identified 36 networks with significant differences in three conditions in FC matrices. Post hoc comparison showed no differences between the three conditions, indicating that all three had the same networks. Among the 36 networks, strength of FC of 8 networks was modified under physiological noise correction. Connectivity between left and right anterior medial frontal regions increased strength of connectivity. These areas are located on the medial cerebral hemisphere, close to the sagittal sinus and arteries in the cerebral hemispheres, suggesting that medial frontal areas may be sensitive to cardiac rhythm close to arteries. The other networks observed temporal regions and showed a decrease in their connectivity strength by removing physiological noise, indicating that physiological noise, especially respiration, may be sensitive to BOLD signal in the temporal regions during resting state. Temporal lobe was highly correlated with anxiety-related respiration changes (Masaoka and Homma, 2000), speech processing, and respiratory sensation. These factors may affect the rs-fMRI signaling sensitivity.

Natural scene representations in the gamma band are prototypical across subjects

Prototypical brain responses describe similarity in neural representations between subjects in response to a natural stimulus. During natural movie viewing, for example, inter-subject correlation (ISC) measured by fMRI is high in visual areas (Hasson et al., 2004). But the electrophysiological basis for this fMRI ISC has been controversial. Previous reports have only found ISC in low frequency bands-below 12 ​Hz (Chang et al., 2015). These findings stand in contrast to reports that gamma band oscillations-30 to 90 ​Hz-are highly stimulus-driven in visual cortex (Perry et al., 2015). To resolve this discrepancy, we carried out both ISC estimation and a novel inter-subject representational correlation analysis across six frequency bands extracted from MEG data of 24 subjects who each viewed four 5-min clips of an underwater documentary. Region-of-interest-based and vertex-based temporal ISC estimates confirmed that low-frequency bands are significantly synchronized in visual areas and that gamma band has low temporal correlation. We also found the representational geometry of movie scenes were related to structural statistics from the stimuli. Crucially, our results show that the gamma band oscillations also reflect prototypical brain response in scene representations formed in response to naturalistic stimuli as revealed by inter-subject representational correlation.

Neuromagnetic activation and oscillatory dynamics of stimulus-locked processing during naturalistic viewing

Naturalistic stimuli such as watching a movie while in the scanner provide an ecologically valid paradigm that has the potential of extracting valuable information on how the brain processes complex stimuli in realistic visual and auditory contexts. Naturalistic viewing is also easier to conduct with challenging participant groups including patients and children. Given the high temporal resolution of MEG, in the present study, we demonstrate how a short movie clip can be used to map distinguishable activation and connectivity dynamics underlying the processing of specific classes of visual stimuli such as face and hand manipulations, as well as contrasting activation dynamics for auditory words and non-words. MEG data were collected from 22 healthy volunteers (6 females, 3 left handed, mean age - 27.7 ​± ​5.28 years) during the presentation of naturalistic audiovisual stimuli. The MEG data were split into trials with the onset of the stimuli belonging to classes of interest (words, non-words, faces, hand manipulations). Based on the components of the averaged sensor ERFs time-locked to the visual and auditory stimulus onset, four and three time-windows, respectively, were defined to explore brain activation dynamics. Pseudo-Z, defined as the ratio of the source-projected time-locked power to the projected noise power for each vertex, was computed and used as a proxy of time-locked brain activation. Statistical testing using the mean-centered Partial Least Squares analysis indicated periods where a given visual or auditory stimuli had higher activation. Based on peak pseudo-Z differences between the visual conditions, time-frequency resolved analyses were performed to assess beta band desynchronization in motor-related areas, and inter-trial phase synchronization between face processing areas. Our results provide the first evidence that activation and connectivity dynamics in canonical brain regions associated with the processing of particular classes of visual and auditory stimuli can be reliably mapped using MEG during presentation of naturalistic stimuli. Given the strength of MEG for brain mapping in temporal and frequency domains, the use of naturalistic stimuli may open new techniques in analyzing brain dynamics during ecologically valid sensation and perception.

Mapping brain function during naturalistic viewing using high-density diffuse optical tomography

Naturalistic stimuli, such as movies, more closely recapitulate "real life" sensory processing and behavioral demands relative to paradigms that rely on highly distilled and repetitive stimulus presentations. The rich complexity inherent in naturalistic stimuli demands an imaging system capable of measuring spatially distributed brain responses, and analysis tools optimized for unmixing responses to concurrently presented features. In this work, the combination of passive movie viewing with high-density diffuse optical tomography (HD-DOT) is developed as a platform for naturalistic brain mapping. We imaged healthy young adults during free viewing of a feature film using HD-DOT and observed reproducible, synchronized cortical responses across a majority of the field-of-view, most prominently in hierarchical cortical areas related to visual and auditory processing, both within and between individuals. In order to more precisely interpret broad patterns of cortical synchronization, we extracted visual and auditory features from the movie stimulus and mapped the cortical responses to the features. The results demonstrate the sensitivity of HD-DOT to evoked responses during naturalistic viewing, and that feature-based decomposition strategies enable functional mapping of naturalistic stimulus processing, including human-generated speech.

Perception of social synchrony induces mother-child gamma coupling in the social brain

The recent call to move from focus on one brain’s functioning to two-brain communication initiated a search for mechanisms that enable two humans to coordinate brain response during social interactions. Here, we utilized the mother–child context as a developmentally salient setting to study two-brain coupling. Mothers and their 9-year-old children were videotaped at home in positive and conflictual interactions. Positive interactions were microcoded for social synchrony and conflicts for overall dialogical style. Following, mother and child underwent magnetoencephalography while observing the positive vignettes. Episodes of behavioral synchrony, compared to non-synchrony, increased gamma-band power in the superior temporal sulcus (STS), hub of social cognition, mirroring and mentalizing. This neural pattern was coupled between mother and child. Brain-to-brain coordination was anchored in behavioral synchrony; only during episodes of behavioral synchrony, but not during non-synchronous moments, mother’s and child's STS gamma power was coupled. Importantly, neural synchrony was not found during observation of unfamiliar mother-child interaction Maternal empathic/dialogical conflict style predicted mothers’ STS activations whereas child withdrawal predicted attenuated STS response in both partners. Results define a novel neural marker for brain-to-brain synchrony, highlight the role of rapid bottom-up oscillatory mechanisms for neural coupling and indicate that behavior-based processes may drive synchrony between two brains during social interactions.

Brain-to-Brain Synchrony and Learning Outcomes Vary by Student-Teacher Dynamics: Evidence from a Real-world Classroom Electroencephalography Study

How does the human brain support real-world learning? We used wireless electroencephalography to collect neurophysiological data from a group of 12 senior high school students and their teacher during regular biology lessons. Six scheduled classes over the course of the semester were organized such that class materials were presented using different teaching styles (videos and lectures), and students completed a multiple-choice quiz after each class to measure their retention of that lesson's content. Both students' brain-to-brain synchrony and their content retention were higher for videos than lectures across the six classes. Brain-to-brain synchrony between the teacher and students varied as a function of student engagement as well as teacher likeability: Students who reported greater social closeness to the teacher showed higher brain-to-brain synchrony with the teacher, but this was only the case for lectures-that is, when the teacher is an integral part of the content presentation. Furthermore, students' retention of the class content correlated with student-teacher closeness, but not with brain-to-brain synchrony. These findings expand on existing social neuroscience research by showing that social factors such as perceived closeness are reflected in brain-to-brain synchrony in real-world group settings and can predict cognitive outcomes such as students' academic performance.

Sharing the social world via intersubject neural synchronisation

Sociability and capability of shared mental states are hallmarks of the human species, and pursuing shared goals oftentimes requires coordinating both behaviour and mental states. Here we review recent work using indices of intersubject neural synchronisation for measuring similarity of mental states across individuals. We discuss the methodological advances and limitations in the analyses based on intersubject synchrony, and discuss how these kinds of model-free analysis techniques enable the investigation of the brain basis of complex social processes. We argue that similarity of brain activity across individuals can be used, under certain conditions, to index the similarity of their subjective states of consciousness, and thus be used for investigating brain basis of mutual understanding and cooperation.

A Review of Issues Related to Data Acquisition and Analysis in EEG/MEG Studies

Electroencephalography (EEG) and magnetoencephalography (MEG) are non-invasive electrophysiological methods, which record electric potentials and magnetic fields due to electric currents in synchronously-active neurons. With MEG being more sensitive to neural activity from tangential currents and EEG being able to detect both radial and tangential sources, the two methods are complementary. Over the years, neurophysiological studies have changed considerably: high-density recordings are becoming de rigueur; there is interest in both spontaneous and evoked activity; and sophisticated artifact detection and removal methods are available. Improved head models for source estimation have also increased the precision of the current estimates, particularly for EEG and combined EEG/MEG. Because of their complementarity, more investigators are beginning to perform simultaneous EEG/MEG studies to gain more complete information about neural activity. Given the increase in methodological complexity in EEG/MEG, it is important to gather data that are of high quality and that are as artifact free as possible. Here, we discuss some issues in data acquisition and analysis of EEG and MEG data. Practical considerations for different types of EEG and MEG studies are also discussed.

The impact of MEG source reconstruction method on source-space connectivity estimation: A comparison between minimum-norm solution and beamforming

Despite numerous important contributions, the investigation of brain connectivity with magnetoencephalography (MEG) still faces multiple challenges. One critical aspect of source-level connectivity, largely overlooked in the literature, is the putative effect of the choice of the inverse method on the subsequent cortico-cortical coupling analysis. We set out to investigate the impact of three inverse methods on source coherence detection using simulated MEG data. To this end, thousands of randomly located pairs of sources were created. Several parameters were manipulated, including inter- and intra-source correlation strength, source size and spatial configuration. The simulated pairs of sources were then used to generate sensor-level MEG measurements at varying signal-to-noise ratios (SNR). Next, the source level power and coherence maps were calculated using three methods (a) L2-Minimum-Norm Estimate (MNE), (b) Linearly Constrained Minimum Variance (LCMV) beamforming, and (c) Dynamic Imaging of Coherent Sources (DICS) beamforming. The performances of the methods were evaluated using Receiver Operating Characteristic (ROC) curves. The results indicate that beamformers perform better than MNE for coherence reconstructions if the interacting cortical sources consist of point-like sources. On the other hand, MNE provides better connectivity estimation than beamformers, if the interacting sources are simulated as extended cortical patches, where each patch consists of dipoles with identical time series (high intra-patch coherence). However, the performance of the beamformers for interacting patches improves substantially if each patch of active cortex is simulated with only partly coherent time series (partial intra-patch coherence). These results demonstrate that the choice of the inverse method impacts the results of MEG source-space coherence analysis, and that the optimal choice of the inverse solution depends on the spatial and synchronization profile of the interacting cortical sources. The insights revealed here can guide method selection and help improve data interpretation regarding MEG connectivity estimation.

EEG in the classroom: Synchronised neural recordings during video presentation

We performed simultaneous recordings of electroencephalography (EEG) from multiple students in a classroom, and measured the inter-subject correlation (ISC) of activity evoked by a common video stimulus. The neural reliability, as quantified by ISC, has been linked to engagement and attentional modulation in earlier studies that used high-grade equipment in laboratory settings. Here we reproduce many of the results from these studies using portable low-cost equipment, focusing on the robustness of using ISC for subjects experiencing naturalistic stimuli. The present data shows that stimulus-evoked neural responses, known to be modulated by attention, can be tracked for groups of students with synchronized EEG acquisition. This is a step towards real-time inference of engagement in the classroom.

Visual cortex responses reflect temporal structure of continuous quasi-rhythmic sensory stimulation

Neural processing of dynamic continuous visual input, and cognitive influences thereon, are frequently studied in paradigms employing strictly rhythmic stimulation. However, the temporal structure of natural stimuli is hardly ever fully rhythmic but possesses certain spectral bandwidths (e.g. lip movements in speech, gestures). Examining periodic brain responses elicited by strictly rhythmic stimulation might thus represent ideal, yet isolated cases. Here, we tested how the visual system reflects quasi-rhythmic stimulation with frequencies continuously varying within ranges of classical theta (4–7 Hz), alpha (8–13 Hz) and beta bands (14–20 Hz) using EEG. Our findings substantiate a systematic and sustained neural phase-locking to stimulation in all three frequency ranges. Further, we found that allocation of spatial attention enhances EEG-stimulus locking to theta- and alpha-band stimulation. Our results bridge recent findings regarding phase locking (“entrainment”) to quasi-rhythmic visual input and “frequency-tagging” experiments employing strictly rhythmic stimulation. We propose that sustained EEG-stimulus locking can be considered as a continuous neural signature of processing dynamic sensory input in early visual cortices. Accordingly, EEG-stimulus locking serves to trace the temporal evolution of rhythmic as well as quasi-rhythmic visual input and is subject to attentional bias.

•

Dynamic visual stimuli constitute large parts of our perceptual experience.

•

Strictly rhythmic dynamics condense in EEG-recorded mass-neural activity.

•

We tested how stimuli with fluctuating rhythms reflect in the EEG.

•

We found that the EEG allows tracing two quasi-rhythmic stimuli in parallel.

•

Dynamics of attended stimuli may be tracked with greater temporal precision.

Time-domain analysis of neural tracking of hierarchical linguistic structures

When listening to continuous speech, cortical activity measured by MEG concurrently follows the rhythms of multiple linguistic structures, e.g., syllables, phrases, and sentences. This phenomenon was previously characterized in the frequency domain. Here, we investigate the waveform of neural activity tracking linguistic structures in the time domain and quantify the coherence of neural response phases over subjects listening to the same stimulus. These analyses are achieved by decomposing the multi-channel MEG recordings into components that maximize the correlation between neural response waveforms across listeners. Each MEG component can be viewed as the recording from a virtual sensor that is spatially tuned to a cortical network showing coherent neural activity over subjects. This analysis reveals information not available from previous frequency-domain analysis of MEG global field power: First, concurrent neural tracking of hierarchical linguistic structures emerges at the beginning of the stimulus, rather than slowly building up after repetitions of the same sentential structure. Second, neural tracking of the sentential structure is reflected by slow neural fluctuations, rather than, e.g., a series of short-lasting transient responses at sentential boundaries. Lastly and most importantly, it shows that the MEG responses tracking the syllabic rhythm are spatially separable from the MEG responses tracking the sentential and phrasal rhythms.

Adolescents growing up amidst intractable conflict attenuate brain response to pain of outgroup

Adolescents' participation in intergroup conflicts comprises an imminent global risk, and understanding its neural underpinnings may open new perspectives. We assessed Jewish-Israeli and Arab-Palestinian adolescents for brain response to the pain of ingroup/outgroup protagonists using magnetoencephalography (MEG), one-on-one positive and conflictual interactions with an outgroup member, attitudes toward the regional conflict, and oxytocin levels. A neural marker of ingroup bias emerged, expressed via alpha modulations in the somatosensory cortex (S1) that characterized an automatic response to the pain of all protagonists followed by rebound/enhancement to ingroup pain only. Adolescents' hostile social interactions with outgroup members and uncompromising attitudes toward the conflict influenced this neural marker. Furthermore, higher oxytocin levels in the Jewish-Israeli majority and tighter brain-to-brain synchrony among group members in the Arab-Palestinian minority enhanced the neural ingroup bias. Findings suggest that in cases of intractable intergroup conflict, top-down control mechanisms may block the brain's evolutionary-ancient resonance to outgroup pain, pinpointing adolescents' interpersonal and sociocognitive processes as potential targets for intervention.

Influences of High-Level Features, Gaze, and Scene Transitions on the Reliability of BOLD Responses to Natural Movie Stimuli

Complex, sustained, dynamic, and naturalistic visual stimulation can evoke distributed brain activities that are highly reproducible within and across individuals. However, the precise origins of such reproducible responses remain incompletely understood. Here, we employed concurrent functional magnetic resonance imaging (fMRI) and eye tracking to investigate the experimental and behavioral factors that influence fMRI activity and its intra- and inter-subject reproducibility during repeated movie stimuli. We found that widely distributed and highly reproducible fMRI responses were attributed primarily to the high-level natural content in the movie. In the absence of such natural content, low-level visual features alone in a spatiotemporally scrambled control stimulus evoked significantly reduced degree and extent of reproducible responses, which were mostly confined to the primary visual cortex (V1). We also found that the varying gaze behavior affected the cortical response at the peripheral part of V1 and in the oculomotor network, with minor effects on the response reproducibility over the extrastriate visual areas. Lastly, scene transitions in the movie stimulus due to film editing partly caused the reproducible fMRI responses at widespread cortical areas, especially along the ventral visual pathway. Therefore, the naturalistic nature of a movie stimulus is necessary for driving highly reliable visual activations. In a movie-stimulation paradigm, scene transitions and individuals’ gaze behavior should be taken as potential confounding factors in order to properly interpret cortical activity that supports natural vision.

MEG Connectivity and Power Detections with Minimum Norm Estimates Require Different Regularization Parameters

Minimum Norm Estimation (MNE) is an inverse solution method widely used to reconstruct the source time series that underlie magnetoencephalography (MEG) data. MNE addresses the ill-posed nature of MEG source estimation through regularization (e.g., Tikhonov regularization). Selecting the best regularization parameter is a critical step. Generally, once set, it is common practice to keep the same coefficient throughout a study. However, it is yet to be known whether the optimal lambda for spectral power analysis of MEG source data coincides with the optimal regularization for source-level oscillatory coupling analysis. We addressed this question via extensive Monte-Carlo simulations of MEG data, where we generated 21,600 configurations of pairs of coupled sources with varying sizes, signal-to-noise ratio (SNR), and coupling strengths. Then, we searched for the Tikhonov regularization coefficients (lambda) that maximize detection performance for (a) power and (b) coherence. For coherence, the optimal lambda was two orders of magnitude smaller than the best lambda for power. Moreover, we found that the spatial extent of the interacting sources and SNR, but not the extent of coupling, were the main parameters affecting the best choice for lambda. Our findings suggest using less regularization when measuring oscillatory coupling compared to power estimation.