Future trends in Neuroimaging: Neural processes as expressed within real-life contexts

Brain dynamics and spatiotemporal trajectories during threat processing

In the past decades, functional MRI research has investigated mental states and their brain bases in largely static fashion based on evoked responses during blocked and event-related designs. Despite some progress in naturalistic designs, our understanding of threat processing remains largely limited to those obtained with standard paradigms. In the present paper, we applied Switching Linear Dynamical Systems to uncover the dynamics of threat processing during a continuous threat-of-shock paradigm. Importantly, unlike studies in systems neuroscience that frequently assume that systems are decoupled from external inputs, we characterized both endogenous and exogenous contributions to dynamics. First, we demonstrated that the SLDS model learned the regularities of the experimental paradigm, such that states and state transitions estimated from fMRI time series data from 85 ROIs reflected both the proximity of the circles and their direction (approach vs. retreat). After establishing that the model captured key properties of threat-related processing, we characterized the dynamics of the states and their transitions. The results revealed that threat processing can profitably be viewed in terms of dynamic multivariate patterns whose trajectories are a combination of intrinsic and extrinsic factors that jointly determine how the brain temporally evolves during dynamic threat. We propose that viewing threat processing through the lens of dynamical systems offers important avenues to uncover properties of the dynamics of threat that are not unveiled with standard experimental designs and analyses.

Word Frequency and Predictability Dissociate in Naturalistic Reading

Many studies of human language processing have shown that readers slow down at less frequent or less predictable words, but there is debate about whether frequency and predictability effects reflect separable cognitive phenomena: are cognitive operations that retrieve words from the mental lexicon based on sensory cues distinct from those that predict upcoming words based on context? Previous evidence for a frequency-predictability dissociation is mostly based on small samples (both for estimating predictability and frequency and for testing their effects on human behavior), artificial materials (e.g., isolated constructed sentences), and implausible modeling assumptions (discrete-time dynamics, linearity, additivity, constant variance, and invariance over time), which raises the question: do frequency and predictability dissociate in ordinary language comprehension, such as story reading? This study leverages recent progress in open data and computational modeling to address this question at scale. A large collection of naturalistic reading data (six datasets, >2.2 M datapoints) is analyzed using nonlinear continuous-time regression, and frequency and predictability are estimated using statistical language models trained on more data than is currently typical in psycholinguistics. Despite the use of naturalistic data, strong predictability estimates, and flexible regression models, results converge with earlier experimental studies in supporting dissociable and additive frequency and predictability effects.

Frequency-specific functional difference between gyri and sulci in naturalistic paradigm fMRI

Disentangling functional difference between cortical folding patterns of gyri and sulci provides novel insights into the relationship between brain structure and function. Previous studies using resting-state functional magnetic resonance imaging (rsfMRI) have revealed that sulcal signals exhibit stronger high-frequency but weaker low-frequency components compared to gyral ones, suggesting that gyri may serve as functional integration centers while sulci are segregated local processing units. In this study, we utilize naturalistic paradigm fMRI (nfMRI) to explore the functional difference between gyri and sulci as it has proven to record stronger functional integrations compared to rsfMRI. We adopt a convolutional neural network (CNN) to classify gyral and sulcal fMRI signals in the whole brain (the global model) and within functional brain networks (the local models). The frequency-specific difference between gyri and sulci is then inferred from the power spectral density (PSD) profiles of the learned filters in the CNN model. Our experimental results show that nfMRI shows higher gyral-sulcal PSD contrast effect sizes in the global model compared to rsfMRI. In the local models, the effect sizes are either increased or decreased depending on frequency bands and functional complexity of the FBNs. This study highlights the advantages of nfMRI in depicting the functional difference between gyri and sulci, and provides novel insights into unraveling the relationship between brain structure and function.

High-dimensional Metaverse Platforms and the Virtually Extended Self

The study of cognition has traditionally used low-dimensional measures and stimulus presentations that emphasize laboratory control over high-dimensional (i.e., ecologically valid) tools that reflect the activities and interactions in everyday living. Although controlled experimental presentations in laboratories have enhanced our understanding of cognition for both healthy and clinical cohorts, high dimensionality may extend reality and cognition. High-dimensional Metaverse approaches use extended reality (XR) platforms with dynamic stimulus presentations that couple humans and simulation technologies to extend cognition. The plan for this paper is as follows: The "Extending from low to high-dimensional studies of cognition" section discusses current needs for high-dimensional stimulus presentations that reflect everyday cognitive activities. In the "Algorithmic devices and digital extension of cognition" section, technologies of the extended mind are introduced with the Metaverse as a candidate cognitive process for extension. Next, in the "A neurocognitive framework for understanding technologies of the extended mind" section, a framework and model are proposed for understanding the neural correlates of human technology couplings in terms of automatic algorithmic processes (limbic-ventral striatal loop); reflective cognition (prefrontal-dorsal striatal loop); and algorithmic processing (insular cortex). The algorithmic processes of human-technology interactions can, over time, become an automated and algorithmic coupling of brain and technology. The manuscript ends with a brief summary and discussion of the ways in which the Metaverse can be used for studying how persons respond to high-dimensional stimuli in simulations that approximate real-world activities and interactions.

Mobile fNIRS for exploring inter-brain synchrony across generations and time

While still relatively rare, longitudinal hyperscanning studies are exceptionally valuable for documenting changes in inter-brain synchrony, which may in turn underpin how behaviors develop and evolve in social settings. The generalizability and ecological validity of this experimental approach hinges on the selected imaging technique being mobile-a requirement met by functional near-infrared spectroscopy (fNIRS). fNIRS has most frequently been used to examine the development of inter-brain synchrony and behavior in child-parent dyads. In this position paper, we contend that dedicating attention to longitudinal and intergenerational hyperscanning stands to benefit the fields of social and cognitive neuroscience more broadly. We argue that this approach is particularly relevant for understanding the neural mechanisms underpinning intergenerational social dynamics, and potentially for benchmarking progress in psychological and social interventions, many of which are situated in intergenerational contexts. In line with our position, we highlight areas of intergenerational research that stand to be enhanced by longitudinal hyperscanning with mobile devices, describe challenges that may arise from measuring across generations in the real world, and offer potential solutions.

Interpersonal alignment of neural evidence accumulation to social exchange of confidence

Private, subjective beliefs about uncertainty have been found to have idiosyncratic computational and neural substrates yet, humans share such beliefs seamlessly and cooperate successfully. Bringing together decision making under uncertainty and interpersonal alignment in communication, in a discovery plus pre-registered replication design, we examined the neuro-computational basis of the relationship between privately held and socially shared uncertainty. Examining confidence-speed-accuracy trade-off in uncertainty-ridden perceptual decisions under social vs isolated context, we found that shared (i.e. reported confidence) and subjective (inferred from pupillometry) uncertainty dynamically followed social information. An attractor neural network model incorporating social information as top-down additive input captured the observed behavior and demonstrated the emergence of social alignment in virtual dyadic simulations. Electroencephalography showed that social exchange of confidence modulated the neural signature of perceptual evidence accumulation in the central parietal cortex. Our findings offer a neural population model for interpersonal alignment of shared beliefs.

A normative model of brain responses to social scenarios reflects the maturity of children and adolescents' social-emotional abilities

The rapid brain maturation in childhood and adolescence accompanies the development of socio-emotional functioning. However, it is unclear how the maturation of the neural activity drives the development of socio-emotional functioning and individual differences. This study aimed to reflect the age dependence of inter-individual differences in brain responses to socio-emotional scenarios and to develop naturalistic imaging indicators to assess the maturity of socio-emotional ability at the individual level. Using three independent naturalistic imaging datasets containing healthy participants (n = 111, 21 and 122), we found and validated that age-modulated inter-individual concordance of brain responses to socio-emotional movies in specific brain regions. The similarity of an individual's brain response to the average response of older participants was defined as response typicality, which predicted an individual's emotion regulation strategies in adolescence and theory of mind (ToM) in childhood. Its predictive power was not superseded by age, sex, cognitive performance or executive function. We further showed that the movie's valence and arousal ratings grounded the response typicality. The findings highlight that forming typical brain response patterns may be a neural phenotype underlying the maturation of socio-emotional ability. The proposed response typicality represents a neuroimaging approach to measure individuals' maturity of cognitive reappraisal and ToM.

Narrative simulation of social experiences in naturalistic context - A neurocinematic approach

Narratives may be regarded as simulations of everyday social situations. They are key to studying the human mind in socio-culturally determined contexts as they allow anchoring to the common ground of embodied and environmentally-engaged cognition. Here we review recent findings from naturalistic neuroscience on neural functions in conditions that mimic lifelike situations. We will focus particularly on neurocinematics, a research field that applies mediated narratives as stimuli for neuroimaging experiments. During the last two decades, this paradigm has contributed to an accumulation of insights about the neural underpinnings of behavior and sense-making in various narratively contextualized situations particularly pertaining to socio-emotional encounters. One of the key questions in neurocinematics is, how do intersubjectively synchronized brain activations relate to subjective experiences? Another question we address is how to bring natural contexts into experimental studies. Seeking to respond to both questions, we suggest neurocinematic studies to examine three manifestations of the same phenomenon side-by-side: subjective experiences of narrative situations, unfolding of narrative stimulus structure, and neural processes that co-constitute the experience. This approach facilitates identifying experientially meaningful activity patterns in the brain and points out what they may mean in relation to shared and communicable contents. Via rich-featured and temporally contextualized narrative stimuli, neurocinematics attempts to contribute to emerging holistic theories of neural dynamics and connectomics explaining typical and atypical interindividual variability.

A new paradigm for investigating real-world social behavior and its neural underpinnings

Eye tracking and other behavioral measurements collected from patient-participants in their hospital rooms afford a unique opportunity to study natural behavior for basic and clinical translational research. We describe an immersive social and behavioral paradigm implemented in patients undergoing evaluation for surgical treatment of epilepsy, with electrodes implanted in the brain to determine the source of their seizures. Our studies entail collecting eye tracking with other behavioral and psychophysiological measurements from patient-participants during unscripted behavior, including social interactions with clinical staff, friends, and family in the hospital room. This approach affords a unique opportunity to study the neurobiology of natural social behavior, though it requires carefully addressing distinct logistical, technical, and ethical challenges. Collecting neurophysiological data synchronized to behavioral and psychophysiological measures helps us to study the relationship between behavior and physiology. Combining across these rich data sources while participants eat, read, converse with friends and family, etc., enables clinical-translational research aimed at understanding the participants' disorders and clinician-patient interactions, as well as basic research into natural, real-world behavior. We discuss data acquisition, quality control, annotation, and analysis pipelines that are required for our studies. We also discuss the clinical, logistical, and ethical and privacy considerations critical to working in the hospital setting.

Temporal continuity shapes visual responses of macaque face patch neurons

Macaque inferior temporal cortex neurons respond selectively to complex visual images, with recent work showing that they are also entrained reliably by the evolving content of natural movies. To what extent does temporal continuity itself shape the responses of high-level visual neurons? We addressed this question by measuring how cells in face-selective regions of the macaque visual cortex were affected by the manipulation of a movie's temporal structure. Sampling a 5-min movie at 1 s intervals, we measured neural responses to randomized, brief stimuli of different lengths, ranging from 800 ms dynamic movie snippets to 100 ms static frames. We found that the disruption of temporal continuity strongly altered neural response profiles, particularly in the early response period after stimulus onset. The results suggest that models of visual system function based on discrete and randomized visual presentations may not translate well to the brain's natural modes of operation.

Connectivity alterations of mesostriatal pathways in first episode psychosis

BACKGROUND AND HYPOTHESIS

Pathogenic understanding of the psychotic disorders converges on regulation of dopaminergic signaling in mesostriatocortical pathways. Functional connectivity of the mesostriatal pathways may inform us of the neuronal networks involved.

STUDY DESIGN

This longitudinal study of first episode psychosis (FEP) (49 patients, 43 controls) employed seed-based functional connectivity analyses of fMRI data collected during a naturalistic movie stimulus.

STUDY RESULTS

We identified hypoconnectivity of the dorsal striatum with the midbrain, associated with antipsychotic medication dose in FEP, in comparison with the healthy control group. The midbrain regions that showed hypoconnectivity with the dorsal striatum also showed hypoconnectivity with cerebellar regions suggested to be involved in regulation of the mesostriatocortical dopaminergic pathways. None of the baseline hypoconnectivity detected was seen at follow-up.

CONCLUSIONS

These findings extend earlier resting state findings on mesostriatal connectivity in psychotic disorders and highlight the potential for cerebellar regulation of the mesostriatocortical pathways as a target of treatment trials.

Integrating media content analysis, reception analysis, and media effects studies

Every day, the world of media is at our fingertips, whether it is watching movies, listening to the radio, or browsing online media. On average, people spend over 8 h per day consuming messages from the mass media, amounting to a total lifetime dose of more than 20 years in which conceptual content stimulates our brains. Effects from this flood of information range from short-term attention bursts (e.g., by breaking news features or viral 'memes') to life-long memories (e.g., of one's favorite childhood movie), and from micro-level impacts on an individual's memory, attitudes, and behaviors to macro-level effects on nations or generations. The modern study of media's influence on society dates back to the 1940s. This body of mass communication scholarship has largely asked, "what is media's effect on the individual?" Around the time of the cognitive revolution, media psychologists began to ask, "what cognitive processes are involved in media processing?" More recently, neuroimaging researchers started using real-life media as stimuli to examine perception and cognition under more natural conditions. Such research asks: "what can media tell us about brain function?" With some exceptions, these bodies of scholarship often talk past each other. An integration offers new insights into the neurocognitive mechanisms through which media affect single individuals and entire audiences. However, this endeavor faces the same challenges as all interdisciplinary approaches: Researchers with different backgrounds have different levels of expertise, goals, and foci. For instance, neuroimaging researchers label media stimuli as "naturalistic" although they are in many ways rather artificial. Similarly, media experts are typically unfamiliar with the brain. Neither media creators nor neuroscientifically oriented researchers approach media effects from a social scientific perspective, which is the domain of yet another species. In this article, we provide an overview of approaches and traditions to studying media, and we review the emerging literature that aims to connect these streams. We introduce an organizing scheme that connects the causal paths from media content → brain responses → media effects and discuss network control theory as a promising framework to integrate media content, reception, and effects analyses.

Robust Effects of Working Memory Demand during Naturalistic Language Comprehension in Language-Selective Cortex

To understand language, we must infer structured meanings from real-time auditory or visual signals. Researchers have long focused on word-by-word structure building in working memory as a mechanism that might enable this feat. However, some have argued that language processing does not typically involve rich word-by-word structure building, and/or that apparent working memory effects are underlyingly driven by surprisal (how predictable a word is in context). Consistent with this alternative, some recent behavioral studies of naturalistic language processing that control for surprisal have not shown clear working memory effects. In this fMRI study, we investigate a range of theory-driven predictors of word-by-word working memory demand during naturalistic language comprehension in humans of both sexes under rigorous surprisal controls. In addition, we address a related debate about whether the working memory mechanisms involved in language comprehension are language specialized or domain general. To do so, in each participant, we functionally localize (1) the language-selective network and (2) the "multiple-demand" network, which supports working memory across domains. Results show robust surprisal-independent effects of memory demand in the language network and no effect of memory demand in the multiple-demand network. Our findings thus support the view that language comprehension involves computationally demanding word-by-word structure building operations in working memory, in addition to any prediction-related mechanisms. Further, these memory operations appear to be primarily conducted by the same neural resources that store linguistic knowledge, with no evidence of involvement of brain regions known to support working memory across domains.SIGNIFICANCE STATEMENT This study uses fMRI to investigate signatures of working memory (WM) demand during naturalistic story listening, using a broad range of theoretically motivated estimates of WM demand. Results support a strong effect of WM demand in the brain that is distinct from effects of word predictability. Further, these WM demands register primarily in language-selective regions, rather than in "multiple-demand" regions that have previously been associated with WM in nonlinguistic domains. Our findings support a core role for WM in incremental language processing, using WM resources that are specialized for language.

Decision thresholding on fMRI activation maps using the Hilbert-Huang transform

OBJECTIVE

Functional magnetic resonance imaging (fMRI) requires thresholds by which to identify brain regions with significant activation, particularly for experiments involving real-life paradigms. One conventional non-parametric approach to generating surrogate data involves decomposition of the original fMRI time series using the Fourier transform, after which the phase is randomized without altering the magnitude of individual frequency components. However, it has been reported that spontaneous brain signals could be non-stationary, which, if true, could lead to false-positive results.

APPROACH

This paper introduces a randomization procedure based on the Hilbert-Huang transform by which to account for non-stationarity in fMRI time series derived from two fMRI datasets (stationary or non-stationary). The significance of individual voxels was determined by comparing the distribution of empirical data versus a surrogate distribution.

MAIN RESULTS

In a comparison with conventional phase-randomization and wavelet-based permutation methods, the proposed method proved highly effective in generating activation maps indicating essential brain regions, while filtering out noise in the white matter.

SIGNIFICANCE

This work demonstrated the importance of considering the non-stationary nature of fMRI time series when selecting resampling methods by which to probe brain activity or identify functional networks in real-life fMRI experiments. We propose a statistical testing method to deal with the non-stationarity of continuous brain signals.

A 10-hour within-participant magnetoencephalography narrative dataset to test models of language comprehension

Recently, cognitive neuroscientists have increasingly studied the brain responses to narratives. At the same time, we are witnessing exciting developments in natural language processing where large-scale neural network models can be used to instantiate cognitive hypotheses in narrative processing. Yet, they learn from text alone and we lack ways of incorporating biological constraints during training. To mitigate this gap, we provide a narrative comprehension magnetoencephalography (MEG) data resource that can be used to train neural network models directly on brain data. We recorded from 3 participants, 10 separate recording hour-long sessions each, while they listened to audiobooks in English. After story listening, participants answered short questions about their experience. To minimize head movement, the participants wore MEG-compatible head casts, which immobilized their head position during recording. We report a basic evoked-response analysis showing that the responses accurately localize to primary auditory areas. The responses are robust and conserved across 10 sessions for every participant. We also provide usage notes and briefly outline possible future uses of the resource.

A Comparison of Single- and Multi-Echo Processing of Functional MRI Data During Overt Autobiographical Recall

Recent years have seen an increase in the use of multi-echo fMRI designs by cognitive neuroscientists. Acquiring multiple echoes allows one to increase contrast-to-noise; reduce signal dropout and thermal noise; and identify nuisance signal components in BOLD data. At the same time, multi-echo acquisitions increase data processing complexity and may incur a cost to the temporal and spatial resolution of the acquired data. Here, we re-examine a multi-echo dataset previously analyzed using multi-echo independent components analysis (ME-ICA) and focused on hippocampal activity during the overtly spoken recall of recent and remote autobiographical memories. The goal of the present series of analyses was to determine if ME-ICA’s theoretical denoising benefits might lead to a practical difference in the overall conclusions reached. Compared to single-echo (SE) data, ME-ICA led to qualitatively different findings regarding hippocampal contributions to autobiographical recall: whereas the SE analysis largely failed to reveal hippocampal activity relative to an active baseline, ME-ICA results supported predictions of the Standard Model of Consolidation and a time limited hippocampal involvement. These data provide a practical example of the benefits multi-echo denoising in a naturalistic memory paradigm and demonstrate how they can be used to address long-standing theoretical questions.

Naturalistic viewing conditions can increase task engagement and aesthetic preference but have only minimal impact on EEG quality

Free gaze and moving images are typically avoided in EEG experiments due to the expected generation of artifacts and noise. Yet for a growing number of research questions, loosening these rigorous restrictions would be beneficial. Among these is research on visual aesthetic experiences, which often involve open-ended exploration of highly variable stimuli. Here we systematically compare the effect of conservative vs. more liberal experimental settings on various measures of behavior, brain activity and physiology in an aesthetic rating task. Our primary aim was to assess EEG signal quality. 43 participants either maintained fixation or were allowed to gaze freely, and viewed either static images or dynamic (video) stimuli consisting of dance performances or nature scenes. A passive auditory background task (auditory steady-state response; ASSR) was added as a proxy measure for overall EEG recording quality. We recorded EEG, ECG and eyetracking data, and participants rated their aesthetic preference and state of boredom on each trial. Whereas both behavioral ratings and gaze behavior were affected by task and stimulus manipulations, EEG SNR was barely affected and generally robust across all conditions, despite only minimal preprocessing and no trial rejection. In particular, we show that using video stimuli does not necessarily result in lower EEG quality and can, on the contrary, significantly reduce eye movements while increasing both the participants' aesthetic response and general task engagement. We see these as encouraging results indicating that - at least in the lab - more liberal experimental conditions can be adopted without significant loss of signal quality.

Exploring Hierarchical Auditory Representation via a Neural Encoding Model

By integrating hierarchical feature modeling of auditory information using deep neural networks (DNNs), recent functional magnetic resonance imaging (fMRI) encoding studies have revealed the hierarchical neural auditory representation in the superior temporal gyrus (STG). Most of these studies adopted supervised DNNs (e.g., for audio classification) to derive the hierarchical feature representation of external auditory stimuli. One possible limitation is that the extracted features could be biased toward discriminative features while ignoring general attributes shared by auditory information in multiple categories. Consequently, the hierarchy of neural acoustic processing revealed by the encoding model might be biased toward classification. In this study, we explored the hierarchical neural auditory representation via an fMRI encoding framework in which an unsupervised deep convolutional auto-encoder (DCAE) model was adopted to derive the hierarchical feature representations of the stimuli (naturalistic auditory excerpts in different categories) in fMRI acquisition. The experimental results showed that the neural representation of hierarchical auditory features is not limited to previously reported STG, but also involves the bilateral insula, ventral visual cortex, and thalamus. The current study may provide complementary evidence to understand the hierarchical auditory processing in the human brain.

BrainIAK: The Brain Imaging Analysis Kit

Functional magnetic resonance imaging (fMRI) offers a rich source of data for studying the neural basis of cognition. Here, we describe the Brain Imaging Analysis Kit (BrainIAK), an open-source, free Python package that provides computationally optimized solutions to key problems in advanced fMRI analysis. A variety of techniques are presently included in BrainIAK: intersubject correlation (ISC) and intersubject functional connectivity (ISFC), functional alignment via the shared response model (SRM), full correlation matrix analysis (FCMA), a Bayesian version of representational similarity analysis (BRSA), event segmentation using hidden Markov models, topographic factor analysis (TFA), inverted encoding models (IEMs), an fMRI data simulator that uses noise characteristics from real data (fmrisim), and some emerging methods. These techniques have been optimized to leverage the efficiencies of high-performance compute (HPC) clusters, and the same code can be seamlessly transferred from a laptop to a cluster. For each of the aforementioned techniques, we describe the data analysis problem that the technique is meant to solve and how it solves that problem; we also include an example Jupyter notebook for each technique and an annotated bibliography of papers that have used and/or described that technique. In addition to the sections describing various analysis techniques in BrainIAK, we have included sections describing the future applications of BrainIAK to real-time fMRI, tutorials that we have developed and shared online to facilitate learning the techniques in BrainIAK, computational innovations in BrainIAK, and how to contribute to BrainIAK. We hope that this manuscript helps readers to understand how BrainIAK might be useful in their research.

Functional Subdivisions of the Cerebellum in Naturalistic Paradigm Functional Magnetic Resonance Imaging

Compelling evidence has suggested that the human cerebellum is engaged in a wide range of cognitive tasks besides traditional opinions of motor control, and it is organized into a set of distinct functional subregions. The existing model-driven cerebellum parcellations through resting-state functional MRI (rsfMRI) and task-fMRI are relatively coarse, introducing challenges in resolving the functions of the cerebellum especially when the brain is exposed to naturalistic environments. The current study took the advantages of the naturalistic paradigm (i.e., movie viewing) fMRI (nfMRI) to derive fine parcellations via a data-driven dual-regression-like sparse representation framework. The parcellations were quantitatively evaluated by functional homogeneity, and global and local boundary confidence. In addition, the differences of cerebellum–cerebrum functional connectivities between rsfMRI and nfMRI for some exemplar parcellations were compared to provide qualitatively functional validations. Our experimental results demonstrated that the proposed study successfully identified distinct subregions of the cerebellum. This fine parcellation may serve as a complementary solution to existing cerebellum parcellations, providing an alternative template for exploring neural activities of the cerebellum in naturalistic environments.

The "Narratives" fMRI dataset for evaluating models of naturalistic language comprehension

The "Narratives" collection aggregates a variety of functional MRI datasets collected while human subjects listened to naturalistic spoken stories. The current release includes 345 subjects, 891 functional scans, and 27 diverse stories of varying duration totaling ~4.6 hours of unique stimuli (~43,000 words). This data collection is well-suited for naturalistic neuroimaging analysis, and is intended to serve as a benchmark for models of language and narrative comprehension. We provide standardized MRI data accompanied by rich metadata, preprocessed versions of the data ready for immediate use, and the spoken story stimuli with time-stamped phoneme- and word-level transcripts. All code and data are publicly available with full provenance in keeping with current best practices in transparent and reproducible neuroimaging.

Incremental Language Comprehension Difficulty Predicts Activity in the Language Network but Not the Multiple Demand Network

What role do domain-general executive functions play in human language comprehension? To address this question, we examine the relationship between behavioral measures of comprehension and neural activity in the domain-general "multiple demand" (MD) network, which has been linked to constructs like attention, working memory, inhibitory control, and selection, and implicated in diverse goal-directed behaviors. Specifically, functional magnetic resonance imaging data collected during naturalistic story listening are compared with theory-neutral measures of online comprehension difficulty and incremental processing load (reading times and eye-fixation durations). Critically, to ensure that variance in these measures is driven by features of the linguistic stimulus rather than reflecting participant- or trial-level variability, the neuroimaging and behavioral datasets were collected in nonoverlapping samples. We find no behavioral-neural link in functionally localized MD regions; instead, this link is found in the domain-specific, fronto-temporal "core language network," in both left-hemispheric areas and their right hemispheric homotopic areas. These results argue against strong involvement of domain-general executive circuits in language comprehension.

Incremental Language Comprehension Difficulty Predicts Activity in the Language Network but Not the Multiple Demand Network

What role do domain-general executive functions play in human language comprehension? To address this question, we examine the relationship between behavioral measures of comprehension and neural activity in the domain-general "multiple demand" (MD) network, which has been linked to constructs like attention, working memory, inhibitory control, and selection, and implicated in diverse goal-directed behaviors. Specifically, functional magnetic resonance imaging data collected during naturalistic story listening are compared with theory-neutral measures of online comprehension difficulty and incremental processing load (reading times and eye-fixation durations). Critically, to ensure that variance in these measures is driven by features of the linguistic stimulus rather than reflecting participant- or trial-level variability, the neuroimaging and behavioral datasets were collected in nonoverlapping samples. We find no behavioral-neural link in functionally localized MD regions; instead, this link is found in the domain-specific, fronto-temporal "core language network," in both left-hemispheric areas and their right hemispheric homotopic areas. These results argue against strong involvement of domain-general executive circuits in language comprehension.

Continuous-time deconvolutional regression for psycholinguistic modeling

The influence of stimuli in psycholinguistic experiments diffuses across time because the human response to language is not instantaneous. The linear models typically used to analyze psycholinguistic data are unable to account for this phenomenon due to strong temporal independence assumptions, while existing deconvolutional methods for estimating diffuse temporal structure model time discretely and therefore cannot be directly applied to natural language stimuli where events (words) have variable duration. In light of evidence that continuous-time deconvolutional regression (CDR) can address these issues (Shain & Schuler, 2018), this article motivates the use of CDR for many experimental settings, exposits some of its mathematical properties, and empirically evaluates the influence of various experimental confounds (noise, multicollinearity, and impulse response misspecification), hyperparameter settings, and response types (behavioral and fMRI). Results show that CDR (1) yields highly consistent estimates across a variety of hyperparameter configurations, (2) faithfully recovers the data-generating model on synthetic data, even under adverse training conditions, and (3) outperforms widely-used statistical approaches when applied to naturalistic reading and fMRI data. In addition, procedures for testing scientific hypotheses using CDR are defined and demonstrated, and empirically-motivated best-practices for CDR modeling are proposed. Results support the use of CDR for analyzing psycholinguistic time series, especially in a naturalistic experimental paradigm.

The Treachery of Images: How Realism Influences Brain and Behavior

Although the cognitive sciences aim to ultimately understand behavior and brain function in the real world, for historical and practical reasons, the field has relied heavily on artificial stimuli, typically pictures. We review a growing body of evidence that both behavior and brain function differ between image proxies and real, tangible objects. We also propose a new framework for immersive neuroscience to combine two approaches: (i) the traditional build-up approach of gradually combining simplified stimuli, tasks, and processes; and (ii) a newer tear-down approach that begins with reality and compelling simulations such as virtual reality to determine which elements critically affect behavior and brain processing.

Team-work, Team-brain: Exploring synchrony and team interdependence in a nine-person drumming task via multiparticipant hyperscanning and inter-brain network topology with fNIRS

Teamwork is indispensable in human societies. However, due to the complexity of studying ecologically valid synchronous team actions, requiring multiple members and a range of subjective and objective measures, the mechanism underlying the impact of synchrony on team performance is still unclear. In this paper, we simultaneously measured groups of nine-participants' (total N = 180) fronto-temporal activations during a drum beating task using functional near infrared spectroscopy (fNIRS)-based hyperscanning and multi-brain network modeling, which can assess patterns of shared neural synchrony and attention/information sharing across entire teams. Participants (1) beat randomly without considering others' drumming (random condition), (2) actively coordinated their beats with the entire group without other external cue (team-focus condition), and (3) beat together based on a metronome (shared-focus condition). Behavioral data revealed higher subjective and objective measures of drum-beat synchronization in the team-focus condition, as well as higher felt interdependence. The fNIRS data revealed that participants in the team-focus condition also showed higher interpersonal neural synchronization (INS) and higher Global Network Efficiency in their left TPJ and mPFC. Higher left TPJ Global Network Efficiency also predicted higher actual synchrony in the team-focus condition, with an effect size roughly 1.5 times that of subjective measures, but not in the metronome-enabled shared-focus condition. This result suggests that shared mental representations with high efficiency of information exchange across the entire team may be a key component of synchrony, adding to the understanding of the actual relation to team work.

Intra-Brain Connectivity vs. Inter-Brain Connectivity in Gestures Reproduction: What Relationship?

Recently, the neurosciences have become interested in the investigation of neural responses associated with the use of gestures. This study focuses on the relationship between the intra-brain and inter-brain connectivity mechanisms underlying the execution of different categories of gestures (positive and negative affective, social, and informative) characterizing non-verbal interactions between thirteen couples of subjects, each composed of an encoder and a decoder. The study results underline a similar modulation of intra- and inter-brain connectivity for alpha, delta, and theta frequency bands in specific areas (frontal or posterior regions) depending on the type of gesture. Moreover, taking into account the gestures' valence (positive or negative), a similar modulation of intra- and inter-brain connectivity in the left and right sides was observed. This study showed congruence in the intra-brain and inter-brain connectivity trend during the execution of different gestures, underlining how non-verbal exchanges might be characterized by intra-brain phase alignment and implicit mechanisms of mirroring and synchronization between the two individuals involved in the social exchange.

From scanners to cell phones: neural and real-world responses to social evaluation in adolescent girls

While expanded use of neuroimaging seemed promising to elucidate typical and atypical elements of social sensitivity, in many ways progress in this space has stalled. This is in part due to a disconnection between neurobiological measurements and behavior outside of the laboratory. The present study uses a developmentally salient fMRI computer task and novel ecological momentary assessment protocol to examine whether early adolescent females (n = 76; ages 11–13) with greater neural reactivity to social rejection actually report greater emotional reactivity following negative interactions with peers in daily life. As hypothesized, associations were found between reactivity to perceived social threat in daily life and neural activity in threat-related brain regions, including the left amygdala and bilateral insula, to peer rejection relative to a control condition. Additionally, daily life reactivity to perceived social threat was associated with functional connectivity between the left amygdala and dorsomedial prefrontal cortex during rejection feedback. Unexpectedly, daily life social threat reactivity was also related to heightened amygdala and insula activation to peer acceptance relative to a control condition. These findings may inform key brain–behavior associations supporting sensitivity to social evaluation in adolescence.

Failure to engage the temporoparietal junction/posterior superior temporal sulcus predicts impaired naturalistic social cognition in schizophrenia

Schizophrenia is associated with marked impairments in social cognition. However, the neural correlates of these deficits remain unclear. Here we use naturalistic stimuli to examine the role of the right temporoparietal junction/posterior superior temporal sulcus (TPJ-pSTS)-an integrative hub for the cortical networks pertinent to the understanding complex social situations-in social inference, a key component of social cognition, in schizophrenia. Twenty-seven schizophrenia participants and 21 healthy control subjects watched a clip of the film The Good, the Bad and the Ugly while high resolution multiband functional MRI images were collected. We used inter-subject correlation to measure the evoked activity, which we then compared to social cognition as measured by The Awareness of Social Inference Test (TASIT). We also compared between groups the TPJ-pSTS blood oxygen level-dependent activity (i) relationship with the motion content in the film; (ii) synchronization with other cortical areas involved in the viewing of the movie; and (iii) relationship with the frequency of saccades made during the movie. Activation deficits were greatest in middle TPJ (TPJm) and correlated significantly with impaired TASIT performance across groups. Follow-up analyses of the TPJ-pSTS revealed decreased synchronization with other cortical areas, decreased correlation with the motion content of the movie, and decreased correlation with the saccades made during the movie. The functional impairment of the TPJm, a hub area in the middle of the TPJ-pSTS, predicts deficits in social inference in schizophrenia participants by disrupting the integration of visual motion processing into the TPJ. This disrupted integration then affects the use of the TPJ to guide saccades during the visual scanning of the movie clip. These findings suggest that the TPJ may be a treatment target for improving deficits in a key component of social cognition in schizophrenia participants.

Extracting human cortical responses to sound onsets and acoustic feature changes in real music, and their relation to event rate

Evoked cortical responses (ERs) have mainly been studied in controlled experiments using simplified stimuli. Though, an outstanding question is how the human cortex responds to the complex stimuli encountered in realistic situations. Few electroencephalography (EEG) studies have used Music Information Retrieval (MIR) tools to extract cortical P1/N1/P2 to acoustical changes in real music. However, less than ten events per music piece could be detected leading to ERs due to limitations in automatic detection of sound onsets. Also, the factors influencing a successful extraction of the ERs have not been identified. Finally, previous studies did not localize the sources of the cortical generators. This study is based on an EEG/MEG dataset from 48 healthy normal hearing participants listening to three real music pieces. Acoustic features were computed from the audio signal of the music with the MIR Toolbox. To overcome limits in automatic methods, sound onsets were also manually detected. The chance of obtaining detectable ERs based on ten randomly picked onset points was less than 1:10,000. For the first time, we show that naturalistic P1/N1/P2 ERs can be reliably measured across 100 manually identified sound onsets, substantially improving the signal-to-noise level compared to <10 trials. More ERs were measurable in musical sections with slow event rates (0.2 Hz-2.5 Hz) than with fast event rates (>2.5 Hz). Furthermore, during monophonic sections of the music only P1/P2 were measurable, and during polyphonic sections only N1. Finally, MEG source analysis revealed that naturalistic P2 is located in core areas of the auditory cortex.

Brain signatures predict communicative function of speech production in interaction

People normally know what they want to communicate before they start speaking. However, brain indicators of communication are typically observed only after speech act onset, and it is unclear when any anticipatory brain activity prior to speaking might first emerge, along with the communicative intentions it possibly reflects. Here, we investigated brain activity prior to the production of different speech act types, request and naming actions performed by uttering single words embedded into language games with a partner, similar to natural communication. Starting ca. 600 msec before speech onset, an event-related potential maximal at fronto-central electrodes, which resembled the Readiness Potential, was larger when preparing requests compared to naming actions. Analysis of the cortical sources of this anticipatory brain potential suggests a relatively stronger involvement of fronto-central motor regions for requests, which may reflect the speaker's expectation of the partner actions typically following requests, e.g., the handing over of a requested object. Our results indicate that different neuronal circuits underlying the processing of different speech act types activate already before speaking. Results are discussed in light of previous work addressing the neural basis of speech act understanding and predictive brain indexes of language comprehension.

Keep it real: rethinking the primacy of experimental control in cognitive neuroscience

Naturalistic experimental paradigms in neuroimaging arose from a pressure to test the validity of models we derive from highly-controlled experiments in real-world contexts. In many cases, however, such efforts led to the realization that models developed under particular experimental manipulations failed to capture much variance outside the context of that manipulation. The critique of non-naturalistic experiments is not a recent development; it echoes a persistent and subversive thread in the history of modern psychology. The brain has evolved to guide behavior in a multidimensional world with many interacting variables. The assumption that artificially decoupling and manipulating these variables will lead to a satisfactory understanding of the brain may be untenable. We develop an argument for the primacy of naturalistic paradigms, and point to recent developments in machine learning as an example of the transformative power of relinquishing control. Naturalistic paradigms should not be deployed as an afterthought if we hope to build models of brain and behavior that extend beyond the laboratory into the real world.

A naturalistic neuroimaging database for understanding the brain using ecological stimuli

Neuroimaging has advanced our understanding of human psychology using reductionist stimuli that often do not resemble information the brain naturally encounters. It has improved our understanding of the network organization of the brain mostly through analyses of 'resting-state' data for which the functions of networks cannot be verifiably labelled. We make a 'Naturalistic Neuroimaging Database' (NNDb v1.0) publically available to allow for a more complete understanding of the brain under more ecological conditions during which networks can be labelled. Eighty-six participants underwent behavioural testing and watched one of 10 full-length movies while functional magnetic resonance imaging was acquired. Resulting timeseries data are shown to be of high quality, with good signal-to-noise ratio, few outliers and low movement. Data-driven functional analyses provide further evidence of data quality. They also demonstrate accurate timeseries/movie alignment and how movie annotations might be used to label networks. The NNDb can be used to answer questions previously unaddressed with standard neuroimaging approaches, progressing our knowledge of how the brain works in the real world.

Social cognition in context: A naturalistic imaging approach

Social processing occurs within dynamic, complex, and multimodal contexts, but the study of social cognition typically involves static, artificial stimuli. Naturalistic approaches (e.g., movie viewing) can recapture the richness and complexity of real-world interactions. Novel analytic approaches allow for the investigation of functional brain organization in response to contextually embedded and extended events with a complex temporal structure during movie viewing or narrative processing. In addition to these within-brain measures, movies afford between-brain analyses such as inter-subject correlation, which allows for identification of stimulus-specific brain response through the correlation of brain activity between participants' brains. Research using these approaches offers both practical and theoretical advantages in understanding how we navigate our social world. Practically, movies are engaging stimuli that allow for more rapid presentation of multiple event types and improve compliance even in very young populations. Theoretically, studies have validated the use of these measures by demonstrating functional selectivity to contextually embedded stimuli. Naturalistic approaches also allow for novel insights. For example, regions associated with social cognition have longer temporal receptive windows, making them well suited to social-cognitive processes that require integration of information over longer timescales. Furthermore, the similarity in the temporal and spatial brain response between individuals during naturalistic viewing is related to age, predictive of friendships, and reduced in autism spectrum disorder. These findings offer first glimpses into the power of using these naturalistic, dynamic approaches to understand how we perceive, reason about, and interact with others.

Temporal fluctuations in the brain's modular architecture during movie-watching

Brain networks are flexible and reconfigure over time to support ongoing cognitive processes. However, tracking statistically meaningful reconfigurations across time has proven difficult. This has to do largely with issues related to sampling variability, making instantaneous estimation of network organization difficult, along with increased reliance on task-free (cognitively unconstrained) experimental paradigms, limiting the ability to interpret the origin of changes in network structure over time. Here, we address these challenges using time-varying network analysis in conjunction with a naturalistic viewing paradigm. Specifically, we developed a measure of inter-subject network similarity and used this measure as a coincidence filter to identify synchronous fluctuations in network organization across individuals. Applied to movie-watching data, we found that periods of high inter-subject similarity coincided with reductions in network modularity and increased connectivity between cognitive systems. In contrast, low inter-subject similarity was associated with increased system segregation and more rest-like architectures. We then used a data-driven approach to uncover clusters of functional connections that follow similar trajectories over time and are more strongly correlated during movie-watching than at rest. Finally, we show that synchronous fluctuations in network architecture over time can be linked to a subset of features in the movie. Our findings link dynamic fluctuations in network integration and segregation to patterns of inter-subject similarity, and suggest that moment-to-moment fluctuations in functional connectivity reflect shared cognitive processing across individuals.

Neural dynamics of sentiment processing during naturalistic sentence reading

When we read, our eyes move through the text in a series of fixations and high-velocity saccades to extract visual information. This process allows the brain to obtain meaning, e.g., about sentiment, or the emotional valence, expressed in the written text. How exactly the brain extracts the sentiment of single words during naturalistic reading is largely unknown. This is due to the challenges of naturalistic imaging, which has previously led researchers to employ highly controlled, timed word-by-word presentations of custom reading materials that lack ecological validity. Here, we aimed to assess the electrical neural correlates of word sentiment processing during naturalistic reading of English sentences. We used a publicly available dataset of simultaneous electroencephalography (EEG), eye-tracking recordings, and word-level semantic annotations from 7129 words in 400 sentences (Zurich Cognitive Language Processing Corpus; Hollenstein et al., 2018). We computed fixation-related potentials (FRPs), which are evoked electrical responses time-locked to the onset of fixations. A general linear mixed model analysis of FRPs cleaned from visual- and motor-evoked activity showed a topographical difference between the positive and negative sentiment condition in the 224-304 ​ms interval after fixation onset in left-central and right-posterior electrode clusters. An additional analysis that included word-, phrase-, and sentence-level sentiment predictors showed the same FRP differences for the word-level sentiment, but no additional FRP differences for phrase- and sentence-level sentiment. Furthermore, decoding analysis that classified word sentiment (positive or negative) from sentiment-matched 40-trial average FRPs showed a 0.60 average accuracy (95% confidence interval: [0.58, 0.61]). Control analyses ruled out that these results were based on differences in eye movements or linguistic features other than word sentiment. Our results extend previous research by showing that the emotional valence of lexico-semantic stimuli evoke a fast electrical neural response upon word fixation during naturalistic reading. These results provide an important step to identify the neural processes of lexico-semantic processing in ecologically valid conditions and can serve to improve computer algorithms for natural language processing.

fMRI reveals language-specific predictive coding during naturalistic sentence comprehension

Much research in cognitive neuroscience supports prediction as a canonical computation of cognition across domains. Is such predictive coding implemented by feedback from higher-order domain-general circuits, or is it locally implemented in domain-specific circuits? What information sources are used to generate these predictions? This study addresses these two questions in the context of language processing. We present fMRI evidence from a naturalistic comprehension paradigm (1) that predictive coding in the brain's response to language is domain-specific, and (2) that these predictions are sensitive both to local word co-occurrence patterns and to hierarchical structure. Using a recently developed continuous-time deconvolutional regression technique that supports data-driven hemodynamic response function discovery from continuous BOLD signal fluctuations in response to naturalistic stimuli, we found effects of prediction measures in the language network but not in the domain-general multiple-demand network, which supports executive control processes and has been previously implicated in language comprehension. Moreover, within the language network, surface-level and structural prediction effects were separable. The predictability effects in the language network were substantial, with the model capturing over 37% of explainable variance on held-out data. These findings indicate that human sentence processing mechanisms generate predictions about upcoming words using cognitive processes that are sensitive to hierarchical structure and specialized for language processing, rather than via feedback from high-level executive control mechanisms.

Cortical temporal hierarchy is immature in middle childhood

The development of successful social-cognitive abilities requires one to track, accumulate, and integrate knowledge of other people's mental states across time. Regions of the brain differ in their temporal scale (i.e., a cortical temporal hierarchy) and those receptive to long temporal windows may facilitate social-cognitive abilities; however, the cortical development of long timescale processing remains to be investigated. The current study utilized naturalistic viewing to examine cortical development of long timescale processing and its relation to social-cognitive abilities in middle childhood - a time of expanding social spheres and increasing social-cognitive abilities. We found that, compared to adults, children exhibited reduced low-frequency power in the temporo-parietal junction (TPJ) and reduced specialization for long timescale processing within the TPJ and other regions broadly implicated in the default mode network and higher-order visual processing. Further, specialization for long timescales within the right dorsal medial prefrontal cortex became more 'adult-like' as a function of children's comprehension of character mental states. These results suggest that cortical temporal hierarchy in middle childhood is immature and may be important for an accurate representation of complex naturalistic social stimuli during this age.

Direct Fit to Nature: An Evolutionary Perspective on Biological and Artificial Neural Networks

Evolution is a blind fitting process by which organisms become adapted to their environment. Does the brain use similar brute-force fitting processes to learn how to perceive and act upon the world? Recent advances in artificial neural networks have exposed the power of optimizing millions of synaptic weights over millions of observations to operate robustly in real-world contexts. These models do not learn simple, human-interpretable rules or representations of the world; rather, they use local computations to interpolate over task-relevant manifolds in a high-dimensional parameter space. Counterintuitively, similar to evolutionary processes, over-parameterized models can be simple and parsimonious, as they provide a versatile, robust solution for learning a diverse set of functions. This new family of direct-fit models present a radical challenge to many of the theoretical assumptions in psychology and neuroscience. At the same time, this shift in perspective establishes unexpected links with developmental and ecological psychology.

Functional Imaging of Visuospatial Attention in Complex and Naturalistic Conditions

One of the ultimate goals of cognitive neuroscience is to understand how the brain works in the real world. Functional imaging with naturalistic stimuli provides us with the opportunity to study the brain in situations similar to the everyday life. This includes the processing of complex stimuli that can trigger many types of signals related both to the physical characteristics of the external input and to the internal knowledge that we have about natural objects and environments. In this chapter, I will first outline different types of stimuli that have been used in naturalistic imaging studies. These include static pictures, short video clips, full-length movies, and virtual reality, each comprising specific advantages and disadvantages. Next, I will turn to the main issue of visual-spatial orienting in naturalistic conditions and its neural substrates. I will discuss different classes of internal signals, related to objects, scene structure, and long-term memory. All of these, together with external signals about stimulus salience, have been found to modulate the activity and the connectivity of the frontoparietal attention networks. I will conclude by pointing out some promising future directions for functional imaging with naturalistic stimuli. Despite this field of research is still in its early days, I consider that it will play a major role in bridging the gap between standard laboratory paradigms and mechanisms of brain functioning in the real world.

Educational fMRI: From the Lab to the Classroom

Functional MRI (fMRI) findings hold many potential applications for education, and yet, the translation of fMRI findings to education has not flowed. Here, we address the types of fMRI that could better support applications of neuroscience to the classroom. This 'educational fMRI' comprises eight main challenges: (1) collecting artifact-free fMRI data in school-aged participants and in vulnerable young populations, (2) investigating heterogenous cohorts with wide variability in learning abilities and disabilities, (3) studying the brain under natural and ecological conditions, given that many practical topics of interest for education can be addressed only in ecological contexts, (4) depicting complex age-dependent associations of brain and behaviour with multi-modal imaging, (5) assessing changes in brain function related to developmental trajectories and instructional intervention with longitudinal designs, (6) providing system-level mechanistic explanations of brain function, so that useful individualized predictions about learning can be generated, (7) reporting negative findings, so that resources are not wasted on developing ineffective interventions, and (8) sharing data and creating large-scale longitudinal data repositories to ensure transparency and reproducibility of fMRI findings for education. These issues are of paramount importance to the development of optimal fMRI practices for educational applications.

Emotional Transfer in Human-Horse Interaction: New Perspectives on Equine Assisted Interventions

Equine assisted interventions (EAIs) include all therapeutic interventions aimed at improving human wellbeing through the involvement of horses. Due to the prominent emotional involvement traditionally characterizing their relation with humans, horses developed sophisticated communicative skills, which fostered their ability to respond to human emotional states. In this review, we hypothesize that the proximate causation of successful interventions could be human-animal mutual coordination, through which the subjects bodily and, most importantly, emotionally come into contact. We propose that detecting emotions of other individuals and developing the capacity to fine-tune one's own emotional states accordingly (emotional transfer mechanism), could represent the key engine triggering the positive effects of EAIs. We provide a comprehensive analysis of horses' socio-emotional competences according to recent literature and we propose a multidisciplinary approach to investigate this inter-specific match. By considering human and horse as a unique coupling system during the interaction, it would be possible to objectively measure the degree of coordination through the analysis of physiological variables of both human and animal. Merging the state of art on human-horse relationship with the application of novel methodologies, could help to improve standardized protocols for animal assisted interventions, with particular regard to the emotional states of subjects involved.

Getting into sync: Data-driven analyses reveal patterns of neural coupling that distinguish among different social exchanges

In social interactions, each individual's brain drives an action that, in turn, elicits systematic neural responses in their partner that drive a reaction. Consequently, the brain responses of both interactants become temporally contingent upon one another through the actions they generate, and different interaction dynamics will be underpinned by distinct forms of between‐brain coupling. In this study, we investigated this by “performing functional magnetic resonance imaging on two individuals simultaneously (dual‐fMRI) while they competed or cooperated with one another in a turn‐based or concurrent fashion.” To assess whether distinct patterns of neural coupling were associated with these different interactions, we combined two data‐driven, model‐free analytical techniques: group‐independent component analysis and inter‐subject correlation. This revealed four distinct patterns of brain responses that were temporally aligned between interactants: one emerged during co‐operative exchanges and encompassed brain regions involved in social cognitive processing, such as the temporo‐parietal cortex. The other three were associated with competitive exchanges and comprised brain systems implicated in visuo‐motor processing and social decision‐making, including the cerebellum and anterior cingulate cortex. Interestingly, neural coupling was significantly stronger in concurrent relative to turn‐based exchanges. These results demonstrate the utility of data‐driven approaches applied to “dual‐fMRI” data in elucidating the interpersonal neural processes that give rise to the two‐in‐one dynamic characterizing social interaction.

Real-time Mental State Recognition using a Wearable EEG

The increasing quality and availability of low-cost EEG systems offer new possibilities for non-medical purposes. Existing openly available algorithms to assess the user's mental state in real-time have been mainly performed with medical-grade equipment. In this paper, an approach to assess the user's Focus or Relax states in real-time using a consumer-grade, wearable EEG headband is evaluated. One naive measure and four entropy-based measures, computed using relative frequency band powers in the EEG signal, were introduced. Classifiers for relax and focus state detection, based on the estimation of probability distributions, were developed and evaluated in a user study. Results showed that the Tsallis entropy-based measure performed best for the Focus score, whereas the Renyi measure performed best for the Relax score. Sensitivities of 82.0% and 80.4% with specificities of 82.8% and 80.8% were achieved for the Focus and Relax scores, respectively. The results demonstrated the possibilities of using a wearable EEG system for real-time mental state recognition.

Real-Life Neuroscience: An Ecological Approach to Brain and Behavior Research

Owing to advances in neuroimaging technology, the past couple of decades have witnessed a surge of research on brain mechanisms that underlie human cognition. Despite the immense development in cognitive neuroscience, the vast majority of neuroimaging experiments examine isolated agents carrying out artificial tasks in sensory and socially deprived environments. Thus, the understanding of the mechanisms of various domains in cognitive neuroscience, including social cognition and episodic memory, is sorely lacking. Here we focus on social and memory research as representatives of cognitive functions and propose that mainstream, lab-based experimental designs in these fields suffer from two fundamental limitations, pertaining to person-dependent and situation-dependent factors. The person-dependent factor addresses the issue of limiting the active role of the participants in lab-based paradigms that may interfere with their sense of agency and embodiment. The situation-dependent factor addresses the issue of the artificial decontextualized environment in most available paradigms. Building on recent findings showing that real-life as opposed to controlled experimental paradigms involve different mechanisms, we argue that adopting a real-life approach may radically change our understanding of brain and behavior. Therefore, we advocate in favor of a paradigm shift toward a nonreductionist approach, exploiting portable technology in semicontrolled environments, to explore behavior in real life.