Naturalistic imaging: The use of ecologically valid conditions to study brain function

Brain decoding of spontaneous thought: Predictive modeling of self-relevance and valence using personal narratives

The contents and dynamics of spontaneous thought are important factors for personality traits and mental health. However, assessing spontaneous thoughts is challenging due to their unconstrained nature, and directing participants' attention to report their thoughts may fundamentally alter them. Here, we aimed to decode two key content dimensions of spontaneous thought-self-relevance and valence-directly from brain activity. To train functional MRI-based predictive models, we used individually generated personal stories as stimuli in a story-reading task to mimic narrative-like spontaneous thoughts (n = 49). We then tested these models on multiple test datasets (total n = 199). The default mode, ventral attention, and frontoparietal networks played key roles in the predictions, with the anterior insula and midcingulate cortex contributing to self-relevance prediction and the left temporoparietal junction and dorsomedial prefrontal cortex contributing to valence prediction. Overall, this study presents brain models of internal thoughts and emotions, highlighting the potential for the brain decoding of spontaneous thought.

Prediction of individual brain age using movie and resting-state fMRI

Brain age is a promising biomarker for predicting chronological age based on brain imaging data. Although movie and resting-state functional MRI techniques have attracted much research interest for the investigation of brain function, whether the 2 different imaging paradigms show similarities and differences in terms of their capabilities and properties for predicting brain age remains largely unexplored. Here, we used movie and resting-state functional MRI data from 528 participants aged from 18 to 87 years old in the Cambridge Centre for Ageing and Neuroscience data set for functional network construction and further used elastic net for age prediction model building. The connectivity properties of movie and resting-state functional MRI were evaluated based on the connections supporting predictive model building. We found comparable predictive abilities of movie and resting-state connectivity in estimating brain age of individuals, as evidenced by correlation coefficients of 0.868 and 0.862 between actual and predicted age, respectively. Despite some similarities, notable differences in connectivity properties were observed between the predictive models using movie and resting-state functional MRI data, primarily involving components of the default mode network. Our results highlight that both movie and resting-state functional MRI are effective and promising techniques for predicting brain age. Leveraging its data acquisition advantages, such as improved child and patient compliance resulting in reduced motion artifacts, movie functional MRI is emerging as an important paradigm for studying brain function in pediatric and clinical populations.

A high-density diffuse optical tomography dataset of naturalistic viewing

Traditional laboratory tasks offer tight experimental control but lack the richness of our everyday human experience. As a result many cognitive neuroscientists have been motivated to adopt experimental paradigms that are more natural, such as stories and movies. Here we describe data collected from 58 healthy adult participants (aged 18-76 years) who viewed 10 minutes of a movie (The Good, the Bad, and the Ugly, 1966). Most (36) participants viewed the clip more than once, resulting in 106 sessions of data. Cortical responses were mapped using high-density diffuse optical tomography (first- through fourth nearest neighbor separations of 1.3, 3.0, 3.9, and 4.7 cm), covering large portions of superficial occipital, temporal, parietal, and frontal lobes. Consistency of measured activity across subjects was quantified using intersubject correlation analysis. Data are provided in both channel format (SNIRF) and projected to standard space (NIfTI), using an atlas-based light model. These data are suitable for methods exploration as well as investigating a wide variety of cognitive phenomena.

Combining wearable fNIRS and immersive virtual reality to study preschoolers' social development: a proof-of-principle study on preschoolers' social preference

A child's social world is complex and rich, but has traditionally been assessed with conventional experiments where children are presented with repeated stimuli on a screen. These assessments are impoverished relative to the dynamics of social interactions in real life, and can be challenging to implement with preschoolers, who struggle to comply with strict lab rules. The current work meets the need to develop new platforms to assess preschoolers' social development, by presenting a unique virtual-reality set-up combined with wearable functional near-infrared spectroscopy (fNIRS). As a proof-of-principle, we validated this platform by measuring brain activity during self-guided social interaction in 3-to-5-year-olds, which is under-investigated, yet crucial to understand the basis of social interactions in preschoolers. 37 preschoolers chose an interaction partner from one of 4 human-like avatars of different gender and age. We recorded spontaneous brain fluctuations from the frontal and temporoparietal regions (notably engaged in social-categorization and preference) while children played a bubble-popping game with a preferred and an assigned avatar. 60% of the participants chose to play with the same-gender and same-age avatar. However, this result was driven by females (>80% vs. 50% in males). Different fronto-temporoparietal connectivity patterns when playing with the two avatars were observed, especially in females. We showed the feasibility of using a novel set-up to naturalistically assess social preference in preschoolers, which was assessed at the behavioural and functional connectivity level. This work provides a first proof-of-principle for using cutting-edge technologies and naturalistic experiments to study social development, opening new avenues of research.

The abilities of movie-watching functional connectivity in individual identifications and individualized predictions

Quite a few studies have been performed based on movie-watching functional connectivity (FC). As compared to its resting-state counterpart, however, there is still much to know about its abilities in individual identifications and individualized predictions. To pave the way for appropriate usage of movie-watching FC, we systemically evaluated the minimum number of time points, as well as the exact functional networks, supporting individual identifications and individualized predictions of apparent traits based on it. We performed the study based on the 7T movie-watching fMRI data included in the HCP S1200 Release, and took IQ as the test case for the prediction analyses. The results indicate that movie-watching FC based on only 15 time points can support successful individual identifications (99.47%), and the connectivity contributed more to identifications were much associated with higher-order cognitive processes (the secondary visual network, the frontoparietal network and the posterior multimodal network). For individualized predictions of IQ, it was found that successful predictions necessitated 60 time points (predicted vs. actual IQ correlation significant at P < 0.05, based on 5,000 permutations), and the prediction accuracy increased logarithmically with the number of time points used for connectivity calculation. Furthermore, the connectivity that contributed more to individual identifications exhibited the strongest prediction ability. Collectively, our findings demonstrate that movie-watching FC can capture rich information about human brain function, and its ability in individualized predictions depends heavily on the length of fMRI scans.

Optimizing network neuroscience computation of individual differences in human spontaneous brain activity for test-retest reliability

A rapidly emerging application of network neuroscience in neuroimaging studies has provided useful tools to understand individual differences in intrinsic brain function by mapping spontaneous brain activity, namely intrinsic functional network neuroscience (ifNN). However, the variability of methodologies applied across the ifNN studies-with respect to node definition, edge construction, and graph measurements-makes it difficult to directly compare findings and also challenging for end users to select the optimal strategies for mapping individual differences in brain networks. Here, we aim to provide a benchmark for best ifNN practices by systematically comparing the measurement reliability of individual differences under different ifNN analytical strategies using the test-retest design of the Human Connectome Project. The results uncovered four essential principles to guide ifNN studies: (1) use a whole brain parcellation to define network nodes, including subcortical and cerebellar regions; (2) construct functional networks using spontaneous brain activity in multiple slow bands; and (3) optimize topological economy of networks at individual level; and (4) characterize information flow with specific metrics of integration and segregation. We built an interactive online resource of reliability assessments for future ifNN (https://ibraindata.com/research/ifNN).

Arousal modulates the amygdala-insula reciprocal connectivity during naturalistic emotional movie watching

Emotional arousal is a complex state recruiting distributed cortical and subcortical structures, in which the amygdala and insula play an important role. Although previous neuroimaging studies have showed that the amygdala and insula manifest reciprocal connectivity, the effective connectivities and modulatory patterns on the amygdala-insula interactions underpinning arousal are still largely unknown. One of the reasons may be attributed to static and discrete laboratory brain imaging paradigms used in most existing studies. In this study, by integrating naturalistic-paradigm (i.e., movie watching) functional magnetic resonance imaging (fMRI) with a computational affective model that predicts dynamic arousal for the movie stimuli, we investigated the effective amygdala-insula interactions and the modulatory effect of the input arousal on the effective connections. Specifically, the predicted dynamic arousal of the movie served as regressors in general linear model (GLM) analysis and brain activations were identified accordingly. The regions of interest (i.e., the bilateral amygdala and insula) were localized according to the GLM activation map. The effective connectivity and modulatory effect were then inferred by using dynamic causal modeling (DCM). Our experimental results demonstrated that amygdala was the site of driving arousal input and arousal had a modulatory effect on the reciprocal connections between amygdala and insula. Our study provides novel evidence to the underlying neural mechanisms of arousal in a dynamical naturalistic setting.

Beyond simple laboratory studies: Developing sophisticated models to study rich behavior

Psychology and neuroscience are concerned with the study of behavior, of internal cognitive processes, and their neural foundations. However, most laboratory studies use constrained experimental settings that greatly limit the range of behaviors that can be expressed. While focusing on restricted settings ensures methodological control, it risks impoverishing the object of study: by restricting behavior, we might miss key aspects of cognitive and neural functions. In this article, we argue that psychology and neuroscience should increasingly adopt innovative experimental designs, measurement methods, analysis techniques and sophisticated computational models to probe rich, ecologically valid forms of behavior, including social behavior. We discuss the challenges of studying rich forms of behavior as well as the novel opportunities offered by state-of-the-art methodologies and new sensing technologies, and we highlight the importance of developing sophisticated formal models. We exemplify our arguments by reviewing some recent streams of research in psychology, neuroscience and other fields (e.g., sports analytics, ethology and robotics) that have addressed rich forms of behavior in a model-based manner. We hope that these "success cases" will encourage psychologists and neuroscientists to extend their toolbox of techniques with sophisticated behavioral models - and to use them to study rich forms of behavior as well as the cognitive and neural processes that they engage.

A Novel Exploratory Graph-Based Analytical Tool for Functional Near-Infrared Spectroscopy in Naturalistic Experiments: An Illustrative Application in Typically Developing Children

Naturalistic paradigms are being increasingly applied to investigate human brain function. Compared with resting-state and task-based paradigms in neuroimaging, naturalistic stimuli and situations can be potentially more readily translated to daily-life applications. Among neuroimaging modalities, functional near-infrared spectroscopy (fNIRS) is particularly suitable for naturalistic investigations and applications. However, specific and tailored statistical analysis to interrogate brain function using naturalistic fNIRS is warranted. Here, we describe an exploratory graph-centrality-based approach to investigating participants' spatiotemporal similarities from the fNIRS signal. We illustrate the usefulness of our approach in a sample of typically developing children (10 males and 9 females; mean age of 5.2 years old; sd = 0.78) while they watch the Inscapes movie designed for neuroimaging acquisition. A node in the left dorsal prefrontal cortex presented similar responses across children, and those fNIRS responses were in line with scene transitions in the movie stimulus. Our results suggest the feasibility of applying centrality graph-based measures to investigate brain function in naturalistic fNIRS during development.

Using multi-task experiments to test principles of hippocampal function

Investigations of hippocampal functions have revealed a dizzying array of findings, from lesion-based behavioral deficits, to a diverse range of characterized neural activations, to computational models of putative functionality. Across these findings, there remains an ongoing debate about the core function of the hippocampus and the generality of its representation. Researchers have debated whether the hippocampus's primary role relates to the representation of space, the neural basis of (episodic) memory, or some more general computation that generalizes across various cognitive domains. Within these different perspectives, there is much debate about the nature of feature encodings. Here, we suggest that in order to evaluate hippocampal responses-investigating, for example, whether neuronal representations are narrowly targeted to particular tasks or if they subserve domain-general purposes-a promising research strategy may be the use of multi-task experiments, or more generally switching between multiple task contexts while recording from the same neurons in a given session. We argue that this strategy-when combined with explicitly defined theoretical motivations that guide experiment design-could be a fruitful approach to better understand how hippocampal representations support different behaviors. In doing so, we briefly review key open questions in the field, as exemplified by articles in this special issue, as well as previous work using multi-task experiments, and extrapolate to consider how this strategy could be further applied to probe fundamental questions about hippocampal function.

Toward naturalistic neuroscience: Mechanisms underlying the flattening of brain hierarchy in movie-watching compared to rest and task

Identifying the functional specialization of the brain has moved from using cognitive tasks and resting state to using ecological relevant, naturalistic movies. We leveraged a large-scale neuroimaging dataset to directly investigate the hierarchical reorganization of functional brain activity when watching naturalistic films compared to performing seven cognitive tasks and resting. A thermodynamics-inspired whole-brain model paradigm revealed the generative underlying mechanisms for changing the balance in causal interactions between brain regions in different conditions. Paradoxically, the hierarchy is flatter for movie-watching, and the level of nonreversibility is significantly smaller in comparison to both rest and tasks, where the latter in turn have the highest levels of hierarchy and nonreversibility. The underlying mechanisms were revealed by the model-based generative effective connectivity (GEC). Naturalistic films could therefore provide a fast and convenient way to measure important changes in GEC (integrating functional and anatomical connectivity) found in, for example, neuropsychiatric disorders. Overall, this study demonstrates the benefits of moving toward a more naturalistic neuroscience.

Integrating media content analysis, reception analysis, and media effects studies

Every day, the world of media is at our fingertips, whether it is watching movies, listening to the radio, or browsing online media. On average, people spend over 8 h per day consuming messages from the mass media, amounting to a total lifetime dose of more than 20 years in which conceptual content stimulates our brains. Effects from this flood of information range from short-term attention bursts (e.g., by breaking news features or viral 'memes') to life-long memories (e.g., of one's favorite childhood movie), and from micro-level impacts on an individual's memory, attitudes, and behaviors to macro-level effects on nations or generations. The modern study of media's influence on society dates back to the 1940s. This body of mass communication scholarship has largely asked, "what is media's effect on the individual?" Around the time of the cognitive revolution, media psychologists began to ask, "what cognitive processes are involved in media processing?" More recently, neuroimaging researchers started using real-life media as stimuli to examine perception and cognition under more natural conditions. Such research asks: "what can media tell us about brain function?" With some exceptions, these bodies of scholarship often talk past each other. An integration offers new insights into the neurocognitive mechanisms through which media affect single individuals and entire audiences. However, this endeavor faces the same challenges as all interdisciplinary approaches: Researchers with different backgrounds have different levels of expertise, goals, and foci. For instance, neuroimaging researchers label media stimuli as "naturalistic" although they are in many ways rather artificial. Similarly, media experts are typically unfamiliar with the brain. Neither media creators nor neuroscientifically oriented researchers approach media effects from a social scientific perspective, which is the domain of yet another species. In this article, we provide an overview of approaches and traditions to studying media, and we review the emerging literature that aims to connect these streams. We introduce an organizing scheme that connects the causal paths from media content → brain responses → media effects and discuss network control theory as a promising framework to integrate media content, reception, and effects analyses.

Unraveling the functional attributes of the language connectome: crucial subnetworks, flexibility and variability

Language processing is a highly integrative function, intertwining linguistic operations (processing the language code intentionally used for communication) and extra-linguistic processes (e.g., attention monitoring, predictive inference, long-term memory). This synergetic cognitive architecture requires a distributed and specialized neural substrate. Brain systems have mainly been examined at rest. However, task-related functional connectivity provides additional and valuable information about how information is processed when various cognitive states are involved. We gathered thirteen language fMRI tasks in a unique database of one hundred and fifty neurotypical adults (InLang [Interactive networks of Language] database), providing the opportunity to assess language features across a wide range of linguistic processes. Using this database, we applied network theory as a computational tool to model the task-related functional connectome of language (LANG atlas). The organization of this data-driven neurocognitive atlas of language was examined at multiple levels, uncovering its major components (or crucial subnetworks), and its anatomical and functional correlates. In addition, we estimated its reconfiguration as a function of linguistic demand (flexibility) or several factors such as age or gender (variability). We observed that several discrete networks could be specifically shaped to promote key functional features of language: coding-decoding (Net1), control-executive (Net2), abstract-knowledge (Net3), and sensorimotor (Net4) functions. The architecture of these systems and the functional connectivity of the pivotal brain regions varied according to the nature of the linguistic process, gender, or age. By accounting for the multifaceted nature of language and modulating factors, this study can contribute to enriching and refining existing neurocognitive models of language. The LANG atlas can also be considered a reference for comparative or clinical studies involving various patients and conditions.

Leveraging the Power of Media to Drive Cognition: A Media-Informed Approach to Naturalistic Neuroscience

So-called ‘naturalistic’ stimuli have risen in popularity in cognitive, social and affective neuroscience over the last 15 years. However, a critical property of these stimuli is frequently overlooked: Media—like film, television, books and podcasts—are ‘fundamentally not natural’. They are deliberately crafted products meant to elicit particular human thought, emotion and behavior. Here, we argue for a more informed approach to adopting media stimuli in experimental paradigms. We discuss the pitfalls of combining stimuli that are designed for research with those that are designed for other purposes (e.g. entertainment) under the umbrella term of ‘naturalistic’ and present strategies to improve rigor in the stimulus selection process. We assert that experiencing media should be considered a task akin to any other experimental task(s) and explain how this shift in perspective will compel more nuanced and generalizable research using these stimuli. Throughout, we offer theoretical and practical knowledge from multidisciplinary media research to raise the standard for the treatment of media stimuli in neuroscience research.