Predicting attention across time and contexts with functional brain connectivity

Youth Generalized Anxiety and Brain Activation States During Socioemotional Processing

Importance

The brain enters distinct activation states to support differential cognitive and emotional processes, but little is known about how brain activation states differ in youths with clinical anxiety.

Objective

To characterize brain activation states during socioemotional processing (movie stimuli) and assess associations between state characteristics and movie features and anxiety symptoms.

Design, Setting, and Participants

The Healthy Brain Network is an ongoing cross-sectional study of individuals aged 5 to 21 years experiencing difficulties in school, of whom approximately 45% met criteria for a lifetime anxiety disorder diagnosis. Data used in this study are from the first 9 releases (collected in a nonclinical research setting in the New York City metropolitan area from 2015 to 2020) and include 620 youths aged 5 to 15 years (53% of whom met criteria for a lifetime anxiety disorder diagnosis) who watched an emotional video during functional magnetic resonance imaging and completed questionnaires and clinical evaluation. Of those with functional magnetic resonance imaging data, 432 youths aged 7 to 15 years also self-reported on anxiety symptoms. Data were processed and analyzed between February 2020 and August 2024.

Main Outcomes and Measures

A hidden Markov model was trained to identify brain activation states across participants during video watching. Time spent in each state and the moment-to-moment probability of being in each state were extracted. Videos were annotated for emotion-specific and nonspecific information using the EmoCodes system. Self-reported anxiety symptoms were assessed using the Screen for Child Anxiety Related Disorders. Time spent in each state across the video and during and outside of peaks in negative content correlated with generalized and social anxiety scores.

Results

Among the 620 youths in the overall analysis, 369 were male and the mean (SD) age was 10.4 (2.8) years. In the anxiety symptom analysis, 263 of 432 youths were male and the mean (SD) age was 11.5 (2.2) years. Three brain activation states were identified: a high somatomotor activation state (state 1), a high cingulo-opercular network activation state (state 2), and a high ventral attention and default mode state (state 3). The probability of being in state 3 was correlated with video content that was more negative, quieter, and with less visual motion (ρ < 0.08; P < .001). Increased generalized anxiety was associated with greater time in state 3 (B, 0.10; 95% CI, 0.01 to 0.20; false discovery rate [FDR]-corrected P = .048) and less time in state 2 (B, -0.11; 95% CI, -0.21 to -0.02; FDR-corrected P = .048) when negative social cues were present.

Conclusions and Relevance

Youths entered 3 distinct brain activation states during movie watching, and youths with anxiety spent more time in a state with high ventral attention and default activation during negative socioemotional processing. Youths high in generalized anxiety may be more engaged in deeply processing negative emotional content, which may influence self-regulation. Interventions that focus on changing physiological and psychological state during negative social interactions in youths with anxiety should be considered.

Recent trends in multiple metrics and multimodal analysis for neural activity and pupillometry

Recent studies focusing on neural activity captured by neuroimaging modalities have provided various metrics for elucidating the functional networks and dynamics of the entire brain. Functional magnetic resonance imaging (fMRI) can depict spatiotemporal functional neural networks and dynamic characteristics due to its excellent spatial resolution. However, its temporal resolution is limited. Neuroimaging modalities such as electroencephalography (EEG) and magnetoencephalography (MEG), which have higher temporal resolutions, are utilized for multi-temporal scale and multi-frequency-band analyzes. With this advantage, numerous EEG/MEG-bases studies have revealed the frequency-band specific functional networks involving dynamic functional connectivity and multiple temporal-scale time-series patterns of neural activity. In addition to analyzing neural data, the examination of behavioral data can unveil additional aspects of brain activity through unimodal and multimodal data analyzes performed using appropriate integration techniques. Among the behavioral data assessments, pupillometry can provide comprehensive spatial-temporal-specific features of neural activity. In this perspective, we summarize the recent progress in the development of metrics for analyzing neural data obtained from neuroimaging modalities such as fMRI, EEG, and MEG, as well as behavioral data, with a special focus on pupillometry data. First, we review the typical metrics of neural activity, emphasizing functional connectivity, complexity, dynamic functional connectivity, and dynamic state transitions of whole-brain activity. Second, we examine the metrics related to the time-series data of pupillary diameters and discuss the possibility of multimodal metrics that combine neural and pupillometry data. Finally, we discuss future perspectives on these multiple and multimodal metrics.

Longitudinal single-subject neuroimaging study reveals effects of daily environmental, physiological, and lifestyle factors on functional brain connectivity

Our behavior and mental states are constantly shaped by our environment and experiences. However, little is known about the response of brain functional connectivity to environmental, physiological, and behavioral changes on different timescales, from days to months. This gives rise to an urgent need for longitudinal studies that collect high-frequency data. To this end, for a single subject, we collected 133 days of behavioral data with smartphones and wearables and performed 30 functional magnetic resonance imaging (fMRI) scans measuring attention, memory, resting state, and the effects of naturalistic stimuli. We find traces of past behavior and physiology in brain connectivity that extend up as far as 15 days. While sleep and physical activity relate to brain connectivity during cognitively demanding tasks, heart rate variability and respiration rate are more relevant for resting-state connectivity and movie-watching. This unique data set is openly accessible, offering an exceptional opportunity for further discoveries. Our results demonstrate that we should not study brain connectivity in isolation, but rather acknowledge its interdependence with the dynamics of the environment, changes in lifestyle, and short-term fluctuations such as transient illnesses or restless sleep. These results reflect a prolonged and sustained relationship between external factors and neural processes. Overall, precision mapping designs such as the one employed here can help to better understand intraindividual variability, which may explain some of the observed heterogeneity in fMRI findings. The integration of brain connectivity, physiology data and environmental cues will propel future environmental neuroscience research and support precision healthcare.

Uncovering a stability signature of brain dynamics associated with meditation experience using massive time-series feature extraction

Previous research has examined resting electroencephalographic (EEG) data to explore brain activity related to meditation. However, previous research has mostly examined power in different frequency bands. The practical objective of this study was to comprehensively test whether other types of time-series analysis methods are better suited to characterize brain activity related to meditation. To achieve this, we compared >7000 time-series features of the EEG signal to comprehensively characterize brain activity differences in meditators, using many measures that are novel in meditation research. Eyes-closed resting-state EEG data from 49 meditators and 46 non-meditators was decomposed into the top eight principal components (PCs). We extracted 7381 time-series features from each PC and each participant and used them to train classification algorithms to identify meditators. Highly differentiating individual features from successful classifiers were analysed in detail. Only the third PC (which had a central-parietal maximum) showed above-chance classification accuracy (67 %, pFDR = 0.007), for which 405 features significantly distinguished meditators (all pFDR < 0.05). Top-performing features indicated that meditators exhibited more consistent statistical properties across shorter subsegments of their EEG time-series (higher stationarity) and displayed an altered distributional shape of values about the mean. By contrast, classifiers trained with traditional band-power measures did not distinguish the groups (pFDR > 0.05). Our novel analysis approach suggests the key signatures of meditators' brain activity are higher temporal stability and a distribution of time-series values suggestive of longer, larger, or more frequent non-outlying voltage deviations from the mean within the third PC of their EEG data. The higher temporal stability observed in this EEG component might underpin the higher attentional stability associated with meditation. The novel time-series properties identified here have considerable potential for future exploration in meditation research and the analysis of neural dynamics more broadly.

Data leakage inflates prediction performance in connectome-based machine learning models

Predictive modeling is a central technique in neuroimaging to identify brain-behavior relationships and test their generalizability to unseen data. However, data leakage undermines the validity of predictive models by breaching the separation between training and test data. Leakage is always an incorrect practice but still pervasive in machine learning. Understanding its effects on neuroimaging predictive models can inform how leakage affects existing literature. Here, we investigate the effects of five forms of leakage-involving feature selection, covariate correction, and dependence between subjects-on functional and structural connectome-based machine learning models across four datasets and three phenotypes. Leakage via feature selection and repeated subjects drastically inflates prediction performance, whereas other forms of leakage have minor effects. Furthermore, small datasets exacerbate the effects of leakage. Overall, our results illustrate the variable effects of leakage and underscore the importance of avoiding data leakage to improve the validity and reproducibility of predictive modeling.

The effects of data leakage on connectome-based machine learning models

Predictive modeling has now become a central technique in neuroimaging to identify complex brain-behavior relationships and test their generalizability to unseen data. However, data leakage, which unintentionally breaches the separation between data used to train and test the model, undermines the validity of predictive models. Previous literature suggests that leakage is generally pervasive in machine learning, but few studies have empirically evaluated the effects of leakage in neuroimaging data. Although leakage is always an incorrect practice, understanding the effects of leakage on neuroimaging predictive models provides insight into the extent to which leakage may affect the literature. Here, we investigated the effects of leakage on machine learning models in two common neuroimaging modalities, functional and structural connectomes. Using over 400 different pipelines spanning four large datasets and three phenotypes, we evaluated five forms of leakage fitting into three broad categories: feature selection, covariate correction, and lack of independence between subjects. As expected, leakage via feature selection and repeated subjects drastically inflated prediction performance. Notably, other forms of leakage had only minor effects (e.g., leaky site correction) or even decreased prediction performance (e.g., leaky covariate regression). In some cases, leakage affected not only prediction performance, but also model coefficients, and thus neurobiological interpretations. Finally, we found that predictive models using small datasets were more sensitive to leakage. Overall, our results illustrate the variable effects of leakage on prediction pipelines and underscore the importance of avoiding data leakage to improve the validity and reproducibility of predictive modeling.

Task and Resting-State Functional Connectivity Predict Driving Violations

Aberrant driving behaviors cause accidents; however, there is a lack of understanding of the neural mechanisms underlying these behaviors. To address this issue, a task and resting-state functional connectivity was used to predict aberrant driving behavior and associated personality traits. The study included 29 right-handed participants with driving licenses issued for more than 1 year. During the functional magnetic resonance imaging experiment, participants first recorded their resting state and then watched a driving video while continuously rating the risk and speed on each block. Functional connectome-based predictive modeling was employed for whole brain tasks and resting-state functional connectivity to predict driving behavior (violation, error, and lapses), sensation-seeking, and impulsivity. Resting state and task-based functional connectivity were found to significantly predict driving violations, with resting state significantly predicting lapses and task-based functional connectivity showing a tendency to predict errors. Conversely, neither impulsivity nor sensation-seeking was associated with functional connectivity. The results suggest a significant association between aberrant driving behavior, but a nonsignificant association between impulsivity and sensation-seeking, and task-based or resting state functional connectivity. This could provide a deeper understanding of the neural processing underlying reckless driving that may ultimately be used to prevent accidents.

Perception and memory retrieval states are reflected in distributed patterns of background functional connectivity

The same visual input can serve as the target of perception or as a trigger for memory retrieval depending on whether cognitive processing is externally oriented (perception) or internally oriented (memory retrieval). While numerous human neuroimaging studies have characterized how visual stimuli are differentially processed during perception versus memory retrieval, perception and memory retrieval may also be associated with distinct neural states that are independent of stimulus-evoked neural activity. Here, we combined human fMRI with full correlation matrix analysis (FCMA) to reveal potential differences in "background" functional connectivity across perception and memory retrieval states. We found that perception and retrieval states could be discriminated with high accuracy based on patterns of connectivity across (1) the control network, (2) the default mode network (DMN), and (3) retrosplenial cortex (RSC). In particular, clusters in the control network increased connectivity with each other during the perception state, whereas clusters in the DMN were more strongly coupled during the retrieval state. Interestingly, RSC switched its coupling between networks as the cognitive state shifted from retrieval to perception. Finally, we show that background connectivity (1) was fully independent from stimulus-related variance in the signal and, further, (2) captured distinct aspects of cognitive states compared to traditional classification of stimulus-evoked responses. Together, our results reveal that perception and memory retrieval are associated with sustained cognitive states that manifest as distinct patterns of connectivity among large-scale brain networks.

Exploring the Neural Processes behind Narrative Engagement: An EEG Study

Past cognitive neuroscience studies using naturalistic stimuli have considered narratives holistically and focused on cognitive processes. In this study, we incorporated the narrative structure, the dramatic arc, as an object of investigation, to examine how engagement levels fluctuate across a narrative-aligned dramatic arc. We explored the possibility of predicting self-reported engagement ratings from neural activity and investigated the idiosyncratic effects of each phase of the dramatic arc on brain responses as well as the relationship between engagement and brain responses. We presented a movie excerpt following the six-phase narrative arc structure to female and male participants while collecting EEG signals. We then asked this group of participants to recall the excerpt, another group to segment the video based on the dramatic arc model, and a third to rate their engagement levels while watching the movie. The results showed that the self-reported engagement ratings followed the pattern of the narrative dramatic arc. Moreover, while EEG amplitude could not predict group-averaged engagement ratings, other features comprising dynamic intersubject correlation (dISC), including certain frequency bands, dynamic functional connectivity patterns and graph features were able to achieve this. Furthermore, neural activity in the last two phases of the dramatic arc significantly predicted engagement patterns. This study is the first to explore the cognitive processes behind the dramatic arc and its phases. By demonstrating how neural activity predicts self-reported engagement, which itself aligns with the narrative structure, this study provides insights on the interrelationships between narrative structure, neural responses, and viewer engagement.

Differences in the functional brain architecture of sustained attention and working memory in youth and adults

Sustained attention (SA) and working memory (WM) are critical processes, but the brain networks supporting these abilities in development are unknown. We characterized the functional brain architecture of SA and WM in 9- to 11-year-old children and adults. First, we found that adult network predictors of SA generalized to predict individual differences and fluctuations in SA in youth. A WM model predicted WM performance both across and within children-and captured individual differences in later recognition memory-but underperformed in youth relative to adults. We next characterized functional connections differentially related to SA and WM in youth compared to adults. Results revealed 2 network configurations: a dominant architecture predicting performance in both age groups and a secondary architecture, more prominent for WM than SA, predicting performance in each age group differently. Thus, functional connectivity (FC) predicts SA and WM in youth, with networks predicting WM performance differing more between youths and adults than those predicting SA.

Predicting visual memory across images and within individuals

We only remember a fraction of what we see-including images that are highly memorable and those that we encounter during highly attentive states. However, most models of human memory disregard both an image's memorability and an individual's fluctuating attentional states. Here, we build the first model of memory synthesizing these two disparate factors to predict subsequent image recognition. We combine memorability scores of 1100 images (Experiment 1, n = 706) and attentional state indexed by response time on a continuous performance task (Experiments 2 and 3, n = 57 total). Image memorability and sustained attentional state explained significant variance in image memory, and a joint model of memory including both factors outperformed models including either factor alone. Furthermore, models including both factors successfully predicted memory in an out-of-sample group. Thus, building models based on individual- and image-specific factors allows for directed forecasting of our memories. SIGNIFICANCE STATEMENT: Although memory is a fundamental cognitive process, much of the time memory failures cannot be predicted until it is too late. However, in this study, we show that much of memory is surprisingly pre-determined ahead of time, by factors shared across the population and highly specific to each individual. Specifically, we build a new multidimensional model that predicts memory based just on the images a person sees and when they see them. This research synthesizes findings from disparate domains ranging from computer vision, attention, and memory into a predictive model. These findings have resounding implications for domains such as education, business, and marketing, where it is a top priority to predict (and even manipulate) what information people will remember.

Readiness to remember: predicting variability in episodic memory

Learning and remembering are fundamental to our lives, so what causes us to forget? Answers often highlight preparatory processes that precede learning, as well as mnemonic processes during the act of encoding or retrieval. Importantly, evidence now indicates that preparatory processes that precede retrieval attempts also have powerful influences on memory success or failure. Here, we review recent work from neuroimaging, electroencephalography, pupillometry, and behavioral science to propose an integrative framework of retrieval-period dynamics that explains variance in remembering in the moment and across individuals as a function of interactions among preparatory attention, goal coding, and mnemonic processes. Extending this approach, we consider how a 'readiness to remember' (R2R) framework explains variance in high-level functions of memory and mnemonic disruptions in aging.

Deep Learning Architecture Reduction for fMRI Data

In recent years, deep learning models have demonstrated an inherently better ability to tackle non-linear classification tasks, due to advances in deep learning architectures. However, much remains to be achieved, especially in designing deep convolutional neural network (CNN) configurations. The number of hyper-parameters that need to be optimized to achieve accuracy in classification problems increases with every layer used, and the selection of kernels in each CNN layer has an impact on the overall CNN performance in the training stage, as well as in the classification process. When a popular classifier fails to perform acceptably in practical applications, it may be due to deficiencies in the algorithm and data processing. Thus, understanding the feature extraction process provides insights to help optimize pre-trained architectures, better generalize the models, and obtain the context of each layer’s features. In this work, we aim to improve feature extraction through the use of a texture amortization map (TAM). An algorithm was developed to obtain characteristics from the filters amortizing the filter’s effect depending on the texture of the neighboring pixels. From the initial algorithm, a novel geometric classification score (GCS) was developed, in order to obtain a measure that indicates the effect of one class on another in a classification problem, in terms of the complexity of the learnability in every layer of the deep learning architecture. For this, we assume that all the data transformations in the inner layers still belong to a Euclidean space. In this scenario, we can evaluate which layers provide the best transformations in a CNN, allowing us to reduce the weights of the deep learning architecture using the geometric hypothesis.

Functional ultrasound imaging: A useful tool for functional connectomics?

Functional ultrasound (fUS) is a hemodynamic-based functional neuroimaging technique, primarily used in animal models, that combines a high spatiotemporal resolution, a large field of view, and compatibility with behavior. These assets make fUS especially suited to interrogating brain activity at the systems level. In this review, we describe the technical capabilities offered by fUS and discuss how this technique can contribute to the field of functional connectomics. First, fUS can be used to study intrinsic functional connectivity, namely patterns of correlated activity between brain regions. In this area, fUS has made the most impact by following connectivity changes in disease models, across behavioral states, or dynamically. Second, fUS can also be used to map brain-wide pathways associated with an external event. For example, fUS has helped obtain finer descriptions of several sensory systems, and uncover new pathways implicated in specific behaviors. Additionally, combining fUS with direct circuit manipulations such as optogenetics is an attractive way to map the brain-wide connections of defined neuronal populations. Finally, technological improvements and the application of new analytical tools promise to boost fUS capabilities. As brain coverage and the range of behavioral contexts that can be addressed with fUS keep on increasing, we believe that fUS-guided connectomics will only expand in the future. In this regard, we consider the incorporation of fUS into multimodal studies combining diverse techniques and behavioral tasks to be the most promising research avenue.

State-related neural influences on fMRI connectivity estimation

The spatiotemporal structure of functional magnetic resonance imaging (fMRI) signals has provided a valuable window into the network underpinnings of human brain function and dysfunction. Although some cross-regional temporal correlation patterns (functional connectivity; FC) exhibit a high degree of stability across individuals and species, there is growing acknowledgment that measures of FC can exhibit marked changes over a range of temporal scales. Further, FC can covary with experimental task demands and ongoing neural processes linked to arousal, consciousness and perception, cognitive and affective state, and brain-body interactions. The increased recognition that such interrelated neural processes modulate FC measurements has raised both challenges and new opportunities in using FC to investigate brain function. Here, we review recent advances in the quantification of neural effects that shape fMRI FC and discuss the broad implications of these findings in the design and analysis of fMRI studies. We also discuss how a more complete understanding of the neural factors that shape FC measurements can resolve apparent inconsistencies in the literature and lead to more interpretable conclusions from fMRI studies.

Is it time to put rest to rest?

The so-called resting state, in which participants lie quietly with no particular inputs or outputs, represented a paradigm shift from conventional task-based studies in human neuroimaging. Our foray into rest was fruitful from both a scientific and methodological perspective, but at this point, how much more can we learn from rest on its own? While rest still dominates in many subfields, data from tasks have empirically demonstrated benefits, as well as the potential to provide insights about the mind in addition to the brain. I argue that we can accelerate progress in human neuroscience by de-emphasizing rest in favor of more grounded experiments, including promising integrated designs that respect the prominence of self-generated activity while offering enhanced control and interpretability.

Neural signatures of attentional engagement during narratives and its consequences for event memory

As we comprehend narratives, our attentional engagement fluctuates over time. Despite theoretical conceptions of narrative engagement as emotion-laden attention, little empirical work has characterized the cognitive and neural processes that comprise subjective engagement in naturalistic contexts or its consequences for memory. Here, we relate fluctuations in narrative engagement to patterns of brain coactivation and test whether neural signatures of engagement predict subsequent memory. In behavioral studies, participants continuously rated how engaged they were as they watched a television episode or listened to a story. Self-reported engagement was synchronized across individuals and driven by the emotional content of the narratives. In functional MRI datasets collected as different individuals watched the same show or listened to the same story, engagement drove neural synchrony, such that default mode network activity was more synchronized across individuals during more engaging moments of the narratives. Furthermore, models based on time-varying functional brain connectivity predicted evolving states of engagement across participants and independent datasets. The functional connections that predicted engagement overlapped with a validated neuromarker of sustained attention and predicted recall of narrative events. Together, our findings characterize the neural signatures of attentional engagement in naturalistic contexts and elucidate relationships among narrative engagement, sustained attention, and event memory.