Test-Retest Reliability of fMRI Brain Activity during Memory Encoding

Sex is predicted by spatial memory multivariate activation patterns

Whether sex differences exist in the brain at the macroscopic level, as measured with magnetic resonance imaging (MRI), is a topic of debate. The present spatial long-term memory functional MRI (fMRI) study predicted sex based on event-related patterns of brain activity. Within spatial memory regions of interest, patterns of activity associated with females and males were used to predict the sex of each member of left-out female-male pairs at above-chance accuracy. The current results provide evidence for sex differences in the brain processes underlying spatial long-term memory. This is the first time that sex has been predicted using event-related fMRI activation patterns. The present findings contribute to a growing body of evidence that there are functional and anatomic sex differences in the brain and, more broadly, question the widespread practice of collapsing across sex in the field of cognitive neuroscience.

Brain Reactions to Opening and Closing the Eyes: Salivary Cortisol and Functional Connectivity

This study empirically assessed the strength and duration of short-term effects induced by brain reactions to closing/opening the eyes on a few well-known resting-state networks. We also examined the association between these reactions and subjects’ cortisol levels. A total of 55 young adults underwent 8-min resting-state fMRI (rs-fMRI) scans under 4-min eyes-closed and 4-min eyes-open conditions. Saliva samples were collected from 25 of the 55 subjects before and after the fMRI sessions and assayed for cortisol levels. Our empirical results indicate that when the subjects were relaxed with their eyes closed, the effect of opening the eyes on conventional resting-state networks (e.g., default-mode, frontal-parietal, and saliency networks) lasted for roughly 60-s, during which we observed a short-term increase in activity in rs-fMRI time courses. Moreover, brain reactions to opening the eyes had a pronounced effect on time courses in the temporo-parietal lobes and limbic structures, both of which presented a prolonged decrease in activity. After controlling for demographic factors, we observed a significantly positive correlation between pre-scan cortisol levels and connectivity in the limbic structures under both conditions. Under the eyes-closed condition, the temporo-parietal lobes presented significant connectivity to limbic structures and a significantly positive correlation with pre-scan cortisol levels. Future research on rs-fMRI could consider the eyes-closed condition when probing resting-state connectivity and its neuroendocrine correlates, such as cortisol levels. It also appears that abrupt instructions to open the eyes while the subject is resting quietly with eyes closed could be used to probe brain reactivity to aversive stimuli in the ventral hippocampus and other limbic structures.

Test-retest reliability of fMRI during an emotion processing task: Investigating the impact of analytical approaches on ICC values

Test-retest reliability of fMRI is often assessed using the intraclass correlation coefficient (ICC), a numerical representation of reliability. Reports of low reliability at the individual level may be attributed to analytical approaches and inherent bias/error in the measures used to calculate ICC. It is unclear whether low reliability at the individual level is related to methodological decisions or if fMRI is inherently unreliable. The purpose of this study was to investigate methodological considerations when calculating ICC to improve understanding of fMRI reliability. fMRI data were collected from adolescent females (N=23) at pre- and post-cognitive behavioral therapy. Participants completed an emotion processing task during fMRI. We calculated ICC values using contrasts and β coefficients separately from voxelwise and network (ICA) analyses of the task-based fMRI data. For both voxelwise analysis and ICA, ICC values were higher when calculated using β coefficients. This work provides support for the use of β coefficients over contrasts when assessing reliability of fMRI, and the use of contrasts may underlie low reliability estimates reported in the existing literature. Continued research in this area is warranted to establish fMRI as a reliable measure to draw conclusions and utilize fMRI in clinical settings.

Test–Retest Reliability of Synchrony and Metastability in Resting State fMRI

In recent years, interest has been growing in dynamic characteristic of brain signals from resting-state functional magnetic resonance imaging (rs-fMRI). Synchrony and metastability, as neurodynamic indexes, are considered as one of methods for analyzing dynamic characteristics. Although much research has studied the analysis of neurodynamic indices, few have investigated its reliability. In this paper, the datasets from the Human Connectome Project have been used to explore the test–retest reliabilities of synchrony and metastability from multiple angles through intra-class correlation (ICC). The results showed that both of these indexes had fair test–retest reliability, but they are strongly affected by the field strength, the spatial resolution, and scanning interval, less affected by the temporal resolution. Denoising processing can help improve their ICC values. In addition, the reliability of neurodynamic indexes was affected by the node definition strategy, but these effects were not apparent. In particular, by comparing the test–retest reliability of different resting-state networks, we found that synchrony of different networks was basically stable, but the metastability varied considerably. Among these, DMN and LIM had a relatively higher test–retest reliability of metastability than other networks. This paper provides a methodological reference for exploring the brain dynamic neural activity by using synchrony and metastability in fMRI signals.

Reliability of subsequent memory effects in children and adults: The good, the bad, and the hopeful

Functional MRI (fMRI) is a key tool for investigating neural underpinnings of cognitive development. Yet, in recent years, the reliability of fMRI effects has come into question and with it, the feasibility of using task-based fMRI to identify developmental changes related to cognition. Here, we investigated the reliability of task-based fMRI activations with a widely used subsequent memory paradigm using two developmental samples: a cross-sectional sample (n = 85, age 8-25 years) and a test-retest sample (n = 24, one-month follow up, age 8-20 years). In the large cross-sectional sample, we found good to excellent group-level reliability when assessing activation patterns related to the encoding task and subsequent memory effects. In the test-retest sample, while group-level reliability was excellent, the consistency of activation patterns within individuals was low, particularly for subsequent memory effects. We observed consistent activation patterns in frontal, parietal, and occipital cortices, but comparatively lower test-retest reliability in subcortical regions and the hippocampus. Together, these findings highlight the limitations of interpreting task-based fMRI effects and the importance of incorporating reliability analyses in developmental studies. Leveraging larger and densely collected longitudinal data may help contribute to increased reproducibility and the accumulation of knowledge in developmental sciences.

Early post-treatment blood oxygenation level-dependent responses to emotion processing associated with clinical response to pharmacological treatment in major depressive disorder

INTRODUCTION

Pre-treatment blood oxygenation level-dependent (BOLD) functional magnetic resonance imaging (fMRI) has been used for the early identification of patients with major depressive disorder (MDD) who later respond or fail to respond to medication. However, BOLD responses early after treatment initiation may offer insight into early neural changes associated with later clinical response. The present study evaluated both pre-treatment and early post-treatment fMRI responses to an emotion processing task, to further our understanding of neural changes associated with a successful response to pharmacological intervention.

METHODS

MDD patients who responded (n = 22) and failed to respond (n = 12) after 8 weeks of treatment with either citalopram or quetiapine extended release, and healthy controls (n = 18) underwent two fMRI scans, baseline (pre-treatment), and early post-treatment (one week after treatment commencement). Participants completed an emotional face matching task at both scans.

RESULTS

Using threshold-free cluster enhancement (TFCE) and non-parametric permutation testing, fMRI activation maps showed that after one week of treatment, responders demonstrated increased activation in the left parietal lobule, precentral gyrus, and bilateral insula (all P < 0.05 threshold-free cluster enhancement (TFCE) family-wise error-corrected) to negative facial expressions. Non-responders showed some small increases in the precentral gyrus, while controls showed no differences between scans. Compared to non-responders, responders showed some increased activation in the superior parietal lobule and middle temporal gyrus at the post-treatment scan. There were no group differences between responders, non-responders, and controls at baseline.

CONCLUSIONS

One week after treatment commencement, BOLD signal changes in the parietal lobules, insula, and middle temporal gyrus were related to clinical response to pharmacological treatment.

Neural adaptation of cingulate and insular activity during delayed fear extinction: A replicable pattern across assessment sites and repeated measurements

Adapting threat-related memories towards changing environments is a fundamental ability of organisms. One central process of fear reduction is suggested to be extinction learning, experimentally modeled by extinction training that is repeated exposure to a previously conditioned stimulus (CS) without providing the expected negative consequence (unconditioned stimulus, US). Although extinction training is well investigated, evidence regarding process-related changes in neural activation over time is still missing. Using optimized delayed extinction training in a multicentric trial we tested whether: 1) extinction training elicited decreasing CS-specific neural activation and subjective ratings, 2) extinguished conditioned fear would return after presentation of the US (reinstatement), and 3) results are comparable across different assessment sites and repeated measures. We included 100 healthy subjects (measured twice, 13-week-interval) from six sites. 24 h after fear acquisition training, extinction training, including a reinstatement test, was applied during fMRI. Alongside, participants had to rate subjective US-expectancy, arousal and valence. In the course of the extinction training, we found decreasing neural activation in the insula and cingulate cortex as well as decreasing US-expectancy, arousal and negative valence towards CS+. Re-exposure to the US after extinction training was associated with a temporary increase in neural activation in the anterior cingulate cortex (exploratory analysis) and changes in US-expectancy and arousal ratings. While ICCs-values were low, findings from small groups suggest highly consistent effects across time-points and sites. Therefore, this delayed extinction fMRI-paradigm provides a solid basis for the investigation of differences in neural fear-related mechanisms as a function of anxiety-pathology and exposure-based treatment.

A Functional MRI Paradigm for Efficient Mapping of Memory Encoding Across Sensory Conditions

We introduce a new and time-efficient memory-encoding paradigm for functional magnetic resonance imaging (fMRI). This paradigm is optimized for mapping multiple contrasts using a mixed design, using auditory (environmental/vocal) and visual (scene/face) stimuli. We demonstrate that the paradigm evokes robust neuronal activity in typical sensory and memory networks. We were able to detect auditory and visual sensory-specific encoding activities in auditory and visual cortices. Also, we detected stimulus-selective activation in environmental-, voice-, scene-, and face-selective brain regions (parahippocampal place and fusiform face area). A subsequent recognition task allowed the detection of sensory-specific encoding success activity (ESA) in both auditory and visual cortices, as well as sensory-unspecific positive ESA in the hippocampus. Further, sensory-unspecific negative ESA was observed in the precuneus. Among others, the parallel mixed design enabled sustained and transient activity comparison in contrast to rest blocks. Sustained and transient activations showed great overlap in most sensory brain regions, whereas several regions, typically associated with the default-mode network, showed transient rather than sustained deactivation. We also show that the use of a parallel mixed model had relatively little influence on positive or negative ESA. Together, these results demonstrate a feasible, versatile, and brief memory-encoding task, which includes multiple sensory stimuli to guarantee a comprehensive measurement. This task is especially suitable for large-scale clinical or population studies, which aim to test task-evoked sensory-specific and sensory-unspecific memory-encoding performance as well as broad sensory activity across the life span within a very limited time frame.

Reliability of Frontal Eye Fields Activation and Very Low-Frequency Oscillations Observed during Vergence Eye Movements: an fNIRS Study

Functional near-infrared spectroscopy (fNIRS), an imaging tool that utilizes infrared light to measure changes within the concentration of oxygenated (HbO) and deoxygenated (HbR) hemoglobin, holds promise to study functional activity from motor, visual, and memory cortical regions using stimulus-induced tasks. This study investigated the reliability for fNIRS to examine cortical activations within the frontal eye fields (FEF) while initiating vergence eye movements, the inward and outward rotation of the eyes. FNIRS data were collected from twenty participants with normal binocular vision while performing vergence eye movements compared to sustained gaze fixation within a block design during two different sessions. Reliability of the experimental protocol was assessed using the intraclass correlation coefficient (ICC). The ICC values ranged from 0.6 to 0.7 for measuring the HbO activation within the vicinity of the FEF. A frequency power spectrum analysis revealed two predominant frequencies within the functional activation signals from the FEF. One high-intensity signal was present at 0.029 Hz, centering around the block design frequency. The peak-intensity signal was observed between 0.012 and 0.018 Hz where this very low-frequency oscillation (VLFO) was hypothesized to be generated by the macrovasculature present near the FEF and should be avoided as a block design frequency in future fNIRS studies to avoid false positive results.

A Guide to Designing a Memory fMRI Paradigm for Pre-surgical Evaluation in Temporal Lobe Epilepsy

There has been increasing interest in the clinical and experimental use of memory functional Magnetic Resonance Imaging (fMRI). The 2017 American Academy of Neurology practice guidelines on the use of pre-surgical cognitive fMRI suggests that verbal memory fMRI could be used to lateralize memory functions in people with Temporal Lobe Epilepsy (TLE) and should be used to predict post-operative verbal memory outcome. There are however technical and methodological considerations, to optimize both the sensitivity and specificity of this imaging modality. Below we discuss these constraints and suggest recommendations to consider when designing a memory fMRI paradigm.

Test-retest reliability of brain responses to risk-taking during the balloon analogue risk task

The Balloon Analogue Risk Task (BART) provides a reliable and ecologically valid model for the assessment of individual risk-taking propensity and is frequently used in neuroimaging and developmental research. Although the test-retest reliability of risk-taking behavior during the BART is well established, the reliability of brain activation patterns in response to risk-taking during the BART remains elusive. In this study, we used functional magnetic resonance imaging (fMRI) and evaluated the test-retest reliability of brain responses in 34 healthy adults during a modified BART by calculating the intraclass correlation coefficients (ICC) and Dice’s similarity coefficients (DSC). Analyses revealed that risk-induced brain activation patterns showed good test-retest reliability (median ICC = 0.62) and moderate to high spatial consistency, while brain activation patterns associated with win or loss outcomes only had poor to fair reliability (median ICC = 0.33 for win and 0.42 for loss). These findings have important implications for future utility of the BART in fMRI to examine brain responses to risk-taking and decision-making.

Brain dynamics and connectivity networks under natural auditory stimulation

The analysis of functional magnetic resonance imaging (fMRI) data is challenging when subjects are under exposure to natural sensory stimulation. In this study, a two-stage approach was developed to enable the identification of connectivity networks involved in the processing of information in the brain under natural sensory stimulation. In the first stage, the degree of concordance between the results of inter-subject and intra-subject correlation analyses is assessed statistically. The microstructurally (i.e., cytoarchitectonically) defined brain areas are designated either as concordant in which the results of both correlation analyses are in agreement, or as discordant in which one analysis method shows a higher proportion of supra-threshold voxels than does the other. In the second stage, connectivity networks are identified using the time courses of supra-threshold voxels in brain areas contingent upon the classifications derived in the first stage. In an empirical study, fMRI data were collected from 40 young adults (19 males, average age 22.76 ± 3.25), who underwent auditory stimulation involving sound clips of human voices and animal vocalizations under two operational conditions (i.e., eyes-closed and eyes-open). The operational conditions were designed to assess confounding effects due to auditory instructions or visual perception. The proposed two-stage analysis demonstrated that stress modulation (affective) and language networks in the limbic and cortical structures were respectively engaged during sound stimulation, and presented considerable variability among subjects. The network involved in regulating visuomotor control was sensitive to the eyes-open instruction, and presented only small variations among subjects. A high degree of concordance was observed between the two analyses in the primary auditory cortex which was highly sensitive to the pitch of sound clips. Our results have indicated that brain areas can be identified as concordant or discordant based on the two correlation analyses. This may further facilitate the search for connectivity networks involved in the processing of information under natural sensory stimulation.

Brainglance: Visualizing Group Level MRI Data at One Glance

The vast majority of studies using functional magnetic resonance imaging (fMRI) are analyzed on the group level. Standard group-level analyses, however, come with severe drawbacks: First, they assume functional homogeneity within the group, building on the idea that we use our brains in similar ways. Second, group-level analyses require spatial warping and substantial smoothing to accommodate for anatomical variability across subjects. Such procedures massively distort the underlying fMRI data, which hampers the spatial specificity. Taken together, group statistics capture the effective overlap, rendering the modeling of individual deviations impossible – a major source of false positivity and negativity. The alternative analysis approach is to leave the data in the native subject space, but this makes comparison across individuals difficult. Here, we propose a new framework for visualizing group-level information, better preserving the information of individual subjects. Our proposal is to limit the use of invasive data procedures such as spatial smoothing and warping and rather extract regional information from the individuals. This information is then visualized for all subjects and brain areas at one glance – hence we term the method brainglance. Additionally, our method incorporates a means for clustering individuals to further identify common traits. We showcase our method on two publicly available data sets and discuss our findings.

Test-retest reliability in an fMRI study of naming in dementia

fMRI has been used as an outcome measure in dementia treatment studies, with many previous studies comparing only single pre- and post-treatment fMRI scans to determine treatment-induced neural changes, while utilizing single subject experimental designs. The purpose of the current study was to evaluate fMRI test-retest reliability in dementia patients and typical older adults using noun and verb confrontation naming to evaluate the validity of using a single pre/post-treatment scan comparison. Seven individuals with dementia and 9 control participants were tested three times over two months using the same fMRI procedures. Differences in individual and group level activation patterns were observed that varied across time. Additionally, the extent of variability fluctuated across individuals, groups, and the grammatical category of target words. Our findings suggested that one time fMRI scanning may inadequately represent an individual's typical brain activation pattern, particularly an individual with dementia. Thus, multiple imaging baselines are recommended.

How to Enhance the Power to Detect Brain-Behavior Correlations With Limited Resources

Neuroscience has been diagnosed with a pervasive lack of statistical power and, in turn, reliability. One remedy proposed is a massive increase of typical sample sizes. Parts of the neuroimaging community have embraced this recommendation and actively push for a reallocation of resources toward fewer but larger studies. This is especially true for neuroimaging studies focusing on individual differences to test brain-behavior correlations. Here, I argue for a more efficient solution. Ad hoc simulations show that statistical power crucially depends on the choice of behavioral and neural measures, as well as on sampling strategy. Specifically, behavioral prescreening and the selection of extreme groups can ascertain a high degree of robust in-sample variance. Due to the low cost of behavioral testing compared to neuroimaging, this is a more efficient way of increasing power. For example, prescreening can achieve the power boost afforded by an increase of sample sizes from n = 30 to n = 100 at ∼5% of the cost. This perspective article briefly presents simulations yielding these results, discusses the strengths and limitations of prescreening and addresses some potential counter-arguments. Researchers can use the accompanying online code to simulate the expected power boost of prescreening for their own studies.

Inter-trial variability in brain activity as an indicator of synergistic effects of HIV-1 and drug abuse

BACKGROUND

The objective of this investigation was to detect evidence of the synergism in the effects of HIV-1 and drug abuse on brain function that has been hypothesized but rarely shown. The investigation incorporated several noteworthy improvements in the approach. It used urine toxicology tests to exclude participants complicated by recent methadone use and illicit drug use. Also, it defined drug abuse on a scale that considered symptom severity. Most importantly, it examined inter-trial variability in brain activity as a potentially more sensitive indicator of group differences and functional impairment than the across-trial average.

METHODS

173 participants were assigned to groups defined by their HIV-1 serostatus and Drug Abuse Screening Test score (DAST < vs. > = 6). They completed a simple letter discrimination task including rare target and rare nontarget stimuli. Event-related electroencephalographic responses and key press responses were measured on each trial. During a separate assessment, posturographic measures were recorded.

RESULTS

The inter-trial standard deviation of P300-like activity was superior to the mean amplitude of this activity in differentiating the groups. Unlike the mean, it revealed synergistic statistical effects of HIV and drug abuse. It also correlated significantly with static ataxia.

CONCLUSIONS

Inter-trial variability in P300-like activity is a useful marker for detecting subtle and episodic disruptions in brain function. It demonstrates greater sensitivity than the mean amplitude for detecting differences across groups. Also, as a putative indicator of a disruption in the attentional monitoring of behavior, it predicts subtle impairments in gross motor function.

Intraclass correlation: Improved modeling approaches and applications for neuroimaging

Intraclass correlation (ICC) is a reliability metric that gauges similarity when, for example, entities are measured under similar, or even the same, well-controlled conditions, which in MRI applications include runs/sessions, twins, parent/child, scanners, sites, and so on. The popular definitions and interpretations of ICC are usually framed statistically under the conventional ANOVA platform. Here, we provide a comprehensive overview of ICC analysis in its prior usage in neuroimaging, and we show that the standard ANOVA framework is often limited, rigid, and inflexible in modeling capabilities. These intrinsic limitations motivate several improvements. Specifically, we start with the conventional ICC model under the ANOVA platform, and extend it along two dimensions: first, fixing the failure in ICC estimation when negative values occur under degenerative circumstance, and second, incorporating precision information of effect estimates into the ICC model. These endeavors lead to four modeling strategies: linear mixed-effects (LME), regularized mixed-effects (RME), multilevel mixed-effects (MME), and regularized multilevel mixed-effects (RMME). Compared to ANOVA, each of these four models directly provides estimates for fixed effects and their statistical significances, in addition to the ICC estimate. These new modeling approaches can also accommodate missing data and fixed effects for confounding variables. More importantly, we show that the MME and RMME approaches offer more accurate characterization and decomposition among the variance components, leading to more robust ICC computation. Based on these theoretical considerations and model performance comparisons with a real experimental dataset, we offer the following general-purpose recommendations. First, ICC estimation through MME or RMME is preferable when precision information (i.e., weights that more accurately allocate the variances in the data) is available for the effect estimate; when precision information is unavailable, ICC estimation through LME or the RME is the preferred option. Second, even though the absolute agreement version, ICC(2,1), is presently more popular in the field, the consistency version, ICC(3,1), is a practical and informative choice for whole-brain ICC analysis that achieves a well-balanced compromise when all potential fixed effects are accounted for. Third, approaches for clear, meaningful, and useful result reporting in ICC analysis are discussed. All models, ICC formulations, and related statistical testing methods have been implemented in an open source program 3dICC, which is publicly available as part of the AFNI suite. Even though our work here focuses on the whole-brain level, the modeling strategy and recommendations can be equivalently applied to other situations such as voxel, region, and network levels.

Perceptual Characterization of the Macronutrient Picture System (MaPS) for Food Image fMRI

Food image fMRI paradigms are used widely for investigating the neural basis of ingestive behavior. However, these paradigms have not been validated in terms of ingestive behavior constructs, engagement of food-relevant neural systems, or test-retest reliability, making the generalizability of study findings unclear. Therefore, we validated the Macronutrient Picture System (MaPS) (McClernon et al., 2013), which includes food images from the six categories represented in the Geiselman Food Preference Questionnaire (FPQ) (Geiselman et al., 1998). Twenty-five healthy young adults (n = 21 female, mean age = 20.6 ± 1.1 years, mean BMI = 22.1 ± 1.9 kg/m²) rated the MaPS images in terms of visual interest, appetitive quality, nutrition, emotional valence, liking, and frequency of consumption, and completed the FPQ. In a second study, 12 individuals (n=8 female, mean age = 25.0 ± 6.5 years, mean BMI = 28.2 ± 8.7 kg/m²) viewed MaPS and control images (vegetables and non-food) during two separate 3T BOLD fMRI scans after fasting overnight. Intuitively, high fat/high sugar (HF/HS) and high fat/high complex carbohydrate (HF/HCCHO) images achieved higher liking and appetitive ratings, and lower nutrition ratings, than low fat/low complex carbohydrate/high protein (LF/LCHO/HP) images on average. Within each food category, FPQ scores correlated strongly with MaPS image liking ratings (p < 0.001). Brain activation differences between viewing images of HF/HS and vegetables, and between HF/HCCHO and vegetables, were seen in several reward-related brain regions (e.g., putamen, insula, and medial frontal gyrus). Intra-individual, inter-scan agreement in a summary measure of brain activation differences in seven reward network regions of interest was high (ICC = 0.61), and was even higher when two distinct sets of food images with matching visual ratings were shown in the two scans (ICC = 0.74). These results suggest that the MaPS provides valid representation of food categories and reliably activates food-reward-relevant neural systems.

Comparison of fMRI paradigms assessing visuospatial processing: Robustness and reproducibility

The development of brain imaging techniques, in particular functional magnetic resonance imaging (fMRI), made it possible to non-invasively study the hemispheric lateralization of cognitive brain functions in large cohorts. Comprehensive models of hemispheric lateralization are, however, still missing and should not only account for the hemispheric specialization of individual brain functions, but also for the interactions among different lateralized cognitive processes (e.g., language and visuospatial processing). This calls for robust and reliable paradigms to study hemispheric lateralization for various cognitive functions. While numerous reliable imaging paradigms have been developed for language, which represents the most prominent left-lateralized brain function, the reliability of imaging paradigms investigating typically right-lateralized brain functions, such as visuospatial processing, has received comparatively less attention. In the present study, we aimed to establish an fMRI paradigm that robustly and reliably identifies right-hemispheric activation evoked by visuospatial processing in individual subjects. In a first study, we therefore compared three frequently used paradigms for assessing visuospatial processing and evaluated their utility to robustly detect right-lateralized brain activity on a single-subject level. In a second study, we then assessed the test-retest reliability of the so-called Landmark task–the paradigm that yielded the most robust results in study 1. At the single-voxel level, we found poor reliability of the brain activation underlying visuospatial attention. This suggests that poor signal-to-noise ratios can become a limiting factor for test-retest reliability. This represents a common detriment of fMRI paradigms investigating visuospatial attention in general and therefore highlights the need for careful considerations of both the possibilities and limitations of the respective fMRI paradigm–in particular, when being interested in effects at the single-voxel level. Notably, however, when focusing on the reliability of measures of hemispheric lateralization (which was the main goal of study 2), we show that hemispheric dominance (quantified by the lateralization index, LI, with |LI| >0.4) of the evoked activation could be robustly determined in more than 62% and, if considering only two categories (i.e., left, right), in more than 93% of our subjects. Furthermore, the reliability of the lateralization strength (LI) was “fair” to “good”. In conclusion, our results suggest that the degree of right-hemispheric dominance during visuospatial processing can be reliably determined using the Landmark task, both at the group and single-subject level, while at the same time stressing the need for future refinements of experimental paradigms and more sophisticated fMRI data acquisition techniques.

Test-retest reliability of longitudinal task-based fMRI: Implications for developmental studies

Great advances have been made in functional Magnetic Resonance Imaging (fMRI) studies, including the use of longitudinal design to more accurately identify changes in brain development across childhood and adolescence. While longitudinal fMRI studies are necessary for our understanding of typical and atypical patterns of brain development, the variability observed in fMRI blood-oxygen-level dependent (BOLD) signal and its test-retest reliability in developing populations remain a concern. Here we review the current state of test-retest reliability for child and adolescent fMRI studies (ages 5–18 years) as indexed by intraclass correlation coefficients (ICC). In addition to highlighting ways to improve fMRI test-retest reliability in developmental cognitive neuroscience research, we hope to open a platform for dialogue regarding longitudinal fMRI study designs, analyses, and reporting of results.

Reliability in adolescent fMRI within two years - a comparison of three tasks

Longitudinal developmental fMRI studies just recently began to focus on within-subject reliability using the intraclass coefficient (ICC). It remains largely unclear which degree of reliability can be achieved in developmental studies and whether this depends on the type of task used. Therefore, we aimed to systematically investigate the reliability of three well-classified tasks: an emotional attention, a cognitive control, and an intertemporal choice paradigm. We hypothesized to find higher reliability in the cognitive task than in the emotional or reward-related task. 104 healthy mid-adolescents were scanned at age 14 and again at age 16 within M = 1.8 years using the same paradigms, scanner, and scanning protocols. Overall, we found both variability and stability (i.e. poor to excellent ICCs) depending largely on the region of interest (ROI) and task. Contrary to our hypothesis, whole brain reliability was fair for the cognitive control task but good for the emotional attention and intertemporal choice task. Subcortical ROIs (ventral striatum, amygdala) resulted in lower ICCs than visual ROIs. Current results add to the yet sparse overall ICC literature in both developing samples and adults. This study shows that analyses of stability, i.e. reliability, are helpful benchmarks for longitudinal studies and their implications for adolescent development.

Large-scale functional network overlap is a general property of brain functional organization: Reconciling inconsistent fMRI findings from general-linear-model-based analyses

Functional magnetic resonance imaging (fMRI) studies regularly use univariate general-linear-model-based analyses (GLM). Their findings are often inconsistent across different studies, perhaps because of several fundamental brain properties including functional heterogeneity, balanced excitation and inhibition (E/I), and sparseness of neuronal activities. These properties stipulate heterogeneous neuronal activities in the same voxels and likely limit the sensitivity and specificity of GLM. This paper selectively reviews findings of histological and electrophysiological studies and fMRI spatial independent component analysis (sICA) and reports new findings by applying sICA to two existing datasets. The extant and new findings consistently demonstrate several novel features of brain functional organization not revealed by GLM. They include overlap of large-scale functional networks (FNs) and their concurrent opposite modulations, and no significant modulations in activity of most FNs across the whole brain during any task conditions. These novel features of brain functional organization are highly consistent with the brain's properties of functional heterogeneity, balanced E/I, and sparseness of neuronal activity, and may help reconcile inconsistent GLM findings.

Emergence of representations through repeated training on pronouncing novel letter combinations leads to efficient reading

Printed text can be decoded by utilizing different processing routes depending on the familiarity of the script. A predominant use of word-level decoding strategies can be expected in the case of a familiar script, and an almost exclusive use of letter-level decoding strategies for unfamiliar scripts. Behavioural studies have revealed that frequently occurring words are read more efficiently, suggesting that these words are read in a more holistic way at the word-level, than infrequent and unfamiliar words. To test whether repeated exposure to specific letter combinations leads to holistic reading, we monitored both behavioural and neural responses during novel script decoding and examined changes related to repeated exposure. We trained a group of Dutch university students to decode pseudowords written in an unfamiliar script, i.e., Korean Hangul characters. We compared behavioural and neural responses to pronouncing trained versus untrained two-character pseudowords (equivalent to two-syllable pseudowords). We tested once shortly after the initial training and again after a four days' delay that included another training session. We found that trained pseudowords were pronounced faster and more accurately than novel combinations of radicals (equivalent to letters). Imaging data revealed that pronunciation of trained pseudowords engaged the posterior temporo-parietal region, and engagement of this network was predictive of reading efficiency a month later. The results imply that repeated exposure to specific combinations of graphemes can lead to emergence of holistic representations that result in efficient reading. Furthermore, inter-individual differences revealed that good learners retained efficiency more than bad learners one month later.

Reliability of Task-Based fMRI for Preoperative Planning: A Test-Retest Study in Brain Tumor Patients and Healthy Controls

Background

Functional magnetic resonance imaging (fMRI) continues to develop as a clinical tool for patients with brain cancer, offering data that may directly influence surgical decisions. Unfortunately, routine integration of preoperative fMRI has been limited by concerns about reliability. Many pertinent studies have been undertaken involving healthy controls, but work involving brain tumor patients has been limited. To develop fMRI fully as a clinical tool, it will be critical to examine these reliability issues among patients with brain tumors. The present work is the first to extensively characterize differences in activation map quality between brain tumor patients and healthy controls, including the effects of tumor grade and the chosen behavioral testing paradigm on reliability outcomes.

Method

Test-retest data were collected for a group of low-grade (n = 6) and high-grade glioma (n = 6) patients, and for matched healthy controls (n = 12), who performed motor and language tasks during a single fMRI session. Reliability was characterized by the spatial overlap and displacement of brain activity clusters, BOLD signal stability, and the laterality index. Significance testing was performed to assess differences in reliability between the patients and controls, and low-grade and high-grade patients; as well as between different fMRI testing paradigms.

Results

There were few significant differences in fMRI reliability measures between patients and controls. Reliability was significantly lower when comparing high-grade tumor patients to controls, or to low-grade tumor patients. The motor task produced more reliable activation patterns than the language tasks, as did the rhyming task in comparison to the phonemic fluency task.

Conclusion

In low-grade glioma patients, fMRI data are as reliable as healthy control subjects. For high-grade glioma patients, further investigation is required to determine the underlying causes of reduced reliability. To maximize reliability outcomes, testing paradigms should be carefully selected to generate robust activation patterns.

Reliability of 7T (1) H-MRS measured human prefrontal cortex glutamate, glutamine, and glutathione signals using an adapted echo time optimized PRESS sequence: A between- and within-sessions investigation

PURPOSE

To ascertain the mechanisms of neuropsychiatric illnesses and their treatment, accurate and reliable imaging techniques are required; proton magnetic resonance spectroscopy ((1) H-MRS) can noninvasively measure glutamatergic function. Evidence suggests that aberrant glutamatergic signaling plays a role in numerous psychopathologies. Until recently, overlapping glutamatergic signals (glutamate, glutamine, and glutathione) could not easily be separated. However, the advent of novel pulse sequences and higher field magnetic resonance imaging (MRI) allows more precise resolution of overlapping glutamatergic signals, although the question of signal reliability remains undetermined.

MATERIALS AND METHODS

At 7T MR, we acquired (1) H-MRS data from the medial pregenual anterior cingulate cortex of healthy volunteers (n = 26) twice on two separate days. An adapted echo time optimized point-resolved spectroscopy sequence, modified with the addition of a J-suppression pulse to attenuate N-acetyl-aspartate multiplet signals at 2.49 ppm, was used to excite and acquire the spectra. In-house software was used to model glutamate, glutamine, and glutathione, among other metabolites, referenced to creatine. Intraclass correlation coefficients (ICCs) were computed for within- and between-session measurements.

RESULTS

Within-session measurements of glutamate, glutamine, and glutathione were on average reliable (ICCs ≥0.7). As anticipated, ICCs for between-session values of glutamate, glutamine, and glutathione were slightly lower but nevertheless reliable (ICC >0.62). A negative correlation was observed between glutathione concentration and age (r(24)  = -0.37; P < 0.05), and a gender effect was noted on glutamine and glutathione.

CONCLUSION

The adapted sequence provides good reliability to measure glutamate, glutamine, and glutathione signals.

Test-retest reliability of effective connectivity in the face perception network

Computational approaches have great potential for moving neuroscience toward mechanistic models of the functional integration among brain regions. Dynamic causal modeling (DCM) offers a promising framework for inferring the effective connectivity among brain regions and thus unraveling the neural mechanisms of both normal cognitive function and psychiatric disorders. While the benefit of such approaches depends heavily on their reliability, systematic analyses of the within-subject stability are rare. Here, we present a thorough investigation of the test-retest reliability of an fMRI paradigm for DCM analysis dedicated to unraveling intra- and interhemispheric integration among the core regions of the face perception network. First, we examined the reliability of face-specific BOLD activity in 25 healthy volunteers, who performed a face perception paradigm in two separate sessions. We found good to excellent reliability of BOLD activity within the DCM-relevant regions. Second, we assessed the stability of effective connectivity among these regions by analyzing the reliability of Bayesian model selection and model parameter estimation in DCM. Reliability was excellent for the negative free energy and good for model parameter estimation, when restricting the analysis to parameters with substantial effect sizes. Third, even when the experiment was shortened, reliability of BOLD activity and DCM results dropped only slightly as a function of the length of the experiment. This suggests that the face perception paradigm presented here provides reliable estimates for both conventional activation and effective connectivity measures. We conclude this paper with an outlook on potential clinical applications of the paradigm for studying psychiatric disorders. Hum Brain Mapp 37:730-744, 2016. © 2015 Wiley Periodicals, Inc.

Test-retest reliability of graph metrics of resting state MRI functional brain networks: A review

The employment of graph theory to analyze spontaneous fluctuations in resting state BOLD fMRI data has become a dominant theme in brain imaging studies and neuroscience. Analysis of resting state functional brain networks based on graph theory has proven to be a powerful tool to quantitatively characterize functional architecture of the brain and it has provided a new platform to explore the overall structure of local and global functional connectivity in the brain. Due to its increased use and possible expansion to clinical use, it is essential that the reliability of such a technique is very strongly assessed. In this review, we explore the outcome of recent studies in network reliability which apply graph theory to analyze connectome resting state networks. Therefore, we investigate which preprocessing steps may affect reproducibility the most. In order to investigate network reliability, we compared the test-retest (TRT) reliability of functional data of published neuroimaging studies with different preprocessing steps. In particular we tested influence of global signal regression, correlation metric choice, binary versus weighted link definition, frequency band selection and length of time-series. Statistical analysis shows that only frequency band selection and length of time-series seem to affect TRT reliability. Our results highlight the importance of the choice of the preprocessing steps to achieve more reproducible measurements.

Tutorial on use of intraclass correlation coefficients for assessing intertest reliability and its application in functional near-infrared spectroscopy-based brain imaging

Test-retest reliability of neuroimaging measurements is an important concern in the investigation of cognitive functions in the human brain. To date, intraclass correlation coefficients (ICCs), originally used in interrater reliability studies in behavioral sciences, have become commonly used metrics in reliability studies on neuroimaging and functional near-infrared spectroscopy (fNIRS). However, as there are six popular forms of ICC, the adequateness of the comprehensive understanding of ICCs will affect how one may appropriately select, use, and interpret ICCs toward a reliability study. We first offer a brief review and tutorial on the statistical rationale of ICCs, including their underlying analysis of variance models and technical definitions, in the context of assessment on intertest reliability. Second, we provide general guidelines on the selection and interpretation of ICCs. Third, we illustrate the proposed approach by using an actual research study to assess interest reliability of fNIRS-based, volumetric diffuse optical tomography of brain activities stimulated by a risk decision-making protocol. Last, special issues that may arise in reliability assessment using ICCs are discussed and solutions are suggested.

Reliability of an fMRI paradigm for emotional processing in a multisite longitudinal study

Multisite neuroimaging studies can facilitate the investigation of brain-related changes in many contexts, including patient groups that are relatively rare in the general population. Though multisite studies have characterized the reliability of brain activation during working memory and motor functional magnetic resonance imaging tasks, emotion processing tasks, pertinent to many clinical populations, remain less explored. A traveling participants study was conducted with eight healthy volunteers scanned twice on consecutive days at each of the eight North American Longitudinal Prodrome Study sites. Tests derived from generalizability theory showed excellent reliability in the amygdala ( Eρ2 = 0.82), inferior frontal gyrus (IFG; Eρ2 = 0.83), anterior cingulate cortex (ACC; Eρ2 = 0.76), insula ( Eρ2 = 0.85), and fusiform gyrus ( Eρ2 = 0.91) for maximum activation and fair to excellent reliability in the amygdala ( Eρ2 = 0.44), IFG ( Eρ2 = 0.48), ACC ( Eρ2 = 0.55), insula ( Eρ2 = 0.42), and fusiform gyrus ( Eρ2 = 0.83) for mean activation across sites and test days. For the amygdala, habituation ( Eρ2 = 0.71) was more stable than mean activation. In a second investigation, data from 111 healthy individuals across sites were aggregated in a voxelwise, quantitative meta-analysis. When compared with a mixed effects model controlling for site, both approaches identified robust activation in regions consistent with expected results based on prior single-site research. Overall, regions central to emotion processing showed strong reliability in the traveling participants study and robust activation in the aggregation study. These results support the reliability of blood oxygen level-dependent signal in emotion processing areas across different sites and scanners and may inform future efforts to increase efficiency and enhance knowledge of rare conditions in the population through multisite neuroimaging paradigms.

Studying memory encoding to promote reliable engagement of the medial temporal lobe at the single-subject level

The medial temporal lobe (MTL)—comprising hippocampus and the surrounding neocortical regions—is a targeted brain area sensitive to several neurological diseases. Although functional magnetic resonance imaging (fMRI) has been widely used to assess brain functional abnormalities, detecting MTL activation has been technically challenging. The aim of our study was to provide an fMRI paradigm that reliably activates MTL regions at the individual level, thus providing a useful tool for future research in clinical memory-related studies. Twenty young healthy adults underwent an event-related fMRI study consisting of three encoding conditions: word-pairs, face-name associations and complex visual scenes. A region-of-interest analysis at the individual level comparing novel and repeated stimuli independently for each task was performed. The results of this analysis yielded activations in the hippocampal and parahippocampal regions in most of the participants. Specifically, 95% and 100% of participants showed significant activations in the left hippocampus during the face-name encoding and in the right parahippocampus, respectively, during scene encoding. Additionally, a whole brain analysis, also comparing novel versus repeated stimuli at the group level, showed mainly left frontal activation during the word task. In this group analysis, the face-name association engaged the HP and fusiform gyri bilaterally, along with the left inferior frontal gyrus, and the complex visual scenes activated mainly the parahippocampus and hippocampus bilaterally. In sum, our task design represents a rapid and reliable manner to study and explore MTL activity at the individual level, thus providing a useful tool for future research in clinical memory-related fMRI studies.

Neural responses to dynamic multimodal stimuli and pathology-specific impairments of social cognition in schizophrenia and depression

BACKGROUND

Individuals with schizophrenia and people with depression both show abnormal behavioural and neural responses when perceiving and responding to emotional stimuli, but pathology-specific differences and commonalities remain mostly unclear.

AIMS

To directly compare empathic responses to dynamic multimodal emotional stimuli in a group with schizophrenia and a group with depression, and to investigate their neural correlates using functional magnetic resonance imaging (fMRI).

METHOD

The schizophrenia group (n = 20), the depression group (n = 24) and a control group (n = 24) were presented with portrait-shot video clips expressing emotion through three possible communication channels: facial expression, prosody and content. Participants rated their own and the actor's emotional state as an index of empathy.

RESULTS

Although no group differences were found in empathy ratings, characteristic differences emerged in the fMRI activation patterns. The schizophrenia group demonstrated aberrant activation patterns during the neutral speech content condition in regions implicated in multimodal integration and formation of semantic constructs. Those in the depression group were most affected during conditions with trimodal emotional and trimodal neutral stimuli, in key regions of the mentalising network.

CONCLUSIONS

Our findings reveal characteristic differences in patients with schizophrenia compared with those with depression in their cortical responses to dynamic affective stimuli. These differences indicate that impairments in responding to emotional stimuli may be caused by pathology-specific problems in social cognition.

The frequency and reliability of cortical activity using a novel strategy to present pressure pain stimulus over the lumbar spine

BACKGROUND

The blocked stimulus presentation strategy, in fMRI study designs, is an important means to study brain function related to a particular stimulus. Specifically, applying pressure stimuli perceived as painful to different anatomical regions has been used to improve our understanding of central sensitization, which is an important clinical phenomenon in chronic pain.

NEW METHOD

This paper introduces a novel MR-compatible device used to apply pressure pain stimuli to the lumbar spine of 13 subjects in the supine position. We present the frequency of individuals and within-subject reliability of cortical activity in the following brain regions: the primary somatosensory cortex, insula and anterior cingulate cortex bilaterally.

RESULTS

Using the novel MR-compatible device, a high frequency of individuals showed cortical activity within the a priori brain regions. There was good to excellent run-to-run reliability for peak voxel, while cluster size was less reliable. We found a higher than expected association between stimulus presentation and movement artifacts.

COMPARISON WITH EXISTING METHOD(S)

Unlike previous methods, the current strategy can apply pressure stimuli to subjects over the lumbar spine while they lay supine. Previous methods required subjects to lay prone.

CONCLUSIONS

This strategy could be used for evaluating pressure stimuli related central sensitization associated with back pain.

Test-retest reliability of pain-related brain activity in healthy controls undergoing experimental thermal pain

Although functional magnetic resonance imaging (fMRI) has been proposed as a method to elucidate pain-related biomarkers, little information exists related to psychometric properties of fMRI findings. This knowledge is essential for potential translation of this technology to clinical settings. The purpose of this study was to assess the test-retest reliability of pain-related brain activity and how it compares to the reliability of self-report. Twenty-two healthy controls (mean age = 22.6 years, standard deviation = 2.9) underwent 3 runs of an fMRI paradigm that used thermal stimuli to elicit experimental pain. Functional MRI summary statistics related to brain activity during thermal stimulation periods were extracted from bilateral anterior cingulate cortices and anterior insula. Intraclass correlations (ICCs) were conducted on these summary statistics and generally showed "good" test-retest reliability in all regions of interest (ICC range = .32-.88; mean = .71); however, these results did not surpass ICC values from pain ratings, which fell within the "excellent" range (ICC range = .93-.96; mean = .94). Findings suggest that fMRI is a valuable tool for measuring pain mechanisms but did not show an adequate level of test-retest reliability for fMRI to potentially act as a surrogate for individuals' self-report of pain.

PERSPECTIVE

This study is one of the first reports to demonstrate the test-retest reliability of fMRI findings related to pain processing and provides a comparison to the reliability of subjective reports of pain. This information is essential for determining whether fMRI technology should be potentially translated for clinical use.