Test-retest reliability of graph metrics of resting state MRI functional brain networks: A review

Within-subject test-retest reliability of the atlas-based cortical volume measurement in the rat brain: A voxel-based morphometry study

BACKGROUND

Various neurological and psychological disorders are related to cortical volume changes in specific brain regions, which can be measured in vivo using structural magnetic resonance imaging (sMRI). There is an increasing interest in MRI studies using rat models, especially in longitudinal studies of brain disorders and pharmacologic interventions. However, morphometric changes observed in sMRI are only meaningful if the measurements are reliable. To date, a systematic evaluation of the test-retest reliability of the morphometric measures in the rat brain is still lacking.

NEW METHOD

We rigorously evaluated the test-retest reliability of morphometric measures derived from the voxel-based morphometry (VBM) analysis. 37 Sprague-Dawley rats were scanned twice at an interval of six hours and the gray matter volume was estimated using the VBM-DARTEL method. The intraclass coefficient, percent volume change and Pearson correlation coefficient were used to evaluate the reliability in 96 subregions of the rat brain.

RESULTS

Most subregions showed excellent test-retest reliabilities within an interval of 6 h while a few regions demonstrated lower reliability, especially in the retrosplenial granular cortex. The results were consistent between different methods of reliability assessment.

COMPARISON WITH EXISTING METHOD

To the best of our knowledge, this is the first study to quantify the test-retest reliability of the VBM measurements of the rat brain.

CONCLUSION

Atlas-based cortical volume of the rat brain can be reliably estimated using the VBM-DARTEL method in most subregions. However, findings in subregions with lower reliability must be interpreted with caution.

Intra- and Inter-Scanner Reliability of Voxel-Wise Whole-Brain Analytic Metrics for Resting State fMRI

As the multi-center studies with resting-state functional magnetic resonance imaging (RS-fMRI) have been more and more applied to neuropsychiatric studies, both intra- and inter-scanner reliability of RS-fMRI are becoming increasingly important. The amplitude of low frequency fluctuation (ALFF), regional homogeneity (ReHo), and degree centrality (DC) are 3 main RS-fMRI metrics in a way of voxel-wise whole-brain (VWWB) analysis. Although the intra-scanner reliability (i.e., test-retest reliability) of these metrics has been widely investigated, few studies has investigated their inter-scanner reliability. In the current study, 21 healthy young subjects were enrolled and scanned with blood oxygenation level dependent (BOLD) RS-fMRI in 3 visits (V1 - V3), with V1 and V2 scanned on a GE MR750 scanner and V3 on a Siemens Prisma. RS-fMRI data were collected under two conditions, eyes open (EO) and eyes closed (EC), each lasting 8 minutes. We firstly evaluated the intra- and inter-scanner reliability of ALFF, ReHo, and DC. Secondly, we measured systematic difference between two scanning visits of the same scanner as well as between two scanners. Thirdly, to account for the potential difference of intra- and inter-scanner local magnetic field inhomogeneity, we measured the difference of relative BOLD signal intensity to the mean BOLD signal intensity of the whole brain between each pair of visits. Last, we used percent amplitude of fluctuation (PerAF) to correct the difference induced by relative BOLD signal intensity. The inter-scanner reliability was much worse than intra-scanner reliability; Among the VWWB metrics, DC showed the worst (both for intra-scanner and inter-scanner comparisons). PerAF showed similar intra-scanner reliability with ALFF and the best reliability among all the 4 metrics. PerAF reduced the influence of BOLD signal intensity and hence increase the inter-scanner reliability of ALFF. For multi-center studies, inter-scanner reliability should be taken into account.

Brain Network Connectivity and Executive Function in Long-Term Survivors of Childhood Acute Lymphoblastic Leukemia

Chemotherapeutic agents used to treat acute lymphoblastic leukemia (ALL), the most common cancer affecting young children, have been associated with long-term cognitive impairments that reduce quality of life. Executive dysfunction is one of the most consistently observed deficits and can have substantial and pervasive effects on academic success, occupational achievement, psychosocial function, and psychiatric status. We examined the neural mechanisms of executive dysfunction by measuring structural and functional connectomes in 161 long-term survivors of pediatric ALL, age 8-21 years, who were treated on a single contemporary chemotherapy-only protocol for standard/high- or low-risk disease. Lower global efficiency, a measure of information exchange and network integration, of both structural and functional connectomes was found in survivors with executive dysfunction compared with those without dysfunction (p < 0.046). Patients with standard/high- versus low-risk disease and those who received greater number of intrathecal treatments containing methotrexate had the lowest network efficiencies. Patients with executive dysfunction also showed hyperconnectivity in sensorimotor, visual, and auditory-processing regions (p = 0.037) and poor separation between sensorimotor, executive/attention, salience, and default mode networks (p < 0.0001). Connectome disruption was consistent with a pattern of delayed neurodevelopment that may be associated with reduced resilience, adaptability, and flexibility of the brain network. These findings highlight the need for interventions that will prevent or manage cognitive impairment in survivors of pediatric acute lymphoblastic leukemia.

GraphVar 2.0: A user-friendly toolbox for machine learning on functional connectivity measures

BACKGROUND

We previously presented GraphVar as a user-friendly MATLAB toolbox for comprehensive graph analyses of functional brain connectivity. Here we introduce a comprehensive extension of the toolbox allowing users to seamlessly explore easily customizable decoding models across functional connectivity measures as well as additional features.

NEW METHOD

GraphVar 2.0 provides machine learning (ML) model construction, validation and exploration. Machine learning can be performed across any combination of graph measures and additional variables, allowing for a flexibility in neuroimaging applications.

RESULTS

In addition to previously integrated functionalities, such as network construction and graph-theoretical analyses of brain connectivity with a high-speed general linear model (GLM), users can now perform customizable ML across connectivity matrices, graph measures and additionally imported variables. The new extension also provides parametric and nonparametric testing of classifier and regressor performance, data export, figure generation and high quality export.

COMPARISON WITH EXISTING METHODS

Compared to other existing toolboxes, GraphVar 2.0 offers (1) comprehensive customization, (2) an all-in-one user friendly interface, (3) customizable model design and manual hyperparameter entry, (4) interactive results exploration and data export, (5) automated queue system for modelling multiple outcome variables within the same session, (6) an easy to follow introductory review.

CONCLUSIONS

GraphVar 2.0 allows comprehensive, user-friendly exploration of encoding (GLM) and decoding (ML) modelling approaches on functional connectivity measures making big data neuroscience readily accessible to a broader audience of neuroimaging investigators.

Predicting future learning from baseline network architecture

Human behavior and cognition result from a complex pattern of interactions between brain regions. The flexible reconfiguration of these patterns enables behavioral adaptation, such as the acquisition of a new motor skill. Yet, the degree to which these reconfigurations depend on the brain's baseline sensorimotor integration is far from understood. Here, we asked whether spontaneous fluctuations in sensorimotor networks at baseline were predictive of individual differences in future learning. We analyzed functional MRI data from 19 participants prior to six weeks of training on a new motor skill. We found that visual-motor connectivity was inversely related to learning rate: sensorimotor autonomy at baseline corresponded to faster learning in the future. Using three additional scans, we found that visual-motor connectivity at baseline is a relatively stable individual trait. These results suggest that individual differences in motor skill learning can be predicted from sensorimotor autonomy at baseline prior to task execution.

A BRIEF INTRODUCTION TO THE NEUROGENETICS OF COGNITION-EMOTION INTERACTIONS

Neuroscience research has demonstrated that cognition, emotion, and their dynamic interactions emerge from complex and flexible patterns of activity across distributed neural circuits. A parallel branch of research in genetics has begun to identify common variation in the human DNA sequence (i.e., genome) that may shape individual differences in cognition-emotion interactions by altering molecular and cellular pathways that modulate the activity of these neural circuits. Here we provide a brief introduction to such neurogenetics research and how it may usefully inform our understanding of the biological mechanisms through which dynamic cognition-emotion interactions emerge and, subsequently, help shape normal and abnormal behavior.

On the Reliability of Individual Brain Activity Networks

There is intense interest in fMRI research on whole-brain functional connectivity, and however, two fundamental issues are still unresolved: the impact of spatiotemporal data resolution (spatial parcellation and temporal sampling) and the impact of the network construction method on the reliability of functional brain networks. In particular, the impact of spatiotemporal data resolution on the resulting connectivity findings has not been sufficiently investigated. In fact, a number of studies have already observed that functional networks often give different conclusions across different parcellation scales. If the interpretations from functional networks are inconsistent across spatiotemporal scales, then the whole validity of the functional network paradigm is called into question. This paper investigates the consistency of resting state network structure when using different temporal sampling or spatial parcellation, or different methods for constructing the networks. To pursue this, we develop a novel network comparison framework based on persistent homology from a topological data analysis. We use the new network comparison tools to characterize the spatial and temporal scales under which consistent functional networks can be constructed. The methods are illustrated on Human Connectome Project data, showing that the DISCOH₂ network construction method outperforms other approaches at most data spatiotemporal resolutions.

Probabilistic thresholding of functional connectomes: Application to schizophrenia

Functional connectomes are commonly analysed as sparse graphs, constructed by thresholding cross-correlations between regional neurophysiological signals. Thresholding generally retains the strongest edges (correlations), either by retaining edges surpassing a given absolute weight, or by constraining the edge density. The latter (more widely used) method risks inclusion of false positive edges at high edge densities and exclusion of true positive edges at low edge densities. Here we apply new wavelet-based methods, which enable construction of probabilistically-thresholded graphs controlled for type I error, to a dataset of resting-state fMRI scans of 56 patients with schizophrenia and 71 healthy controls. By thresholding connectomes to fixed edge-specific P value, we found that functional connectomes of patients with schizophrenia were more dysconnected than those of healthy controls, exhibiting a lower edge density and a higher number of (dis)connected components. Furthermore, many participants' connectomes could not be built up to the fixed edge densities commonly studied in the literature (∼5-30%), while controlling for type I error. Additionally, we showed that the topological randomisation previously reported in the schizophrenia literature is likely attributable to "non-significant" edges added when thresholding connectomes to fixed density based on correlation. Finally, by explicitly comparing connectomes thresholded by increasing P value and decreasing correlation, we showed that probabilistically thresholded connectomes show decreased randomness and increased consistency across participants. Our results have implications for future analysis of functional connectivity using graph theory, especially within datasets exhibiting heterogenous distributions of edge weights (correlations), between groups or across participants.

Effects of spatial smoothing on functional brain networks

Graph-theoretical methods have rapidly become a standard tool in studies of the structure and function of the human brain. Whereas the structural connectome can be fairly straightforwardly mapped onto a complex network, there are more degrees of freedom in constructing networks that represent functional connections between brain areas. For functional magnetic resonance imaging (fMRI) data, such networks are typically built by aggregating the blood-oxygen-level dependent signal time series of voxels into larger entities (such as Regions of Interest in some brain atlas) and determining their connection strengths from some measure of time-series correlations. Although it is evident that the outcome must be affected by how the voxel-level time series are treated at the preprocessing stage, there is a lack of systematic studies of the effects of preprocessing on network structure. Here, we focus on the effects of spatial smoothing, a standard preprocessing method for fMRI. We apply various levels of spatial smoothing to resting-state fMRI data and measure the changes induced in functional networks. We show that the level of spatial smoothing clearly affects the degrees and other centrality measures of functional network nodes; these changes are non-uniform, systematic, and depend on the geometry of the brain. The composition of the largest connected network component is also affected in a way that artificially increases the similarity of the networks of different subjects. Our conclusion is that wherever possible, spatial smoothing should be avoided when preprocessing fMRI data for network analysis.

Consistency of Regions of Interest as nodes of fMRI functional brain networks

The functional network approach, where fMRI BOLD time series are mapped to networks depicting functional relationships between brain areas, has opened new insights into the function of the human brain. In this approach, the choice of network nodes is of crucial importance. One option is to consider fMRI voxels as nodes. This results in a large number of nodes, making network analysis and interpretation of results challenging. A common alternative is to use predefined clusters of anatomically close voxels, Regions of Interest (ROIs). This approach assumes that voxels within ROIs are functionally similar. Because these two approaches result in different network structures, it is crucial to understand what happens to network connectivity when moving from the voxel level to the ROI level. We show that the consistency of ROIs, defined as the mean Pearson correlation coefficient between the time series of their voxels, varies widely in resting-state experimental data. Therefore the assumption of similar voxel dynamics within each ROI does not generally hold. Further, the time series of low-consistency ROIs may be highly correlated, resulting in spurious links in ROI-level networks. Based on these results, we recommend that averaging BOLD signals over anatomically defined ROIs should be carefully considered.

Reproducibility of frequency-dependent low frequency fluctuations in reaction time over time and across tasks

Objective

Increased levels of reaction time variability (RTV) are characteristics of sustained attention deficits. The clinical significance of RTV has been widely recognized. However, the reliability of RTV measurements has not been widely studied. The present study aimed to assess the test-retest reliability of RTV conventional measurements, e.g., the standard deviation (SD), the coefficient of variation (CV), and a new measurement called the amplitude of low frequency fluctuation (ALFF) of RT. In addition, we aimed to assess differences and similarities of these measurements between different tasks.

Method

Thirty-seven healthy college students participated in 2 tasks, i.e., an Eriksen flanker task (EFT) and a simple reaction task (SRT), twice over a mean interval of 56 days. Conventional measurements of RTV including RT-SD and RT-CV were assessed first. Then the RT time series were converted into frequency domains, and RT-ALFF was further calculated for the whole frequency band (0.0023–0.167 Hz) and for a few sub-frequency bands including Slow-6 (<0.01 Hz), Slow-5 (0.01–0.027 Hz), Slow-4 (0.027–0.073 Hz), and Slow-3 (0.073–0.167 Hz). The test-retest reliability of these measurements was evaluated through intra-class correlation (ICC) tests. Differences and correlations between each EFT and SRT measurement were further examined during both visits.

Results

1) The RT-ALFF of the Slow-5/4/3 and conventional measurements of RT-SD and RT-CV showed moderate to high levels of test-retest reliability. EFT RT-ALFF patterns generated slightly higher ICC values than SRT values in higher frequency bands (Slow-3), but SRT RT-ALFF values showed slightly higher ICC values than EFT values in lower frequency bands (Slow-5 and Slow-4). 2) RT-ALFF magnitudes in each sub-frequency band were greater for the SRT than those for the EFT. 3) The RT-ALFF in the Slow-4 of the EFT was found to be correlated with the RT-ALFF in the Slow-5 of the SRT for both two visits, but no consistently significant correlation was found between the same frequency bands.

Conclusions

These findings reveal good test-retest reliability for conventional measurements and for the RT-ALFF of RTV. The RT-ALFF presented frequency-dependent similarities across tasks. All of our results reveal the presence of different frequency structures between the two tasks, and thus the frequency-dependent characteristics of different tasks deserve more attention in future studies.

Reliability of EEG Measures of Interaction: A Paradigm Shift Is Needed to Fight the Reproducibility Crisis

Measures of interaction (connectivity) of the EEG are at the forefront of current neuroscientific research. Unfortunately, test-retest reliability can be very low, depending on the measure and its estimation, the EEG-frequency of interest, the length of the signal, and the population under investigation. In addition, artifacts can hamper the continuity of the EEG signal, and in some clinical situations it is impractical to exclude artifacts. We aimed to examine factors that moderate test-retest reliability of measures of interaction. The study involved 40 patients with a range of neurological diseases and memory impairments (age median: 60; range 21–76; 40% female; 22 mild cognitive impairment, 5 subjective cognitive complaints, 13 temporal lobe epilepsy), and 20 healthy controls (age median: 61.5; range 23–74; 70% female). We calculated 14 measures of interaction based on the multivariate autoregressive model from two EEG-recordings separated by 2 weeks. We characterized test-retest reliability by correlating the measures between the two EEG-recordings for variations of data length, data discontinuity, artifact exclusion, model order, and frequency over all combinations of channels and all frequencies, individually for each subject, yielding a correlation coefficient for each participant. Excluding artifacts had strong effects on reliability of some measures, such as classical, real valued coherence (~0.1 before, ~0.9 after artifact exclusion). Full frequency directed transfer function was highly reliable and robust against artifacts. Variation of data length decreased reliability in relation to poor adjustment of model order and signal length. Variation of discontinuity had no effect, but reliabilities were different between model orders, frequency ranges, and patient groups depending on the measure. Pathology did not interact with variation of signal length or discontinuity. Our results emphasize the importance of documenting reliability, which may vary considerably between measures of interaction. We recommend careful selection of measures of interaction in accordance with the properties of the data. When only short data segments are available and when the signal length varies strongly across subjects after exclusion of artifacts, reliability becomes an issue. Finally, measures which show high reliability irrespective of the presence of artifacts could be extremely useful in clinical situations when exclusion of artifacts is impractical.

Fluctuations between high- and low-modularity topology in time-resolved functional connectivity

Modularity is an important topological attribute for functional brain networks. Recent human fMRI studies have reported that modularity of functional networks varies not only across individuals being related to demographics and cognitive performance, but also within individuals co-occurring with fluctuations in network properties of functional connectivity, estimated over short time intervals. However, characteristics of these time-resolved functional networks during periods of high and low modularity have remained largely unexplored. In this study we investigate basic spatiotemporal properties of time-resolved networks in the high and low modularity periods during rest, with a particular focus on their spatial connectivity patterns, temporal homogeneity and test-retest reliability. We show that spatial connectivity patterns of time-resolved networks in the high and low modularity periods are represented by increased and decreased dissociation of the default mode network module from task-positive network modules, respectively. We also find that the instances of time-resolved functional connectivity sampled from within the high (respectively, low) modularity period are relatively homogeneous (respectively, heterogeneous) over time, indicating that during the low modularity period the default mode network interacts with other networks in a variable manner. We confirmed that the occurrence of the high and low modularity periods varies across individuals with moderate inter-session test-retest reliability and that it is correlated with previously-reported individual differences in the modularity of functional connectivity estimated over longer timescales. Our findings illustrate how time-resolved functional networks are spatiotemporally organized during periods of high and low modularity, allowing one to trace individual differences in long-timescale modularity to the variable occurrence of network configurations at shorter timescales.

Reliability of EEG Interactions Differs between Measures and Is Specific for Neurological Diseases

Alterations of interaction (connectivity) of the EEG reflect pathological processes in patients with neurologic disorders. Nevertheless, it is questionable whether these patterns are reliable over time in different measures of interaction and whether this reliability of the measures is the same across different patient populations. In order to address this topic we examined 22 patients with mild cognitive impairment, five patients with subjective cognitive complaints, six patients with right-lateralized temporal lobe epilepsy, seven patients with left lateralized temporal lobe epilepsy, and 20 healthy controls. We calculated 14 measures of interaction from two EEG-recordings separated by 2 weeks. In order to characterize test-retest reliability, we correlated these measures for each group and compared the correlations between measures and between groups. We found that both measures of interaction as well as groups differed from each other in terms of reliability. The strongest correlation coefficients were found for spectrum, coherence, and full frequency directed transfer function (average rho > 0.9). In the delta (2–4 Hz) range, reliability was lower for mild cognitive impairment compared to healthy controls and left lateralized temporal lobe epilepsy. In the beta (13–30 Hz), gamma (31–80 Hz), and high gamma (81–125 Hz) frequency ranges we found decreased reliability in subjective cognitive complaints compared to mild cognitive impairment. In the gamma and high gamma range we found increased reliability in left lateralized temporal lobe epilepsy patients compared to healthy controls. Our results emphasize the importance of documenting reliability of measures of interaction, which may vary considerably between measures, but also between patient populations. We suggest that studies claiming clinical usefulness of measures of interaction should provide information on the reliability of the results. In addition, differences between patient groups in reliability of interactions in the EEG indicate the potential of reliability to serve as a new biomarker for pathological memory decline as well as for epilepsy. While the brain concert of information flow is generally variable, high reliability, and thus, low variability may reflect abnormal firing patterns.

Extraction of dynamic functional connectivity from brain grey matter and white matter for MCI classification

Brain functional connectivity (FC) extracted from resting-state fMRI (RS-fMRI) has become a popular approach for diagnosing various neurodegenerative diseases, including Alzheimer's disease (AD) and its prodromal stage, mild cognitive impairment (MCI). Current studies mainly construct the FC networks between grey matter (GM) regions of the brain based on temporal co-variations of the blood oxygenation level-dependent (BOLD) signals, which reflects the synchronized neural activities. However, it was rarely investigated whether the FC detected within the white matter (WM) could provide useful information for diagnosis. Motivated by the recently proposed functional correlation tensors (FCT) computed from RS-fMRI and used to characterize the structured pattern of local FC in the WM, we propose in this article a novel MCI classification method based on the information conveyed by both the FC between the GM regions and that within the WM regions. Specifically, in the WM, the tensor-based metrics (e.g., fractional anisotropy [FA], similar to the metric calculated based on diffusion tensor imaging [DTI]) are first calculated based on the FCT and then summarized along each of the major WM fiber tracts connecting each pair of the brain GM regions. This could capture the functional information in the WM, in a similar network structure as the FC network constructed for the GM, based only on the same RS-fMRI data. Moreover, a sliding window approach is further used to partition the voxel-wise BOLD signal into multiple short overlapping segments. Then, both the FC and FCT between each pair of the brain regions can be calculated based on the BOLD signal segments in the GM and WM, respectively. In such a way, our method can generate dynamic FC and dynamic FCT to better capture functional information in both GM and WM and further integrate them together by using our developed feature extraction, selection, and ensemble learning algorithms. The experimental results verify that the dynamic FCT can provide valuable functional information in the WM; by combining it with the dynamic FC in the GM, the diagnosis accuracy for MCI subjects can be significantly improved even using RS-fMRI data alone. Hum Brain Mapp 38:5019-5034, 2017. © 2017 Wiley Periodicals, Inc.

Emerging Frontiers of Neuroengineering: A Network Science of Brain Connectivity

Neuroengineering is faced with unique challenges in repairing or replacing complex neural systems that are composed of many interacting parts. These interactions form intricate patterns over large spatiotemporal scales and produce emergent behaviors that are difficult to predict from individual elements. Network science provides a particularly appropriate framework in which to study and intervene in such systems by treating neural elements (cells, volumes) as nodes in a graph and neural interactions (synapses, white matter tracts) as edges in that graph. Here, we review the emerging discipline of network neuroscience, which uses and develops tools from graph theory to better understand and manipulate neural systems from micro- to macroscales. We present examples of how human brain imaging data are being modeled with network analysis and underscore potential pitfalls. We then highlight current computational and theoretical frontiers and emphasize their utility in informing diagnosis and monitoring, brain-machine interfaces, and brain stimulation. A flexible and rapidly evolving enterprise, network neuroscience provides a set of powerful approaches and fundamental insights that are critical for the neuroengineer's tool kit.

Improving the Test-Retest Reliability of Resting State fMRI by Removing the Impact of Sleep

Resting state functional magnetic resonance imaging (rs-fMRI) provides a powerful tool to examine large-scale neural networks in the human brain and their disturbances in neuropsychiatric disorders. Thanks to its low demand and high tolerance, resting state paradigms can be easily acquired from clinical population. However, due to the unconstrained nature, resting state paradigm is associated with excessive head movement and proneness to sleep. Consequently, the test-retest reliability of rs-fMRI measures is moderate at best, falling short of widespread use in the clinic. Here, we characterized the effect of sleep on the test-retest reliability of rs-fMRI. Using measures of heart rate variability (HRV) derived from simultaneous electrocardiogram (ECG) recording, we identified portions of fMRI data when subjects were more alert or sleepy, and examined their effects on the test-retest reliability of functional connectivity measures. When volumes of sleep were excluded, the reliability of rs-fMRI is significantly improved, and the improvement appears to be general across brain networks. The amount of improvement is robust with the removal of as much as 60% volumes of sleepiness. Therefore, test-retest reliability of rs-fMRI is affected by sleep and could be improved by excluding volumes of sleepiness as indexed by HRV. Our results suggest a novel and practical method to improve test-retest reliability of rs-fMRI measures.

Biomarkers, designs, and interpretations of resting-state fMRI in translational pharmacological research: A review of state-of-the-Art, challenges, and opportunities for studying brain chemistry

A decade of research and development in resting-state functional MRI (RSfMRI) has opened new translational and clinical research frontiers. This review aims to bridge between technical and clinical researchers who seek reliable neuroimaging biomarkers for studying drug interactions with the brain. About 85 pharma-RSfMRI studies using BOLD signal (75% of all) or arterial spin labeling (ASL) were surveyed to investigate the acute effects of psychoactive drugs. Experimental designs and objectives include drug fingerprinting dose-response evaluation, biomarker validation and calibration, and translational studies. Common biomarkers in these studies include functional connectivity, graph metrics, cerebral blood flow and the amplitude and spectrum of BOLD fluctuations. Overall, RSfMRI-derived biomarkers seem to be sensitive to spatiotemporal dynamics of drug interactions with the brain. However, drugs cause both central and peripheral effects, thus exacerbate difficulties related to biological confounds, structured noise from motion and physiological confounds, as well as modeling and inference testing. Currently, these issues are not well explored, and heterogeneities in experimental design, data acquisition and preprocessing make comparative or meta-analysis of existing reports impossible. A unifying collaborative framework for data-sharing and data-mining is thus necessary for investigating the commonalities and differences in biomarker sensitivity and specificity, and establishing guidelines. Multimodal datasets including sham-placebo or active control sessions and repeated measurements of various psychometric, physiological, metabolic and neuroimaging phenotypes are essential for pharmacokinetic/pharmacodynamic modeling and interpretation of the findings. We provide a list of basic minimum and advanced options that can be considered in design and analyses of future pharma-RSfMRI studies. Hum Brain Mapp 38:2276-2325, 2017. © 2017 Wiley Periodicals, Inc.

Test-retest reliability of functional connectivity networks during naturalistic fMRI paradigms

Functional connectivity analysis has become a powerful tool for probing the human brain function and its breakdown in neuropsychiatry disorders. So far, most studies adopted resting-state paradigm to examine functional connectivity networks in the brain, thanks to its low demand and high tolerance that are essential for clinical studies. However, the test-retest reliability of resting-state connectivity measures is moderate, potentially due to its low behavioral constraint. On the other hand, naturalistic neuroimaging paradigms, an emerging approach for cognitive neuroscience with high ecological validity, could potentially improve the reliability of functional connectivity measures. To test this hypothesis, we characterized the test-retest reliability of functional connectivity measures during a natural viewing condition, and benchmarked it against resting-state connectivity measures acquired within the same functional magnetic resonance imaging (fMRI) session. We found that the reliability of connectivity and graph theoretical measures of brain networks is significantly improved during natural viewing conditions over resting-state conditions, with an average increase of almost 50% across various connectivity measures. Not only sensory networks for audio-visual processing become more reliable, higher order brain networks, such as default mode and attention networks, but also appear to show higher reliability during natural viewing. Our results support the use of natural viewing paradigms in estimating functional connectivity of brain networks, and have important implications for clinical application of fMRI. Hum Brain Mapp 38:2226-2241, 2017. © 2017 Wiley Periodicals, Inc.

Environmental factors linked to depression vulnerability are associated with altered cerebellar resting-state synchronization

Hosting nearly eighty percent of all human neurons, the cerebellum is functionally connected to large regions of the brain. Accumulating data suggest that some cerebellar resting-state alterations may constitute a key candidate mechanism for depressive psychopathology. While there is some evidence linking cerebellar function and depression, two topics remain largely unexplored. First, the genetic or environmental roots of this putative association have not been elicited. Secondly, while different mathematical representations of resting-state fMRI patterns can embed diverse information of relevance for health and disease, many of them have not been studied in detail regarding the cerebellum and depression. Here, high-resolution fMRI scans were examined to estimate functional connectivity patterns across twenty-six cerebellar regions in a sample of 48 identical twins (24 pairs) informative for depression liability. A network-based statistic approach was employed to analyze cerebellar functional networks built using three methods: the conventional approach of filtered BOLD fMRI time-series, and two analytic components of this oscillatory activity (amplitude envelope and instantaneous phase). The findings indicate that some environmental factors may lead to depression vulnerability through alterations of the neural oscillatory activity of the cerebellum during resting-state. These effects may be observed particularly when exploring the amplitude envelope of fMRI oscillations.

Towards a consensus regarding global signal regression for resting state functional connectivity MRI

The number of resting state functional connectivity MRI studies continues to expand at a rapid rate along with the options for data processing. Of the processing options, few have generated as much controversy as global signal regression and the subsequent observation of negative correlations (anti-correlations). This debate has motivated new processing strategies and advancement in the field, but has also generated significant confusion and contradictory guidelines. In this article, we work towards a consensus regarding global signal regression. We highlight several points of agreement including the fact that there is not a single "right" way to process resting state data that reveals the "true" nature of the brain. Although further work is needed, different processing approaches likely reveal complementary insights about the brain's functional organisation.

Multisite reliability of MR-based functional connectivity

Recent years have witnessed an increasing number of multisite MRI functional connectivity (fcMRI) studies. While multisite studies provide an efficient way to accelerate data collection and increase sample sizes, especially for rare clinical populations, any effects of site or MRI scanner could ultimately limit power and weaken results. Little data exists on the stability of functional connectivity measurements across sites and sessions. In this study, we assess the influence of site and session on resting state functional connectivity measurements in a healthy cohort of traveling subjects (8 subjects scanned twice at each of 8 sites) scanned as part of the North American Prodrome Longitudinal Study (NAPLS). Reliability was investigated in three types of connectivity analyses: (1) seed-based connectivity with posterior cingulate cortex (PCC), right motor cortex (RMC), and left thalamus (LT) as seeds; (2) the intrinsic connectivity distribution (ICD), a voxel-wise connectivity measure; and (3) matrix connectivity, a whole-brain, atlas-based approach to assessing connectivity between nodes. Contributions to variability in connectivity due to subject, site, and day-of-scan were quantified and used to assess between-session (test-retest) reliability in accordance with Generalizability Theory. Overall, no major site, scanner manufacturer, or day-of-scan effects were found for the univariate connectivity analyses; instead, subject effects dominated relative to the other measured factors. However, summaries of voxel-wise connectivity were found to be sensitive to site and scanner manufacturer effects. For all connectivity measures, although subject variance was three times the site variance, the residual represented 60-80% of the variance, indicating that connectivity differed greatly from scan to scan independent of any of the measured factors (i.e., subject, site, and day-of-scan). Thus, for a single 5min scan, reliability across connectivity measures was poor (ICC=0.07-0.17), but increased with increasing scan duration (ICC=0.21-0.36 at 25min). The limited effects of site and scanner manufacturer support the use of multisite studies, such as NAPLS, as a viable means of collecting data on rare populations and increasing power in univariate functional connectivity studies. However, the results indicate that aggregation of fcMRI data across longer scan durations is necessary to increase the reliability of connectivity estimates at the single-subject level.

Uses, misuses, new uses and fundamental limitations of magnetic resonance imaging in cognitive science

When blood oxygenation level-dependent (BOLD) contrast functional magnetic resonance imaging (fMRI) was discovered in the early 1990s, it provoked an explosion of interest in exploring human cognition, using brain mapping techniques based on MRI. Standards for data acquisition and analysis were rapidly put in place, in order to assist comparison of results across laboratories. Recently, MRI data acquisition capabilities have improved dramatically, inviting a rethink of strategies for relating functional brain activity at the systems level with its neuronal substrates and functional connections. This paper reviews the established capabilities of BOLD contrast fMRI, the perceived weaknesses of major methods of analysis, and current results that may provide insights into improved brain modelling. These results have inspired the use of in vivo myeloarchitecture for localizing brain activity, individual subject analysis without spatial smoothing and mapping of changes in cerebral blood volume instead of BOLD activation changes. The apparent fundamental limitations of all methods based on nuclear magnetic resonance are also discussed.This article is part of the themed issue 'Interpreting BOLD: a dialogue between cognitive and cellular neuroscience'.

Reliability of graph analysis of resting state fMRI using test-retest dataset from the Human Connectome Project

The exploration of brain networks with resting-state fMRI (rs-fMRI) combined with graph theoretical approaches has become popular, with the perspective of finding network graph metrics as biomarkers in the context of clinical studies. A preliminary requirement for such findings is to assess the reliability of the graph based connectivity metrics. In previous test-retest (TRT) studies, this reliability has been explored using intraclass correlation coefficient (ICC) with heterogeneous results. But the issue of sample size has not been addressed. Using the large TRT rs-fMRI dataset from the Human Connectome Project (HCP), we computed ICCs and their corresponding p-values (applying permutation and bootstrap techniques) and varied the number of subjects (from 20 to 100), the scan duration (from 400 to 1200 time points), the cost and the graph metrics, using the Anatomic-Automatic Labelling (AAL) parcellation scheme. We quantified the reliability of the graph metrics computed both at global and regional level depending, at optimal cost, on two key parameters, the sample size and the number of time points or scan duration. In the cost range between 20% to 35%, most of the global graph metrics are reliable with 40 subjects or more with long scan duration (14min 24s). In large samples (for instance, 100 subjects), most global and regional graph metrics are reliable for a minimum scan duration of 7min 14s. Finally, for 40 subjects and long scan duration (14min 24s), the reliable regions are located in the main areas of the default mode network (DMN), the motor and the visual networks.

Building a Science of Individual Differences from fMRI

To date, fMRI research has been concerned primarily with evincing generic principles of brain function through averaging data from multiple subjects. Given rapid developments in both hardware and analysis tools, the field is now poised to study fMRI-derived measures in individual subjects, and to relate these to psychological traits or genetic variations. We discuss issues of validity, reliability and statistical assessment that arise when the focus shifts to individual subjects and that are applicable also to other imaging modalities. We emphasize that individual assessment of neural function with fMRI presents specific challenges and necessitates careful consideration of anatomical and vascular between-subject variability as well as sources of within-subject variability.

Studying Sub-Dendrograms of Resting-State Functional Networks with Voxel-Wise Hierarchical Clustering

Hierarchical clustering is a useful data-driven approach to classify complex data and has been used to analyze resting-state functional magnetic resonance imaging (fMRI) data and derive functional networks of the human brain at very large scale, such as the entire visual or sensory-motor cortex. In this study, we developed a voxel-wise, whole-brain hierarchical clustering framework to perform multi-stage analysis of group-averaged resting-state fMRI data in different levels of detail. With the framework we analyzed particularly the somatosensory motor and visual systems in fine details and constructed the corresponding sub-dendrograms, which corroborate consistently with the known modular organizations from previous clinical and experimental studies. The framework provides a useful tool for data-driven analysis of resting-state fMRI data to gain insight into the hierarchical organization and degree of functional modulation among the sub-units.