Optimizing preprocessing and analysis pipelines for single-subject fMRI: 2. Interactions with ICA, PCA, task contrast and inter-subject heterogeneity

The multiverse of data preprocessing and analysis in graph-based fMRI: A systematic literature review of analytical choices fed into a decision support tool for informed analysis

The large number of different analytical choices used by researchers is partly responsible for the challenge of replication in neuroimaging studies. For an exhaustive robustness analysis, knowledge of the full space of analytical options is essential. We conducted a systematic literature review to identify the analytical decisions in functional neuroimaging data preprocessing and analysis in the emerging field of cognitive network neuroscience. We found 61 different steps, with 17 of them having debatable parameter choices. Scrubbing, global signal regression, and spatial smoothing are among the controversial steps. There is no standardized order in which different steps are applied, and the parameter settings within several steps vary widely across studies. By aggregating the pipelines across studies, we propose three taxonomic levels to categorize analytical choices: 1) inclusion or exclusion of specific steps, 2) parameter tuning within steps, and 3) distinct sequencing of steps. We have developed a decision support application with high educational value called METEOR to facilitate access to the data in order to design well-informed robustness (multiverse) analysis.

Large Individual Differences in Functional Connectivity in the Context of Major Depression and Antidepressant Pharmacotherapy

Clinical studies of major depression (MD) generally focus on group effects, yet interindividual differences in brain function are increasingly recognized as important and may even impact effect sizes related to group effects. Here, we examine the magnitude of individual differences in relation to group differences that are commonly investigated (e.g., related to MD diagnosis and treatment response). Functional MRI data from 107 participants (63 female, 44 male) were collected at baseline, 2, and 8 weeks during which patients received pharmacotherapy (escitalopram, N = 68) and controls (N = 39) received no intervention. The unique contributions of different sources of variation were examined by calculating how much variance in functional connectivity was shared across all participants and sessions, within/across groups (patients vs controls, responders vs nonresponders, female vs male participants), recording sessions, and individuals. Individual differences and common connectivity across groups, sessions, and participants contributed most to the explained variance (>95% across analyses). Group differences related to MD diagnosis, treatment response, and biological sex made significant but small contributions (0.3-1.2%). High individual variation was present in cognitive control and attention areas, while low individual variation characterized primary sensorimotor regions. Group differences were much smaller than individual differences in the context of MD and its treatment. These results could be linked to the variable findings and difficulty translating research on MD to clinical practice. Future research should examine brain features with low and high individual variation in relation to psychiatric symptoms and treatment trajectories to explore the clinical relevance of the individual differences identified here.

Open and reproducible neuroimaging: From study inception to publication

Empirical observations of how labs conduct research indicate that the adoption rate of open practices for transparent, reproducible, and collaborative science remains in its infancy. This is at odds with the overwhelming evidence for the necessity of these practices and their benefits for individual researchers, scientific progress, and society in general. To date, information required for implementing open science practices throughout the different steps of a research project is scattered among many different sources. Even experienced researchers in the topic find it hard to navigate the ecosystem of tools and to make sustainable choices. Here, we provide an integrated overview of community-developed resources that can support collaborative, open, reproducible, replicable, robust and generalizable neuroimaging throughout the entire research cycle from inception to publication and across different neuroimaging modalities. We review tools and practices supporting study inception and planning, data acquisition, research data management, data processing and analysis, and research dissemination. An online version of this resource can be found at https://oreoni.github.io. We believe it will prove helpful for researchers and institutions to make a successful and sustainable move towards open and reproducible science and to eventually take an active role in its future development.

Baseline Functional Connectivity in Resting State Networks Associated with Depression and Remission Status after 16 Weeks of Pharmacotherapy: A CAN-BIND Report

Understanding the neural underpinnings of major depressive disorder (MDD) and its treatment could improve treatment outcomes. So far, findings are variable and large sample replications scarce. We aimed to replicate and extend altered functional connectivity associated with MDD and pharmacotherapy outcomes in a large, multisite sample. Resting-state fMRI data were collected from 129 patients and 99 controls through the Canadian Biomarker Integration Network in Depression. Symptoms were assessed with the Montgomery-Åsberg Depression Rating Scale (MADRS). Connectivity was measured as correlations between four seeds (anterior and posterior cingulate cortex, insula and dorsolateral prefrontal cortex) and all other brain voxels. Partial least squares was used to compare connectivity prior to treatment between patients and controls, and between patients reaching remission (MADRS ≤ 10) early (within 8 weeks), late (within 16 weeks), or not at all. We replicated previous findings of altered connectivity in patients. In addition, baseline connectivity of the anterior/posterior cingulate and insula seeds differentiated patients with different treatment outcomes. The stability of these differences was established in the largest single-site subsample. Our replication and extension of altered connectivity highlighted previously reported and new differences between patients and controls, and revealed features that might predict remission prior to pharmacotherapy. Trial registration:  ClinicalTrials.gov: NCT01655706.

Resting state fMRI scanner instabilities revealed by longitudinal phantom scans in a multi-center study

Quality assurance (QA) is crucial in longitudinal and/or multi-site studies, which involve the collection of data from a group of subjects over time and/or at different locations. It is important to regularly monitor the performance of the scanners over time and at different locations to detect and control for intrinsic differences (e.g., due to manufacturers) and changes in scanner performance (e.g., due to gradual component aging, software and/or hardware upgrades, etc.). As part of the Ontario Neurodegenerative Disease Research Initiative (ONDRI) and the Canadian Biomarker Integration Network in Depression (CAN-BIND), QA phantom scans were conducted approximately monthly for three to four years at 13 sites across Canada with 3T research MRI scanners. QA parameters were calculated for each scan using the functional Biomarker Imaging Research Network's (fBIRN) QA phantom and pipeline to capture between- and within-scanner variability. We also describe a QA protocol to measure the full-width-at-half-maximum (FWHM) of slice-wise point spread functions (PSF), used in conjunction with the fBIRN QA parameters. Variations in image resolution measured by the FWHM are a primary source of variance over time for many sites, as well as between sites and between manufacturers. We also identify an unexpected range of instabilities affecting individual slices in a number of scanners, which may amount to a substantial contribution of unexplained signal variance to their data. Finally, we identify a preliminary preprocessing approach to reduce this variance and/or alleviate the slice anomalies, and in a small human data set show that this change in preprocessing can have a significant impact on seed-based connectivity measurements for some individual subjects. We expect that other fMRI centres will find this approach to identifying and controlling scanner instabilities useful in similar studies.

Multisite Comparison of MRI Defacing Software Across Multiple Cohorts

With improvements to both scan quality and facial recognition software, there is an increased risk of participants being identified by a 3D render of their structural neuroimaging scans, even when all other personal information has been removed. To prevent this, facial features should be removed before data are shared or openly released, but while there are several publicly available software algorithms to do this, there has been no comprehensive review of their accuracy within the general population. To address this, we tested multiple algorithms on 300 scans from three neuroscience research projects, funded in part by the Ontario Brain Institute, to cover a wide range of ages (3-85 years) and multiple patient cohorts. While skull stripping is more thorough at removing identifiable features, we focused mainly on defacing software, as skull stripping also removes potentially useful information, which may be required for future analyses. We tested six publicly available algorithms (afni_refacer, deepdefacer, mri_deface, mridefacer, pydeface, quickshear), with one skull stripper (FreeSurfer) included for comparison. Accuracy was measured through a pass/fail system with two criteria; one, that all facial features had been removed and two, that no brain tissue was removed in the process. A subset of defaced scans were also run through several preprocessing pipelines to ensure that none of the algorithms would alter the resulting outputs. We found that the success rates varied strongly between defacers, with afni_refacer (89%) and pydeface (83%) having the highest rates, overall. In both cases, the primary source of failure came from a single dataset that the defacer appeared to struggle with - the youngest cohort (3-20 years) for afni_refacer and the oldest (44-85 years) for pydeface, demonstrating that defacer performance not only depends on the data provided, but that this effect varies between algorithms. While there were some very minor differences between the preprocessing results for defaced and original scans, none of these were significant and were within the range of variation between using different NIfTI converters, or using raw DICOM files.

Resolution-based spectral clustering for brain parcellation using functional MRI

BACKGROUND

Brain parcellation is important for exploiting neuroimaging data. Variability in physiology between individuals has led to the need for data-driven approaches to parcellation, with recent research focusing on simultaneously estimating and partitioning the network structure of the brain.

NEW METHOD

We view data preprocessing, parcellation, and parcel validation from the perspective of predictive modeling. The goal is to identify parcels in a way that best generalizes to unseen data. We utilize an uncertainty quantification approach from image science to define parcels as groups of unresolvable variables in the predictive model. Model parameters are chosen via cross-validation. Parcellation results are compared based on both their repeatability as well as their ability to describe held-out data.

RESULTS

The approach provides insight and strategies for open questions such as the choice of evaluation metrics, selection of model order, and the optimal tuning of preprocessing steps for functional imaging data.

COMPARISON WITH EXISTING METHODS

We compare new and established approaches using functional imaging data, where we find the proposed approach produces parcellations which are both more accurate and more repeatable than the current baseline clustering method. The metrics also demonstrate potential problems with overfitting for the baseline method, and with bias for other methods.

CONCLUSIONS

Clustering of resolution provides a principled and robust approach to brain parcellation which improves on the current baseline.

Resting State fMRI: Going Through the Motions

Resting state functional magnetic resonance imaging (rs-fMRI) has become an indispensable tool in neuroscience research. Despite this, rs-fMRI signals are easily contaminated by artifacts arising from movement of the head during data collection. The artifacts can be problematic even for motions on the millimeter scale, with complex spatiotemporal properties that can lead to substantial errors in functional connectivity estimates. Effective correction methods must be employed, therefore, to distinguish true functional networks from motion-related noise. Research over the last three decades has produced numerous correction methods, many of which must be applied in combination to achieve satisfactory data quality. Subject instruction, training, and mild restraints are helpful at the outset, but usually insufficient. Improvements come from applying multiple motion correction algorithms retrospectively after rs-fMRI data are collected, although residual artifacts can still remain in cases of elevated motion, which are especially prevalent in patient populations. Although not commonly adopted at present, “real-time” correction methods are emerging that can be combined with retrospective methods and that promise better correction and increased rs-fMRI signal sensitivity. While the search for the ideal motion correction protocol continues, rs-fMRI research will benefit from good disclosure practices, such as: (1) reporting motion-related quality control metrics to provide better comparison between studies; and (2) including motion covariates in group-level analyses to limit the extent of motion-related confounds when studying group differences.

Functional MRI of Letter Cancellation Task Performance in Older Adults

The Letter Cancellation Task (LCT) is a widely used pen-and-paper probe of attention in clinical and research settings. Despite its popularity, the neural correlates of the task are not well understood. The present study uses functional magnetic resonance imaging (fMRI) and specialized tablet technology to identify the neural correlates of the LCT in 32 healthy older adults between 50-85 years of age, and further investigates the effect of healthy aging on performance. Subjects performed the LCT in its standard pen-and-paper administration and with the tablet during fMRI. Performance on the tablet was significantly slower than on pen-and-paper, with both response modes showing slower performance as a function of age. Across all ages, bilateral brain activation was observed in the cerebellum, superior temporal lobe, precentral gyrus, frontal gyrus, and occipital and parietal areas. Increasing age correlated with reduced brain activity in the supplementary motor area, middle occipital gyrus, medial and inferior frontal gyrus, cerebellum and putamen. Better LCT performance was correlated with increased activity in the middle frontal gyrus, and reduced activity in the cerebellum. The brain regions activated are associated with visuospatial attention and motor control, and are consistent with the neural correlates of LCT performance previously identified in lesion studies.

BOLD signal variability and complexity in children and adolescents with and without autism spectrum disorder

•

Resting-state BOLD signal variability and complexity were examined.

•

No significant group differences were observed in youth with and without autism.

•

A continuum of brain-behavior relationships was observed across diagnostic groups.

•

Positive correlations were found between brain measures, age and global efficiency.

•

Negative correlations were found between the brain measures and behavioral severity.

Variability of neural signaling is an important index of healthy brain functioning, as is signal complexity, which relates to information processing capacity. Alterations in variability and complexity may underlie certain brain dysfunctions. Here, resting-state fMRI was used to examine variability and complexity in children and adolescents with and without autism spectrum disorder (ASD). Variability was measured using the mean square successive difference (MSSD) of the time series, and complexity was assessed using sample entropy. A categorical approach was implemented to determine if the brain measures differed between diagnostic groups (ASD and controls). A dimensional approach was used to examine the continuum of relationships between each brain measure and behavioral severity, age, IQ, and the global efficiency (GE) of each participant’s structural connectome, which reflects the structural capacity for information processing. Using the categorical approach, no significant group differences were found for neither MSSD nor entropy. The dimensional approach revealed significant positive correlations between each brain measure, GE, and age. Negative correlations were observed between each brain measure and the severity of ASD behaviors across all participants. These results reveal the nature of variability and complexity of BOLD signals in children and adolescents with and without ASD.

Brain connectivity tracks effects of chemotherapy separately from behavioral measures

Several studies in cancer research have suggested that cognitive dysfunction following chemotherapy, referred to in lay terms as "chemobrain", is a serious problem. At present, the changes in integrative brain function that underlie such dysfunction remain poorly understood. Recent developments in neuroimaging suggest that patterns of functional connectivity can provide a broadly applicable neuromarker of cognitive performance and other psychometric measures. The current study used multivariate analysis methods to identify patterns of disruption in resting state functional connectivity of the brain due to chemotherapy and the degree to which the disruptions can be linked to behavioral measures of distress and cognitive performance. Sixty two women (22 healthy control, 18 patients treated with adjuvant chemotherapy, and 22 treated without chemotherapy) were evaluated with neurocognitive measures followed by self-report questionnaires and open eyes resting-state fMRI scanning at three time points: diagnosis (M0, pre-adjuvant treatment), 1 month (M1), and 7 months (M7) after treatment. The results indicated deficits in cognitive health of breast cancer patients immediately after chemotherapy that improved over time. This psychological trajectory was paralleled by a disruption and later recovery of resting-state functional connectivity, mostly in the parietal and frontal brain regions. Mediation analysis showed that the functional connectivity alteration pattern is a separable treatment symptom from the decreased cognitive health. Current study indicates that more targeted support for patients should be developed to ameliorate these multi-faceted side effects of chemotherapy treatment on neural functioning and cognitive health.

Different neural activities support auditory working memory in musicians and bilinguals

Musical training and bilingualism benefit executive functioning and working memory (WM)-however, the brain networks supporting this advantage are not well specified. Here, we used functional magnetic resonance imaging and the n-back task to assess WM for spatial (sound location) and nonspatial (sound category) auditory information in musician monolingual (musicians), nonmusician bilinguals (bilinguals), and nonmusician monolinguals (controls). Musicians outperformed bilinguals and controls on the nonspatial WM task. Overall, spatial and nonspatial WM were associated with greater activity in dorsal and ventral brain regions, respectively. Increasing WM load yielded similar recruitment of the anterior-posterior attention network in all three groups. In both tasks and both levels of difficulty, musicians showed lower brain activity than controls in superior prefrontal frontal gyrus and dorsolateral prefrontal cortex (DLPFC) bilaterally, a finding that may reflect improved and more efficient use of neural resources. Bilinguals showed enhanced activity in language-related areas (i.e., left DLPFC and left supramarginal gyrus) relative to musicians and controls, which could be associated with the need to suppress interference associated with competing semantic activations from multiple languages. These findings indicate that the auditory WM advantage in musicians and bilinguals is mediated by different neural networks specific to each life experience.

Functional MRI of Handwriting Tasks: A Study of Healthy Young Adults Interacting with a Novel Touch-Sensitive Tablet

Handwriting is a complex human activity that engages a blend of cognitive and visual motor skills. Current understanding of the neural correlates of handwriting has largely come from lesion studies of patients with impaired handwriting. Task-based fMRI studies would be useful to supplement this work. To address concerns over ecological validity, previously we developed a fMRI-compatible, computerized tablet system for writing and drawing including visual feedback of hand position and an augmented reality display. The purpose of the present work is to use the tablet system in proof-of-concept to characterize brain activity associated with clinically relevant handwriting tasks, originally developed to characterize handwriting impairments in Alzheimer’s disease patients. As a prelude to undertaking fMRI studies of patients, imaging was performed of twelve young healthy subjects who copied sentences, phone numbers, and grocery lists using the fMRI-compatible tablet. Activation maps for all handwriting tasks consisted of a distributed network of regions in reasonable agreement with previous studies of handwriting performance. In addition, differences in brain activity were observed between the test subcomponents consistent with different demands of neural processing for successful task performance, as identified by investigating three quantitative behavioral metrics (writing speed, stylus contact force and stylus in air time). This study provides baseline behavioral and brain activity results for fMRI studies that adopt this handwriting test to characterize patients with brain impairments.

ICN_Atlas: Automated description and quantification of functional MRI activation patterns in the framework of intrinsic connectivity networks

Generally, the interpretation of functional MRI (fMRI) activation maps continues to rely on assessing their relationship to anatomical structures, mostly in a qualitative and often subjective way. Recently, the existence of persistent and stable brain networks of functional nature has been revealed; in particular these so-called intrinsic connectivity networks (ICNs) appear to link patterns of resting state and task-related state connectivity. These networks provide an opportunity of functionally-derived description and interpretation of fMRI maps, that may be especially important in cases where the maps are predominantly task-unrelated, such as studies of spontaneous brain activity e.g. in the case of seizure-related fMRI maps in epilepsy patients or sleep states. Here we present a new toolbox (ICN_Atlas) aimed at facilitating the interpretation of fMRI data in the context of ICN. More specifically, the new methodology was designed to describe fMRI maps in function-oriented, objective and quantitative way using a set of 15 metrics conceived to quantify the degree of 'engagement' of ICNs for any given fMRI-derived statistical map of interest. We demonstrate that the proposed framework provides a highly reliable quantification of fMRI activation maps using a publicly available longitudinal (test-retest) resting-state fMRI dataset. The utility of the ICN_Atlas is also illustrated on a parametric task-modulation fMRI dataset, and on a dataset of a patient who had repeated seizures during resting-state fMRI, confirmed on simultaneously recorded EEG. The proposed ICN_Atlas toolbox is freely available for download at http://icnatlas.com and at http://www.nitrc.org for researchers to use in their fMRI investigations.

Tablet-Based Functional MRI of the Trail Making Test: Effect of Tablet Interaction Mode

The Trail Making Test (TMT) is widely used for assessing executive function, frontal lobe abilities, and visual motor skills. Part A of this pen-and-paper test (TMT-A) involves linking numbers randomly distributed in space, in ascending order. Part B (TMT-B) alternates between linking numbers and letters. TMT-B is more demanding than TMT-A, but the mental processing that supports the performance of this test remains incompletely understood. Functional MRI (fMRI) may help to clarify the relationship between TMT performance and brain activity, but providing an environment that supports real-world pen-and-paper interactions during fMRI is challenging. Previously, an fMRI-compatible tablet system was developed for writing and drawing with two modes of interaction: the original cursor-based, proprioceptive approach, and a new mode involving augmented reality to provide visual feedback of hand position (VFHP) for enhanced user interaction. This study characterizes the use of the tablet during fMRI of young healthy adults (n = 22), with half of the subjects performing TMT with VFHP and the other half performing TMT without VFHP. Activation maps for both TMT-A and TMT-B performance showed considerable overlap between the two tablet modes, and no statistically differences in brain activity were detected when contrasting TMT-B vs. TMT-A for the two tablet modes. Behavioral results also showed no statistically different interaction effects for TMT-B vs. TMT-A for the two tablet modes. Tablet-based TMT scores showed reasonable convergent validity with those obtained by administering the standard pen-and-paper TMT to the same subjects. Overall, the results suggest that despite the slightly different mechanisms involved for the two modes of tablet interaction, both are suitable for use in fMRI studies involving TMT performance. This study provides information for using tablet-based TMT methods appropriately in future fMRI studies involving patients and healthy individuals.

Methods for cleaning the BOLD fMRI signal

Blood oxygen-level-dependent functional magnetic resonance imaging (BOLD fMRI) has rapidly become a popular technique for the investigation of brain function in healthy individuals, patients as well as in animal studies. However, the BOLD signal arises from a complex mixture of neuronal, metabolic and vascular processes, being therefore an indirect measure of neuronal activity, which is further severely corrupted by multiple non-neuronal fluctuations of instrumental, physiological or subject-specific origin. This review aims to provide a comprehensive summary of existing methods for cleaning the BOLD fMRI signal. The description is given from a methodological point of view, focusing on the operation of the different techniques in addition to pointing out the advantages and limitations in their application. Since motion-related and physiological noise fluctuations are two of the main noise components of the signal, techniques targeting their removal are primarily addressed, including both data-driven approaches and using external recordings. Data-driven approaches, which are less specific in the assumed model and can simultaneously reduce multiple noise fluctuations, are mainly based on data decomposition techniques such as principal and independent component analysis. Importantly, the usefulness of strategies that benefit from the information available in the phase component of the signal, or in multiple signal echoes is also highlighted. The use of global signal regression for denoising is also addressed. Finally, practical recommendations regarding the optimization of the preprocessing pipeline for the purpose of denoising and future venues of research are indicated. Through the review, we summarize the importance of signal denoising as an essential step in the analysis pipeline of task-based and resting state fMRI studies.

BIDS apps: Improving ease of use, accessibility, and reproducibility of neuroimaging data analysis methods

The rate of progress in human neurosciences is limited by the inability to easily apply a wide range of analysis methods to the plethora of different datasets acquired in labs around the world. In this work, we introduce a framework for creating, testing, versioning and archiving portable applications for analyzing neuroimaging data organized and described in compliance with the Brain Imaging Data Structure (BIDS). The portability of these applications (BIDS Apps) is achieved by using container technologies that encapsulate all binary and other dependencies in one convenient package. BIDS Apps run on all three major operating systems with no need for complex setup and configuration and thanks to the comprehensiveness of the BIDS standard they require little manual user input. Previous containerized data processing solutions were limited to single user environments and not compatible with most multi-tenant High Performance Computing systems. BIDS Apps overcome this limitation by taking advantage of the Singularity container technology. As a proof of concept, this work is accompanied by 22 ready to use BIDS Apps, packaging a diverse set of commonly used neuroimaging algorithms.

Modulation of the sensorimotor system by sustained manual pressure stimulation

In Vojta physiotherapy, also known as reflex locomotion therapy, prolonged peripheral pressure stimulation induces complex generalized involuntary motor responses and modifies subsequent behavior, but its neurobiological basis remains unknown. We hypothesized that the stimulation would induce sensorimotor activation changes in functional magnetic resonance imaging (fMRI) during sequential finger opposition. Thirty healthy volunteers (mean age 24.2) underwent two randomized fMRI sessions involving manual pressure stimulation applied either at the right lateral heel according to Vojta, or at the right lateral ankle (control site). Participants were scanned before and after the stimulation when performing auditory-paced sequential finger opposition with their right hand. Despite an extensive activation decrease following both stimulation paradigms, the stimulation of the heel specifically led to an increase in task-related activation in the predominantly contralateral pontomedullary reticular formation and bilateral posterior cerebellar hemisphere and vermis. Our findings suggest that sustained pressure stimulation of the foot is associated with differential short-term changes in hand motor task-related activation depending on the stimulation. This is the first evidence for brainstem modulation after peripheral pressure stimulation, suggesting that the after-effects of reflex locomotion physiotherapy involve a modulation of the pontomedullary reticular formation.

Optimizing fMRI preprocessing pipelines for block-design tasks as a function of age

Functional Magnetic Resonance Imaging (fMRI) is a powerful neuroimaging tool, which is often hampered by significant noise confounds. There is evidence that our ability to detect activations in task fMRI is highly dependent on the preprocessing steps used to control noise and artifact. However, the vast majority of studies examining preprocessing pipelines in fMRI have focused on young adults. Given the widespread use of fMRI for characterizing the neurobiology of aging, it is critical to examine how the impact of preprocessing choices varies as a function of age. In this study, we employ the NPAIRS cross-validation framework, which optimizes pipelines based on metrics of prediction accuracy (P) and spatial reproducibility (R), to compare the effects of pipeline optimization between young (21-33 years) and older (61-82 years) cohorts, for three different block-design contrasts. Motion is shown to be a greater issue in the older cohort, and we introduce new statistical approaches to control for potential biases due to head motion during pipeline optimization. In comparison, data-driven methods of physiological noise correction show comparable benefits for both young and old cohorts. Using our optimization framework, we demonstrate that the optimal pipelines tend to be highly similar across age cohorts. In addition, there is a comparable, significant benefit of pipeline optimization across age cohorts, for (P, R) metrics and independent validation measures of activation overlap (both between-subject, within-session and within-subject, between-session). The choice of task contrast consistently shows a greater impact than the age cohort, for (P, R) metrics and activation overlap. Finally, adaptive pipeline optimization per task run shows improved sensitivity to age-related changes in brain activity, particularly for weaker, more complex cognitive contrasts. The current study provides the first detailed examination of preprocessing pipelines across age cohorts, demonstrating a significant benefit of adaptive pipeline optimization across age groups.

Scanning the horizon: towards transparent and reproducible neuroimaging research

Functional neuroimaging techniques have transformed our ability to probe the neurobiological basis of behaviour and are increasingly being applied by the wider neuroscience community. However, concerns have recently been raised that the conclusions that are drawn from some human neuroimaging studies are either spurious or not generalizable. Problems such as low statistical power, flexibility in data analysis, software errors and a lack of direct replication apply to many fields, but perhaps particularly to functional MRI. Here, we discuss these problems, outline current and suggested best practices, and describe how we think the field should evolve to produce the most meaningful and reliable answers to neuroscientific questions.

A Hitchhiker's Guide to Functional Magnetic Resonance Imaging

Functional Magnetic Resonance Imaging (fMRI) studies have become increasingly popular both with clinicians and researchers as they are capable of providing unique insights into brain functions. However, multiple technical considerations (ranging from specifics of paradigm design to imaging artifacts, complex protocol definition, and multitude of processing and methods of analysis, as well as intrinsic methodological limitations) must be considered and addressed in order to optimize fMRI analysis and to arrive at the most accurate and grounded interpretation of the data. In practice, the researcher/clinician must choose, from many available options, the most suitable software tool for each stage of the fMRI analysis pipeline. Herein we provide a straightforward guide designed to address, for each of the major stages, the techniques, and tools involved in the process. We have developed this guide both to help those new to the technique to overcome the most critical difficulties in its use, as well as to serve as a resource for the neuroimaging community.

Functional changes in brain activity after hypnosis in patients with dental phobia

Visiting the dentist is often accompanied by apprehension or anxiety. People, who suffer from specific dental phobia (a disproportional fear of dental) procedures show psychological and physiological symptoms which make dental treatments difficult or impossible. For such purposes, hypnosis is often used in dental practice as an alternative for a number of treatments adjuvant or instead of sedation or general anaesthetics, as medication is often associated with risks and side effects. This is the first study to address the effects of a brief dental hypnosis on the fear processing structures of the brain in dental phobics using functional magnetic resonance imaging (fMRI). 12 dental phobics (DP; mean 34.9years) and 12 healthy controls (CO; mean 33.2years) were scanned with a 3T MRI whole body-scanner observing brain activity changes after a brief hypnotic invervention. An fMRI event-related design symptom provocation task applying animated audio-visual pseudorandomized strong phobic stimuli was presented in order to maximize the fearful reactions during scanning. Control videos showed the use of familiar electronic household equipment. In DP group, main effects of fear condition were found in the left amygdala and bilaterally in the anterior cingulate cortex (ACC), insula and hippocampu (R<L). During hypnosis DP showed a significantly reduced activation in all of these areas. Reduced neural activity patterns were also found in the control group. No amygdala activation was detected in healthy subjects in the two experimental conditions. Compared to DP, CO showed less bilateral activation in the insula and ACC in the awake condition. Findings show that anxiety-provoking stimuli such as undergoing dental surgery, endodontic treatments or insufficient anaesthetics, can be effectively reduced under hypnosis. The present study gives scientific evidence that hypnosis is a powerful and successful method for inhibiting the reaction of the fear circuitry structures.

Building a Science of Individual Differences from fMRI

To date, fMRI research has been concerned primarily with evincing generic principles of brain function through averaging data from multiple subjects. Given rapid developments in both hardware and analysis tools, the field is now poised to study fMRI-derived measures in individual subjects, and to relate these to psychological traits or genetic variations. We discuss issues of validity, reliability and statistical assessment that arise when the focus shifts to individual subjects and that are applicable also to other imaging modalities. We emphasize that individual assessment of neural function with fMRI presents specific challenges and necessitates careful consideration of anatomical and vascular between-subject variability as well as sources of within-subject variability.

An Automated, Adaptive Framework for Optimizing Preprocessing Pipelines in Task-Based Functional MRI

BOLD fMRI is sensitive to blood-oxygenation changes correlated with brain function; however, it is limited by relatively weak signal and significant noise confounds. Many preprocessing algorithms have been developed to control noise and improve signal detection in fMRI. Although the chosen set of preprocessing and analysis steps (the “pipeline”) significantly affects signal detection, pipelines are rarely quantitatively validated in the neuroimaging literature, due to complex preprocessing interactions. This paper outlines and validates an adaptive resampling framework for evaluating and optimizing preprocessing choices by optimizing data-driven metrics of task prediction and spatial reproducibility. Compared to standard “fixed” preprocessing pipelines, this optimization approach significantly improves independent validation measures of within-subject test-retest, and between-subject activation overlap, and behavioural prediction accuracy. We demonstrate that preprocessing choices function as implicit model regularizers, and that improvements due to pipeline optimization generalize across a range of simple to complex experimental tasks and analysis models. Results are shown for brief scanning sessions (<3 minutes each), demonstrating that with pipeline optimization, it is possible to obtain reliable results and brain-behaviour correlations in relatively small datasets.

A Novel Brain Networks Enhancement Model (BNEM) for BOLD fMRI Data Analysis With Highly Spatial Reproducibility

Independent component analysis aiming at detecting the functional connectivity among discrete cortical brain regions has been extensively used to explore the functional magnetic resonance imaging data. Although the independent components (ICs) were with relatively high quality, the noise embedding in ICs has a great impact on the true active/inactive region inference and the reproducibility, in postprocessing stage, e.g., the extraction of statistical parametrical maps (SPMs). In this paper, a novel brain network enhancement model (BNEM) is proposed, which mainly consists of two key techniques: 1) 3-D wavelet noise filter (3DWNF) for the meaningful ICs, which greatly suppresses noise and enforces the real activation inference of SPMs; and 2) a spatial reproducibility enhancement algorithm (SREA), aiming to improve the reproducibility of SPMs. The simulated experiment demonstrated that the postfiltering signals by 3DWNF were with higher correlation and less normalized mean square error to the ground truths than the prefiltering ones; SREA could further enhance the quality of most postfiltering ones, preserving the consistency with 3DWNF. The real data experiments also revealed that 1) 3DWNF could lead to more accurate preservation of the true positive voxels by correctly identifying the high proportionally misclassified voxels of the nonenhanced SPMs; 2) SREA could further improve the classification accuracy of the active/inactive voxels of SPMs corresponding to the 3DWNF denoised ICs; and 3) both 3DWNF and SREA contribute to the reproducibility enhancement of the reproduced SPMs by BNEM. Thus, BNEM is expected to have wide applicability in the neuroscience and clinical domain.

Is fMRI "noise" really noise? Resting state nuisance regressors remove variance with network structure

Noise correction is a critical step towards accurate mapping of resting state BOLD fMRI connectivity. Noise sources related to head motion or physiology are typically modelled by nuisance regressors, and a generalised linear model is applied to regress out the associated signal variance. In this study, we use independent component analysis (ICA) to characterise the data variance typically discarded in this pre-processing stage in a cohort of 12 healthy volunteers. The signal variance removed by 24, 12, 6, or only 3 head motion parameters demonstrated network structure typically associated with functional connectivity, and certain networks were discernable in the variance extracted by as few as 2 physiologic regressors. Simulated nuisance regressors, unrelated to the true data noise, also removed variance with network structure, indicating that any group of regressors that randomly sample variance may remove highly structured “signal” as well as “noise.” Furthermore, to support this we demonstrate that random sampling of the original data variance continues to exhibit robust network structure, even when as few as 10% of the original volumes are considered. Finally, we examine the diminishing returns of increasing the number of nuisance regressors used in pre-processing, showing that excessive use of motion regressors may do little better than chance in removing variance within a functional network. It remains an open challenge to understand the balance between the benefits and confounds of noise correction using nuisance regressors.

•

Data variance removed by nuisance regressors contains network structure.

•

Simulated regressors unrelated to noise also extract data with network structure.

•

Random sampling of original data (as few as 10% of volumes) reveals robust networks.

•

After optimal number, motion regressors remove similar variance as simulated ones.

•

Excessive nuisance regressors extract random signal variance with network structure.

A computerized tablet with visual feedback of hand position for functional magnetic resonance imaging

Neuropsychological tests behavioral tasks that very commonly involve handwriting and drawing are widely used in the clinic to detect abnormal brain function. Functional magnetic resonance imaging (fMRI) may be useful in increasing the specificity of such tests. However, performing complex pen-and-paper tests during fMRI involves engineering challenges. Previously, we developed an fMRI-compatible, computerized tablet system to address this issue. However, the tablet did not include visual feedback of hand position (VFHP), a human factors component that may be important for fMRI of certain patient populations. A real-time system was thus developed to provide VFHP and integrated with the tablet in an augmented reality display. The effectiveness of the system was initially tested in young healthy adults who performed various handwriting tasks in front of a computer display with and without VFHP. Pilot fMRI of writing tasks were performed by two representative individuals with and without VFHP. Quantitative analysis of the behavioral results indicated improved writing performance with VFHP. The pilot fMRI results suggest that writing with VFHP requires less neural resources compared to the without VFHP condition, to maintain similar behavior. Thus, the tablet system with VFHP is recommended for future fMRI studies involving patients with impaired brain function and where ecologically valid behavior is important.

An open science resource for establishing reliability and reproducibility in functional connectomics

Efforts to identify meaningful functional imaging-based biomarkers are limited by the ability to reliably characterize inter-individual differences in human brain function. Although a growing number of connectomics-based measures are reported to have moderate to high test-retest reliability, the variability in data acquisition, experimental designs, and analytic methods precludes the ability to generalize results. The Consortium for Reliability and Reproducibility (CoRR) is working to address this challenge and establish test-retest reliability as a minimum standard for methods development in functional connectomics. Specifically, CoRR has aggregated 1,629 typical individuals’ resting state fMRI (rfMRI) data (5,093 rfMRI scans) from 18 international sites, and is openly sharing them via the International Data-sharing Neuroimaging Initiative (INDI). To allow researchers to generate various estimates of reliability and reproducibility, a variety of data acquisition procedures and experimental designs are included. Similarly, to enable users to assess the impact of commonly encountered artifacts (for example, motion) on characterizations of inter-individual variation, datasets of varying quality are included.

Data-analytical stability of cluster-wise and peak-wise inference in fMRI data analysis

BACKGROUND

Carp (2012) demonstrated the large variability that is present in the method sections of fMRI studies. This methodological variability between studies limits reproducible research.

NEW METHOD

Evaluation protocols for methods used in fMRI should include data-analytical stability measures quantifying the variability in results following choices in the methods. Data-analytical stability can be seen as a proxy for reproducibility. To illustrate how one can perform such evaluations, we study two competing approaches for topological feature based inference (random field theory and permutation based testing) and two competing methods for smoothing (Gaussian smoothing and adaptive smoothing). We compare these approaches from the perspective of data-analytical stability in real data, and additionally consider validity and reliability in simulations.

RESULTS

There is clear evidence that choices in the methods impact the validity, reliability and stability of the results. For the particular comparison studied, we find that permutation based methods render the most valid results. For stability and reliability, the performance of different smoothing and inference types depends on the setting. However, while being more reliable, adaptive smoothing can evoke less stable results when using larger kernel width, especially with cluster size based permutation inference.

COMPARISON WITH EXISTING METHODS

While existing evaluation methods focus on validity and reliability, we show that data-analytical stability enables to further distinguish between performance of different methods.

CONCLUSION

Data-analytical stability is an important additional criterion that can easily be incorporated in evaluation protocols.

Comparing within-subject classification and regularization methods in fMRI for large and small sample sizes

In recent years, a variety of multivariate classifier models have been applied to fMRI, with different modeling assumptions. When classifying high-dimensional fMRI data, we must also regularize to improve model stability, and the interactions between classifier and regularization techniques are still being investigated. Classifiers are usually compared on large, multisubject fMRI datasets. However, it is unclear how classifier/regularizer models perform for within-subject analyses, as a function of signal strength and sample size. We compare four standard classifiers: Linear and Quadratic Discriminants, Logistic Regression and Support Vector Machines. Classification was performed on data in the linear kernel (covariance) feature space, and classifiers are tuned with four commonly-used regularizers: Principal Component and Independent Component Analysis, and penalization of kernel features using L₁ and L₂ norms. We evaluated prediction accuracy (P) and spatial reproducibility (R) of all classifier/regularizer combinations on single-subject analyses, over a range of three different block task contrasts and sample sizes for a BOLD fMRI experiment. We show that the classifier model has a small impact on signal detection, compared to the choice of regularizer. PCA maximizes reproducibility and global SNR, whereas Lp -norms tend to maximize prediction. ICA produces low reproducibility, and prediction accuracy is classifier-dependent. However, trade-offs in (P,R) depend partly on the optimization criterion, and PCA-based models are able to explore the widest range of (P,R) values. These trends are consistent across task contrasts and data sizes (training samples range from 6 to 96 scans). In addition, the trends in classifier performance are consistent for ROI-based classifier analyses.

Model specification and the reliability of fMRI results: implications for longitudinal neuroimaging studies in psychiatry

Functional Magnetic Resonance Imagine (fMRI) is an important assessment tool in longitudinal studies of mental illness and its treatment. Understanding the psychometric properties of fMRI-based metrics, and the factors that influence them, will be critical for properly interpreting the results of these efforts. The current study examined whether the choice among alternative model specifications affects estimates of test-retest reliability in key emotion processing regions across a 6-month interval. Subjects (N = 46) performed an emotional-faces paradigm during fMRI in which neutral faces dynamically morphed into one of four emotional faces. Median voxelwise intraclass correlation coefficients (mvICCs) were calculated to examine stability over time in regions showing task-related activity as well as in bilateral amygdala. Four modeling choices were evaluated: a default model that used the canonical hemodynamic response function (HRF), a flexible HRF model that included additional basis functions, a modified CompCor (mCompCor) model that added corrections for physiological noise in the global signal, and a final model that combined the flexible HRF and mCompCor models. Model residuals were examined to determine the degree to which each pipeline met modeling assumptions. Results indicated that the choice of modeling approaches impacts both the degree to which model assumptions are met and estimates of test-retest reliability. ICC estimates in the visual cortex increased from poor (mvICC = 0.31) in the default pipeline to fair (mvICC = 0.45) in the full alternative pipeline – an increase of 45%. In nearly all tests, the models with the fewest assumption violations generated the highest ICC estimates. Implications for longitudinal treatment studies that utilize fMRI are discussed.

Pattern classification of fMRI data: applications for analysis of spatially distributed cortical networks

The field of fMRI data analysis is rapidly growing in sophistication, particularly in the domain of multivariate pattern classification. However, the interaction between the properties of the analytical model and the parameters of the BOLD signal (e.g. signal magnitude, temporal variance and functional connectivity) is still an open problem. We addressed this problem by evaluating a set of pattern classification algorithms on simulated and experimental block-design fMRI data. The set of classifiers consisted of linear and quadratic discriminants, linear support vector machine, and linear and nonlinear Gaussian naive Bayes classifiers. For linear discriminant, we used two methods of regularization: principal component analysis, and ridge regularization. The classifiers were used (1) to classify the volumes according to the behavioral task that was performed by the subject, and (2) to construct spatial maps that indicated the relative contribution of each voxel to classification. Our evaluation metrics were: (1) accuracy of out-of-sample classification and (2) reproducibility of spatial maps. In simulated data sets, we performed an additional evaluation of spatial maps with ROC analysis. We varied the magnitude, temporal variance and connectivity of simulated fMRI signal and identified the optimal classifier for each simulated environment. Overall, the best performers were linear and quadratic discriminants (operating on principal components of the data matrix) and, in some rare situations, a nonlinear Gaussian naïve Bayes classifier. The results from the simulated data were supported by within-subject analysis of experimental fMRI data, collected in a study of aging. This is the first study that systematically characterizes interactions between analysis model and signal parameters (such as magnitude, variance and correlation) on the performance of pattern classifiers for fMRI.

On the definition of signal-to-noise ratio and contrast-to-noise ratio for FMRI data

Signal-to-noise ratio, the ratio between signal and noise, is a quantity that has been well established for MRI data but is still subject of ongoing debate and confusion when it comes to fMRI data. fMRI data are characterised by small activation fluctuations in a background of noise. Depending on how the signal of interest and the noise are identified, signal-to-noise ratio for fMRI data is reported by using many different definitions. Since each definition comes with a different scale, interpreting and comparing signal-to-noise ratio values for fMRI data can be a very challenging job. In this paper, we provide an overview of existing definitions. Further, the relationship with activation detection power is investigated. Reference tables and conversion formulae are provided to facilitate comparability between fMRI studies.

PHYCAA+: an optimized, adaptive procedure for measuring and controlling physiological noise in BOLD fMRI

The presence of physiological noise in functional MRI can greatly limit the sensitivity and accuracy of BOLD signal measurements, and produce significant false positives. There are two main types of physiological confounds: (1) high-variance signal in non-neuronal tissues of the brain including vascular tracts, sinuses and ventricles, and (2) physiological noise components which extend into gray matter tissue. These physiological effects may also be partially coupled with stimuli (and thus the BOLD response). To address these issues, we have developed PHYCAA+, a significantly improved version of the PHYCAA algorithm (Churchill et al., 2011) that (1) down-weights the variance of voxels in probable non-neuronal tissue, and (2) identifies the multivariate physiological noise subspace in gray matter that is linked to non-neuronal tissue. This model estimates physiological noise directly from EPI data, without requiring external measures of heartbeat and respiration, or manual selection of physiological components. The PHYCAA+ model significantly improves the prediction accuracy and reproducibility of single-subject analyses, compared to PHYCAA and a number of commonly-used physiological correction algorithms. Individual subject denoising with PHYCAA+ is independently validated by showing that it consistently increased between-subject activation overlap, and minimized false-positive signal in non gray-matter loci. The results are demonstrated for both block and fast single-event task designs, applied to standard univariate and adaptive multivariate analysis models.

On the plurality of (methodological) worlds: estimating the analytic flexibility of FMRI experiments

How likely are published findings in the functional neuroimaging literature to be false? According to a recent mathematical model, the potential for false positives increases with the flexibility of analysis methods. Functional MRI (fMRI) experiments can be analyzed using a large number of commonly used tools, with little consensus on how, when, or whether to apply each one. This situation may lead to substantial variability in analysis outcomes. Thus, the present study sought to estimate the flexibility of neuroimaging analysis by submitting a single event-related fMRI experiment to a large number of unique analysis procedures. Ten analysis steps for which multiple strategies appear in the literature were identified, and two to four strategies were enumerated for each step. Considering all possible combinations of these strategies yielded 6,912 unique analysis pipelines. Activation maps from each pipeline were corrected for multiple comparisons using five thresholding approaches, yielding 34,560 significance maps. While some outcomes were relatively consistent across pipelines, others showed substantial methods-related variability in activation strength, location, and extent. Some analysis decisions contributed to this variability more than others, and different decisions were associated with distinct patterns of variability across the brain. Qualitative outcomes also varied with analysis parameters: many contrasts yielded significant activation under some pipelines but not others. Altogether, these results reveal considerable flexibility in the analysis of fMRI experiments. This observation, when combined with mathematical simulations linking analytic flexibility with elevated false positive rates, suggests that false positive results may be more prevalent than expected in the literature. This risk of inflated false positive rates may be mitigated by constraining the flexibility of analytic choices or by abstaining from selective analysis reporting.

An improved framework for confound regression and filtering for control of motion artifact in the preprocessing of resting-state functional connectivity data

Several recent reports in large, independent samples have demonstrated the influence of motion artifact on resting-state functional connectivity MRI (rsfc-MRI). Standard rsfc-MRI preprocessing typically includes regression of confounding signals and band-pass filtering. However, substantial heterogeneity exists in how these techniques are implemented across studies, and no prior study has examined the effect of differing approaches for the control of motion-induced artifacts. To better understand how in-scanner head motion affects rsfc-MRI data, we describe the spatial, temporal, and spectral characteristics of motion artifacts in a sample of 348 adolescents. Analyses utilize a novel approach for describing head motion on a voxelwise basis. Next, we systematically evaluate the efficacy of a range of confound regression and filtering techniques for the control of motion-induced artifacts. Results reveal that the effectiveness of preprocessing procedures on the control of motion is heterogeneous, and that improved preprocessing provides a substantial benefit beyond typical procedures. These results demonstrate that the effect of motion on rsfc-MRI can be substantially attenuated through improved preprocessing procedures, but not completely removed.

The secret lives of experiments: methods reporting in the fMRI literature

Replication of research findings is critical to the progress of scientific understanding. Accordingly, most scientific journals require authors to report experimental procedures in sufficient detail for independent researchers to replicate their work. To what extent do research reports in the functional neuroimaging literature live up to this standard? The present study evaluated methods reporting and methodological choices across 241 recent fMRI articles. Many studies did not report critical methodological details with regard to experimental design, data acquisition, and analysis. Further, many studies were underpowered to detect any but the largest statistical effects. Finally, data collection and analysis methods were highly flexible across studies, with nearly as many unique analysis pipelines as there were studies in the sample. Because the rate of false positive results is thought to increase with the flexibility of experimental designs, the field of functional neuroimaging may be particularly vulnerable to false positives. In sum, the present study documented significant gaps in methods reporting among fMRI studies. Improved methodological descriptions in research reports would yield significant benefits for the field.