1
|
Bossier H, Roels SP, Seurinck R, Banaschewski T, Barker GJ, Bokde ALW, Quinlan EB, Desrivières S, Flor H, Grigis A, Garavan H, Gowland P, Heinz A, Ittermann B, Martinot JL, Artiges E, Nees F, Orfanos DP, Poustka L, Fröhner Dipl-Psych JH, Smolka MN, Walter H, Whelan R, Schumann G, Moerkerke B. The empirical replicability of task-based fMRI as a function of sample size. Neuroimage 2020; 212:116601. [PMID: 32036019 DOI: 10.1016/j.neuroimage.2020.116601] [Citation(s) in RCA: 36] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2018] [Revised: 01/25/2020] [Accepted: 02/01/2020] [Indexed: 11/30/2022] Open
Abstract
Replicating results (i.e. obtaining consistent results using a new independent dataset) is an essential part of good science. As replicability has consequences for theories derived from empirical studies, it is of utmost importance to better understand the underlying mechanisms influencing it. A popular tool for non-invasive neuroimaging studies is functional magnetic resonance imaging (fMRI). While the effect of underpowered studies is well documented, the empirical assessment of the interplay between sample size and replicability of results for task-based fMRI studies remains limited. In this work, we extend existing work on this assessment in two ways. Firstly, we use a large database of 1400 subjects performing four types of tasks from the IMAGEN project to subsample a series of independent samples of increasing size. Secondly, replicability is evaluated using a multi-dimensional framework consisting of 3 different measures: (un)conditional test-retest reliability, coherence and stability. We demonstrate not only a positive effect of sample size, but also a trade-off between spatial resolution and replicability. When replicability is assessed voxelwise or when observing small areas of activation, a larger sample size than typically used in fMRI is required to replicate results. On the other hand, when focussing on clusters of voxels, we observe a higher replicability. In addition, we observe variability in the size of clusters of activation between experimental paradigms or contrasts of parameter estimates within these.
Collapse
Affiliation(s)
- Han Bossier
- Department of Data Analysis, Ghent University, Ghent, Belgium.
| | - Sanne P Roels
- Department of Data Analysis, Ghent University, Ghent, Belgium
| | - Ruth Seurinck
- Department of Data Analysis, Ghent University, Ghent, Belgium
| | - Tobias Banaschewski
- Department of Child and Adolescent Psychiatry and Psychotherapy, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Square J5, 68159, Mannheim, Germany
| | - Gareth J Barker
- Department of Neuroimaging, Institute of Psychiatry, Psychology & Neuroscience, King's College London, United Kingdom
| | - Arun L W Bokde
- Discipline of Psychiatry, School of Medicine and Trinity College Institute of Neuroscience, Trinity College Dublin, Ireland
| | - Erin Burke Quinlan
- Medical Research Council - Social, Genetic and Developmental Psychiatry Centre, Institute of Psychiatry, Psychology & Neuroscience, King's College London, United Kingdom
| | - Sylvane Desrivières
- Medical Research Council - Social, Genetic and Developmental Psychiatry Centre, Institute of Psychiatry, Psychology & Neuroscience, King's College London, United Kingdom
| | - Herta Flor
- Department of Cognitive and Clinical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Square J5, Mannheim, Germany; Department of Psychology, School of Social Sciences, University of Mannheim, 68131, Mannheim, Germany
| | - Antoine Grigis
- NeuroSpin, CEA, Université Paris-Saclay, F-91191, Gif-sur-Yvette, France
| | - Hugh Garavan
- Departments of Psychiatry and Psychology, University of Vermont, 05405, Burlington, VT, USA
| | - Penny Gowland
- Sir Peter Mansfield Imaging Centre School of Physics and Astronomy, University of Nottingham, University Park, Nottingham, United Kingdom
| | - Andreas Heinz
- Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of Health, Department of Psychiatry and Psychotherapy, Campus Charité Mitte, Charitéplatz 1, Berlin, Germany
| | - Bernd Ittermann
- Physikalisch-Technische Bundesanstalt (PTB), Braunschweig and Berlin, Germany
| | - Jean-Luc Martinot
- Institut National de la Santé et de la Recherche Médicale, INSERM Unit 1000 "Neuroimaging & Psychiatry", University Paris Sud, University Paris Descartes, Sorbonne Paris Cité; and Maison de Solenn, Paris, France
| | - Eric Artiges
- Institut National de la Santé et de la Recherche Médicale, INSERM Unit 1000 "Neuroimaging & Psychiatry", University Paris Sud, University Paris Descartes, Sorbonne Paris Cité; and Psychiatry Department 91G16, Orsay Hospital, France
| | - Frauke Nees
- Department of Child and Adolescent Psychiatry and Psychotherapy, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Square J5, 68159, Mannheim, Germany; Department of Cognitive and Clinical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Square J5, Mannheim, Germany
| | | | - Luise Poustka
- Department of Child and Adolescent Psychiatry and Psychotherapy, University Medical Centre Göttingen, von-Siebold-Str. 5, 37075, Göttingen, Germany
| | | | - Michael N Smolka
- Department of Psychiatry and Neuroimaging Center, Technische Universität Dresden, Dresden, Germany
| | - Henrik Walter
- Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin Institute of Health, Department of Psychiatry and Psychotherapy, Campus Charité Mitte, Charitéplatz 1, Berlin, Germany
| | - Robert Whelan
- School of Psychology and Global Brain Health Institute, Trinity College Dublin, Ireland
| | - Gunter Schumann
- Medical Research Council - Social, Genetic and Developmental Psychiatry Centre, Institute of Psychiatry, Psychology & Neuroscience, King's College London, United Kingdom
| | | | | |
Collapse
|
2
|
Lu W, Dong K, Cui D, Jiao Q, Qiu J. Quality assurance of human functional magnetic resonance imaging: a literature review. Quant Imaging Med Surg 2019; 9:1147-1162. [PMID: 31367569 DOI: 10.21037/qims.2019.04.18] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/26/2023]
Abstract
Functional magnetic resonance imaging (fMRI) has been a popular approach in brain research over the past 20 years. It offers a noninvasive method to probe the brain and uses blood oxygenation level dependent (BOLD) signal changes to access brain function. However, the BOLD signal only represents a small fraction of the total MR signal. System instability and various noise have a strong impact on the BOLD signal. Additionally, fMRI applies fast imaging technique to record brain cognitive process over time, requiring high temporal stability of MR scanners. Furthermore, data acquisition, image quality, processing, and statistical analysis methods also have a great effect on the results of fMRI studies. Quality assurance (QA) programs for fMRI can test the stability of MR scanners, evaluate the quality of fMRI and help to find errors during fMRI scanning, thereby greatly enhancing the success rate of fMRI. In this review, we focus on previous studies which developed QA programs and methods in SCI/SCIE citation peer-reviewed publications over the last 20 years, including topics on existing fMRI QA programs, QA phantoms, image QA metrics, quality evaluation of existing preprocessing pipelines and fMRI statistical analysis methods. The summarized studies were classified into four categories: QA of fMRI systems, QA of fMRI data, quality evaluation of data processing pipelines and statistical methods and QA of task-related fMRI. Summary tables and figures of QA programs and metrics have been developed based on the comprehensive review of the literature.
Collapse
Affiliation(s)
- Weizhao Lu
- Medical Engineering and Technical Center, Shandong First Medical University & Shandong Academy of Medical Sciences, Tai'an 271016, China.,Department of Radiology, Shandong First Medical University & Shandong Academy of Medical Sciences, Tai'an 271016, China
| | - Kejiang Dong
- Department of Radiology, Shandong First Medical University & Shandong Academy of Medical Sciences, Tai'an 271016, China
| | - Dong Cui
- Department of Radiology, Shandong First Medical University & Shandong Academy of Medical Sciences, Tai'an 271016, China
| | - Qing Jiao
- Department of Radiology, Shandong First Medical University & Shandong Academy of Medical Sciences, Tai'an 271016, China
| | - Jianfeng Qiu
- Medical Engineering and Technical Center, Shandong First Medical University & Shandong Academy of Medical Sciences, Tai'an 271016, China.,Department of Radiology, Shandong First Medical University & Shandong Academy of Medical Sciences, Tai'an 271016, China
| |
Collapse
|
3
|
Stevens MTR, Clarke DB, Stroink G, Beyea SD, D'Arcy RC. Improving fMRI reliability in presurgical mapping for brain tumours. J Neurol Neurosurg Psychiatry 2016; 87:267-74. [PMID: 25814491 DOI: 10.1136/jnnp-2015-310307] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/06/2015] [Accepted: 02/27/2015] [Indexed: 11/04/2022]
Abstract
PURPOSE Functional MRI (fMRI) is becoming increasingly integrated into clinical practice for presurgical mapping. Current efforts are focused on validating data quality, with reliability being a major factor. In this paper, we demonstrate the utility of a recently developed approach that uses receiver operating characteristic-reliability (ROC-r) to: (1) identify reliable versus unreliable data sets; (2) automatically select processing options to enhance data quality; and (3) automatically select individualised thresholds for activation maps. METHODS Presurgical fMRI was conducted in 16 patients undergoing surgical treatment for brain tumours. Within-session test-retest fMRI was conducted, and ROC-reliability of the patient group was compared to a previous healthy control cohort. Individually optimised preprocessing pipelines were determined to improve reliability. Spatial correspondence was assessed by comparing the fMRI results to intraoperative cortical stimulation mapping, in terms of the distance to the nearest active fMRI voxel. RESULTS The average ROC-r reliability for the patients was 0.58±0.03, as compared to 0.72±0.02 in healthy controls. For the patient group, this increased significantly to 0.65±0.02 by adopting optimised preprocessing pipelines. Co-localisation of the fMRI maps with cortical stimulation was significantly better for more reliable versus less reliable data sets (8.3±0.9 vs 29±3 mm, respectively). CONCLUSIONS We demonstrated ROC-r analysis for identifying reliable fMRI data sets, choosing optimal postprocessing pipelines, and selecting patient-specific thresholds. Data sets with higher reliability also showed closer spatial correspondence to cortical stimulation. ROC-r can thus identify poor fMRI data at time of scanning, allowing for repeat scans when necessary. ROC-r analysis provides optimised and automated fMRI processing for improved presurgical mapping.
Collapse
Affiliation(s)
- M Tynan R Stevens
- Department of Physics, Dalhousie University, Halifax, Nova Scotia, Canada Biomedical Translational Imaging Centre, IWK Health Sciences Centre, Halifax, Nova Scotia, Canada
| | - David B Clarke
- Division of Neurosurgery, QEII Health Sciences Centre, Halifax, Nova Scotia, Canada Division of Surgery, QEII Health Sciences Centre, Halifax, Nova Scotia, Canada
| | - Gerhard Stroink
- Department of Physics, Dalhousie University, Halifax, Nova Scotia, Canada
| | - Steven D Beyea
- Department of Physics, Dalhousie University, Halifax, Nova Scotia, Canada Biomedical Translational Imaging Centre, IWK Health Sciences Centre, Halifax, Nova Scotia, Canada
| | - Ryan Cn D'Arcy
- Department of Computing Science, Simon Fraser University, Burnaby, British Columbia, Canada
| |
Collapse
|
4
|
Morrison MA, Churchill NW, Cusimano MD, Schweizer TA, Das S, Graham SJ. Reliability of Task-Based fMRI for Preoperative Planning: A Test-Retest Study in Brain Tumor Patients and Healthy Controls. PLoS One 2016; 11:e0149547. [PMID: 26894279 PMCID: PMC4760755 DOI: 10.1371/journal.pone.0149547] [Citation(s) in RCA: 37] [Impact Index Per Article: 4.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2015] [Accepted: 02/02/2016] [Indexed: 11/25/2022] Open
Abstract
Background Functional magnetic resonance imaging (fMRI) continues to develop as a clinical tool for patients with brain cancer, offering data that may directly influence surgical decisions. Unfortunately, routine integration of preoperative fMRI has been limited by concerns about reliability. Many pertinent studies have been undertaken involving healthy controls, but work involving brain tumor patients has been limited. To develop fMRI fully as a clinical tool, it will be critical to examine these reliability issues among patients with brain tumors. The present work is the first to extensively characterize differences in activation map quality between brain tumor patients and healthy controls, including the effects of tumor grade and the chosen behavioral testing paradigm on reliability outcomes. Method Test-retest data were collected for a group of low-grade (n = 6) and high-grade glioma (n = 6) patients, and for matched healthy controls (n = 12), who performed motor and language tasks during a single fMRI session. Reliability was characterized by the spatial overlap and displacement of brain activity clusters, BOLD signal stability, and the laterality index. Significance testing was performed to assess differences in reliability between the patients and controls, and low-grade and high-grade patients; as well as between different fMRI testing paradigms. Results There were few significant differences in fMRI reliability measures between patients and controls. Reliability was significantly lower when comparing high-grade tumor patients to controls, or to low-grade tumor patients. The motor task produced more reliable activation patterns than the language tasks, as did the rhyming task in comparison to the phonemic fluency task. Conclusion In low-grade glioma patients, fMRI data are as reliable as healthy control subjects. For high-grade glioma patients, further investigation is required to determine the underlying causes of reduced reliability. To maximize reliability outcomes, testing paradigms should be carefully selected to generate robust activation patterns.
Collapse
Affiliation(s)
- Melanie A. Morrison
- Physical Sciences Platform, Sunnybrook Research Institute, Toronto, ON, Canada
- Department of Medical Biophysics, University of Toronto, Toronto, ON, Canada
- * E-mail:
| | | | - Michael D. Cusimano
- Keenan Research Centre, St. Michael's Hospital, Toronto, ON, Canada
- Division of Neurosurgery, St. Michael's Hospital, Toronto, ON, Canada
- Department of Surgery, University of Toronto, Toronto, ON, Canada
| | - Tom A. Schweizer
- Keenan Research Centre, St. Michael's Hospital, Toronto, ON, Canada
- Division of Neurosurgery, St. Michael's Hospital, Toronto, ON, Canada
- Department of Surgery, University of Toronto, Toronto, ON, Canada
| | - Sunit Das
- Keenan Research Centre, St. Michael's Hospital, Toronto, ON, Canada
- Division of Neurosurgery, St. Michael's Hospital, Toronto, ON, Canada
- Department of Surgery, University of Toronto, Toronto, ON, Canada
| | - Simon J. Graham
- Physical Sciences Platform, Sunnybrook Research Institute, Toronto, ON, Canada
- Department of Medical Biophysics, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
5
|
Stevens MTR, D’Arcy RC, Stroink G, Clarke DB, Beyea SD. Thresholds in fMRI studies: Reliable for single subjects? J Neurosci Methods 2013; 219:312-23. [DOI: 10.1016/j.jneumeth.2013.08.005] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2013] [Revised: 08/02/2013] [Accepted: 08/05/2013] [Indexed: 12/31/2022]
|
6
|
Liu J, Nan J, Xiong S, Li G, Qin W, Tian J. Additional evidence for the sustained effect of acupuncture at the vision-related acupuncture point, GB37. Acupunct Med 2013; 31:185-94. [PMID: 23369811 DOI: 10.1136/acupmed-2012-010251] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/30/2023]
Abstract
OBJECTIVE To investigate the dynamics underlying the sustained effect of acupuncture as a possible explanation of earlier findings that acupuncture stimulation at the vision-related acupuncture point, GB37, cannot specifically change the functional MRI (fMRI) signals of the visual cortex compared with stimulation at an adjacent non-meridian point. METHODS The 'on-off' experimental design was separated into four series conditions: 1 min of baseline scanning at the beginning, then two stimulation epochs separated by a 50 s 'rest' period, and then a 1 min 'rest' epoch. The standard General Linear Model (GLM) approach and multi-conditions analysis were used. RESULTS Results from the multi-conditions analysis were different from those from the standard GLM analysis. We found that the neural signal of the limbic-paralimbic-neocortical system after acupuncture stimulus lasted longer than the putative period. Furthermore, the fMRI signal changes in the occipital cortex showed different temporal patterns between GB37 and the non-meridian point. CONCLUSIONS Owing to the sustained effect of acupuncture, standard GLM analysis may be unsuitable for 'on-off' design acupuncture studies and lead to uncertain and contradictory results. The findings from this study suggest that acupuncture at GB37 can induce complex brain activity in the vision cortex. The state-related neural signal may reflect one of the significant characteristics underlying acupuncture.
Collapse
Affiliation(s)
- Jixin Liu
- Life Sciences Research Center, School of Life Sciences and Technology, Xidian University, Xi'an, China.
| | | | | | | | | | | |
Collapse
|
7
|
Enhancing reproducibility of fMRI statistical maps using generalized canonical correlation analysis in NPAIRS framework. Neuroimage 2012; 60:1970-81. [DOI: 10.1016/j.neuroimage.2012.01.137] [Citation(s) in RCA: 23] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2010] [Revised: 01/20/2012] [Accepted: 01/28/2012] [Indexed: 11/18/2022] Open
|
8
|
Ou W, Wells WM, Golland P. Combining spatial priors and anatomical information for fMRI detection. Med Image Anal 2010; 14:318-31. [PMID: 20362488 DOI: 10.1016/j.media.2010.02.007] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2008] [Revised: 02/07/2010] [Accepted: 02/12/2010] [Indexed: 10/19/2022]
Abstract
In this paper, we analyze Markov Random Field (MRF) as a spatial regularizer in fMRI detection. The low signal-to-noise ratio (SNR) in fMRI images presents a serious challenge for detection algorithms, making regularization necessary to achieve good detection accuracy. Gaussian smoothing, traditionally employed to boost SNR, often produces over-smoothed activation maps. Recently, the use of MRF priors has been suggested as an alternative regularization approach. However, solving for an optimal configuration of the MRF is NP-hard in general. In this work, we investigate fast inference algorithms based on the Mean Field approximation in application to MRF priors for fMRI detection. Furthermore, we propose a novel way to incorporate anatomical information into the MRF-based detection framework and into the traditional smoothing methods. Intuitively speaking, the anatomical evidence increases the likelihood of activation in the gray matter and improves spatial coherency of the resulting activation maps within each tissue type. Validation using the receiver operating characteristic (ROC) analysis and the confusion matrix analysis on simulated data illustrates substantial improvement in detection accuracy using the anatomically guided MRF spatial regularizer. We further demonstrate the potential benefits of the proposed method in real fMRI signals of reduced length. The anatomically guided MRF regularizer enables significant reduction of the scan length while maintaining the quality of the resulting activation maps.
Collapse
Affiliation(s)
- Wanmei Ou
- Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, United States.
| | | | | |
Collapse
|
9
|
Van Horn JD, Toga AW. Is it time to re-prioritize neuroimaging databases and digital repositories? Neuroimage 2009; 47:1720-34. [PMID: 19371790 PMCID: PMC2754579 DOI: 10.1016/j.neuroimage.2009.03.086] [Citation(s) in RCA: 59] [Impact Index Per Article: 3.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2008] [Revised: 03/30/2009] [Accepted: 03/31/2009] [Indexed: 11/16/2022] Open
Abstract
The development of in vivo brain imaging has lead to the collection of large quantities of digital information. In any individual research article, several tens of gigabytes-worth of data may be represented-collected across normal and patient samples. With the ease of collecting such data, there is increased desire for brain imaging datasets to be openly shared through sophisticated databases. However, very often the raw and pre-processed versions of these data are not available to researchers outside of the team that collected them. A range of neuroimaging databasing approaches has streamlined the transmission, storage, and dissemination of data from such brain imaging studies. Though early sociological and technical concerns have been addressed, they have not been ameliorated altogether for many in the field. In this article, we review the progress made in neuroimaging databases, their role in data sharing, data management, potential for the construction of brain atlases, recording data provenance, and value for re-analysis, new publication, and training. We feature the LONI IDA as an example of an archive being used as a source for brain atlas workflow construction, list several instances of other successful uses of image databases, and comment on archive sustainability. Finally, we suggest that, given these developments, now is the time for the neuroimaging community to re-prioritize large-scale databases as a valuable component of brain imaging science.
Collapse
Affiliation(s)
- John Darrell Van Horn
- Laboratory of Neuro Imaging (LONI), Department of Neurology, UCLA School of Medicine, University of California Los Angeles, 635 Charles E. Young Drive SW, Suite 225, Los Angeles, CA 90095-7334. Phone: (310) 206-2101 (voice), Fax: (310) 206-5518 (fax)
| | - Arthur W. Toga
- Laboratory of Neuro Imaging (LONI), Department of Neurology, UCLA School of Medicine, University of California Los Angeles, 635 Charles E. Young Drive SW, Suite 225, Los Angeles, CA 90095-7334. Phone: (310) 206-2101 (voice), Fax: (310) 206-5518 (fax)
| |
Collapse
|
10
|
Liou M, Su HR, Savostyanov AN, Lee JD, Aston JAD, Chuang CH, Cheng PE. Beyond p-values: Averaged and reproducible evidence in fMRI experiments. Psychophysiology 2009; 46:367-78. [DOI: 10.1111/j.1469-8986.2008.00780.x] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
11
|
Friedman L, Stern H, Brown GG, Mathalon DH, Turner J, Glover GH, Gollub RL, Lauriello J, Lim KO, Cannon T, Greve DN, Bockholt HJ, Belger A, Mueller B, Doty MJ, He J, Wells W, Smyth P, Pieper S, Kim S, Kubicki M, Vangel M, Potkin SG. Test-retest and between-site reliability in a multicenter fMRI study. Hum Brain Mapp 2008; 29:958-72. [PMID: 17636563 PMCID: PMC3670112 DOI: 10.1002/hbm.20440] [Citation(s) in RCA: 193] [Impact Index Per Article: 12.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022] Open
Abstract
In the present report, estimates of test-retest and between-site reliability of fMRI assessments were produced in the context of a multicenter fMRI reliability study (FBIRN Phase 1, www.nbirn.net). Five subjects were scanned on 10 MRI scanners on two occasions. The fMRI task was a simple block design sensorimotor task. The impulse response functions to the stimulation block were derived using an FIR-deconvolution analysis with FMRISTAT. Six functionally-derived ROIs covering the visual, auditory and motor cortices, created from a prior analysis, were used. Two dependent variables were compared: percent signal change and contrast-to-noise-ratio. Reliability was assessed with intraclass correlation coefficients derived from a variance components analysis. Test-retest reliability was high, but initially, between-site reliability was low, indicating a strong contribution from site and site-by-subject variance. However, a number of factors that can markedly improve between-site reliability were uncovered, including increasing the size of the ROIs, adjusting for smoothness differences, and inclusion of additional runs. By employing multiple steps, between-site reliability for 3T scanners was increased by 123%. Dropping one site at a time and assessing reliability can be a useful method of assessing the sensitivity of the results to particular sites. These findings should provide guidance toothers on the best practices for future multicenter studies.
Collapse
Affiliation(s)
- Lee Friedman
- Department of Psychiatry and Human Behavior, University of California Irvine, Irvine, California, USA.
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
12
|
Zhang J, Liang L, Anderson JR, Gatewood L, Rottenberg DA, Strother SC. Evaluation and comparison of GLM- and CVA-based fMRI processing pipelines with Java-based fMRI processing pipeline evaluation system. Neuroimage 2008; 41:1242-52. [PMID: 18482849 PMCID: PMC4277234 DOI: 10.1016/j.neuroimage.2008.03.034] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2007] [Revised: 03/11/2008] [Accepted: 03/17/2008] [Indexed: 10/22/2022] Open
Abstract
Activation patterns identified by fMRI processing pipelines or fMRI software packages are usually determined by the preprocessing options, parameters, and statistical models used. Previous studies that evaluated options of GLM (general linear model)--based fMRI processing pipelines are mainly based on simulated data with receiver operating characteristics (ROC) analysis, but evaluation of such fMRI processing pipelines on real fMRI data is rare. To understand the effect of processing options on performance of GLM-based fMRI processing pipelines with real fMRI data, we investigated the impact of commonly-used fMRI preprocessing steps; optimized the associated GLM-based single-subject processing pipelines; and quantitatively compared univariate GLM (in FSL.FEAT and NPAIRS.GLM) and multivariate CVA (canonical variates analysis) (in NPAIRS.CVA)-based analytic models in single-subject analysis with a recently developed fMRI processing pipeline evaluation system based on prediction accuracy (classification accuracy) and reproducibility performance metrics. For block-design data, we found that with GLM analysis (1) slice timing correction and global intensity normalization have little consistent impact on fMRI processing pipelines, spatial smoothing and high-pass filtering or temporal detrending significantly increases pipeline performance and thus are essential for robust fMRI statistical analysis; (2) combined optimization of spatial smoothing and temporal detrending improves pipeline performance; and (3) in general, the prediction performance of multivariate CVA is higher than that of the univariate GLM, while univariate GLM is more reproducible than multivariate CVA. Because of the different bias-variance trade-offs of univariate and multivariate models, it may be necessary to consider a consensus approach to obtain more accurate activation patterns in fMRI data.
Collapse
Affiliation(s)
- Jing Zhang
- Health Informatics Graduate Program, University of Minnesota, Minneapolis, MN 55455, USA.
| | | | | | | | | | | |
Collapse
|
13
|
Van Horn JD, Ishai A. Mapping the human brain: new insights from FMRI data sharing. Neuroinformatics 2008; 5:146-53. [PMID: 17917125 DOI: 10.1007/s12021-007-0011-6] [Citation(s) in RCA: 26] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/1999] [Revised: 11/30/1999] [Accepted: 11/30/1999] [Indexed: 11/29/2022]
Abstract
The sharing of primary data in the field of neuroscience has received considerable scrutiny from scientific societies and from science journals. Many see this as value added for science publishing that can enhance and inform secondary examination of data and results. Still others worry that data sharing is an undue burden for researchers with little long term value to science. But examples of how data sharing can be done successfully do exist. The fMRI Data Center, established at Dartmouth College in 2000 and now based at the University of California Santa Barbara, has worked to facilitate the open sharing of neuroimaging data from peer-reviewed papers to foster progress in cognitive science. The fMRI study on the representation of objects in the human occipital and temporal cortex, published in 2000 in the Journal of Cognitive Neuroscience (JOCN), marked the first deposition in the new database. Despite initial concerns about fMRI data sharing, this data set was frequently downloaded. We describe the original results of distributed brain activation patterns elicited by faces and objects in the human visual system, and overview several secondary analyses by independent investigators. A philosopher tested Husserl's temporal components of consciousness, whereas other brain imagers deployed new analytic tools, from Dynamic Causal Modeling, which estimates the neural interactions between cortical regions, to a novel method for constructing reproducibility maps. These re-analyses revealed new findings not reported in the original study, provided new perspectives on visual perception, generated new predictions, and resulted in new collaborations and publications in high profile journals.
Collapse
Affiliation(s)
- John Darrell Van Horn
- Laboratory of Neuro Imaging, Department of Neurology, David Geffen School of Medicine, University of California Los Angeles, 635 Charles E. Young Drive SW, Suite 225, Los Angeles, CA 90095-7334, USA.
| | | |
Collapse
|
14
|
Van De Ville D, Seghier ML, Lazeyras F, Blu T, Unser M. WSPM: Wavelet-based statistical parametric mapping. Neuroimage 2007; 37:1205-17. [PMID: 17689101 DOI: 10.1016/j.neuroimage.2007.06.011] [Citation(s) in RCA: 22] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2007] [Revised: 05/23/2007] [Accepted: 06/03/2007] [Indexed: 10/23/2022] Open
Abstract
Recently, we have introduced an integrated framework that combines wavelet-based processing with statistical testing in the spatial domain. In this paper, we propose two important enhancements of the framework. First, we revisit the underlying paradigm; i.e., that the effect of the wavelet processing can be considered as an adaptive denoising step to "improve" the parameter map, followed by a statistical detection procedure that takes into account the non-linear processing of the data. With an appropriate modification of the framework, we show that it is possible to reduce the spatial bias of the method with respect to the best linear estimate, providing conservative results that are closer to the original data. Second, we propose an extension of our earlier technique that compensates for the lack of shift-invariance of the wavelet transform. We demonstrate experimentally that both enhancements have a positive effect on performance. In particular, we present a reproducibility study for multi-session data that compares WSPM against SPM with different amounts of smoothing. The full approach is available as a toolbox, named WSPM, for the SPM2 software; it takes advantage of multiple options and features of SPM such as the general linear model.
Collapse
Affiliation(s)
- Dimitri Van De Ville
- Biomedical Imaging Group, Ecole Polytechnique Fédérale de Lausanne (EPFL), and Department of Radiology and Medical Informatics, University Hospital Geneva, Switzerland.
| | | | | | | | | |
Collapse
|
15
|
Lindquist MA, Waugh C, Wager TD. Modeling state-related fMRI activity using change-point theory. Neuroimage 2007; 35:1125-41. [DOI: 10.1016/j.neuroimage.2007.01.004] [Citation(s) in RCA: 54] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2006] [Revised: 12/19/2006] [Accepted: 01/04/2007] [Indexed: 10/23/2022] Open
|
16
|
Saxe R, Brett M, Kanwisher N. Divide and conquer: a defense of functional localizers. Neuroimage 2006; 30:1088-96; discussion 1097-9. [PMID: 16635578 DOI: 10.1016/j.neuroimage.2005.12.062] [Citation(s) in RCA: 363] [Impact Index Per Article: 20.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2005] [Revised: 12/15/2005] [Accepted: 12/22/2005] [Indexed: 11/25/2022] Open
Abstract
Numerous functionally distinct regions of cortex (e.g., V1, MT, the fusiform face area) can be easily identified in any normal human subject in just a few minutes of fMRI scanning. However, the locations of these regions vary across subjects. Investigations of these regions have therefore often used a functional region of interest (fROI) approach in which the region is first identified functionally in each subject individually, before subsequent scans in the same subjects test specific hypotheses concerning that region. This fROI method, which resembled long-established practice in visual neurophysiology, has methodological, statistical, and theoretical advantages over standard alternatives (such as whole-brain analyses of group data): (i) because functional properties are more consistently and robustly associated with fROIs than with locations in stereotaxic space, functional hypotheses concerning fROIs are often the most straightforward to frame, motivate, and test, (ii) because hypotheses are tested in only a handful of fROIs (instead of in tens of thousands of voxels), advance specification of fROIs provides a massive increase in statistical power over whole-brain analyses, and (iii) some fROIs may serve as candidate distinct components of the mind/brain worth investigation as such. Of course fROIs can be productively used in conjunction with other complementary methods. Here, we explain the motivation for and advantages of the fROI approach, and we rebut the criticism of this method offered by Friston et al. (Friston, K., Rotshtein, P., Geng, J., Sterzer, P., Henson, R., in press. A critique of functional localizers. NeuroImage).
Collapse
|