Sharing neuroimaging studies of human cognition

Image harmonization: A review of statistical and deep learning methods for removing batch effects and evaluation metrics for effective harmonization

Magnetic resonance imaging and computed tomography from multiple batches (e.g. sites, scanners, datasets, etc.) are increasingly used alongside complex downstream analyses to obtain new insights into the human brain. However, significant confounding due to batch-related technical variation, called batch effects, is present in this data; direct application of downstream analyses to the data may lead to biased results. Image harmonization methods seek to remove these batch effects and enable increased generalizability and reproducibility of downstream results. In this review, we describe and categorize current approaches in statistical and deep learning harmonization methods. We also describe current evaluation metrics used to assess harmonization methods and provide a standardized framework to evaluate newly-proposed methods for effective harmonization and preservation of biological information. Finally, we provide recommendations to end-users to advocate for more effective use of current methods and to methodologists to direct future efforts and accelerate development of the field.

Highlight results, don't hide them: Enhance interpretation, reduce biases and improve reproducibility

Most neuroimaging studies display results that represent only a tiny fraction of the collected data. While it is conventional to present "only the significant results" to the reader, here we suggest that this practice has several negative consequences for both reproducibility and understanding. This practice hides away most of the results of the dataset and leads to problems of selection bias and irreproducibility, both of which have been recognized as major issues in neuroimaging studies recently. Opaque, all-or-nothing thresholding, even if well-intentioned, places undue influence on arbitrary filter values, hinders clear communication of scientific results, wastes data, is antithetical to good scientific practice, and leads to conceptual inconsistencies. It is also inconsistent with the properties of the acquired data and the underlying biology being studied. Instead of presenting only a few statistically significant locations and hiding away the remaining results, studies should "highlight" the former while also showing as much as possible of the rest. This is distinct from but complementary to utilizing data sharing repositories: the initial presentation of results has an enormous impact on the interpretation of a study. We present practical examples and extensions of this approach for voxelwise, regionwise and cross-study analyses using publicly available data that was analyzed previously by 70 teams (NARPS; Botvinik-Nezer, et al., 2020), showing that it is possible to balance the goals of displaying a full set of results with providing the reader reasonably concise and "digestible" findings. In particular, the highlighting approach sheds useful light on the kind of variability present among the NARPS teams' results, which is primarily a varied strength of agreement rather than disagreement. Using a meta-analysis built on the informative "highlighting" approach shows this relative agreement, while one using the standard "hiding" approach does not. We describe how this simple but powerful change in practice-focusing on highlighting results, rather than hiding all but the strongest ones-can help address many large concerns within the field, or at least to provide more complete information about them. We include a list of practical suggestions for results reporting to improve reproducibility, cross-study comparisons and meta-analyses.

Type 2 diabetes mellitus accelerates brain aging and cognitive decline: Complementary findings from UK Biobank and meta-analyses

Background

Type 2 diabetes mellitus (T2DM) is known to be associated with neurobiological and cognitive deficits; however, their extent, overlap with aging effects, and the effectiveness of existing treatments in the context of the brain are currently unknown.

Methods

We characterized neurocognitive effects independently associated with T2DM and age in a large cohort of human subjects from the UK Biobank with cross-sectional neuroimaging and cognitive data. We then proceeded to evaluate the extent of overlap between the effects related to T2DM and age by applying correlation measures to the separately characterized neurocognitive changes. Our findings were complemented by meta-analyses of published reports with cognitive or neuroimaging measures for T2DM and healthy controls (HCs). We also evaluated in a cohort of T2DM-diagnosed individuals using UK Biobank how disease chronicity and metformin treatment interact with the identified neurocognitive effects.

Results

The UK Biobank dataset included cognitive and neuroimaging data (N = 20,314), including 1012 T2DM and 19,302 HCs, aged between 50 and 80 years. Duration of T2DM ranged from 0 to 31 years (mean 8.5 ± 6.1 years); 498 were treated with metformin alone, while 352 were unmedicated. Our meta-analysis evaluated 34 cognitive studies (N = 22,231) and 60 neuroimaging studies: 30 of T2DM (N = 866) and 30 of aging (N = 1088). Compared to age, sex, education, and hypertension-matched HC, T2DM was associated with marked cognitive deficits, particularly in executive functioning and processing speed. Likewise, we found that the diagnosis of T2DM was significantly associated with gray matter atrophy, primarily within the ventral striatum, cerebellum, and putamen, with reorganization of brain activity (decreased in the caudate and premotor cortex and increased in the subgenual area, orbitofrontal cortex, brainstem, and posterior cingulate cortex). The structural and functional changes associated with T2DM show marked overlap with the effects correlating with age but appear earlier, with disease duration linked to more severe neurodegeneration. Metformin treatment status was not associated with improved neurocognitive outcomes.

Conclusions

The neurocognitive impact of T2DM suggests marked acceleration of normal brain aging. T2DM gray matter atrophy occurred approximately 26% ± 14% faster than seen with normal aging; disease duration was associated with increased neurodegeneration. Mechanistically, our results suggest a neurometabolic component to brain aging. Clinically, neuroimaging-based biomarkers may provide a valuable adjunctive measure of T2DM progression and treatment efficacy based on neurological effects.

Funding

The research described in this article was funded by the W. M. Keck Foundation (to LRMP), the White House Brain Research Through Advancing Innovative Technologies (BRAIN) Initiative (NSFNCS-FR 1926781 to LRMP), and the Baszucki Brain Research Fund (to LRMP). None of the funding sources played any role in the design of the experiments, data collection, analysis, interpretation of the results, the decision to publish, or any aspect relevant to the study. DJW reports serving on data monitoring committees for Novo Nordisk. None of the authors received funding or in-kind support from pharmaceutical and/or other companies to write this article.

Advanced Bioelectrical Signal Processing Methods: Past, Present and Future Approach-Part II: Brain Signals

As it was mentioned in the previous part of this work (Part I)-the advanced signal processing methods are one of the quickest and the most dynamically developing scientific areas of biomedical engineering with their increasing usage in current clinical practice. In this paper, which is a Part II work-various innovative methods for the analysis of brain bioelectrical signals were presented and compared. It also describes both classical and advanced approaches for noise contamination removal such as among the others digital adaptive and non-adaptive filtering, signal decomposition methods based on blind source separation, and wavelet transform.

Neuroimaging studies of the early stages of psychosis: A critical review

Psychiatric imaging, in particular functional imaging techniques such as functional magnetic resonance imaging (fMRI) are potentially powerful tools to explore the neurophysiological basis of the early stages of psychosis. Despite this impressive growth, neuroimaging has yet to become an established as diagnostic instrument this area, partly as a result of significant heterogeneity across the findings from research studies. The present review aims to: (i) assess the determinants of inconsistencies in the results from neuroimaging studies of the early stages of psychosis; and (ii) suggest approaches for future imaging research in this field that may reduce methodological differences between studies.

Data management and sharing in neuroimaging: Practices and perceptions of MRI researchers

Neuroimaging methods such as magnetic resonance imaging (MRI) involve complex data collection and analysis protocols, which necessitate the establishment of good research data management (RDM). Despite efforts within the field to address issues related to rigor and reproducibility, information about the RDM-related practices and perceptions of neuroimaging researchers remains largely anecdotal. To inform such efforts, we conducted an online survey of active MRI researchers that covered a range of RDM-related topics. Survey questions addressed the type(s) of data collected, tools used for data storage, organization, and analysis, and the degree to which practices are defined and standardized within a research group. Our results demonstrate that neuroimaging data is acquired in multifarious forms, transformed and analyzed using a wide variety of software tools, and that RDM practices and perceptions vary considerably both within and between research groups, with trainees reporting less consistency than faculty. Ratings of the maturity of RDM practices from ad-hoc to refined were relatively high during the data collection and analysis phases of a project and significantly lower during the data sharing phase. Perceptions of emerging practices including open access publishing and preregistration were largely positive, but demonstrated little adoption into current practice.

Prefrontoparietal dysfunction during emotion regulation in anxiety disorder: a meta-analysis of functional magnetic resonance imaging studies

OBJECTIVE

Impairments in emotion regulation, and more specifically in cognitive reappraisal, are thought to play a key role in the pathogenesis of anxiety disorders. However, the available evidence on such deficits is inconsistent. To further illustrate the neurobiological underpinnings of anxiety disorder, the present meta-analysis summarizes functional magnetic resonance imaging (fMRI) findings for cognitive reappraisal tasks and investigates related brain areas.

METHODS

We performed a comprehensive series of meta-analyses of cognitive reappraisal fMRI studies contrasting patients with anxiety disorder with healthy control (HC) subjects, employing an anisotropic effect-size signed differential mapping approach. We also conducted a subgroup analysis of medication status, anxiety disorder subtype, data-processing software, and MRI field strengths. Meta-regression was used to explore the effects of demographics and clinical characteristics. Eight studies, with 11 datasets including 219 patients with anxiety disorder and 227 HC, were identified.

RESULTS

Compared with HC, patients with anxiety disorder showed relatively decreased activation of the bilateral dorsomedial prefrontal cortex (dmPFC), bilateral dorsal anterior cingulate cortex (dACC), bilateral supplementary motor area (SMA), left ventromedial prefrontal cortex (vmPFC), bilateral parietal cortex, and left fusiform gyrus during cognitive reappraisal. The subgroup analysis, jackknife sensitivity analysis, heterogeneity analysis, and Egger's tests further confirmed these findings.

CONCLUSIONS

Impaired cognitive reappraisal in anxiety disorder may be the consequence of hypo-activation of the prefrontoparietal network, consistent with insufficient top-down control. Our findings provide robust evidence that functional impairment in prefrontoparietal neuronal circuits may have a significant role in the pathogenesis of anxiety disorder.

Resources available for autism research in the big data era: a systematic review

Recently, there has been a move encouraged by many stakeholders towards generating big, open data in many areas of research. One area where big, open data is particularly valuable is in research relating to complex heterogeneous disorders such as Autism Spectrum Disorder (ASD). The inconsistencies of findings and the great heterogeneity of ASD necessitate the use of big and open data to tackle important challenges such as understanding and defining the heterogeneity and potential subtypes of ASD. To this end, a number of initiatives have been established that aim to develop big and/or open data resources for autism research. In order to provide a useful data reference for autism researchers, a systematic search for ASD data resources was conducted using the Scopus database, the Google search engine, and the pages on 'recommended repositories' by key journals, and the findings were translated into a comprehensive list focused on ASD data. The aim of this review is to systematically search for all available ASD data resources providing the following data types: phenotypic, neuroimaging, human brain connectivity matrices, human brain statistical maps, biospecimens, and ASD participant recruitment. A total of 33 resources were found containing different types of data from varying numbers of participants. Description of the data available from each data resource, and links to each resource is provided. Moreover, key implications are addressed and underrepresented areas of data are identified.

DeID - a data sharing tool for neuroimaging studies

Funding institutions and researchers increasingly expect that data will be shared to increase scientific integrity and provide other scientists with the opportunity to use the data with novel methods that may advance understanding in a particular field of study. In practice, sharing human subject data can be complicated because data must be de-identified prior to sharing. Moreover, integrating varied data types collected in a study can be challenging and time consuming. For example, sharing data from structural imaging studies of a complex disorder requires the integration of imaging, demographic and/or behavioral data in a way that no subject identifiers are included in the de-identified dataset and with new subject labels or identification values that cannot be tracked back to the original ones. We have developed a Java program that users can use to remove identifying information in neuroimaging datasets, while still maintaining the association among different data types from the same subject for further studies. This software provides a series of user interaction wizards to allow users to select data variables to be de-identified, implements functions for auditing and validation of de-identified data, and enables the user to share the de-identified data in a single compressed package through various communication protocols, such as FTPS and SFTP. DeID runs with Windows, Linux, and Mac operating systems and its open architecture allows it to be easily adapted to support a broader array of data types, with the goal of facilitating data sharing. DeID can be obtained at http://www.nitrc.org/projects/deid.

Multi-site study of additive genetic effects on fractional anisotropy of cerebral white matter: Comparing meta and megaanalytical approaches for data pooling

Combining datasets across independent studies can boost statistical power by increasing the numbers of observations and can achieve more accurate estimates of effect sizes. This is especially important for genetic studies where a large number of observations are required to obtain sufficient power to detect and replicate genetic effects. There is a need to develop and evaluate methods for joint-analytical analyses of rich datasets collected in imaging genetics studies. The ENIGMA-DTI consortium is developing and evaluating approaches for obtaining pooled estimates of heritability through meta-and mega-genetic analytical approaches, to estimate the general additive genetic contributions to the intersubject variance in fractional anisotropy (FA) measured from diffusion tensor imaging (DTI). We used the ENIGMA-DTI data harmonization protocol for uniform processing of DTI data from multiple sites. We evaluated this protocol in five family-based cohorts providing data from a total of 2248 children and adults (ages: 9-85) collected with various imaging protocols. We used the imaging genetics analysis tool, SOLAR-Eclipse, to combine twin and family data from Dutch, Australian and Mexican-American cohorts into one large "mega-family". We showed that heritability estimates may vary from one cohort to another. We used two meta-analytical (the sample-size and standard-error weighted) approaches and a mega-genetic analysis to calculate heritability estimates across-population. We performed leave-one-out analysis of the joint estimates of heritability, removing a different cohort each time to understand the estimate variability. Overall, meta- and mega-genetic analyses of heritability produced robust estimates of heritability.

Making data sharing work: the FCP/INDI experience

Over a decade ago, the fMRI Data Center (fMRIDC) pioneered open-access data sharing in the task-based functional neuroimaging community. Well ahead of its time, the fMRIDC effort encountered logistical, sociocultural and funding barriers that impeded the field-wise instantiation of open-access data sharing. In 2009, ambitions for open-access data sharing were revived in the resting state functional MRI community in the form of two grassroots initiatives: the 1000 Functional Connectomes Project (FCP) and its successor, the International Neuroimaging Datasharing Initiative (INDI). Beyond providing open access to thousands of clinical and non-clinical imaging datasets, the FCP and INDI have demonstrated the feasibility of large-scale data aggregation for hypothesis generation and testing. Yet, the success of the FCP and INDI should not be confused with widespread embracement of open-access data sharing. Reminiscent of the challenges faced by fMRIDC, key controversies persist and include participant privacy, the role of informatics, and the logistical and cultural challenges of establishing an open science ethos. We discuss the FCP and INDI in the context of these challenges, highlighting the promise of current initiatives and suggesting solutions for possible pitfalls.

What should be the data sharing policy of cognitive science?

There is a growing chorus of voices in the scientific community calling for greater openness in the sharing of raw data that lead to a publication. In this commentary, we discuss the merits of sharing, common concerns that are raised, and practical issues that arise in developing a sharing policy. We suggest that the cognitive science community discuss the topic and establish a data-sharing policy.

The UCLA multimodal connectivity database: a web-based platform for brain connectivity matrix sharing and analysis

Brain connectomics research has rapidly expanded using functional MRI (fMRI) and diffusion-weighted MRI (dwMRI). A common product of these varied analyses is a connectivity matrix (CM). A CM stores the connection strength between any two regions (“nodes”) in a brain network. This format is useful for several reasons: (1) it is highly distilled, with minimal data size and complexity, (2) graph theory can be applied to characterize the network's topology, and (3) it retains sufficient information to capture individual differences such as age, gender, intelligence quotient (IQ), or disease state. Here we introduce the UCLA Multimodal Connectivity Database (http://umcd.humanconnectomeproject.org), an openly available website for brain network analysis and data sharing. The site is a repository for researchers to publicly share CMs derived from their data. The site also allows users to select any CM shared by another user, compute graph theoretical metrics on the site, visualize a report of results, or download the raw CM. To date, users have contributed over 2000 individual CMs, spanning different imaging modalities (fMRI, dwMRI) and disorders (Alzheimer's, autism, Attention Deficit Hyperactive Disorder). To demonstrate the site's functionality, whole brain functional and structural connectivity matrices are derived from 60 subjects' (ages 26–45) resting state fMRI (rs-fMRI) and dwMRI data and uploaded to the site. The site is utilized to derive graph theory global and regional measures for the rs-fMRI and dwMRI networks. Global and nodal graph theoretical measures between functional and structural networks exhibit low correspondence. This example demonstrates how this tool can enhance the comparability of brain networks from different imaging modalities and studies. The existence of this connectivity-based repository should foster broader data sharing and enable larger-scale meta-analyses comparing networks across imaging modality, age group, and disease state.

Publication bias in neuroimaging research: implications for meta-analyses

Neuroimaging and the neurosciences have made notable advances in sharing activation results through detailed databases, making meta-analysis of the published research faster and easier. However, the effect of publication bias in these fields has not been previously addressed or accounted for in the developed meta-analytic methods. In this article, we examine publication bias in functional magnetic resonance imaging (fMRI) for tasks involving working memory in the frontal lobes (Brodmann Areas 4, 6, 8, 9, 10, 37, 45, 46, and 47). Seventy-four studies were selected from the literature and the effect of publication bias was examined using a number of regression-based techniques. Pearson's r correlation coefficient and Cohen's d effect size estimates were computed for the activation in each study and compared to the study sample size using Egger's regression, Macaskill's regression, and the 'Trim and Fill' method. Evidence for publication bias was identified in this body of literature (p < 0.01 for each test), generally, though was neither task- nor sub-region-dependent. While we focused our analysis on this subgroup of brain mapping studies, we believe our findings generalize to the brain imaging literature as a whole and databases seeking to curate their collective results. While neuroimaging databases of summary effects are of enormous value to the community, the potential publication bias should be considered when performing meta-analyses based on database contents.

DICCCOL: dense individualized and common connectivity-based cortical landmarks

Is there a common structural and functional cortical architecture that can be quantitatively encoded and precisely reproduced across individuals and populations? This question is still largely unanswered due to the vast complexity, variability, and nonlinearity of the cerebral cortex. Here, we hypothesize that the common cortical architecture can be effectively represented by group-wise consistent structural fiber connections and take a novel data-driven approach to explore the cortical architecture. We report a dense and consistent map of 358 cortical landmarks, named Dense Individualized and Common Connectivity-based Cortical Landmarks (DICCCOLs). Each DICCCOL is defined by group-wise consistent white-matter fiber connection patterns derived from diffusion tensor imaging (DTI) data. Our results have shown that these 358 landmarks are remarkably reproducible over more than one hundred human brains and possess accurate intrinsically established structural and functional cross-subject correspondences validated by large-scale functional magnetic resonance imaging data. In particular, these 358 cortical landmarks can be accurately and efficiently predicted in a new single brain with DTI data. Thus, this set of 358 DICCCOL landmarks comprehensively encodes the common structural and functional cortical architectures, providing opportunities for many applications in brain science including mapping human brain connectomes, as demonstrated in this work.

Data sharing in neuroimaging research

Significant resources around the world have been invested in neuroimaging studies of brain function and disease. Easier access to this large body of work should have profound impact on research in cognitive neuroscience and psychiatry, leading to advances in the diagnosis and treatment of psychiatric and neurological disease. A trend toward increased sharing of neuroimaging data has emerged in recent years. Nevertheless, a number of barriers continue to impede momentum. Many researchers and institutions remain uncertain about how to share data or lack the tools and expertise to participate in data sharing. The use of electronic data capture (EDC) methods for neuroimaging greatly simplifies the task of data collection and has the potential to help standardize many aspects of data sharing. We review here the motivations for sharing neuroimaging data, the current data sharing landscape, and the sociological or technical barriers that still need to be addressed. The INCF Task Force on Neuroimaging Datasharing, in conjunction with several collaborative groups around the world, has started work on several tools to ease and eventually automate the practice of data sharing. It is hoped that such tools will allow researchers to easily share raw, processed, and derived neuroimaging data, with appropriate metadata and provenance records, and will improve the reproducibility of neuroimaging studies. By providing seamless integration of data sharing and analysis tools within a commodity research environment, the Task Force seeks to identify and minimize barriers to data sharing in the field of neuroimaging.

DICCCOL: dense individualized and common connectivity-based cortical landmarks

Is there a common structural and functional cortical architecture that can be quantitatively encoded and precisely reproduced across individuals and populations? This question is still largely unanswered due to the vast complexity, variability, and nonlinearity of the cerebral cortex. Here, we hypothesize that the common cortical architecture can be effectively represented by group-wise consistent structural fiber connections and take a novel data-driven approach to explore the cortical architecture. We report a dense and consistent map of 358 cortical landmarks, named Dense Individualized and Common Connectivity-based Cortical Landmarks (DICCCOLs). Each DICCCOL is defined by group-wise consistent white-matter fiber connection patterns derived from diffusion tensor imaging (DTI) data. Our results have shown that these 358 landmarks are remarkably reproducible over more than one hundred human brains and possess accurate intrinsically established structural and functional cross-subject correspondences validated by large-scale functional magnetic resonance imaging data. In particular, these 358 cortical landmarks can be accurately and efficiently predicted in a new single brain with DTI data. Thus, this set of 358 DICCCOL landmarks comprehensively encodes the common structural and functional cortical architectures, providing opportunities for many applications in brain science including mapping human brain connectomes, as demonstrated in this work.

Multimodality data integration in epilepsy

An important goal of software development in the medical field is the design of methods which are able to integrate information obtained from various imaging and nonimaging modalities into a cohesive framework in order to understand the results of qualitatively different measurements in a larger context. Moreover, it is essential to assess the various features of the data quantitatively so that relationships in anatomical and functional domains between complementing modalities can be expressed mathematically. This paper presents a clinically feasible software environment for the quantitative assessment of the relationship among biochemical functions as assessed by PET imaging and electrophysiological parameters derived from intracranial EEG. Based on the developed software tools, quantitative results obtained from individual modalities can be merged into a data structure allowing a consistent framework for advanced data mining techniques and 3D visualization. Moreover, an effort was made to derive quantitative variables (such as the spatial proximity index, SPI) characterizing the relationship between complementing modalities on a more generic level as a prerequisite for efficient data mining strategies. We describe the implementation of this software environment in twelve children (mean age 5.2 +/- 4.3 years) with medically intractable partial epilepsy who underwent both high-resolution structural MR and functional PET imaging. Our experiments demonstrate that our approach will lead to a better understanding of the mechanisms of epileptogenesis and might ultimately have an impact on treatment. Moreover, our software environment holds promise to be useful in many other neurological disorders, where integration of multimodality data is crucial for a better understanding of the underlying disease mechanisms.

Large-scale automated synthesis of human functional neuroimaging data

The explosive growth of the human neuroimaging literature has led to major advances in understanding of human brain function, but has also made aggregation and synthesis of neuroimaging findings increasingly difficult. Here we describe and validate an automated brain mapping framework that uses text mining, meta-analysis and machine learning techniques to generate a large database of mappings between neural and cognitive states. We demonstrate the capacity of our approach to automatically conduct large-scale, high-quality neuroimaging meta-analyses, address long-standing inferential problems in the neuroimaging literature, and support accurate ‘decoding’ of broad cognitive states from brain activity in both entire studies and individual human subjects. Collectively, our results validate a powerful and generative framework for synthesizing human neuroimaging data on an unprecedented scale.

SynapticDB, effective web-based management and sharing of data from serial section electron microscopy

Serial section electron microscopy (ssEM) is rapidly expanding as a primary tool to investigate synaptic circuitry and plasticity. The ultrastructural images collected through ssEM are content rich and their comprehensive analysis is beyond the capacity of an individual laboratory. Hence, sharing ultrastructural data is becoming crucial to visualize, analyze, and discover the structural basis of synaptic circuitry and function in the brain. We devised a web-based management system called SynapticDB (http://synapses.clm.utexas.edu/synapticdb/) that catalogues, extracts, analyzes, and shares experimental data from ssEM. The management strategy involves a library with check-in, checkout and experimental tracking mechanisms. We developed a series of spreadsheet templates (MS Excel, Open Office spreadsheet, etc) that guide users in methods of data collection, structural identification, and quantitative analysis through ssEM. SynapticDB provides flexible access to complete templates, or to individual columns with instructional headers that can be selected to create user-defined templates. New templates can also be generated and uploaded. Research progress is tracked via experimental note management and dynamic PDF forms that allow new investigators to follow standard protocols and experienced researchers to expand the range of data collected and shared. The combined use of templates and tracking notes ensures that the supporting experimental information is populated into the database and associated with the appropriate ssEM images and analyses. We anticipate that SynapticDB will serve future meta-analyses towards new discoveries about the composition and circuitry of neurons and glia, and new understanding about structural plasticity during development, behavior, learning, memory, and neuropathology.

Neurocognitive phenotypes and genetic dissection of disorders of brain and behavior

Elucidating the molecular mechanisms underlying quantitative neurocognitive phenotypes will further our understanding of the brain's structural and functional architecture and advance the diagnosis and treatment of the psychiatric disorders that these traits underlie. Although many neurocognitive traits are highly heritable, little progress has been made in identifying genetic variants unequivocally associated with these phenotypes. A major obstacle to such progress is the difficulty in identifying heritable neurocognitive measures that are precisely defined and systematically assessed and represent unambiguous mental constructs, yet are also amenable to the high-throughput phenotyping necessary to obtain adequate power for genetic association studies. In this perspective we compare the current status of genetic investigations of neurocognitive phenotypes to that of other categories of biomedically relevant traits and suggest strategies for genetically dissecting traits that may underlie disorders of brain and behavior.

Cognitive neuroscience 2.0: building a cumulative science of human brain function

Cognitive neuroscientists increasingly recognize that continued progress in understanding human brain function will require not only the acquisition of new data, but also the synthesis and integration of data across studies and laboratories. Here we review ongoing efforts to develop a more cumulative science of human brain function. We discuss the rationale for an increased focus on formal synthesis of the cognitive neuroscience literature, provide an overview of recently developed tools and platforms designed to facilitate the sharing and integration of neuroimaging data, and conclude with a discussion of several emerging developments that hold even greater promise in advancing the study of human brain function.

Is it time to re-prioritize neuroimaging databases and digital repositories?

The development of in vivo brain imaging has lead to the collection of large quantities of digital information. In any individual research article, several tens of gigabytes-worth of data may be represented-collected across normal and patient samples. With the ease of collecting such data, there is increased desire for brain imaging datasets to be openly shared through sophisticated databases. However, very often the raw and pre-processed versions of these data are not available to researchers outside of the team that collected them. A range of neuroimaging databasing approaches has streamlined the transmission, storage, and dissemination of data from such brain imaging studies. Though early sociological and technical concerns have been addressed, they have not been ameliorated altogether for many in the field. In this article, we review the progress made in neuroimaging databases, their role in data sharing, data management, potential for the construction of brain atlases, recording data provenance, and value for re-analysis, new publication, and training. We feature the LONI IDA as an example of an archive being used as a source for brain atlas workflow construction, list several instances of other successful uses of image databases, and comment on archive sustainability. Finally, we suggest that, given these developments, now is the time for the neuroimaging community to re-prioritize large-scale databases as a valuable component of brain imaging science.

Pooling FMRI data: meta-analysis, mega-analysis and multi-center studies

The quantitative analysis of pooled data from related functional magnetic resonance imaging (fMRI) experiments has the potential to significantly accelerate progress in brain mapping. Such data-pooling can be achieved through meta-analysis (the pooled analysis of published results), mega-analysis (the pooled analysis of raw data) or multi-site studies, which can be seen as designed mega-analyses. Current limitations in function-location brain mapping and how data-pooling can be used to remediate them are reviewed, with particular attention to power aggregation and mitigation of false positive results. Some recently developed analysis tools for meta- and mega-analysis are also presented, and recommendations for the conduct of valid fMRI data pooling are formulated.

Lost in localization: a minimal middle way

Commentaries by Derrfuss and Mar (Derrfuss, J., Mar, R., 2009. Lost in localization: the need for a universal coordinate database. Neuroimage.) and Nielsen (Nielsen, F.A., 2009. Lost in localization: a solution with neuroinformatics 2.0? Neuroimage.) outline the need for a universal coordinate database and some possible approaches to creating one. I highlight the issue of minimal or maximal database scope and advocate a bottom-up approach to this problem.

Science of the conscious mind

Human beings have direct access to their own mental states, but can only indirectly observe cosmic radiation and enzyme kinetics. Why then can we measure the temperature of far away galaxies and the activation constant of kinases to the third digit, yet we only gauge our happiness on a scale from 1 to 7? Here we propose a radical research paradigm shift to embrace the subjective conscious mind into the realm of objective empirical science. Key steps are the axiomatic acceptance of first-person experiences as scientific observables; the definition of a quantitative, reliable metric system based on natural language; and the careful distinction of subjective mental states (e.g., interpretation and intent) from physically measurable sensory and motor behaviors (input and output). Using this approach, we propose a series of reproducible experiments that may help define a still largely unexplored branch of science. We speculate that the development of this new discipline will be initially parallel to, and eventually converging with, neurobiology and physics.

Neuroinformatics: from bioinformatics to databasing the brain

Neuroinformatics seeks to create and maintain web-accessible databases of experimental and computational data, together with innovative software tools, essential for understanding the nervous system in its normal function and in neurological disorders. Neuroinformatics includes traditional bioinformatics of gene and protein sequences in the brain; atlases of brain anatomy and localization of genes and proteins; imaging of brain cells; brain imaging by positron emission tomography (PET), functional magnetic resonance imaging (fMRI), electroencephalography (EEG), magnetoencephalography (MEG) and other methods; many electrophysiological recording methods; and clinical neurological data, among others. Building neuroinformatics databases and tools presents difficult challenges because they span a wide range of spatial scales and types of data stored and analyzed. Traditional bioinformatics, by comparison, focuses primarily on genomic and proteomic data (which of course also presents difficult challenges). Much of bioinformatics analysis focus on sequences (DNA, RNA, and protein molecules), as the type of data that are stored, compared, and sometimes modeled. Bioinformatics is undergoing explosive growth with the addition, for example, of databases that catalog interactions between proteins, of databases that track the evolution of genes, and of systems biology databases which contain models of all aspects of organisms. This commentary briefly reviews neuroinformatics with clarification of its relationship to traditional and modern bioinformatics.

Domain-specific data sharing in neuroscience: what do we have to learn from each other?

Molecular biology and genomics have made notable strides in the sharing of primary data and resources. In other domains of neuroscience research, however, there has been resistance to adopting formalized strategies for data exchange, archiving, and availability. In this article, we discuss how neuroscience domains might follow the lead of molecular biology on what has been successful and what has failed in active data sharing. This considers not only the technical challenges but also the sociological concerns in making it possible. Though, not a pain-free process, with increased data availability, scientists from multiple fields can enjoy greater opportunity for novel discoveries about the brain in health and disease.

Provenance in neuroimaging

Provenance, the description of the history of a set of data, has grown more important with the proliferation of research consortia-related efforts in neuroimaging. Knowledge about the origin and history of an image is crucial for establishing data and results quality; detailed information about how it was processed, including the specific software routines and operating systems that were used, is necessary for proper interpretation, high fidelity replication and re-use. We have drafted a mechanism for describing provenance in a simple and easy to use environment, alleviating the burden of documentation from the user while still providing a rich description of an image's provenance. This combination of ease of use and highly descriptive metadata should greatly facilitate the collection of provenance and subsequent sharing of data.

The cell centered database project: an update on building community resources for managing and sharing 3D imaging data

Databases have become integral parts of data management, dissemination, and mining in biology. At the Second Annual Conference on Electron Tomography, held in Amsterdam in 2001, we proposed that electron tomography data should be shared in a manner analogous to structural data at the protein and sequence scales. At that time, we outlined our progress in creating a database to bring together cell level imaging data across scales, The Cell Centered Database (CCDB). The CCDB was formally launched in 2002 as an on-line repository of high-resolution 3D light and electron microscopic reconstructions of cells and subcellular structures. It contains 2D, 3D, and 4D structural and protein distribution information from confocal, multiphoton, and electron microscopy, including correlated light and electron microscopy. Many of the data sets are derived from electron tomography of cells and tissues. In the 5 years since its debut, we have moved the CCDB from a prototype to a stable resource and expanded the scope of the project to include data management and knowledge engineering. Here, we provide an update on the CCDB and how it is used by the scientific community. We also describe our work in developing additional knowledge tools, e.g., ontologies, for annotation and query of electron microscopic data.

Region of interest template for the human basal ganglia: comparing EPI and standardized space approaches

Identifying task-related activation in the basal ganglia (BG) is an important area of interest in normal motor systems and cognitive neuroscience. The purpose of this study was to compare changes in brain activation in the BG using results obtained from two different masking methods: a mask drawn in standardized space from a T1-weighted anatomical image and individual region of interest (ROI) masks drawn from each subject's echo-planar image (EPI) from different tasks with reference to the high resolution fast spin echo image of each subject. Two standardized masks were used: a mask developed in Talairach space (Basal Ganglia Human Area Template (BGHAT)) and a mask developed in Montreal Neurological Institute space (MNI mask). Ten subjects produced fingertip force pulses in five separate contraction tasks during fMRI scanning. ROIs were the left caudate, putamen, external and internal portions of the globus pallidus, and subthalamic nucleus. ANOVA revealed a similar average number of voxels in the EPI mask across tasks in each BG region. The percent signal change (PSC) was consistent within each region regardless of which mask was used. Linear regression analyses between PSC in BGHAT and EPI masks and MNI and EPI masks yielded r(2) values between 0.74-0.99 and 0.70-0.99 across regions, respectively. In conclusion, PSC in different BG ROIs can be compared across studies using these different masking methods. The masking method used does not affect the overall interpretation of results with respect to the effect of task. Use of a mask drawn in standardized space is a valid and time saving method of identifying PSC in the small nuclei of the BG.

Offering to share: how to put heads together in autism neuroimaging

Data sharing in autism neuroimaging presents scientific, technical, and social obstacles. We outline the desiderata for a data-sharing scheme that combines imaging with other measures of phenotype and with genetics, defines requirements for comparability of derived data and recommendations for raw data, outlines a core protocol including multispectral structural and diffusion-tensor imaging and optional extensions, provides for the collection of prospective, confound-free normative data, and extends sharing and collaborative development not only to data but to the analytical tools and methods applied to these data. A theme in these requirements is the need to preserve creative approaches and risk-taking within individual laboratories at the same time as common standards are provided for these laboratories to build on.

The extensible neuroimaging archive toolkit

The Extensible Neuroimaging Archive Toolkit (XNAT) is a software platform designed to facilitate common management and productivity tasks for neuroimaging and associated data. In particular, XNAT enables qualitycontrol procedures and provides secure access to and storage of data. XNAT follows a threetiered architecture that includes a data archive, user interface, and middleware engine. Data can be entered into the archive as XML or through data entry forms. Newly added data are stored in a virtual quarantine until an authorized user has validated it. XNAT subsequently maintains a history profile to track all changes made to the managed data. User access to the archive is provided by a secure web application. The web application provides a number of quality control and productivity features, including data entry forms, data-type-specific searches, searches that combine across data types, detailed reports, and listings of experimental data, upload/download tools, access to standard laboratory workflows, and administration and security tools. XNAT also includes an online image viewer that supports a number of common neuroimaging formats, including DICOM and Analyze. The viewer can be extended to support additional formats and to generate custom displays. By managing data with XNAT, laboratories are prepared to better maintain the long-term integrity of their data, to explore emergent relations across data types, and to share their data with the broader neuroimaging community.

Individual variability in brain activations associated with episodic retrieval: A role for large-scale databases

The localization of brain functions using neuroimaging techniques is commonly dependent on statistical analyses of groups of subjects in order to identify sites of activation, particularly in studies of episodic memory. Exclusive reliance on group analysis may be to the detriment of understanding the true underlying cognitive nature of brain activations. In this overview, we found that the patterns of brain activity associated with episodic retrieval are very distinct for individual subjects from the patterns of brain activity at the group level. These differences appear to go beyond the relatively small variations due to cyctoarchitectonic differences or spatial normalization. We review evidence that individual patterns of brain activity vary widely across subjects and are reliable over time despite extensive variability. We suggest that varied but reliable individual patterns of significant brain activity may be indicative of different cognitive strategies used to produce a recognition response. We argue that individual analyses in conjunction with group analyses are likely to be critical in fully understanding the relationship between retrieval processes and underlying neural systems.

Workflow-based approaches to neuroimaging analysis

Analysis of functional and structural magnetic resonance imaging (MRI) brain images requires a complex sequence of data processing steps to proceed from raw image data to the final statistical tests. Neuroimaging researchers have begun to apply workflow-based computing techniques to automate data analysis tasks. This chapter discusses eight major components of workflow management systems (WFMSs): the workflow description language, editor, task modules, data access, verification, client, engine, and provenance, and their implementation in the Fiswidgets neuroimaging workflow system. Neuroinformatics challenges involved in applying workflow techniques in the domain of neuroimaging are discussed.

A systematic review and quantitative appraisal of fMRI studies of verbal fluency: role of the left inferior frontal gyrus

The left inferior frontal gyrus (LIFG) has consistently been associated with both phonologic and semantic operations in functional neuroimaging studies. Two main theories have proposed a different functional organization in the LIFG for these processes. One theory suggests an anatomic parcellation of phonologic and semantic operations within the LIFG. An alternative theory proposes that both processes are encompassed within a supramodal executive function in a single region in the LIFG. To test these theories, we carried out a systematic review of functional magnetic resonance imaging studies employing phonologic and semantic verbal fluency tasks. Seventeen articles meeting our pre-established criteria were found, consisting of 22 relevant experiments with 197 healthy subjects and a total of 41 peak activations in the LIFG. We determined 95% confidence intervals of the mean location (x, y, and z coordinates) of peaks of blood oxygenation level-dependent (BOLD) responses from published phonologic and semantic verbal fluency studies using the nonparametric technique of bootstrap analysis. Significant differences were revealed in dorsal-ventral (z-coordinate) localizations of the peak BOLD response: phonologic verbal fluency peak BOLD response was significantly more dorsal to the peak associated with semantic verbal fluency (confidence interval of difference: 1.9-17.4 mm). No significant differences were evident in antero-posterior (x-coordinate) or medial-lateral (y-coordinate) positions. The results support distinct dorsal-ventral locations for phonologic and semantic processes within the LIFG. Current limitations to meta-analytic integration of published functional neuroimaging studies are discussed.

Mobilizing the base of neuroscience data: the case of neuronal morphologies

Despite the explosive growth of bioinformatics, data sharing has not yet become routine in neuroscience, possibly because of several broad-spanning issues, from data heterogeneity to privacy regulations. We present the case of neuronal morphology as an ideal example of shareable data. Drawing from recent experience, we argue that the tremendous research potential of existing (and largely unused) digital reconstructions should diffuse any reticence to sharing this type of data.

Skull-stripping magnetic resonance brain images using a model-based level set

The segmentation of brain tissue from nonbrain tissue in magnetic resonance (MR) images, commonly referred to as skull stripping, is an important image processing step in many neuroimage studies. A new mathematical algorithm, a model-based level set (MLS), was developed for controlling the evolution of the zero level curve that is implicitly embedded in the level set function. The evolution of the curve was controlled using two terms in the level set equation, whose values represented the forces that determined the speed of the evolving curve. The first force was derived from the mean curvature of the curve, and the second was designed to model the intensity characteristics of the cortex in MR images. The combination of these forces in a level set framework pushed or pulled the curve toward the brain surface. Quantitative evaluation of the MLS algorithm was performed by comparing the results of the MLS algorithm to those obtained using expert segmentation in 29 sets of pediatric brain MR images and 20 sets of young adult MR images. Another 48 sets of elderly adult MR images were used for qualitatively evaluating the algorithm. The MLS algorithm was also compared to two existing methods, the brain extraction tool (BET) and the brain surface extractor (BSE), using the data from the Internet brain segmentation repository (IBSR). The MLS algorithm provides robust skull-stripping results, making it a promising tool for use in large, multi-institutional, population-based neuroimaging studies.

A method for generating reproducible evidence in fMRI studies

Insights into cognitive neuroscience from neuroimaging techniques are now required to go beyond the localisation of well-known cognitive functions. Fundamental to this is the notion of reproducibility of experimental outcomes. This paper addresses the central issue that functional magnetic resonance imaging (fMRI) experiments will produce more desirable information if researchers begin to search for reproducible evidence rather than only p value significance. The study proposes a methodology for investigating reproducible evidence without conducting separate fMRI experiments. The reproducible evidence is gathered from the separate runs within the study. The associated empirical Bayes and ROC extensions of the linear model provide parameter estimates to determine reproducibility. Empirical applications of the methodology suggest that reproducible evidence is robust to small sample sizes and sensitive to both the magnitude and persistency of brain activation. It is demonstrated that research findings in fMRI studies would be more compelling with supporting reproducible evidence in addition to standard hypothesis testing evidence.

Discovery and integrative neuroscience

Hypothesis driven research has been shown to be an excellent model for pursuing investigations in neuroscience. The Human Genome Project demonstrated the added value of discovery research, especially in areas where large amounts of data are produced. Neuroscience has become a data rich field, and one that would be enhanced by incorporating the discovery approach. Databases, as well as analytical, modeling and simulation tools, will have to be developed, and they will need to be interoperable and federated. This paper presents an overview of the development of the field of neuroscience databases and associate tools: Neuroinformatics. The primary focus is on the impact of NIH funding of this process. The important issues of data sharing, as viewed from the perspective of the scientist and private and public funding organizations, are discussed. Neuroinformatics will provide more than just a sophisticated array of information technologies to help scientists understand and integrate nervous system data. It will make available powerful models of neural functions and facilitate discovery, hypothesis formulation and electronic collaboration.