1
|
Giehl K, Mutsaerts HJ, Aarts K, Barkhof F, Caspers S, Chetelat G, Colin ME, Düzel E, Frisoni GB, Ikram MA, Jovicich J, Morbelli S, Oertel W, Paret C, Perani D, Ritter P, Segura B, Wisse LEM, De Witte E, Cappa SF, van Eimeren T. Sharing brain imaging data in the Open Science era: how and why? Lancet Digit Health 2024; 6:e526-e535. [PMID: 38906618 DOI: 10.1016/s2589-7500(24)00069-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2023] [Revised: 03/18/2024] [Accepted: 04/02/2024] [Indexed: 06/23/2024]
Abstract
The sharing of human neuroimaging data has great potential to accelerate the development of imaging biomarkers in neurological and psychiatric disorders; however, major obstacles remain in terms of how and why to share data in the Open Science context. In this Health Policy by the European Cluster for Imaging Biomarkers, we outline the current main opportunities and challenges based on the results of an online survey disseminated among senior scientists in the field. Although the scientific community fully recognises the importance of data sharing, technical, legal, and motivational aspects often prevent active adoption. Therefore, we provide practical advice on how to overcome the technical barriers. We also call for a harmonised application of the General Data Protection Regulation across EU countries. Finally, we suggest the development of a system that makes data count by recognising the generation and sharing of data as a highly valuable contribution to the community.
Collapse
Affiliation(s)
- Kathrin Giehl
- Department of Nuclear Medicine, Faculty of Medicine and University Hospital Cologne, University of Cologne, Cologne, Germany; Institute of Neurosciences and Medicine (INM-2), Research Center Jülich, Jülich, Germany
| | - Henk-Jan Mutsaerts
- Radiology and Nuclear Medicine, Vrije Universiteit Medical Center, Amsterdam University Medical Center, Amsterdam, Netherlands; Amsterdam Neuroscience, Brain Imaging, Amsterdam, Netherlands
| | | | - Frederik Barkhof
- Radiology and Nuclear Medicine, Vrije Universiteit Medical Center, Amsterdam University Medical Center, Amsterdam, Netherlands; Amsterdam Neuroscience, Brain Imaging, Amsterdam, Netherlands; Queen Square Institute of Neurology and Centre for Medical Image Computing, University College London, London, UK
| | - Svenja Caspers
- Institute of Neuroscience and Medicine (INM-1), Research Center Jülich, Jülich, Germany; Institute for Anatomy I, Medical Faculty and University Hospital Düsseldorf, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
| | - Gaël Chetelat
- Normandie université, UNICAEN, INSERM, U1237, NeuroPresage Team, Cyceron, Caen, France
| | | | - Emrah Düzel
- Faculty of Medicine, Institute for Cognitive Neurology and Dementia Research, University of Magdeburg, Magdeburg, Germany; Center for Neurodegenerative Diseases, Magdeburg, Germany
| | - Giovanni B Frisoni
- Department of Rehabilitation and Geriatrics, Memory Center, Geneva University and University Hospitals, Geneva, Switzerland
| | - M Arfan Ikram
- Department of Epidemiology, Erasmus University Medical Center, Rotterdam, Netherlands
| | - Jorge Jovicich
- Center for Mind/Brain Sciences, University of Trento, Rovereto, Italy
| | - Silvia Morbelli
- Nuclear Medicine Unit, AOU Città Della Salute e Della Scienza di Torino, Turin, Italy; Department of Medical Sciences, University of Turin, Turin, Italy
| | - Wolfgang Oertel
- European Brain Council, Brussels, Belgium; Department of Neurology, University of Marburg, Marburg, Germany
| | - Christian Paret
- Department of Psychosomatic Medicine and Psychotherapy, Central Institute of Mental Health Mannheim, Medical Faculty Mannheim/Heidelberg University, Mannheim, Germany
| | - Daniela Perani
- San Raffaele University and San Raffaele Scientific Institute, Milan, Italy
| | - Petra Ritter
- Berlin Institute of Health, Charité, Universitätsmedizin Berlin, Berlin, Germany; Department of Neurology with Experimental Neurology, Charité, Universitätsmedizin Berlin, Berlin, Germany; Bernstein Focus State Dependencies of Learning and Bernstein Center for Computational Neuroscience, Berlin, Germany; Einstein Center for Neuroscience Berlin, Berlin, Germany; Einstein Center Digital Future, Berlin, Germany
| | - Bàrbara Segura
- Medical Psychology Unit, Department of Medicine, Institute of Neurosciences, University of Barcelona, Barcelona, Spain; Hospital Clinic Foundation for Biomedical Research-August Pi i Sunyer Biomedical Research Institute, Barcelona, Spain; Biomedical Research Networking Center on Neurodegenerative Diseases Barcelona, Spain
| | - Laura E M Wisse
- Department of Clinical Sciences Lund, Lund University, Lund, Sweden
| | - Elke De Witte
- Neurosurgical Department, Erasmus University Medical Center, Rotterdam, Netherlands
| | - Stefano F Cappa
- University Institute of Advanced Studies, Pavia, Italy; IRCCS Mondino Foundation, Pavia, Italy
| | - Thilo van Eimeren
- Department of Nuclear Medicine, Faculty of Medicine and University Hospital Cologne, University of Cologne, Cologne, Germany; Department of Neurology, Faculty of Medicine and University Hospital Cologne, University of Cologne, Cologne, Germany.
| |
Collapse
|
2
|
Weinerova J, Botvinik-Nezer R, Tibon R. Five creative ways to promote reproducible science. Nat Hum Behav 2024; 8:411-413. [PMID: 38287175 DOI: 10.1038/s41562-023-01808-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/31/2024]
Affiliation(s)
| | - Rotem Botvinik-Nezer
- Department of Psychology, the Hebrew University of Jerusalem, Jerusalem, Israel
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA
| | - Roni Tibon
- School of Psychology, University of Nottingham, Nottingham, UK
| |
Collapse
|
3
|
Botvinik-Nezer R, Wager TD. Reproducibility in Neuroimaging Analysis: Challenges and Solutions. BIOLOGICAL PSYCHIATRY. COGNITIVE NEUROSCIENCE AND NEUROIMAGING 2023; 8:780-788. [PMID: 36906444 DOI: 10.1016/j.bpsc.2022.12.006] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/15/2022] [Revised: 11/27/2022] [Accepted: 12/11/2022] [Indexed: 12/23/2022]
Abstract
Recent years have marked a renaissance in efforts to increase research reproducibility in psychology, neuroscience, and related fields. Reproducibility is the cornerstone of a solid foundation of fundamental research-one that will support new theories built on valid findings and technological innovation that works. The increased focus on reproducibility has made the barriers to it increasingly apparent, along with the development of new tools and practices to overcome these barriers. Here, we review challenges, solutions, and emerging best practices with a particular emphasis on neuroimaging studies. We distinguish 3 main types of reproducibility, discussing each in turn. Analytical reproducibility is the ability to reproduce findings using the same data and methods. Replicability is the ability to find an effect in new datasets, using the same or similar methods. Finally, robustness to analytical variability refers to the ability to identify a finding consistently across variation in methods. The incorporation of these tools and practices will result in more reproducible, replicable, and robust psychological and brain research and a stronger scientific foundation across fields of inquiry.
Collapse
Affiliation(s)
- Rotem Botvinik-Nezer
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, New Hampshire.
| | - Tor D Wager
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, New Hampshire
| |
Collapse
|
4
|
El Amin M, Borders JC, Long HL, Keller MA, Kearney E. Open Science Practices in Communication Sciences and Disorders: A Survey. JOURNAL OF SPEECH, LANGUAGE, AND HEARING RESEARCH : JSLHR 2023; 66:1928-1947. [PMID: 36417765 PMCID: PMC10554559 DOI: 10.1044/2022_jslhr-22-00062] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/27/2022] [Revised: 07/15/2022] [Accepted: 08/07/2022] [Indexed: 06/16/2023]
Abstract
PURPOSE Open science is a collection of practices that seek to improve the accessibility, transparency, and replicability of science. Although these practices have garnered interest in related fields, it remains unclear whether open science practices have been adopted in the field of communication sciences and disorders (CSD). This study aimed to survey the knowledge, implementation, and perceived benefits and barriers of open science practices in CSD. METHOD An online survey was disseminated to researchers in the United States actively engaged in CSD research. Four-core open science practices were examined: preregistration, self-archiving, gold open access, and open data. Data were analyzed using descriptive statistics and regression models. RESULTS Two hundred twenty-two participants met the inclusion criteria. Most participants were doctoral students (38%) or assistant professors (24%) at R1 institutions (58%). Participants reported low knowledge of preregistration and gold open access. There was, however, a high level of desire to learn more for all practices. Implementation of open science practices was also low, most notably for preregistration, gold open access, and open data (< 25%). Predictors of knowledge and participation, as well as perceived barriers to implementation, are discussed. CONCLUSION Although participation in open science appears low in the field of CSD, participants expressed a strong desire to learn more in order to engage in these practices in the future. Supplemental Material and Open Science Form: https://doi.org/10.23641/asha.21569040.
Collapse
Affiliation(s)
- Mariam El Amin
- Communication Sciences and Disorders, University of Georgia, Athens
| | - James C. Borders
- Department of Biobehavioral Sciences, Teacher College, Columbia University, New York, NY
| | | | | | - Elaine Kearney
- Department of Speech, Language and Hearing Sciences, Boston University, MA
| |
Collapse
|
5
|
Schroeder PA, Artemenko C, Kosie JE, Cockx H, Stute K, Pereira J, Klein F, Mehler DMA. Using preregistration as a tool for transparent fNIRS study design. NEUROPHOTONICS 2023; 10:023515. [PMID: 36908680 PMCID: PMC9993433 DOI: 10.1117/1.nph.10.2.023515] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/09/2022] [Accepted: 01/11/2023] [Indexed: 05/04/2023]
Abstract
Significance The expansion of functional near-infrared spectroscopy (fNIRS) methodology and analysis tools gives rise to various design and analytical decisions that researchers have to make. Several recent efforts have developed guidelines for preprocessing, analyzing, and reporting practices. For the planning stage of fNIRS studies, similar guidance is desirable. Study preregistration helps researchers to transparently document study protocols before conducting the study, including materials, methods, and analyses, and thus, others to verify, understand, and reproduce a study. Preregistration can thus serve as a useful tool for transparent, careful, and comprehensive fNIRS study design. Aim We aim to create a guide on the design and analysis steps involved in fNIRS studies and to provide a preregistration template specified for fNIRS studies. Approach The presented preregistration guide has a strong focus on fNIRS specific requirements, and the associated template provides examples based on continuous-wave (CW) fNIRS studies conducted in humans. These can, however, be extended to other types of fNIRS studies. Results On a step-by-step basis, we walk the fNIRS user through key methodological and analysis-related aspects central to a comprehensive fNIRS study design. These include items specific to the design of CW, task-based fNIRS studies, but also sections that are of general importance, including an in-depth elaboration on sample size planning. Conclusions Our guide introduces these open science tools to the fNIRS community, providing researchers with an overview of key design aspects and specification recommendations for comprehensive study planning. As such it can be used as a template to preregister fNIRS studies or merely as a tool for transparent fNIRS study design.
Collapse
Affiliation(s)
- Philipp A. Schroeder
- University of Tuebingen, Department of Psychology, Faculty of Science, Tuebingen, Germany
| | - Christina Artemenko
- University of Tuebingen, Department of Psychology, Faculty of Science, Tuebingen, Germany
| | - Jessica E. Kosie
- Princeton University, Social and Natural Sciences, Department of Psychology, Princeton, New Jersey, United States
| | - Helena Cockx
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Biophysics Department, Faculty of Science, Nijmegen, The Netherlands
| | - Katharina Stute
- Chemnitz University of Technology, Institute of Human Movement Science and Health, Faculty of Behavioural and Social Sciences, Chemnitz, Germany
| | - João Pereira
- University of Coimbra, Coimbra Institute for Biomedical Imaging and Translational Research, Coimbra, Portugal
| | - Franziska Klein
- University of Oldenburg, Department of Psychology, Neurocognition and functional Neurorehabilitation Group, Oldenburg (Oldb), Germany
- RWTH Aachen University, Medical School, Department of Psychiatry, Psychotherapy and Psychosomatics, Aachen, Germany
| | - David M. A. Mehler
- RWTH Aachen University, Medical School, Department of Psychiatry, Psychotherapy and Psychosomatics, Aachen, Germany
- University of Münster, Institute for Translational Psychiatry, Medical School, Münster, Germany
| |
Collapse
|
6
|
Reer A, Wiebe A, Wang X, Rieger JW. FAIR human neuroscientific data sharing to advance AI driven research and applications: Legal frameworks and missing metadata standards. Front Genet 2023; 14:1086802. [PMID: 37007976 PMCID: PMC10065194 DOI: 10.3389/fgene.2023.1086802] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2022] [Accepted: 02/21/2023] [Indexed: 03/16/2023] Open
Abstract
Modern AI supported research holds many promises for basic and applied science. However, the application of AI methods is often limited because most labs cannot, on their own, acquire large and diverse datasets, which are best for training these methods. Data sharing and open science initiatives promise some relief to the problem, but only if the data are provided in a usable way. The FAIR principles state very general requirements for useful data sharing: they should be findable, accessible, interoperable, and reusable. This article will focus on two challenges to implement the FAIR framework for human neuroscience data. On the one hand, human data can fall under special legal protection. The legal frameworks regulating how and what data can be openly shared differ greatly across countries which can complicate data sharing or even discourage researchers from doing so. Moreover, openly accessible data require standardization of data and metadata organization and annotation in order to become interpretable and useful. This article briefly introduces open neuroscience initiatives that support the implementation of the FAIR principles. It then reviews legal frameworks, their consequences for accessibility of human neuroscientific data and some ethical implications. We hope this comparison of legal jurisdictions helps to elucidate that some alleged obstacles for data sharing only require an adaptation of procedures but help to protect the privacy of our most generous donors to research … our study participants. Finally, it elaborates on the problem of missing standards for metadata annotation and introduces initiatives that aim at developing tools to make neuroscientific data acquisition and analysis pipelines FAIR by design. While the paper focuses on making human neuroscience data useful for data-intensive AI the general considerations hold for other fields where large amounts of openly available human data would be helpful.
Collapse
Affiliation(s)
- Aaron Reer
- Applied Neurocognitive Psychology Lab, Institute for Medicine and Healthcare, Department of Psychology, Oldenburg University, Oldenburg, Germany
- *Correspondence: Aaron Reer,
| | - Andreas Wiebe
- Chair for Intellectual Property and Information Law, Göttingen University, Göttingen, Germany
| | - Xu Wang
- Chair for Intellectual Property and Information Law, Göttingen University, Göttingen, Germany
| | - Jochem W. Rieger
- Applied Neurocognitive Psychology Lab, Institute for Medicine and Healthcare, Department of Psychology, Oldenburg University, Oldenburg, Germany
| |
Collapse
|
7
|
Priestley DR, Staph J, Koneru SD, Rajtmajer SM, Cwiek A, Vervoordt S, Hillary FG. Establishing ground truth in the traumatic brain injury literature: if replication is the answer, then what are the questions? Brain Commun 2022; 5:fcac322. [PMID: 36601624 PMCID: PMC9806718 DOI: 10.1093/braincomms/fcac322] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2022] [Revised: 09/09/2022] [Accepted: 12/06/2022] [Indexed: 12/13/2022] Open
Abstract
The replication crisis poses important challenges to modern science. Central to this challenge is re-establishing ground truths or the most fundamental theories that serve as the bedrock to a scientific community. However, the goal to identify hypotheses with the greatest support is non-trivial given the unprecedented rate of scientific publishing. In this era of high-volume science, the goal of this study is to sample from one research community within clinical neuroscience (traumatic brain injury) and track major trends that have shaped this literature over the past 50 years. To do so, we first conduct a decade-wise (1980-2019) network analysis to examine the scientific communities that shape this literature. To establish the robustness of our findings, we utilized searches from separate search engines (Web of Science; Semantic Scholar). As a second goal, we sought to determine the most highly cited hypotheses influencing the literature in each decade. In a third goal, we then searched for any papers referring to 'replication' or efforts to reproduce findings within our >50 000 paper dataset. From this search, 550 papers were analysed to determine the frequency and nature of formal replication studies over time. Finally, to maximize transparency, we provide a detailed procedure for the creation and analysis of our dataset, including a discussion of each of our major decision points, to facilitate similar efforts in other areas of neuroscience. We found that the unparalleled rate of scientific publishing within the brain injury literature combined with the scarcity of clear hypotheses in individual publications is a challenge to both evaluating accepted findings and determining paths forward to accelerate science. Additionally, while the conversation about reproducibility has increased over the past decade, the rate of published replication studies continues to be a negligible proportion of the research. Meta-science and computational methods offer the critical opportunity to assess the state of the science and illuminate pathways forward, but ultimately there is structural change needed in the brain injury literature and perhaps others.
Collapse
Affiliation(s)
| | | | - Sai D Koneru
- College of Information Sciences and Technology, Penn State University, University Park, PA 16802, USA
| | - Sarah M Rajtmajer
- College of Information Sciences and Technology, Penn State University, University Park, PA 16802, USA
| | - Andrew Cwiek
- Department of Psychology, Penn State University, University Park, PA 16802, USA
| | - Samantha Vervoordt
- Department of Psychology, Penn State University, University Park, PA 16802, USA
| | - Frank G Hillary
- Correspondence to: Frank G. Hillary Professor of Psychology 313 Bruce V. Moore Building, University Park, PA, USA E-mail:
| |
Collapse
|
8
|
Bush KA, Calvert ML, Kilts CD. Lessons learned: A neuroimaging research center's transition to open and reproducible science. Front Big Data 2022; 5:988084. [PMID: 36105538 PMCID: PMC9464934 DOI: 10.3389/fdata.2022.988084] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2022] [Accepted: 08/10/2022] [Indexed: 11/13/2022] Open
Abstract
Human functional neuroimaging has evolved dramatically in recent years, driven by increased technical complexity and emerging evidence that functional neuroimaging findings are not generally reproducible. In response to these trends, neuroimaging scientists have developed principles, practices, and tools to both manage this complexity as well as to enhance the rigor and reproducibility of neuroimaging science. We group these best practices under four categories: experiment pre-registration, FAIR data principles, reproducible neuroimaging analyses, and open science. While there is growing recognition of the need to implement these best practices there exists little practical guidance of how to accomplish this goal. In this work, we describe lessons learned from efforts to adopt these best practices within the Brain Imaging Research Center at the University of Arkansas for Medical Sciences over 4 years (July 2018-May 2022). We provide a brief summary of the four categories of best practices. We then describe our center's scientific workflow (from hypothesis formulation to result reporting) and detail how each element of this workflow maps onto these four categories. We also provide specific examples of practices or tools that support this mapping process. Finally, we offer a roadmap for the stepwise adoption of these practices, providing recommendations of why and what to do as well as a summary of cost-benefit tradeoffs for each step of the transition.
Collapse
Affiliation(s)
- Keith A. Bush
- Department of Psychiatry, Brain Imaging Research Center, University of Arkansas for Medical Sciences, Little Rock, AR, United States
| | | | | |
Collapse
|
9
|
Paret C, Unverhau N, Feingold F, Poldrack RA, Stirner M, Schmahl C, Sicorello M. Survey on Open Science Practices in Functional Neuroimaging. Neuroimage 2022; 257:119306. [PMID: 35595201 DOI: 10.1016/j.neuroimage.2022.119306] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2021] [Revised: 05/03/2022] [Accepted: 05/10/2022] [Indexed: 12/12/2022] Open
Abstract
Replicability and reproducibility of scientific findings is paramount for sustainable progress in neuroscience. Preregistration of the hypotheses and methods of an empirical study before analysis, the sharing of primary research data, and compliance with data standards such as the Brain Imaging Data Structure (BIDS), are considered effective practices to secure progress and to substantiate quality of research. We investigated the current level of adoption of open science practices in neuroimaging and the difficulties that prevent researchers from using them. Email invitations to participate in the survey were sent to addresses received through a PubMed search of human functional magnetic resonance imaging studies that were published between 2010 and 2020. 283 persons completed the questionnaire. Although half of the participants were experienced with preregistration, the willingness to preregister studies in the future was modest. The majority of participants had experience with the sharing of primary neuroimaging data. Most of the participants were interested in implementing a standardized data structure such as BIDS in their labs. Based on demographic variables, we compared participants on seven subscales, which had been generated through factor analysis. Exploratory analyses found that experienced researchers at lower career level had higher fear of being transparent and researchers with residence in the EU had a higher need for data governance. Additionally, researchers at medical faculties as compared to other university faculties reported a more unsupportive supervisor with regards to open science practices and a higher need for data governance. The results suggest growing adoption of open science practices but also highlight a number of important impediments.
Collapse
Affiliation(s)
- Christian Paret
- Department of Psychosomatic Medicine and Psychotherapy, Central Institute of Mental Health Mannheim, Medical Faculty Mannheim / Heidelberg University, Germany; Sagol Brain Institute, Wohl Institute for Advanced Imaging, Tel-Aviv Sourasky Medical Center and School of Psychological Sciences, Tel-Aviv University, Israel.
| | - Nike Unverhau
- Department of Psychosomatic Medicine and Psychotherapy, Central Institute of Mental Health Mannheim, Medical Faculty Mannheim / Heidelberg University, Germany
| | | | | | - Madita Stirner
- Department of Psychosomatic Medicine and Psychotherapy, Central Institute of Mental Health Mannheim, Medical Faculty Mannheim / Heidelberg University, Germany
| | - Christian Schmahl
- Department of Psychosomatic Medicine and Psychotherapy, Central Institute of Mental Health Mannheim, Medical Faculty Mannheim / Heidelberg University, Germany
| | - Maurizio Sicorello
- Department of Psychosomatic Medicine and Psychotherapy, Central Institute of Mental Health Mannheim, Medical Faculty Mannheim / Heidelberg University, Germany
| |
Collapse
|