1
|
Sandoval-Lentisco A, López-Nicolás R, Tortajada M, López-López JA, Sánchez-Meca J. Transparency in Cognitive Training Meta-analyses: A Meta-review. Neuropsychol Rev 2024:10.1007/s11065-024-09638-2. [PMID: 38639881 DOI: 10.1007/s11065-024-09638-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2022] [Accepted: 03/10/2024] [Indexed: 04/20/2024]
Abstract
Meta-analyses often present flexibility regarding their inclusion criteria, outcomes of interest, statistical analyses, and assessments of the primary studies. For this reason, it is necessary to transparently report all the information that could impact the results. In this meta-review, we aimed to assess the transparency of meta-analyses that examined the benefits of cognitive training, given the ongoing controversy that exists in this field. Ninety-seven meta-analytic reviews were included, which examined a wide range of populations with different clinical conditions and ages. Regarding the reporting, information about the search of the studies, screening procedure, or data collection was detailed by most reviews. However, authors usually failed to report other aspects such as the specific meta-analytic parameters, the formula used to compute the effect sizes, or the data from primary studies that were used to compute the effect sizes. Although some of these practices have improved over the years, others remained the same. Moreover, examining the eligibility criteria of the reviews revealed a great heterogeneity in aspects such as the training duration, age cut-offs, or study designs that were considered. Preregistered meta-analyses often specified poorly how they would deal with the multiplicity of data or assess publication bias in their protocols, and some contained non-disclosed deviations in their eligibility criteria or outcomes of interests. The findings shown here, although they do not question the benefits of cognitive training, illustrate important aspects that future reviews must consider.
Collapse
Affiliation(s)
| | - Rubén López-Nicolás
- Department Basic Psychology and Methodology, University of Murcia, Murcia, Spain
| | - Miriam Tortajada
- Department Basic Psychology and Methodology, University of Murcia, Murcia, Spain
| | | | - Julio Sánchez-Meca
- Department Basic Psychology and Methodology, University of Murcia, Murcia, Spain
| |
Collapse
|
2
|
Black L, Farzinnia R, Humphrey N, Marquez J. Variation in global network properties across risk factors for adolescent internalizing symptoms: evidence of cumulative effects on structure and connectivity. Psychol Med 2024; 54:687-697. [PMID: 37772485 DOI: 10.1017/s0033291723002362] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 09/30/2023]
Abstract
BACKGROUND Identifying adolescents at risk of internalizing problems is a key priority. However, studies have tended to consider such problems in simple ways using diagnoses, or item summaries. Network theory and methods instead allow for more complex interaction between symptoms. Two key hypotheses predict differences in global network properties for those at risk: altered structure and increased connectivity. METHODS The current study evaluated these hypotheses for nine risk factors (e.g. income deprivation and low parent/carer support) individually and cumulatively in a large sample of 12-15 year-olds (N = 34 564). Recursive partitioning and bootstrapped networks were used to evaluate structural and connectivity differences. RESULTS The pattern of network interactions was shown to be significantly different via recursive partitioning for all comparisons across risk-present/absent groups and levels of cumulative risk, except for income deprivation. However, the magnitude of differences appeared small. Most individual risk factors also showed relatively small effects for connectivity. Exceptions were noted for gender and sexual minority risk groups, as well as low parent/carer support, where larger effects were evident. A strong linear trend was observed between increasing cumulative risk exposure and connectivity. CONCLUSIONS A robust approach to considering the effect of risk exposure on global network properties was demonstrated. Results are consistent with the ideas that pathological states are associated with higher connectivity, and that the number of risks, regardless of their nature, is important. Gender/sexual minority status and low parent/carer support had the biggest individual impacts on connectivity, suggesting these are particularly important for identification and prevention.
Collapse
Affiliation(s)
- Louise Black
- Manchester Institute of Education, University of Manchester, Manchester, UK
| | - Reihaneh Farzinnia
- Manchester Institute of Education, University of Manchester, Manchester, UK
| | - Neil Humphrey
- Manchester Institute of Education, University of Manchester, Manchester, UK
| | - Jose Marquez
- Manchester Institute of Education, University of Manchester, Manchester, UK
| |
Collapse
|
3
|
Zulfiqar M, Crusoe MR, König-Ries B, Steinbeck C, Peters K, Gadelha L. Implementation of FAIR Practices in Computational Metabolomics Workflows-A Case Study. Metabolites 2024; 14:118. [PMID: 38393009 PMCID: PMC10891576 DOI: 10.3390/metabo14020118] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2023] [Revised: 01/30/2024] [Accepted: 02/07/2024] [Indexed: 02/25/2024] Open
Abstract
Scientific workflows facilitate the automation of data analysis tasks by integrating various software and tools executed in a particular order. To enable transparency and reusability in workflows, it is essential to implement the FAIR principles. Here, we describe our experiences implementing the FAIR principles for metabolomics workflows using the Metabolome Annotation Workflow (MAW) as a case study. MAW is specified using the Common Workflow Language (CWL), allowing for the subsequent execution of the workflow on different workflow engines. MAW is registered using a CWL description on WorkflowHub. During the submission process on WorkflowHub, a CWL description is used for packaging MAW using the Workflow RO-Crate profile, which includes metadata in Bioschemas. Researchers can use this narrative discussion as a guideline to commence using FAIR practices for their bioinformatics or cheminformatics workflows while incorporating necessary amendments specific to their research area.
Collapse
Affiliation(s)
- Mahnoor Zulfiqar
- Institute for Inorganic and Analytical Chemistry, Friedrich Schiller University Jena, 07743 Jena, Germany;
- Cluster of Excellence Balance of the Microverse, Friedrich Schiller University Jena, 07743 Jena, Germany;
| | - Michael R. Crusoe
- ELIXIR (The European Life-Sciences Infrastructure for Biological Information) Germany, Institute of Bio- and Geosciences (IBG-5)—Computational Metagenomics, Forschungszentrum Jülich GmbH, 52428 Jülich, Germany;
| | - Birgitta König-Ries
- Cluster of Excellence Balance of the Microverse, Friedrich Schiller University Jena, 07743 Jena, Germany;
- Institute for Informatics, Friedrich Schiller University Jena, 07743 Jena, Germany
- iDiv—German Centre for Integrative Biodiversity Research, Halle-Jena-Leipzig, 04103 Leipzig, Germany;
| | - Christoph Steinbeck
- Institute for Inorganic and Analytical Chemistry, Friedrich Schiller University Jena, 07743 Jena, Germany;
- Cluster of Excellence Balance of the Microverse, Friedrich Schiller University Jena, 07743 Jena, Germany;
| | - Kristian Peters
- iDiv—German Centre for Integrative Biodiversity Research, Halle-Jena-Leipzig, 04103 Leipzig, Germany;
- Geobotany and Botanical Gardens, Martin-Luther University of Halle-Wittenberg, 06108 Halle, Germany
- Leibniz Institute of Plant Biochemistry, 06120 Halle, Germany
| | - Luiz Gadelha
- Institute for Inorganic and Analytical Chemistry, Friedrich Schiller University Jena, 07743 Jena, Germany;
- Cluster of Excellence Balance of the Microverse, Friedrich Schiller University Jena, 07743 Jena, Germany;
- Institute for Informatics, Friedrich Schiller University Jena, 07743 Jena, Germany
- German Cancer Research Center (DKFZ), 69120 Heidelberg, Germany
| |
Collapse
|
4
|
Hodges CB, Stone BM, Johnson PK, Carter JH, Sawyers CK, Roby PR, Lindsey HM. Researcher degrees of freedom in statistical software contribute to unreliable results: A comparison of nonparametric analyses conducted in SPSS, SAS, Stata, and R. Behav Res Methods 2023; 55:2813-2837. [PMID: 35953660 DOI: 10.3758/s13428-022-01932-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/11/2022] [Indexed: 11/08/2022]
Abstract
Researcher degrees of freedom can affect the results of hypothesis tests and consequently, the conclusions drawn from the data. Previous research has documented variability in accuracy, speed, and documentation of output across various statistical software packages. In the current investigation, we conducted Pearson's chi-square test of independence, Spearman's rank-ordered correlation, Kruskal-Wallis one-way analysis of variance, Wilcoxon Mann-Whitney U rank-sum tests, and Wilcoxon signed-rank tests, along with estimates of skewness and kurtosis, on large, medium, and small samples of real and simulated data in SPSS, SAS, Stata, and R and compared the results with those obtained through hand calculation using the raw computational formulas. Multiple inconsistencies were found in the results produced between statistical packages due to algorithmic variation, computational error, and statistical output. The most notable inconsistencies were due to algorithmic variations in the computation of Pearson's chi-square test conducted on 2 × 2 tables, where differences in p-values reported by different software packages ranged from .005 to .162, largely as a function of sample size. We discuss how such inconsistencies may influence the conclusions drawn from the results of statistical analyses depending on the statistical software used, and we urge researchers to analyze their data across multiple packages to check for inconsistencies and report details regarding the statistical procedure used for data analysis.
Collapse
Affiliation(s)
- Cooper B Hodges
- Department of Neurology, University of Utah School of Medicine, Salt Lake City, UT, USA
- Department of Psychology, Brigham Young University, Provo, UT, USA
- Department of Physical Medicine and Rehabilitation, Virginia Commonwealth University, Richmond, VA, USA
- School of Social and Behavioral Sciences, Andrews University, Berrien Springs, MI, USA
| | - Bryant M Stone
- Department of Psychology, Southern Illinois University, Carbondale, 1125 Lincoln Drive, Carbondale, IL, 62901, USA.
| | - Paula K Johnson
- Department of Neurology, University of Utah School of Medicine, Salt Lake City, UT, USA
- Neuroscience Center, Brigham Young University, Provo, UT, USA
| | - James H Carter
- Department of Psychology, Stanford University, Stanford, CA, USA
| | - Chelsea K Sawyers
- Department of Psychiatry, Virginia Commonwealth University, Richmond, VA, USA
| | - Patricia R Roby
- Center for Injury Research and Prevention, Children's Hospital of Philadelphia, Philadelphia, PA, USA
| | - Hannah M Lindsey
- Department of Neurology, University of Utah School of Medicine, Salt Lake City, UT, USA
- Department of Psychology, Brigham Young University, Provo, UT, USA
| |
Collapse
|
5
|
Krämer RJ, Koch M, Levacher J, Schmitz F. Testing Replicability and Generalizability of the Time on Task Effect. J Intell 2023; 11:jintelligence11050082. [PMID: 37233332 DOI: 10.3390/jintelligence11050082] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2022] [Revised: 04/24/2023] [Accepted: 04/25/2023] [Indexed: 05/27/2023] Open
Abstract
The time on task (ToT) effect describes the relationship of the time spent on a cognitive task and the probability of successful task completion. The effect has been shown to vary in size and direction across tests and even within tests, depending on the test taker and item characteristics. Specifically, investing more time has a positive effect on response accuracy for difficult items and low ability test-takers, but a negative effect for easy items and high ability test-takers. The present study sought to test the replicability of this result pattern of the ToT effect across samples independently drawn from the same populations of persons and items. Furthermore, its generalizability was tested in terms of differential correlations across ability tests. To this end, ToT effects were estimated for three different reasoning tests and one test measuring natural sciences knowledge in 10 comparable subsamples with a total N = 2640. Results for the subsamples were highly similar, demonstrating that ToT effects are estimated with sufficient reliability. Generally, faster answers tended to be more accurate, suggesting a relatively effortless processing style. However, with increasing item difficulty and decreasing person ability, the effect flipped to the opposite direction, i.e., higher accuracy with longer processing times. The within-task moderation of the ToT effect can be reconciled with an account on effortful processing or cognitive load. By contrast, the generalizability of the ToT effect across different tests was only moderate. Cross-test relations were stronger in relative terms if performance in the respective tasks was more strongly related. This suggests that individual differences in the ToT effect depend on test characteristics such as their reliabilities but also similarities and differences of their processing requirements.
Collapse
Affiliation(s)
- Raimund J Krämer
- Department of Psychology, University of Duisburg-Essen, Universitätsstraße 2, 45141 Essen, Germany
| | - Marco Koch
- Individual Differences & Psychodiagnostics, Saarland University, Campus A1.3, 66123 Saarbrücken, Germany
| | - Julie Levacher
- Individual Differences & Psychodiagnostics, Saarland University, Campus A1.3, 66123 Saarbrücken, Germany
| | - Florian Schmitz
- Department of Psychology, University of Duisburg-Essen, Universitätsstraße 2, 45141 Essen, Germany
| |
Collapse
|
6
|
Bringmann LF, Albers C, Bockting C, Borsboom D, Ceulemans E, Cramer A, Epskamp S, Eronen MI, Hamaker E, Kuppens P, Lutz W, McNally RJ, Molenaar P, Tio P, Voelkle MC, Wichers M. Psychopathological networks: Theory, methods and practice. Behav Res Ther 2021; 149:104011. [PMID: 34998034 DOI: 10.1016/j.brat.2021.104011] [Citation(s) in RCA: 62] [Impact Index Per Article: 20.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2021] [Revised: 11/05/2021] [Accepted: 11/27/2021] [Indexed: 12/19/2022]
Abstract
In recent years, network approaches to psychopathology have sparked much debate and have had a significant impact on how mental disorders are perceived in the field of clinical psychology. However, there are many important challenges in moving from theory to empirical research and clinical practice and vice versa. Therefore, in this article, we bring together different points of view on psychological networks by methodologists and clinicians to give a critical overview on these challenges, and to present an agenda for addressing these challenges. In contrast to previous reviews, we especially focus on methodological issues related to temporal networks. This includes topics such as selecting and assessing the quality of the nodes in the network, distinguishing between- and within-person effects in networks, relating items that are measured at different time scales, and dealing with changes in network structures. These issues are not only important for researchers using network models on empirical data, but also for clinicians, who are increasingly likely to encounter (person-specific) networks in the consulting room.
Collapse
Affiliation(s)
- Laura F Bringmann
- University of Groningen, University Medical Center Groningen, Interdisciplinary Center Psychopathology and Emotion Regulation (ICPE), P.O. Box 30.001 (CC72), 9700 RB, Groningen, the Netherlands; University of Groningen, Faculty of Behavioural and Social Sciences, Department of Psychometrics and Statistics, Grote Kruisstraat 2/1, 9712 TS, Groningen, the Netherlands.
| | - Casper Albers
- University of Groningen, Faculty of Behavioural and Social Sciences, Department of Psychometrics and Statistics, Grote Kruisstraat 2/1, 9712 TS, Groningen, the Netherlands
| | - Claudi Bockting
- Centre for Urban Mental Health, University of Amsterdam, Amsterdam, the Netherlands; Department of Psychiatry, Amsterdam UMC, Location AMC, University of Amsterdam, Amsterdam, the Netherlands
| | - Denny Borsboom
- Department of Psychology, University of Amsterdam, Amsterdam, the Netherlands
| | - Eva Ceulemans
- KU Leuven, Faculty of Psychology and Educational Sciences, Leuven, Belgium
| | - Angélique Cramer
- RIVM National Institute for Public Health and the Environment, the Netherlands
| | - Sacha Epskamp
- Centre for Urban Mental Health, University of Amsterdam, Amsterdam, the Netherlands; Department of Psychology, University of Amsterdam, Amsterdam, the Netherlands
| | - Markus I Eronen
- Department of Theoretical Philosophy, University of Groningen, the Netherlands
| | - Ellen Hamaker
- Department of Methodology and Statistics, Utrecht University, the Netherlands
| | - Peter Kuppens
- KU Leuven, Faculty of Psychology and Educational Sciences, Leuven, Belgium
| | - Wolfgang Lutz
- Department of Psychology, University of Trier, Germany
| | | | - Peter Molenaar
- Department of Human Development and Family Studies, The Pennsylvania State University, USA
| | - Pia Tio
- Department of Psychology, University of Amsterdam, Amsterdam, the Netherlands; Department of Methodology and Statistics, Tilburg University, Tilburg, the Netherlands
| | - Manuel C Voelkle
- Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany
| | - Marieke Wichers
- University of Groningen, University Medical Center Groningen, Interdisciplinary Center Psychopathology and Emotion Regulation (ICPE), P.O. Box 30.001 (CC72), 9700 RB, Groningen, the Netherlands
| |
Collapse
|
7
|
Michalik A, Wójcicka L, Zdun-Ryżewska A, Czerwińska-Osipiak A, Krzemiński M, Olszewska J, Klasa-Mazurkiewicz D, Huizink AC. Polish Adaptation of the Pregnancy-Related Anxiety Questionnaire-Revised 2 for All Pregnant Women. Healthcare (Basel) 2021; 9:917. [PMID: 34356295 PMCID: PMC8307515 DOI: 10.3390/healthcare9070917] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2021] [Revised: 07/14/2021] [Accepted: 07/16/2021] [Indexed: 11/16/2022] Open
Abstract
Pregnancy-related anxiety (PrA) is a specific type of anxiety characteristic of the perinatal period. PrA can affect pregnancy and birth. However, no validated tool exists to measure PrA in Polish obstetric practice. The aim of this study was to translate the Pregnancy-Related Anxiety Questionnaire-Revised 2 (PRAQ-R2) into Polish and to evaluate its reliability and factorial and construct validity. This study was conducted in Poland as an online questionnaire in April 2020 and included 175 healthy women. To validate the PRAQ-R2, we used standardized tools for the measurement of general anxiety: the modified Visual Analog Scale (VAS), the Ten-Item Personality Inventory (TIPI), and the Hospital Anxiety and Depression Scale (HADS). Scale reliability was assessed using Cronbach's alpha. Concurrent validity was evaluated by calculating Spearman's rho correlation coefficients. Statistical analyses were performed using R ver. 4.0.2. Values for comparative fit index >0.90, Tucker-Lewis index >0.90, and root mean square error of approximation <0.08 indicated acceptable model fit, confirming the reliability of the three-factor structure of the translation. The subscales and total scores had good consistency (α > 0.7), and convergent validity was demonstrated. The PRAQ-R2 as translated into Polish represents the first validated tool in Poland to measure PrA for all pregnant women.
Collapse
Affiliation(s)
- Anna Michalik
- Department of Obstetrical and Gynaecological Nursing, Medical University of Gdansk, 80-211 Gdansk, Poland; (A.M.); (L.W.); (A.C.-O.); (J.O.)
| | - Lucyna Wójcicka
- Department of Obstetrical and Gynaecological Nursing, Medical University of Gdansk, 80-211 Gdansk, Poland; (A.M.); (L.W.); (A.C.-O.); (J.O.)
| | - Agata Zdun-Ryżewska
- Department of Quality of Life Research, Medical University of Gdansk, 80-210 Gdansk, Poland;
| | - Agnieszka Czerwińska-Osipiak
- Department of Obstetrical and Gynaecological Nursing, Medical University of Gdansk, 80-211 Gdansk, Poland; (A.M.); (L.W.); (A.C.-O.); (J.O.)
| | - Michał Krzemiński
- Institute of Applied Mathematics, Faculty of Applied Physics and Mathematics, Gdansk University of Technology, 80-233 Gdansk, Poland;
| | - Jolanta Olszewska
- Department of Obstetrical and Gynaecological Nursing, Medical University of Gdansk, 80-211 Gdansk, Poland; (A.M.); (L.W.); (A.C.-O.); (J.O.)
| | - Dagmara Klasa-Mazurkiewicz
- Department of Gynaecology, Oncologic Gynaecology and Gynaecological Endocrinology, Medical University of Gdansk, 80-214 Gdansk, Poland;
| | - Anja C. Huizink
- Department of Clinical Developmental Psychology, Vrije Universiteit Amsterdam, Van der Boechorststraat 1, 1081 BT Amsterdam, The Netherlands
| |
Collapse
|
8
|
A meta-review of transparency and reproducibility-related reporting practices in published meta-analyses on clinical psychological interventions (2000-2020). Behav Res Methods 2021; 54:334-349. [PMID: 34173943 PMCID: PMC8863703 DOI: 10.3758/s13428-021-01644-z] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/02/2021] [Indexed: 12/12/2022]
Abstract
Meta-analysis is a powerful and important tool to synthesize the literature about a research topic. Like other kinds of research, meta-analyses must be reproducible to be compliant with the principles of the scientific method. Furthermore, reproducible meta-analyses can be easily updated with new data and reanalysed applying new and more refined analysis techniques. We attempted to empirically assess the prevalence of transparency and reproducibility-related reporting practices in published meta-analyses from clinical psychology by examining a random sample of 100 meta-analyses. Our purpose was to identify the key points that could be improved, with the aim of providing some recommendations for carrying out reproducible meta-analyses. We conducted a meta-review of meta-analyses of psychological interventions published between 2000 and 2020. We searched PubMed, PsycInfo and Web of Science databases. A structured coding form to assess transparency indicators was created based on previous studies and existing meta-analysis guidelines. We found major issues concerning: completely reproducible search procedures report, specification of the exact method to compute effect sizes, choice of weighting factors and estimators, lack of availability of the raw statistics used to compute the effect size and of interoperability of available data, and practically total absence of analysis script code sharing. Based on our findings, we conclude with recommendations intended to improve the transparency, openness, and reproducibility-related reporting practices of meta-analyses in clinical psychology and related areas.
Collapse
|
9
|
Gilmore RO, Qian Y. An open developmental science will be more rigorous, robust, and impactful. INFANT AND CHILD DEVELOPMENT 2021. [DOI: 10.1002/icd.2254] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Affiliation(s)
- Rick O. Gilmore
- Department of Psychology The Pennsylvania State University University Park Pennsylvania USA
| | - Yiming Qian
- Department of Psychology The Pennsylvania State University University Park Pennsylvania USA
| |
Collapse
|
10
|
Internalizing symptoms, well-being, and correlates in adolescence: A multiverse exploration via cross-lagged panel network models. Dev Psychopathol 2021; 34:1477-1491. [PMID: 34128457 DOI: 10.1017/s0954579421000225] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Internalizing symptoms are the most prevalent mental health problem in adolescents, with sharp increases seen, particularly for girls, and evidence that young people today report more problems than previous generations. It is therefore critical to measure and monitor these states on a large scale and consider correlates. We used novel panel network methodology to explore relationships between internalizing symptoms, well-being, and inter/intrapersonal indicators. A multiverse design was used with 32 conditions to consider the stability of results across arbitrary researcher decisions in a large community sample over three years (N = 15,843, aged 11-12 at Time 1). Networks were consistently similar for girls and boys. Stable trait-like effects within anxiety, attentional, and social indicators were found. Within-person networks were densely connected and suggested mental health and inter/intrapersonal correlates related to one another in similar complex ways. The multiverse design suggested the particular operationalization of items can substantially influence conclusions. Nevertheless, indicators such as thinking clearly, unhappiness, dealing with stress, and worry showed more consistent centrality, suggesting these indicators may play particularly important roles in the development of mental health in adolescence.
Collapse
|
11
|
Wiebels K, Moreau D. Leveraging Containers for Reproducible Psychological Research. ADVANCES IN METHODS AND PRACTICES IN PSYCHOLOGICAL SCIENCE 2021. [DOI: 10.1177/25152459211017853] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Containers have become increasingly popular in computing and software engineering and are gaining traction in scientific research. They allow packaging up all code and dependencies to ensure that analyses run reliably across a range of operating systems and software versions. Despite being a crucial component for reproducible science, containerization has yet to become mainstream in psychology. In this tutorial, we describe the logic behind containers, what they are, and the practical problems they can solve. We walk the reader through the implementation of containerization within a research workflow with examples using Docker and R. Specifically, we describe how to use existing containers, build personalized containers, and share containers alongside publications. We provide a worked example that includes all steps required to set up a container for a research project and can easily be adapted and extended. We conclude with a discussion of the possibilities afforded by the large-scale adoption of containerization, especially in the context of cumulative, open science, toward a more efficient and inclusive research ecosystem.
Collapse
Affiliation(s)
- Kristina Wiebels
- School of Psychology and Centre for Brain Research, University of Auckland, Auckland, New Zealand
| | - David Moreau
- School of Psychology and Centre for Brain Research, University of Auckland, Auckland, New Zealand
| |
Collapse
|
12
|
van Dijk W, Schatschneider C, Hart SA. Open Science in Education Sciences. JOURNAL OF LEARNING DISABILITIES 2021; 54:139-152. [PMID: 32734821 PMCID: PMC7856082 DOI: 10.1177/0022219420945267] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/12/2023]
Abstract
The Open Science movement has gained considerable traction in the last decade. The Open Science movement tries to increase trust in research results and open the access to all elements of a research project to the public. Central to these goals, Open Science has promoted five critical tenets: Open Data, Open Analysis, Open Materials, Preregistration, and Open Access. All Open Science elements can be thought of as extensions to the traditional way of achieving openness in science, which has been scientific publication of research outcomes in journals or books. Open Science in education sciences, however, has the potential to be much more than a safeguard against questionable research. Open Science in education science provides opportunities to (a) increase the transparency and therefore replicability of research and (b) develop and answer research questions about individuals with learning disabilities and learning difficulties that were previously impossible to answer due to complexities in data analysis methods. We will provide overviews of the main tenets of Open Science (i.e., Open Data, Open Analysis, Open Materials, Preregistration, and Open Access), show how they are in line with grant funding agencies' expectations for rigorous research processes, and present resources on best practices for each of the tenets.
Collapse
|
13
|
Hickman L, Thapa S, Tay L, Cao M, Srinivasan P. Text Preprocessing for Text Mining in Organizational Research: Review and Recommendations. ORGANIZATIONAL RESEARCH METHODS 2020. [DOI: 10.1177/1094428120971683] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Recent advances in text mining have provided new methods for capitalizing on the voluminous natural language text data created by organizations, their employees, and their customers. Although often overlooked, decisions made during text preprocessing affect whether the content and/or style of language are captured, the statistical power of subsequent analyses, and the validity of insights derived from text mining. Past methodological articles have described the general process of obtaining and analyzing text data, but recommendations for preprocessing text data were inconsistent. Furthermore, primary studies use and report different preprocessing techniques. To address this, we conduct two complementary reviews of computational linguistics and organizational text mining research to provide empirically grounded text preprocessing decision-making recommendations that account for the type of text mining conducted (i.e., open or closed vocabulary), the research question under investigation, and the data set’s characteristics (i.e., corpus size and average document length). Notably, deviations from these recommendations will be appropriate and, at times, necessary due to the unique characteristics of one’s text data. We also provide recommendations for reporting text mining to promote transparency and reproducibility.
Collapse
Affiliation(s)
- Louis Hickman
- Purdue University College of Health and Human Sciences, West Lafayette, IN, USA
| | - Stuti Thapa
- Purdue University College of Health and Human Sciences, West Lafayette, IN, USA
| | - Louis Tay
- Purdue University College of Health and Human Sciences, West Lafayette, IN, USA
| | | | | |
Collapse
|
14
|
Kunkels YK, Knapen SE, Zuidersma M, Wichers M, Riese H, Emerencia AC. ACTman: Automated preprocessing and analysis of actigraphy data. J Sci Med Sport 2019; 23:481-486. [PMID: 31813761 DOI: 10.1016/j.jsams.2019.11.009] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2019] [Revised: 11/06/2019] [Accepted: 11/13/2019] [Indexed: 12/01/2022]
Abstract
OBJECTIVES To introduce a novel software-library called Actigraphy Manager (ACTman) which automates labor-intensive actigraphy data preprocessing and analyses steps while improving transparency, reproducibility, and scalability over software suites traditionally used in actigraphy research practice. DESIGN Descriptive. METHODS Use cases are described for performing a common actigraphy task in ACTman and alternative actigraphy software. Important inefficiencies in actigraphy workflow are identified and their consequences are described. We explain how these hinder the feasibility of conducting studies with large groups of athletes and/or longer data collection periods. Thereafter, the information flow through the ACTman software is described and we explain how it alleviates aforementioned inefficiencies. Furthermore, transparency, reproducibility, and scalability issues of commonly used actigraphy software packages are discussed and compared with the ACTman package. RESULTS It is shown that from an end-user perspective ACTman offers a compact workflow as it automates many preprocessing and analysis steps that otherwise have to be performed manually. When considering transparency, reproducibility, and scalability the design of the ACTman software is found to outperform proprietary and open-source actigraphy software suites. As such, ACTman alleviates important bottlenecks within actigraphy research practice. CONCLUSIONS ACTman facilitates the current transition towards larger datasets containing data of multiple athletes by automating labor-intensive preprocessing and analyses steps within actigraphy research. Furthermore, ACTman offers many features which enhance user-convenience and analysis customization, such as moving window functionality and period selection options. ACTman is open-source and thus fully verifiable, in contrast with many proprietary software packages which remain a black box for researchers.
Collapse
Affiliation(s)
- Yoram K Kunkels
- University of Groningen, University Medical Center Groningen (UMCG), Department of Psychiatry, Interdisciplinary Center Psychopathology and Emotional Regulation (ICPE), The Netherlands.
| | - Stefan E Knapen
- University of Groningen, University Medical Center Groningen, Department of Psychiatry, Research School of Behavioral and Cognitive Neurosciences (BCN), ICPE, The Netherlands; Department of Neurology, Leiden University Medical Centre (LUMC), The Netherlands
| | - Marij Zuidersma
- University of Groningen, University Medical Center Groningen (UMCG), Department of Psychiatry, Interdisciplinary Center Psychopathology and Emotional Regulation (ICPE), The Netherlands
| | - Marieke Wichers
- University of Groningen, University Medical Center Groningen (UMCG), Department of Psychiatry, Interdisciplinary Center Psychopathology and Emotional Regulation (ICPE), The Netherlands
| | - Harriëtte Riese
- University of Groningen, University Medical Center Groningen (UMCG), Department of Psychiatry, Interdisciplinary Center Psychopathology and Emotional Regulation (ICPE), The Netherlands
| | - Ando C Emerencia
- University of Groningen, Department of Developmental Psychology, The Netherlands
| |
Collapse
|