1
|
van den Akker OR, van Assen MALM, Bakker M, Elsherif M, Wong TK, Wicherts JM. Preregistration in practice: A comparison of preregistered and non-preregistered studies in psychology. Behav Res Methods 2023:10.3758/s13428-023-02277-0. [PMID: 37950113 DOI: 10.3758/s13428-023-02277-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/18/2023] [Indexed: 11/12/2023]
Abstract
Preregistration has gained traction as one of the most promising solutions to improve the replicability of scientific effects. In this project, we compared 193 psychology studies that earned a Preregistration Challenge prize or preregistration badge to 193 related studies that were not preregistered. In contrast to our theoretical expectations and prior research, we did not find that preregistered studies had a lower proportion of positive results (Hypothesis 1), smaller effect sizes (Hypothesis 2), or fewer statistical errors (Hypothesis 3) than non-preregistered studies. Supporting our Hypotheses 4 and 5, we found that preregistered studies more often contained power analyses and typically had larger sample sizes than non-preregistered studies. Finally, concerns about the publishability and impact of preregistered studies seem unwarranted, as preregistered studies did not take longer to publish and scored better on several impact measures. Overall, our data indicate that preregistration has beneficial effects in the realm of statistical power and impact, but we did not find robust evidence that preregistration prevents p-hacking and HARKing (Hypothesizing After the Results are Known).
Collapse
Affiliation(s)
- Olmo R van den Akker
- Department of Methodology and Statistics, Tilburg University, Warandelaan 2, 5037 AB, Tilburg, The Netherlands.
| | - Marcel A L M van Assen
- Department of Methodology and Statistics, Tilburg University, Warandelaan 2, 5037 AB, Tilburg, The Netherlands
- Department of Sociology, Utrecht University, Utrecht, The Netherlands
| | - Marjan Bakker
- Department of Methodology and Statistics, Tilburg University, Warandelaan 2, 5037 AB, Tilburg, The Netherlands
| | - Mahmoud Elsherif
- Department of Neuroscience, Psychology, and Behaviour, University of Leicester, Leicester, UK
| | - Tsz Keung Wong
- Department of Methodology and Statistics, Tilburg University, Warandelaan 2, 5037 AB, Tilburg, The Netherlands
| | - Jelte M Wicherts
- Department of Methodology and Statistics, Tilburg University, Warandelaan 2, 5037 AB, Tilburg, The Netherlands
| |
Collapse
|
2
|
Cariaga-Martinez A, Alelú-Paz R. False data, positive results in neurobiology: moving beyond the epigenetics of blood and saliva samples in mental disorders. J Negat Results Biomed 2016; 15:21. [PMID: 27938391 PMCID: PMC5151128 DOI: 10.1186/s12952-016-0064-x] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2016] [Accepted: 11/08/2016] [Indexed: 11/12/2022] Open
Abstract
Many psychiatric diseases are influenced by a set of several genetic and environmental factors that genetics alone cannot explain. Specifically, in schizophrenia and bipolar disorder the absence of consistently replicated genetic effects together with evidence for lasting changes in gene expression after environmental exposures suggest a role of epigenetic mechanisms in its pathophysiological mechanisms. In this field, the presence of positive results could potentially uncover molecular mechanisms of deregulated gene expression in these complex disorders. In this commentary we have reviewed the positive data obtained over the last 5 years from the scientific literature published in PubMed and we have shown that these results are based on peripheral samples (blood, saliva and other fluids) that do not allow us to obtain reliable and/or valid results, under any circumstances. Finally, we highlight the need to employ human brain samples in the epigenetic study of mental disorders.
Collapse
Affiliation(s)
- A Cariaga-Martinez
- Laboratory for Neuroscience of Mental Disorders Elena Pessino, Department of Medicine and Medical Specialties, School of Medicine, Alcalá University, Madrid, Spain.
| | - R Alelú-Paz
- Laboratory for Neuroscience of Mental Disorders Elena Pessino, Department of Medicine and Medical Specialties, School of Medicine, Alcalá University, Madrid, Spain.,Department of Psychiatry, Ramón y Cajal Hospital, IRYCIS, Madrid, Spain
| |
Collapse
|
3
|
Rothwell E, Johnson E, Mathiesen A, Golden K, Metcalf A, Rose NC, Botkin JR. Experiences among Women with Positive Prenatal Expanded Carrier Screening Results. J Genet Couns 2016; 26:690-696. [PMID: 27796679 DOI: 10.1007/s10897-016-0037-8] [Citation(s) in RCA: 33] [Impact Index Per Article: 4.1] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2016] [Accepted: 10/14/2016] [Indexed: 11/29/2022]
Abstract
The offering and acceptance of expanded carrier screening is increasing among pregnant women including women without an increased risk based on race, ethnicity or family history. The chances of a positive screening test have been reported to be as high as 24 % when multiple conditions are screened. Yet, little is known about the way these tests are offered and how patients are affected by a positive test result. To explore this area of genetic testing, interviews (n = 17) were conducted among women who received positive expanded carrier results in the context of obstetric care. A content analysis was conducted on the transcript data from the interviews. Outcomes of this research suggest that educational interventions are needed to improve maternal understanding of positive carrier screening results. Most of the participants in this study confused the results with other prenatal screening test options. In addition, the way the results were discussed varied greatly, and influenced participants' thoughts about reproductive decisions that led to a range of emotional uncertainty. Our data suggests that genetic counseling improved participants' understanding of positive results. More research is needed to further understand if our results are consistent within a larger, more diverse sample, and to explore how to best provide education about expanded carrier screening.
Collapse
Affiliation(s)
- Erin Rothwell
- Division of Health Systems and Community-Based Care, College of Nursing, School of Medicine, University of Utah, 10 South 2000 East, Salt Lake City, UT, 84112, USA. .,Division of Medical Ethics and Humanities, University of Utah, Salt Lake City, UT, USA.
| | - Erin Johnson
- Division of Health Systems and Community-Based Care, College of Nursing, School of Medicine, University of Utah, 10 South 2000 East, Salt Lake City, UT, 84112, USA
| | - Amber Mathiesen
- Division of Maternal Fetal Medicine, Department of ObGyn, University of Utah, Salt Lake City, UT, USA
| | - Kylie Golden
- Division of Health Systems and Community-Based Care, College of Nursing, School of Medicine, University of Utah, 10 South 2000 East, Salt Lake City, UT, 84112, USA
| | - Audrey Metcalf
- Division of Health Systems and Community-Based Care, College of Nursing, School of Medicine, University of Utah, 10 South 2000 East, Salt Lake City, UT, 84112, USA
| | - Nancy C Rose
- Department of Pediatrics, University of Utah, Salt Lake City, UT, USA
| | - Jeffrey R Botkin
- Division of Medical Ethics and Humanities, University of Utah, Salt Lake City, UT, USA.,Department of Pediatrics, University of Utah, Salt Lake City, UT, USA
| |
Collapse
|