51
|
Wang H, Radomska HS, Phelps MA. Replication Study: Coding-independent regulation of the tumor suppressor PTEN by competing endogenous mRNAs. eLife 2020; 9:56651. [PMID: 33073769 DOI: 10.7554/elife.56651] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2020] [Accepted: 09/18/2020] [Indexed: 12/19/2022] Open
Abstract
As part of the Reproducibility Project: Cancer Biology, we published a Registered Report (Phelps et al., 2016) that described how we intended to replicate selected experiments from the paper 'Coding-independent regulation of the tumor suppressor PTEN by competing endogenous mRNAs' (Tay et al., 2011). Here, we report the results. We found depletion of putative PTEN competing endogenous mRNAs (ceRNAs) in DU145 cells did not impact PTEN 3'UTR regulation using a reporter, while the original study reported decreased activity when SERINC1, VAPA, and CNOT6L were depleted (Figure 3C; Tay et al., 2011). Using the same reporter, we found decreased activity when ceRNA 3'UTRs were overexpressed, while the original study reported increased activity (Figure 3D; Tay et al., 2011). In HCT116 cells, ceRNA depletion resulted in decreased PTEN protein levels, a result similar to the findings reported in the original study (Figure 3G,H; Tay et al., 2011); however, while the original study reported an attenuated ceRNA effect in microRNA deficient (DicerEx5) HCT116 cells, we observed increased PTEN protein levels. Further, we found depletion of the ceRNAs VAPA or CNOT6L did not statistically impact DU145, wild-type HCT116, or DicerEx5 HCT116 cell proliferation. The original study reported increased DU145 and wild-type HCT116 cell proliferation when these ceRNAs were depleted, which was attenuated in the DicerEx5 HCT116 cells (Figure 5B; Tay et al., 2011). Differences between the original study and this replication attempt, such as variance between biological repeats, are factors that might have influenced the results. Finally, we report meta-analyses for each result.
Collapse
|
52
|
Heyman T, Maerten AS. Correction notices in psychology: impactful or inconsequential? ROYAL SOCIETY OPEN SCIENCE 2020; 7:200834. [PMID: 33204456 PMCID: PMC7657932 DOI: 10.1098/rsos.200834] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/13/2020] [Accepted: 09/03/2020] [Indexed: 06/11/2023]
Abstract
Science is self-correcting, or so the adage goes, but to what extent is that indeed the case? Answering this question requires careful consideration of the various approaches to achieve the collective goal of self-correction. One of the most straightforward mechanisms is individual self-correction: researchers rectifying their own mistakes by publishing a correction notice. Although it offers an efficient route to correcting the scientific record, it has received little to no attention from a metascientific point of view. We aim to fill this void by analysing the content of correction notices published from 2010 until 2018 in the three psychology journals featuring the highest number of corrections over that timespan based on the Scopus database (i.e. Psychological Science with N = 58, Frontiers in Psychology with N = 99 and Journal of Affective Disorders with N = 57). More concretely, we examined which aspects of the original papers were affected (e.g. hypotheses, data-analyses, metadata such as author order, affiliations, funding information etc.) as well as the perceived implications for the papers' main findings. Our exploratory analyses showed that many corrections involved inconsequential errors. Furthermore, authors rarely revised their conclusions, even though several corrections concerned changes to the results. We conclude with a discussion of current policies, and suggest ways to improve upon the present situation by (i) preventing mistakes, and (ii) transparently rectifying those mistakes that do find their way into the literature.
Collapse
|
53
|
Roberts SO, Bareket-Shavit C, Dollins FA, Goldie PD, Mortenson E. Racial Inequality in Psychological Research: Trends of the Past and Recommendations for the Future. PERSPECTIVES ON PSYCHOLOGICAL SCIENCE 2020; 15:1295-1309. [PMID: 32578504 DOI: 10.1177/1745691620927709] [Citation(s) in RCA: 295] [Impact Index Per Article: 73.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Race plays an important role in how people think, develop, and behave. In the current article, we queried more than 26,000 empirical articles published between 1974 and 2018 in top-tier cognitive, developmental, and social psychology journals to document how often psychological research acknowledges this reality and to examine whether people who edit, write, and participate in the research are systematically connected. We note several findings. First, across the past five decades, psychological publications that highlight race have been rare, and although they have increased in developmental and social psychology, they have remained virtually nonexistent in cognitive psychology. Second, most publications have been edited by White editors, under which there have been significantly fewer publications that highlight race. Third, many of the publications that highlight race have been written by White authors who employed significantly fewer participants of color. In many cases, we document variation as a function of area and decade. We argue that systemic inequality exists within psychological research and that systemic changes are needed to ensure that psychological research benefits from diversity in editing, writing, and participation. To this end, and in the spirit of the field's recent emphasis on metascience, we offer recommendations for journals and authors.
Collapse
|
54
|
Abstract
The partial success of a study to reproduce experiments that linked pseudogenes and cancer proves that understanding RNA networks is more complicated than expected.
Collapse
|
55
|
Kerwin J, Khan I. Replication Study: A coding-independent function of gene and pseudogene mRNAs regulates tumour biology. eLife 2020; 9:e51019. [PMID: 32314732 PMCID: PMC7185998 DOI: 10.7554/elife.51019] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2019] [Accepted: 02/20/2020] [Indexed: 01/02/2023] Open
Abstract
As part of the Reproducibility Project: Cancer Biology we published a Registered Report (Khan et al., 2015), that described how we intended to replicate selected experiments from the paper "A coding-independent function of gene and pseudogene mRNAs regulates tumour biology" (Poliseno et al., 2010). Here we report the results. We found PTEN depletion in the prostate cancer cell line DU145 did not detectably impact expression of the corresponding pseudogene PTENP1. Similarly, depletion of PTENP1 did not impact PTEN mRNA levels. The original study reported PTEN or PTENP1 depletion statistically reduced the corresponding pseudogene or gene (Figure 2G; Poliseno et al., 2010). PTEN and/or PTENP1 depletion in DU145 cells decreased PTEN protein expression, which was similar to the original study (Figure 2H; Poliseno et al., 2010). Further, depletion of PTEN and/or PTENP1 increased DU145 proliferation compared to non-targeting siRNA, which was in the same direction as the original study (Figure 2F; Poliseno et al., 2010), but not statistically significant. We found PTEN 3'UTR overexpression in DU145 cells did not impact PTENP1 expression, while the original study reported PTEN 3'UTR increased PTENP1 levels (Figure 4A; Poliseno et al., 2010). Overexpression of PTEN 3'UTR also statistically decreased DU145 proliferation compared to controls, which was similar to the findings reported in the original study (Figure 4A; Poliseno et al., 2010). Differences between the original study and this replication attempt, such as level of knockdown efficiency and cellular confluence, are factors that might have influenced the results. Finally, where possible, we report meta-analyses for each result.
Collapse
|
56
|
Krenn M, Zeilinger A. Predicting research trends with semantic and neural networks with an application in quantum physics. Proc Natl Acad Sci U S A 2020; 117:1910-1916. [PMID: 31937664 PMCID: PMC6994972 DOI: 10.1073/pnas.1914370116] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/08/2023] Open
Abstract
The vast and growing number of publications in all disciplines of science cannot be comprehended by a single human researcher. As a consequence, researchers have to specialize in narrow subdisciplines, which makes it challenging to uncover scientific connections beyond the own field of research. Thus, access to structured knowledge from a large corpus of publications could help push the frontiers of science. Here, we demonstrate a method to build a semantic network from published scientific literature, which we call SemNet We use SemNet to predict future trends in research and to inspire personalized and surprising seeds of ideas in science. We apply it in the discipline of quantum physics, which has seen an unprecedented growth of activity in recent years. In SemNet, scientific knowledge is represented as an evolving network using the content of 750,000 scientific papers published since 1919. The nodes of the network correspond to physical concepts, and links between two nodes are drawn when two concepts are concurrently studied in research articles. We identify influential and prize-winning research topics from the past inside SemNet, thus confirming that it stores useful semantic knowledge. We train a neural network using states of SemNet of the past to predict future developments in quantum physics and confirm high-quality predictions using historic data. Using network theoretical tools, we can suggest personalized, out-of-the-box ideas by identifying pairs of concepts, which have unique and extremal semantic network properties. Finally, we consider possible future developments and implications of our findings.
Collapse
|
57
|
Sarma GP, Levey A, Faundez V. Re-examining physician-scientist training through the prism of the discovery-invention cycle. F1000Res 2019; 8:2123. [PMID: 32089830 PMCID: PMC7014578 DOI: 10.12688/f1000research.21448.1] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 12/13/2019] [Indexed: 11/22/2022] Open
Abstract
The training of physician-scientists lies at the heart of future medical research. In this commentary, we apply Narayanamurti and Odumosu’s framework of the “discovery-invention cycle” to analyze the structure and outcomes of the integrated MD/PhD program. We argue that the linear model of “bench-to-bedside” research, which is also reflected in the present training of MD/PhDs, merits continual re-evaluation to capitalize on the richness of opportunities arising in clinical medicine. In addition to measuring objective career outcomes, as existing research has done, we suggest that detailed characterization of researchers’ efforts using both qualitative and quantitative techniques is necessary to understand if dual-degree training is being utilized. As an example, we propose that the application of machine learning and data science to corpora of biomedical literature and anonymized clinical data might allow us to see if there are objective “signatures” of research uniquely enabled by MD/PhD training. We close by proposing several hypotheses for shaping physician-scientist training, the relative merits of which could be assessed using the techniques proposed above. Our overarching message is the importance of deeply understanding individual career trajectories as well as characterizing organizational details and cultural nuances to drive new policy which shapes the future of the physician-scientist workforce.
Collapse
|
58
|
Sheen MR, Fields JL, Northan B, Lacoste J, Ang LH, Fiering S. Replication Study: Biomechanical remodeling of the microenvironment by stromal caveolin-1 favors tumor invasion and metastasis. eLife 2019; 8:45120. [PMID: 31845647 PMCID: PMC6917490 DOI: 10.7554/elife.45120] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/14/2019] [Accepted: 11/06/2019] [Indexed: 01/06/2023] Open
Abstract
As part of the Reproducibility Project: Cancer Biology we published a Registered Report (Fiering et al., 2015) that described how we intended to replicate selected experiments from the paper 'Biomechanical remodeling of the microenvironment by stromal caveolin-1 favors tumor invasion and metastasis' (Goetz et al., 2011). Here we report the results. Primary mouse embryonic fibroblasts (pMEFs) expressing caveolin 1 (Cav1WT) demonstrated increased extracellular matrix remodeling in vitro compared to Cav1 deficient (Cav1KO) pMEFs, similar to the original study (Goetz et al., 2011). In vivo, we found higher levels of intratumoral stroma remodeling, determined by fibronectin fiber orientation, in tumors from cancer cells co-injected with Cav1WT pMEFs compared to cancer cells only or cancer cells plus Cav1KO pMEFs, which were in the same direction as the original study (Supplemental Figure S7C; Goetz et al., 2011), but not statistically significant. Primary tumor growth was similar between conditions, like the original study (Supplemental Figure S7Ca; Goetz et al., 2011). We found metastatic burden was similar between Cav1WT and Cav1KO pMEFs, while the original study found increased metastases with Cav1WT (Figure 7C; Goetz et al., 2011); however, the duration of our in vivo experiments (45 days) were much shorter than in the study by Goetz et al. (2011) (75 days). This makes it difficult to interpret the difference between the studies as it is possible that the cells required more time to manifest the difference between treatments observed by Goetz et al. We also found a statistically significant negative correlation of intratumoral remodeling with metastatic burden, while the original study found a statistically significant positive correlation (Figure 7Cd; Goetz et al., 2011), but again there were differences between the studies in terms of the duration of the metastasis studies and the imaging approaches that could have impacted the outcomes. Finally, we report meta-analyses for each result.
Collapse
|
59
|
Abstract
The partial success of an attempt to repeat findings in cancer biology highlights the need to improve study designs for preclinical research into metastasis and the targeting of cancer cells.
Collapse
|
60
|
DeJesus JM, Callanan MA, Solis G, Gelman SA. Generic language in scientific communication. Proc Natl Acad Sci U S A 2019; 116:18370-18377. [PMID: 31451665 PMCID: PMC6744883 DOI: 10.1073/pnas.1817706116] [Citation(s) in RCA: 28] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/08/2023] Open
Abstract
Scientific communication poses a challenge: To clearly highlight key conclusions and implications while fully acknowledging the limitations of the evidence. Although these goals are in principle compatible, the goal of conveying complex and variable data may compete with reporting results in a digestible form that fits (increasingly) limited publication formats. As a result, authors' choices may favor clarity over complexity. For example, generic language (e.g., "Introverts and extraverts require different learning environments") may mislead by implying general, timeless conclusions while glossing over exceptions and variability. Using generic language is especially problematic if authors overgeneralize from small or unrepresentative samples (e.g., exclusively Western, middle-class). We present 4 studies examining the use and implications of generic language in psychology research articles. Study 1, a text analysis of 1,149 psychology articles published in 11 journals in 2015 and 2016, examined the use of generics in titles, research highlights, and abstracts. We found that generics were ubiquitously used to convey results (89% of articles included at least 1 generic), despite that most articles made no mention of sample demographics. Generics appeared more frequently in shorter units of the paper (i.e., highlights more than abstracts), and generics were not associated with sample size. Studies 2 to 4 (n = 1,578) found that readers judged results expressed with generic language to be more important and generalizable than findings expressed with nongeneric language. We highlight potential unintended consequences of language choice in scientific communication, as well as what these choices reveal about how scientists think about their data.
Collapse
|
61
|
Uhlmann EL, Ebersole CR, Chartier CR, Errington TM, Kidwell MC, Lai CK, McCarthy RJ, Riegelman A, Silberzahn R, Nosek BA. Scientific Utopia III: Crowdsourcing Science. PERSPECTIVES ON PSYCHOLOGICAL SCIENCE 2019; 14:711-733. [PMID: 31260639 DOI: 10.1177/1745691619850561] [Citation(s) in RCA: 40] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Most scientific research is conducted by small teams of investigators who together formulate hypotheses, collect data, conduct analyses, and report novel findings. These teams operate independently as vertically integrated silos. Here we argue that scientific research that is horizontally distributed can provide substantial complementary value, aiming to maximize available resources, promote inclusiveness and transparency, and increase rigor and reliability. This alternative approach enables researchers to tackle ambitious projects that would not be possible under the standard model. Crowdsourced scientific initiatives vary in the degree of communication between project members from largely independent work curated by a coordination team to crowd collaboration on shared activities. The potential benefits and challenges of large-scale collaboration span the entire research process: ideation, study design, data collection, data analysis, reporting, and peer review. Complementing traditional small science with crowdsourced approaches can accelerate the progress of science and improve the quality of scientific research.
Collapse
|
62
|
Caldwell AR, Cheuvront SN. Basic statistical considerations for physiology: The journal Temperature toolbox. Temperature (Austin) 2019; 6:181-210. [PMID: 31608303 PMCID: PMC6773229 DOI: 10.1080/23328940.2019.1624131] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2018] [Revised: 05/19/2019] [Accepted: 05/21/2019] [Indexed: 12/26/2022] Open
Abstract
The average environmental and occupational physiologist may find statistics are difficult to interpret and use since their formal training in statistics is limited. Unfortunately, poor statistical practices can generate erroneous or at least misleading results and distorts the evidence in the scientific literature. These problems are exacerbated when statistics are used as thoughtless ritual that is performed after the data are collected. The situation is worsened when statistics are then treated as strict judgements about the data (i.e., significant versus non-significant) without a thought given to how these statistics were calculated or their practical meaning. We propose that researchers should consider statistics at every step of the research process whether that be the designing of experiments, collecting data, analysing the data or disseminating the results. When statistics are considered as an integral part of the research process, from start to finish, several problematic practices can be mitigated. Further, proper practices in disseminating the results of a study can greatly improve the quality of the literature. Within this review, we have included a number of reminders and statistical questions researchers should answer throughout the scientific process. Rather than treat statistics as a strict rule following procedure we hope that readers will use this review to stimulate a discussion around their current practices and attempt to improve them. The code to reproduce all analyses and figures within the manuscript can be found at https://doi.org/10.17605/OSF.IO/BQGDH.
Collapse
|
63
|
Essex A, Pineda J, Acharya G, Xin H, Evans J. Replication Study: Wnt activity defines colon cancer stem cells and is regulated by the microenvironment. eLife 2019; 8:e45426. [PMID: 31215867 PMCID: PMC6584130 DOI: 10.7554/elife.45426] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2019] [Accepted: 06/07/2019] [Indexed: 01/05/2023] Open
Abstract
As part of the Reproducibility Project: Cancer Biology we published a Registered Report (Evans et al., 2015), that described how we intended to replicate selected experiments from the paper 'Wnt activity defines colon cancer stem cells and is regulated by the microenvironment' (Vermeulen et al., 2010). Here, we report the results. Using three independent primary spheroidal colon cancer cultures that expressed a Wnt reporter construct we observed high Wnt activity was associated with the cell surface markers CD133, CD166, and CD29, but not CD24 and CD44, while the original study found all five markers were correlated with high Wnt activity (Figure 2F; Vermeulen et al., 2010). Clonogenicity was highest in cells with high Wnt activity and clonogenic potential of cells with low Wnt activity were increased by myofibroblast-secreted factors, including HGF. While the effects were in the same direction as the original study (Figure 6D; Vermeulen et al., 2010) whether statistical significance was reached among the different conditions varied. When tested in vivo, we did not find a difference in tumorigenicity between high and low Wnt activity, while the original study found cells with high Wnt activity were more effective in inducing tumors (Figure 7E; Vermeulen et al., 2010). Tumorigenicity, however, was increased with myofibroblast-secreted factors, which was in the same direction as the original study (Figure 7E; Vermeulen et al., 2010), but not statistically significant. Finally, we report meta-analyses for each results where possible.
Collapse
|
64
|
Tiokhin L, Derex M. Competition for novelty reduces information sampling in a research game-a registered report. ROYAL SOCIETY OPEN SCIENCE 2019; 6:180934. [PMID: 31218016 PMCID: PMC6549967 DOI: 10.1098/rsos.180934] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/11/2018] [Accepted: 04/08/2019] [Indexed: 06/09/2023]
Abstract
Incentive structures shape scientists' research practices. One incentive in particular, rewarding priority of publication, is hypothesized to harm scientific reliability by promoting rushed, low-quality research. Here, we develop a laboratory experiment to test whether competition affects information sampling and guessing accuracy in a game that mirrors aspects of scientific investigation. In our experiment, individuals gather data in order to guess true states of the world and face a tradeoff between guessing quickly and increasing accuracy by acquiring more information. To test whether competition affects accuracy, we compare a treatment in which individuals are rewarded for each correct guess to a treatment where individuals face the possibility of being 'scooped' by a competitor. In a second set of conditions, we make information acquisition contingent on solving arithmetic problems to test whether competition increases individual effort (i.e. arithmetic-problem solving speed). We find that competition causes individuals to make guesses using less information, thereby reducing their accuracy (H1a and H1b confirmed). We find no evidence that competition increases individual effort (H2, inconclusive evidence). Our experiment provides proof of concept that rewarding priority of publication can incentivize individuals to acquire less information, producing lower-quality research as a consequence.
Collapse
|
65
|
Soto CJ. How Replicable Are Links Between Personality Traits and Consequential Life Outcomes? The Life Outcomes of Personality Replication Project. Psychol Sci 2019; 30:711-727. [PMID: 30950321 DOI: 10.1177/0956797619831612] [Citation(s) in RCA: 129] [Impact Index Per Article: 25.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022] Open
Abstract
The Big Five personality traits have been linked to dozens of life outcomes. However, metascientific research has raised questions about the replicability of behavioral science. The Life Outcomes of Personality Replication (LOOPR) Project was therefore conducted to estimate the replicability of the personality-outcome literature. Specifically, I conducted preregistered, high-powered (median N = 1,504) replications of 78 previously published trait-outcome associations. Overall, 87% of the replication attempts were statistically significant in the expected direction. The replication effects were typically 77% as strong as the corresponding original effects, which represents a significant decline in effect size. The replicability of individual effects was predicted by the effect size and design of the original study, as well as the sample size and statistical power of the replication. These results indicate that the personality-outcome literature provides a reasonably accurate map of trait-outcome associations but also that it stands to benefit from efforts to improve replicability.
Collapse
|
66
|
Yan X, Tang B, Chen B, Shan Y, Yang H. Replication Study: The microRNA miR-34a inhibits prostate cancer stem cells and metastasis by directly repressing CD44. eLife 2019; 8:43511. [PMID: 30860027 PMCID: PMC6414201 DOI: 10.7554/elife.43511] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2018] [Accepted: 02/25/2019] [Indexed: 12/22/2022] Open
Abstract
As part of the Reproducibility Project: Cancer Biology, we published a Registered Report (Li et al., 2015), that described how we intended to replicate selected experiments from the paper 'The microRNA miR-34a inhibits prostate cancer stem cells and metastasis by directly repressing CD44' (Liu et al., 2011). Here we report the results. We found the microRNA, miR-34a, was expressed at twice the level in CD44+ prostate cancer cells purified from xenograft tumors (LAPC4 cells) compared to CD44- LAPC4 cells, whereas the original study reported miR-34a was underexpressed in CD44+ LAPC4 cells (Figure 1B; Liu et al., 2011). When LAPC4 cells engineered to express miR-34a were injected into mice, we did not observe changes in tumor growth or CD44 expression; however, unexpectedly miR-34a expression was lost in vivo. In the original study, LAPC4 cells expressing miR-34a had a statistically significant reduction in tumor regeneration and reduced CD44 expression compared to control (Figure 4A and Supplemental Figures 4A,B and 5C; Liu et al., 2011). Furthermore, when we tested if miR-34a regulated CD44 through binding sites in the 3'UTR we did not find a statistically significant difference, whereas the original study reported miR-34a decreased CD44 expression that was partially abrogated by mutation of the binding sites in the CD44 3'UTR (Figure 4D; Liu et al., 2011). Finally, where possible, we report meta-analyses for each result.
Collapse
|
67
|
Amaral OB, Neves K, Wasilewska-Sampaio AP, Carneiro CFD. The Brazilian Reproducibility Initiative. eLife 2019; 8:e41602. [PMID: 30720433 PMCID: PMC6374071 DOI: 10.7554/elife.41602] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2018] [Accepted: 01/25/2019] [Indexed: 12/12/2022] Open
Abstract
Most efforts to estimate the reproducibility of published findings have focused on specific areas of research, even though science is usually assessed and funded on a regional or national basis. Here we describe a project to assess the reproducibility of findings in biomedical science published by researchers based in Brazil. The Brazilian Reproducibility Initiative is a systematic, multicenter effort to repeat between 60 and 100 experiments: the project will focus on a set of common methods, repeating each experiment in three different laboratories from a countrywide network. The results, due in 2021, will allow us to estimate the level of reproducibility of biomedical science in Brazil, and to investigate what aspects of the published literature might help to predict whether a finding is reproducible.
Collapse
|
68
|
Kim J, Afshari A, Sengupta R, Sebastiano V, Gupta A, Kim YH. Replication study: Melanoma exosomes educate bone marrow progenitor cells toward a pro-metastatic phenotype through MET. eLife 2018; 7:39944. [PMID: 30526855 PMCID: PMC6289570 DOI: 10.7554/elife.39944] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2018] [Accepted: 12/01/2018] [Indexed: 12/14/2022] Open
Abstract
As part of the Reproducibility Project: Cancer Biology we published a Registered Report (Lesnik et al., 2016) that described how we intended to replicate selected experiments from the paper ‘Melanoma exosomes educate bone marrow progenitor cells toward a pro-metastatic phenotype through MET’ (Peinado et al., 2012). Here we report the results. We regenerated tumor cells stably expressing a short hairpin to reduce Met expression (shMet) using the same highly metastatic mouse melanoma cell line (B16-F10) as the original study, which efficiently downregulated Met in B16F10 cells similar to the original study (Supplementary Figure 5A; Peinado et al., 2012). Exosomes from control cells expressed Met, which was reduced in exosomes from shMet cells; however, we were unable to reliably detect phosphorylated Met in exosomes. We tested the effect of exosome-dependent Met signaling on primary tumor growth and metastasis. Similar to the results in the original study, we did not find a statistically significant change in primary tumor growth. Measuring lung and femur metastases, we found a small increase in metastatic burden with exosomes from control cells that was diminished when Met expression was reduced; however, while the effects were in the same direction as the original study (Figure 4E; Peinado et al., 2012), they were not statistically significant. Differences between the original study and this replication attempt, such as level of knockdown efficiency, cell line genetic drift, sample sizes, study endpoints, and variability of observed metastatic burden, are factors that might have influenced the outcomes. Finally, we report meta-analyses for each result.
Collapse
|
69
|
Eaton K, Pirani A, Snitkin ES. Replication Study: Intestinal inflammation targets cancer-inducing activity of the microbiota. eLife 2018; 7:e34364. [PMID: 30295289 PMCID: PMC6175580 DOI: 10.7554/elife.34364] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2017] [Accepted: 09/19/2018] [Indexed: 01/16/2023] Open
Abstract
As part of the Reproducibility Project: Cancer Biology we published a Registered Report (Eaton et al., 2015) that described how we intended to replicate selected experiments from the paper "Intestinal Inflammation Targets Cancer-Inducing Activity of the Microbiota" (Arthur et al., 2012). Here we report the results. We observed no impact on bacterial growth or colonization capacity when the polyketide synthase (pks) genotoxic island was deleted from E. coli NC101, similar to the original study (Supplementary Figure 7; Arthur et al., 2012). However, for the experiment that compared inflammation, invasion, and neoplasia in azoxymethane (AOM)-treated interleukin-10-deficient mice mono-associated with NC101 or NC101[Formula: see text] pks the experimental timing of the replication attempt was longer than that of the original study. This difference was because in the original study the methodology was not clearly stated and likely led to the increased mortality and severity of inflammation observed in this replication attempt. Additionally, early death occurred during AOM treatment with higher mortality observed in NC101[Formula: see text] pks mono-associated mice compared to NC101, which was in the same direction, but more severe than the original study (Suppleme1ntal Figure 10; Arthur et al., 2012). A meta-analysis suggests that mice mono-associated with NC101[Formula: see text] pks have higher mortality compared to NC101. While these data were unable to address whether, under the conditions of the original study, NC101 and NC101[Formula: see text] pks differ in inflammation, invasion, and neoplasia this replication attempt demonstrates that clear description of experimental methods is essential to ensure accurate reproduction of experimental studies.
Collapse
|
70
|
Sears CL. The who, where and how of fusobacteria and colon cancer. eLife 2018; 7:e28434. [PMID: 29533185 PMCID: PMC5849411 DOI: 10.7554/elife.28434] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2018] [Accepted: 03/05/2018] [Indexed: 01/22/2023] Open
Abstract
The association between the bacterium Fusobacterium nucleatum and human colon cancer is more complicated than it first appeared.
Collapse
|
71
|
Abstract
As part of the <ext-link ext-link-type="uri" xlink:href="https://osf.io/e81xl/wiki/home/">Reproducibility Project: Cancer Biology</ext-link>, we published a Registered Report (Repass et al., 2016), that described how we intended to replicate an experiment from the paper 'Fusobacterium nucleatum infection is prevalent in human colorectal carcinoma' (Castellarin et al., 2012). Here we report the results. When measuring Fusobacterium nucleatum DNA by qPCR in colorectal carcinoma (CRC), adjacent normal tissue, and separate matched control tissue, we did not detect a signal for F. nucleatum in most samples: 25% of CRCs, 15% of adjacent normal, and 0% of matched control tissue were positive based on quantitative PCR (qPCR) and confirmed by sequencing of the qPCR products. When only samples with detectable F. nucleatum in CRC and adjacent normal tissue were compared, the difference was not statistically significant, while the original study reported a statistically significant increase in F. nucleatum expression in CRC compared to adjacent normal tissue (Figure 2; Castellarin et al., 2012). Finally, we report a meta-analysis of the result, which suggests F. nucleatum expression is increased in CRC, but is confounded by the inability to detect F. nucleatum in most samples. The difference in F. nucleatum expression between CRC and adjacent normal tissues was thus smaller than the original study, and not detected in most samples.
Collapse
|
72
|
Abstract
The transcription factor c-Myc amplifies the transcription of many growth-related genes in cancer cells, but its role as an oncogene is not fully understood.
Collapse
|
73
|
Lewis LM, Edwards MC, Meyers ZR, Talbot CC, Hao H, Blum D. Replication Study: Transcriptional amplification in tumor cells with elevated c-Myc. eLife 2018; 7:30274. [PMID: 29313490 PMCID: PMC5760205 DOI: 10.7554/elife.30274] [Citation(s) in RCA: 26] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2017] [Accepted: 11/16/2017] [Indexed: 12/17/2022] Open
Abstract
As part of the Reproducibility Project: Cancer Biology, we published a Registered Report (Blum et al., 2015), that described how we intended to replicate selected experiments from the paper 'Transcriptional amplification in tumor cells with elevated c-Myc' (Lin et al., 2012). Here we report the results. We found overexpression of c-Myc increased total levels of RNA in P493-6 Burkitt's lymphoma cells; however, while the effect was in the same direction as the original study (Figure 3E; Lin et al., 2012), statistical significance and the size of the effect varied between the original study and the two different lots of serum tested in this replication. Digital gene expression analysis for a set of genes was also performed on P493-6 cells before and after c-Myc overexpression. Transcripts from genes that were active before c-Myc induction increased in expression following c-Myc overexpression, similar to the original study (Figure 3F; Lin et al., 2012). Transcripts from genes that were silent before c-Myc induction also increased in expression following c-Myc overexpression, while the original study concluded elevated c-Myc had no effect on silent genes (Figure 3F; Lin et al., 2012). Treating the data as paired, we found a statistically significant increase in gene expression for both active and silent genes upon c-Myc induction, with the change in gene expression greater for active genes compared to silent genes. Finally, we report meta-analyses for each result.
Collapse
|
74
|
Vanden Heuvel JP, Maddox E, Maalouf SW. Replication Study: Systematic identification of genomic markers of drug sensitivity in cancer cells. eLife 2018; 7. [PMID: 29313488 PMCID: PMC5760202 DOI: 10.7554/elife.29747] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2017] [Accepted: 12/13/2017] [Indexed: 12/11/2022] Open
Abstract
In 2016, as part of the Reproducibility Project: Cancer Biology, we published a Registered Report (Vanden Heuvel et al., 2016), that described how we intended to replicate selected experiments from the paper ‘Systematic identification of genomic markers of drug sensitivity in cancer cells’ (Garnett et al., 2012). Here we report the results. We found Ewing’s sarcoma cell lines, overall, were more sensitive to the PARP inhibitor olaparib than osteosarcoma cell lines; however, while the effect was in the same direction as the original study (Figure 4C; Garnett et al., 2012), it was not statistically significant. Further, mouse mesenchymal cells transformed with either the EWS-FLI1 or FUS-CHOP rearrangement displayed similar sensitivities to olaparib, whereas the Ewing’s sarcoma cell line SK-N-MC had increased olaparib sensitivity. In the original study, mouse mesenchymal cells transformed with the EWS-FLI1 rearrangement and SK-N-MC cells were found to have similar sensitivities to olaparib, whereas mesenchymal cells transformed with the FUS-CHOP rearrangement displayed a reduced sensitivity to olaparib (Figure 4E; Garnett et al., 2012). We also studied another Ewing’s sarcoma cell line, A673: A673 cells depleted of EWS-FLI1 or a negative control both displayed similar sensitivities to olaparib, whereas the original study reported a decreased sensitivity to olaparib when EWS-FLI1 was depleted (Figure 4F; Garnett et al., 2012). Differences between the original study and this replication attempt, such as the use of different sarcoma cell lines and level of knockdown efficiency, are factors that might have influenced the outcomes. Finally, where possible, we report meta-analyses for each result.
Collapse
|
75
|
Plavén-Sigray P, Matheson GJ, Schiffler BC, Thompson WH. The readability of scientific texts is decreasing over time. eLife 2017; 6:e27725. [PMID: 28873054 PMCID: PMC5584989 DOI: 10.7554/elife.27725] [Citation(s) in RCA: 75] [Impact Index Per Article: 10.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2017] [Accepted: 08/04/2017] [Indexed: 11/27/2022] Open
Abstract
Clarity and accuracy of reporting are fundamental to the scientific process. Readability formulas can estimate how difficult a text is to read. Here, in a corpus consisting of 709,577 abstracts published between 1881 and 2015 from 123 scientific journals, we show that the readability of science is steadily decreasing. Our analyses show that this trend is indicative of a growing use of general scientific jargon. These results are concerning for scientists and for the wider public, as they impact both the reproducibility and accessibility of research findings.
Collapse
|