1
|
Scoggins B, Robertson MP. Measuring transparency in the social sciences: political science and international relations. ROYAL SOCIETY OPEN SCIENCE 2024; 11:240313. [PMID: 39076374 PMCID: PMC11285849 DOI: 10.1098/rsos.240313] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/22/2024] [Revised: 05/11/2024] [Accepted: 05/23/2024] [Indexed: 07/31/2024]
Abstract
The scientific method is predicated on transparency-yet the pace at which transparent research practices are being adopted by the scientific community is slow. The replication crisis in psychology showed that published findings employing statistical inference are threatened by undetected errors, data manipulation and data falsification. To mitigate these problems and bolster research credibility, open data and preregistration practices have gained traction in the natural and social sciences. However, the extent of their adoption in different disciplines is unknown. We introduce computational procedures to identify the transparency of a research field using large-scale text analysis and machine learning classifiers. Using political science and international relations as an illustrative case, we examine 93 931 articles across the top 160 political science and international relations journals between 2010 and 2021. We find that approximately 21% of all statistical inference papers have open data and 5% of all experiments are preregistered. Despite this shortfall, the example of leading journals in the field shows that change is feasible and can be effected quickly.
Collapse
Affiliation(s)
- Bermond Scoggins
- School of Politics and International Relations, Australian National University, Canberra, Australia
- School of Social Sciences, Monash University, Melbourne, Australia
| | - Matthew P. Robertson
- School of Politics and International Relations, Australian National University, Canberra, Australia
- Department of Social Data Science, University of Mannheim, Mannheim, Germany
| |
Collapse
|
2
|
Samsa G. Fabrication in a study about honesty: A lost episode of columbo illustrating how forensic statistics is performed. Account Res 2024:1-17. [PMID: 38493503 DOI: 10.1080/08989621.2024.2329265] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2023] [Accepted: 03/07/2024] [Indexed: 03/19/2024]
Abstract
The three steps of a typical forensic statistical analysis are (1) verify that the raw data file is correct; (2) verify that the statistical analysis file derived from the raw data file is correct; and (3) verify that the statistical analyses are appropriate. We illustrate applying these three steps to a manuscript which was subsequently retracted, focusing on step 1. In the absence of an external source for comparison, criteria for assessing the raw data file were internal consistency and plausibility. A forensic statistical analysis isn't like a murder mystery, and it many circumstances discovery of a mechanism for falsification or fabrication might not be realistic.
Collapse
Affiliation(s)
- Greg Samsa
- Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, NC, USA
| |
Collapse
|
3
|
El Amin M, Borders JC, Long HL, Keller MA, Kearney E. Open Science Practices in Communication Sciences and Disorders: A Survey. JOURNAL OF SPEECH, LANGUAGE, AND HEARING RESEARCH : JSLHR 2023; 66:1928-1947. [PMID: 36417765 PMCID: PMC10554559 DOI: 10.1044/2022_jslhr-22-00062] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/27/2022] [Revised: 07/15/2022] [Accepted: 08/07/2022] [Indexed: 06/16/2023]
Abstract
PURPOSE Open science is a collection of practices that seek to improve the accessibility, transparency, and replicability of science. Although these practices have garnered interest in related fields, it remains unclear whether open science practices have been adopted in the field of communication sciences and disorders (CSD). This study aimed to survey the knowledge, implementation, and perceived benefits and barriers of open science practices in CSD. METHOD An online survey was disseminated to researchers in the United States actively engaged in CSD research. Four-core open science practices were examined: preregistration, self-archiving, gold open access, and open data. Data were analyzed using descriptive statistics and regression models. RESULTS Two hundred twenty-two participants met the inclusion criteria. Most participants were doctoral students (38%) or assistant professors (24%) at R1 institutions (58%). Participants reported low knowledge of preregistration and gold open access. There was, however, a high level of desire to learn more for all practices. Implementation of open science practices was also low, most notably for preregistration, gold open access, and open data (< 25%). Predictors of knowledge and participation, as well as perceived barriers to implementation, are discussed. CONCLUSION Although participation in open science appears low in the field of CSD, participants expressed a strong desire to learn more in order to engage in these practices in the future. Supplemental Material and Open Science Form: https://doi.org/10.23641/asha.21569040.
Collapse
Affiliation(s)
- Mariam El Amin
- Communication Sciences and Disorders, University of Georgia, Athens
| | - James C. Borders
- Department of Biobehavioral Sciences, Teacher College, Columbia University, New York, NY
| | | | | | - Elaine Kearney
- Department of Speech, Language and Hearing Sciences, Boston University, MA
| |
Collapse
|
4
|
Broderick T, Gelman A, Meager R, Smith AL, Zheng T. Toward a taxonomy of trust for probabilistic machine learning. SCIENCE ADVANCES 2023; 9:eabn3999. [PMID: 36791188 PMCID: PMC9931201 DOI: 10.1126/sciadv.abn3999] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/23/2021] [Accepted: 01/13/2023] [Indexed: 06/18/2023]
Abstract
Probabilistic machine learning increasingly informs critical decisions in medicine, economics, politics, and beyond. To aid the development of trust in these decisions, we develop a taxonomy delineating where trust in an analysis can break down: (i) in the translation of real-world goals to goals on a particular set of training data, (ii) in the translation of abstract goals on the training data to a concrete mathematical problem, (iii) in the use of an algorithm to solve the stated mathematical problem, and (iv) in the use of a particular code implementation of the chosen algorithm. We detail how trust can fail at each step and illustrate our taxonomy with two case studies. Finally, we describe a wide variety of methods that can be used to increase trust at each step of our taxonomy. The use of our taxonomy highlights not only steps where existing research work on trust tends to concentrate and but also steps where building trust is particularly challenging.
Collapse
Affiliation(s)
- Tamara Broderick
- Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Andrew Gelman
- Department of Statistics, Columbia University, New York, NY, USA
- Department of Political Science, Columbia University, New York, NY, USA
| | - Rachael Meager
- Department of Economics, London School of Economics and Political Science, London, UK
| | - Anna L. Smith
- Department of Statistics, University of Kentucky, Lexington, KY, USA
| | - Tian Zheng
- Department of Statistics, Columbia University, New York, NY, USA
| |
Collapse
|
5
|
Morrow M, MaWhinney S, Brooks KM, Huntley R, Castillo-Mancilla JR, Anderson PL, Kiser JJ. Improving the accuracy of adherence data collected using medication monitoring technology in clinical research. Contemp Clin Trials 2023; 125:107051. [PMID: 36528338 DOI: 10.1016/j.cct.2022.107051] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2022] [Revised: 12/09/2022] [Accepted: 12/12/2022] [Indexed: 12/23/2022]
Abstract
Ambiguous adherence data adversely effects the statistical analyses, study conclusions, and generalizability of research findings for clinical trials. Fortunately, technology-based measures of drug dosing provide more objective measures of medication adherence. While adherence data obtained through monitoring technology avoids the well documented shortcomings of self-reported adherence data, there are important limitations and nuances with use of these technologies that should be considered at study inception, conduct, and analysis. This article describes considerations for data collection and management with use of electronic adherence monitoring, specifically mobile-phone video applications or electronic pillbox devices. The overall aim of this communique is to provide research teams the ability to collect more accurate adherence data and ultimately improve the quality and outcome of their research.
Collapse
Affiliation(s)
- Mary Morrow
- Department of Biostatistics and Informatics, Colorado School of Public Health, Anschutz Medical Campus, Aurora, CO, USA
| | - Samantha MaWhinney
- Department of Biostatistics and Informatics, Colorado School of Public Health, Anschutz Medical Campus, Aurora, CO, USA
| | - Kristina M Brooks
- Department of Pharmaceutical Sciences, Skaggs School of Pharmacy and Pharmaceutical Sciences, University of Colorado Anschutz Medical Campus, Aurora, CO, USA
| | - Ryan Huntley
- Department of Pharmaceutical Sciences, Skaggs School of Pharmacy and Pharmaceutical Sciences, University of Colorado Anschutz Medical Campus, Aurora, CO, USA
| | - Jose R Castillo-Mancilla
- Division of Infectious Diseases, School of Medicine, University of Colorado Anschutz Medical Campus, Aurora, CO, USA
| | - Peter L Anderson
- Department of Pharmaceutical Sciences, Skaggs School of Pharmacy and Pharmaceutical Sciences, University of Colorado Anschutz Medical Campus, Aurora, CO, USA
| | - Jennifer J Kiser
- Department of Pharmaceutical Sciences, Skaggs School of Pharmacy and Pharmaceutical Sciences, University of Colorado Anschutz Medical Campus, Aurora, CO, USA.
| |
Collapse
|
6
|
Master SR. The Case for Including Data and Code with ML Publications in Laboratory Medicine. J Appl Lab Med 2023; 8:213-216. [PMID: 36610411 DOI: 10.1093/jalm/jfac088] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2022] [Accepted: 09/21/2022] [Indexed: 01/09/2023]
Affiliation(s)
- Stephen R Master
- Department of Pathology and Laboratory Medicine, Children's Hospital of Philadelphia, Philadelphia, PA, USA.,Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| |
Collapse
|
7
|
Raman R, Utts J, Cohen AI, Hayat MJ. Integrating Ethics into the Guidelines for Assessment and Instruction in Statistics Education (GAISE). AM STAT 2022. [DOI: 10.1080/00031305.2022.2156612] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Affiliation(s)
- Rameela Raman
- Department of Biostatistics, School of Medicine, Vanderbilt University, Nashville, TN, USA
| | - Jessica Utts
- Department of Statistics, University of California, Irvine, CA, USA
| | - Andrew I. Cohen
- Department of Philosophy and Jean Beer Blumenfeld Center for Ethics, Georgia State University, Atlanta, GA, USA
| | - Matthew J. Hayat
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA
| |
Collapse
|
8
|
Herman DS, Rhoads DD, Schulz WL, Durant TJS. Artificial Intelligence and Mapping a New Direction in Laboratory Medicine: A Review. Clin Chem 2021; 67:1466-1482. [PMID: 34557917 DOI: 10.1093/clinchem/hvab165] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2021] [Accepted: 07/26/2021] [Indexed: 12/21/2022]
Abstract
BACKGROUND Modern artificial intelligence (AI) and machine learning (ML) methods are now capable of completing tasks with performance characteristics that are comparable to those of expert human operators. As a result, many areas throughout healthcare are incorporating these technologies, including in vitro diagnostics and, more broadly, laboratory medicine. However, there are limited literature reviews of the landscape, likely future, and challenges of the application of AI/ML in laboratory medicine. CONTENT In this review, we begin with a brief introduction to AI and its subfield of ML. The ensuing sections describe ML systems that are currently in clinical laboratory practice or are being proposed for such use in recent literature, ML systems that use laboratory data outside the clinical laboratory, challenges to the adoption of ML, and future opportunities for ML in laboratory medicine. SUMMARY AI and ML have and will continue to influence the practice and scope of laboratory medicine dramatically. This has been made possible by advancements in modern computing and the widespread digitization of health information. These technologies are being rapidly developed and described, but in comparison, their implementation thus far has been modest. To spur the implementation of reliable and sophisticated ML-based technologies, we need to establish best practices further and improve our information system and communication infrastructure. The participation of the clinical laboratory community is essential to ensure that laboratory data are sufficiently available and incorporated conscientiously into robust, safe, and clinically effective ML-supported clinical diagnostics.
Collapse
Affiliation(s)
- Daniel S Herman
- Department of Pathology and Laboratory Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Daniel D Rhoads
- Department of Laboratory Medicine, Cleveland Clinic, Cleveland, OH, USA.,Department of Pathology, Cleveland Clinic Lerner College of Medicine, Case Western Reserve University, Cleveland, OH, USA
| | - Wade L Schulz
- Department of Laboratory Medicine, Yale University, New Haven, CT, USA
| | - Thomas J S Durant
- Department of Laboratory Medicine, Yale University, New Haven, CT, USA
| |
Collapse
|
9
|
Holmes DT, Mobini M, McCudden CR. Reproducible manuscript preparation with RMarkdown application to JMSACL and other Elsevier Journals. J Mass Spectrom Adv Clin Lab 2021; 22:8-16. [PMID: 34939050 PMCID: PMC8662334 DOI: 10.1016/j.jmsacl.2021.09.002] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022] Open
Abstract
INTRODUCTION With the rising complexity of modern multimarker analytical techniques and notable scientific publication retractions required for erroneous statistical analysis, there is increasing awareness of the importance of research transparency and reproducibility. The development of mature open-source tools for literate programming in multiple langauge paradigms has made fully-reproducible authorship possible. OBJECTIVES We describe the procedure for manuscript preparation using RMarkdown and the R statistical programming language with application to JMSACL or any other Elsevier journal. METHODS An instructional manuscript has been prepared in the RMarkdown markup language with stepwise directions on preparing sections, subsections, lists, tables, figures and reference management in an entirely reproducible format. RESULTS From RMarkdown code, a submission-ready PDF is generated and JMSACL-compatible LaTeX code is generated. These can be uploaded to the Editorial Manager. CONCLUSION A completely reproducible manuscript preparation pipeline using the R and RMarkdown is described.
Collapse
Affiliation(s)
- Daniel T. Holmes
- St. Paul’s Hospital, Department of Pathology and Laboratory Medicine, 1081 Burrard St., Vancouver, BC V6Z 1Y6, Canada
- University of British, Columbia Department of Pathology and Laboratory Medicine, 2211 Wesbrook Mall, Vancouver, BC V6T 1Z7, Canada
| | - Mahdi Mobini
- St. Paul’s Hospital, Department of Pathology and Laboratory Medicine, 1081 Burrard St., Vancouver, BC V6Z 1Y6, Canada
- Providence Health Digital Products, 1190 Hornby St., Vancouver, BC V6Z 2K5, Canada
| | - Christopher R. McCudden
- Department of Pathology and Laboratory Medicine, Ottawa Hospital, General Campus, 501 Smyth Road, Ottawa, ON K1H 8L6, Canada
- Department of Pathology and Laboratory Medicine, University of Ottawa, Canada
- Eastern Ontario Regional Laboratory Association, Canada
| |
Collapse
|
10
|
Abeysooriya M, Soria M, Kasu MS, Ziemann M. Gene name errors: Lessons not learned. PLoS Comput Biol 2021; 17:e1008984. [PMID: 34329294 PMCID: PMC8357140 DOI: 10.1371/journal.pcbi.1008984] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2021] [Revised: 08/11/2021] [Accepted: 07/01/2021] [Indexed: 02/01/2023] Open
Abstract
Erroneous conversion of gene names into other dates and other data types has been a frustration for computational biologists for years. We hypothesized that such errors in supplementary files might diminish after a report in 2016 highlighting the extent of the problem. To assess this, we performed a scan of supplementary files published in PubMed Central from 2014 to 2020. Overall, gene name errors continued to accumulate unabated in the period after 2016. An improved scanning software we developed identified gene name errors in 30.9% (3,436/11,117) of articles with supplementary Excel gene lists; a figure significantly higher than previously estimated. This is due to gene names being converted not just to dates and floating-point numbers, but also to internal date format (five-digit numbers). These findings further reinforce that spreadsheets are ill-suited to use with large genomic data.
Collapse
Affiliation(s)
- Mandhri Abeysooriya
- Deakin University, School of Life and Environmental Sciences, Geelong, Australia
| | - Megan Soria
- Deakin University, School of Life and Environmental Sciences, Geelong, Australia
| | - Mary Sravya Kasu
- Deakin University, School of Life and Environmental Sciences, Geelong, Australia
| | - Mark Ziemann
- Deakin University, School of Life and Environmental Sciences, Geelong, Australia
- * E-mail:
| |
Collapse
|
11
|
From the Digital Data Revolution toward a Digital Society: Pervasiveness of Artificial Intelligence. MACHINE LEARNING AND KNOWLEDGE EXTRACTION 2021. [DOI: 10.3390/make3010014] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/01/2023]
Abstract
Technological progress has led to powerful computers and communication technologies that penetrate nowadays all areas of science, industry and our private lives. As a consequence, all these areas are generating digital traces of data amounting to big data resources. This opens unprecedented opportunities but also challenges toward the analysis, management, interpretation and responsible usage of such data. In this paper, we discuss these developments and the fields that have been particularly effected by the digital revolution. Our discussion is AI-centered showing domain-specific prospects but also intricacies for the method development in artificial intelligence. For instance, we discuss recent breakthroughs in deep learning algorithms and artificial intelligence as well as advances in text mining and natural language processing, e.g., word-embedding methods that enable the processing of large amounts of text data from diverse sources such as governmental reports, blog entries in social media or clinical health records of patients. Furthermore, we discuss the necessity of further improving general artificial intelligence approaches and for utilizing advanced learning paradigms. This leads to arguments for the establishment of statistical artificial intelligence. Finally, we provide an outlook on important aspects of future challenges that are of crucial importance for the development of all fields, including ethical AI and the influence of bias on AI systems. As potential end-point of this development, we define digital society as the asymptotic limiting state of digital economy that emerges from fully connected information and communication technologies enabling the pervasiveness of AI. Overall, our discussion provides a perspective on the elaborate relatedness of digital data and AI systems.
Collapse
|
12
|
Abstract
Advances in computing technology have spurred two extraordinary phenomena in science: large-scale and high-throughput data collection coupled with the creation and implementation of complex statistical algorithms for data analysis. These two phenomena have brought about tremendous advances in scientific discovery but have raised two serious concerns. The complexity of modern data analyses raises questions about the reproducibility of the analyses, meaning the ability of independent analysts to recreate the results claimed by the original authors using the original data and analysis techniques. Reproducibility is typically thwarted by a lack of availability of the original data and computer code. A more general concern is the replicability of scientific findings, which concerns the frequency with which scientific claims are confirmed by completely independent investigations. Although reproducibility and replicability are related, they focus on different aspects of scientific progress. In this review, we discuss the origins of reproducible research, characterize the current status of reproducibility in public health research, and connect reproducibility to current concerns about the replicability of scientific findings. Finally, we describe a path forward for improving both the reproducibility and replicability of public health research in the future.
Collapse
Affiliation(s)
- Roger D Peng
- Department of Biostatistics, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, Maryland 21205, USA; ,
| | - Stephanie C Hicks
- Department of Biostatistics, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, Maryland 21205, USA; ,
| |
Collapse
|
13
|
Robert J. Une fraude exemplaire. Bull Cancer 2021; 108:145-150. [PMID: 33423782 DOI: 10.1016/j.bulcan.2020.12.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2020] [Accepted: 12/16/2020] [Indexed: 10/22/2022]
Affiliation(s)
- Jacques Robert
- Université de Bordeaux, Inserm U1218, 229, cours de l'Argonne, 33076 Bordeaux, France.
| |
Collapse
|
14
|
Encoding Medical Experiments in Jupyter Notebooks. SYSTEMS MEDICINE 2021. [DOI: 10.1016/b978-0-12-801238-3.11687-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022] Open
|
15
|
Inference of Networks from Large Datasets. SYSTEMS MEDICINE 2021. [DOI: 10.1016/b978-0-12-801238-3.11345-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
|
16
|
Hall LM, Hendricks AE. High-throughput analysis suggests differences in journal false discovery rate by subject area and impact factor but not open access status. BMC Bioinformatics 2020; 21:564. [PMID: 33297936 PMCID: PMC7724881 DOI: 10.1186/s12859-020-03817-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2020] [Accepted: 10/13/2020] [Indexed: 12/03/2022] Open
Abstract
Background A low replication rate has been reported in some scientific areas motivating the creation of resource intensive collaborations to estimate the replication rate by repeating individual studies. The substantial resources required by these projects limits the number of studies that can be repeated and consequently the generalizability of the findings. We extend the use of a method from Jager and Leek to estimate the false discovery rate for 94 journals over a 5-year period using p values from over 30,000 abstracts enabling the study of how the false discovery rate varies by journal characteristics.
Results We find that the empirical false discovery rate is higher for cancer versus general medicine journals (p = 9.801E−07, 95% CI: 0.045, 0.097; adjusted mean false discovery rate cancer = 0.264 vs. general medicine = 0.194). We also find that false discovery rate is negatively associated with log journal impact factor. A two-fold decrease in journal impact factor is associated with an average increase of 0.020 in FDR (p = 2.545E−04). Conversely, we find no statistically significant evidence of a higher false discovery rate, on average, for Open Access versus closed access journals (p = 0.320, 95% CI − 0.015, 0.046, adjusted mean false discovery rate Open Access = 0.241 vs. closed access = 0.225). Conclusions Our results identify areas of research that may need additional scrutiny and support to facilitate replicable science. Given our publicly available R code and data, others can complete a broad assessment of the empirical false discovery rate across other subject areas and characteristics of published research.
Collapse
Affiliation(s)
- L M Hall
- Mathematical and Statistical Sciences, University of Colorado Denver, Denver, CO, USA
| | - A E Hendricks
- Mathematical and Statistical Sciences, University of Colorado Denver, Denver, CO, USA. .,Biostatistics and Informatics, Colorado School of Public Health, Aurora, CO, USA.
| |
Collapse
|
17
|
Welty LJ, Rasmussen LV, Baldridge AS, Whitley EW. Facilitating reproducible research through direct connection of data analysis with manuscript preparation: StatTag for connecting statistical software to Microsoft Word. JAMIA Open 2020; 3:342-358. [PMID: 33215069 PMCID: PMC7660954 DOI: 10.1093/jamiaopen/ooaa043] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2020] [Revised: 08/06/2020] [Accepted: 09/09/2020] [Indexed: 12/14/2022] Open
Abstract
Objectives To enhance reproducible research by creating a broadly accessible, free, open-source software tool for connecting Microsoft Word to statistical programs (R/R Markdown, Python, SAS, Stata) so that results may be automatically updated in a manuscript. Materials and Methods We developed StatTag for Windows as a Microsoft Word plug-in using C# and for macOS as a native application using Objective-C. Source code is available under the MIT license at https://github.com/stattag. Results StatTag links analysis file(s) (R/R Markdown, SAS, Stata, or Python) and a Word document, invokes the statistical program(s) to obtain results, and embeds selected output in the document. StatTag can accommodate multiple statistical programs with a single document and features an interface to view, edit, and rerun statistical code directly from Word. Discussion and Conclusion StatTag may facilitate reproducibility within increasingly multidisciplinary research teams, improve research transparency through review and publication, and complement data-sharing initiatives.
Collapse
Affiliation(s)
- Leah J Welty
- Division of Biostatistics, Department of Preventive Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA
| | - Luke V Rasmussen
- Division of Health and Biomedical Informatics, Department of Preventive Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA
| | - Abigail S Baldridge
- Department of Preventive Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA
| | - Eric W Whitley
- Department of Preventive Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA
| |
Collapse
|
18
|
Huang L, Luo H, Li S, Wu FX, Wang J. Drug-drug similarity measure and its applications. Brief Bioinform 2020; 22:5956929. [PMID: 33152756 DOI: 10.1093/bib/bbaa265] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2020] [Revised: 09/13/2020] [Accepted: 09/14/2020] [Indexed: 02/01/2023] Open
Abstract
Drug similarities play an important role in modern biology and medicine, as they help scientists gain deep insights into drugs' therapeutic mechanisms and conduct wet labs that may significantly improve the efficiency of drug research and development. Nowadays, a number of drug-related databases have been constructed, with which many methods have been developed for computing similarities between drugs for studying associations between drugs, human diseases, proteins (drug targets) and more. In this review, firstly, we briefly introduce the publicly available drug-related databases. Secondly, based on different drug features, interaction relationships and multimodal data, we summarize similarity calculation methods in details. Then, we discuss the applications of drug similarities in various biological and medical areas. Finally, we evaluate drug similarity calculation methods with common evaluation metrics to illustrate the important roles of drug similarity measures on different applications.
Collapse
Affiliation(s)
- Lan Huang
- Hunan Provincial Key Lab of Bioinformatics, School of Computer Science and Engineering at Central South University, Hunan, China
| | - Huimin Luo
- School of Computer and Information Engineering at Henan University, Kaifeng, China
| | - Suning Li
- Hunan Provincial Key Lab of Bioinformatics, School of Computer Science and Engineering, Central South University, Changsha, Hunan, China
| | - Fang-Xiang Wu
- College of Engineering and Department of Computer Sciences, University of Saskatchewan, Saskatoon, Canada
| | - Jianxin Wang
- Hunan Provincial Key Lab of Bioinformatics, School of Computer Science and Engineering at Central South University, Hunan, China
| |
Collapse
|
19
|
Quantify and control reproducibility in high-throughput experiments. Nat Methods 2020; 17:1207-1213. [PMID: 33046893 DOI: 10.1038/s41592-020-00978-4] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2020] [Accepted: 09/14/2020] [Indexed: 11/09/2022]
Abstract
Ensuring reproducibility of results in high-throughput experiments is crucial for biomedical research. Here, we propose a set of computational methods, INTRIGUE, to evaluate and control reproducibility in high-throughput settings. Our approaches are built on a new definition of reproducibility that emphasizes directional consistency when experimental units are assessed with signed effect size estimates. The proposed methods are designed to (1) assess the overall reproducible quality of multiple studies and (2) evaluate reproducibility at the individual experimental unit levels. We demonstrate the proposed methods in detecting unobserved batch effects via simulations. We further illustrate the versatility of the proposed methods in transcriptome-wide association studies: in addition to reproducible quality control, they are also suited to investigating genuine biological heterogeneity. Finally, we discuss the potential extensions of the proposed methods in other vital areas of reproducible research (for example, publication bias and conceptual replications).
Collapse
|
20
|
Valdez D, Vorland CJ, Brown AW, Mayo-Wilson E, Otten J, Ball R, Grant S, Levy R, Svetina Valdivia D, Allison DB. Improving open and rigorous science: ten key future research opportunities related to rigor, reproducibility, and transparency in scientific research. F1000Res 2020; 9:1235. [PMID: 33628434 PMCID: PMC7898357 DOI: 10.12688/f1000research.26594.1] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 10/02/2020] [Indexed: 11/20/2022] Open
Abstract
Background: As part of a coordinated effort to expand research activity around rigor, reproducibility, and transparency (RRT) across scientific disciplines, a team of investigators at the Indiana University School of Public Health-Bloomington hosted a workshop in October 2019 with international leaders to discuss key opportunities for RRT research. Objective: The workshop aimed to identify research priorities and opportunities related to RRT. Design: Over two-days, workshop attendees gave presentations and participated in three working groups: (1) Improving Education & Training in RRT, (2) Reducing Statistical Errors and Increasing Analytic Transparency, and (3) Looking Outward: Increasing Truthfulness and Accuracy of Research Communications. Following small-group discussions, the working groups presented their findings, and participants discussed the research opportunities identified. The investigators compiled a list of research priorities, which were circulated to all participants for feedback. Results: Participants identified the following priority research questions: (1) Can RRT-focused statistics and mathematical modeling courses improve statistics practice?; (2) Can specialized training in scientific writing improve transparency?; (3) Does modality (e.g. face to face, online) affect the efficacy RRT-related education?; (4) How can automated programs help identify errors more efficiently?; (5) What is the prevalence and impact of errors in scientific publications (e.g., analytic inconsistencies, statistical errors, and other objective errors)?; (6) Do error prevention workflows reduce errors?; (7) How do we encourage post-publication error correction?; (8) How does 'spin' in research communication affect stakeholder understanding and use of research evidence?; (9) Do tools to aid writing research reports increase comprehensiveness and clarity of research reports?; and (10) Is it possible to inculcate scientific values and norms related to truthful, rigorous, accurate, and comprehensive scientific reporting? Conclusion: Participants identified important and relatively unexplored questions related to improving RRT. This list may be useful to the scientific community and investigators seeking to advance meta-science (i.e. research on research).
Collapse
Affiliation(s)
- Danny Valdez
- Indiana University School of Public Health, Bloomington, IN, 47403, USA
| | - Colby J. Vorland
- Indiana University School of Public Health, Bloomington, IN, 47403, USA
| | - Andrew W. Brown
- Indiana University School of Public Health, Bloomington, IN, 47403, USA
| | - Evan Mayo-Wilson
- Indiana University School of Public Health, Bloomington, IN, 47403, USA
| | - Justin Otten
- Indiana University School of Public Health, Bloomington, IN, 47403, USA
| | - Richard Ball
- Project TIER, Haverford College, Haverford, Pennsylvania, 19041, USA
| | - Sean Grant
- Indiana University Purdue University Indianapolis Fairbanks School of Public Health, Indianapolis, IN, 46223, USA
| | - Rachel Levy
- Rachel Levy, Mathematical Association of America, 1529 18th St. NW, Washington, DC, 20036, USA
| | | | - David B. Allison
- Indiana University School of Public Health, Bloomington, IN, 47403, USA
| |
Collapse
|
21
|
Di Leo G, Sardanelli F. Statistical significance: p value, 0.05 threshold, and applications to radiomics-reasons for a conservative approach. Eur Radiol Exp 2020; 4:18. [PMID: 32157489 PMCID: PMC7064671 DOI: 10.1186/s41747-020-0145-y] [Citation(s) in RCA: 108] [Impact Index Per Article: 27.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2019] [Accepted: 01/23/2020] [Indexed: 12/17/2022] Open
Abstract
Here, we summarise the unresolved debate about p value and its dichotomisation. We present the statement of the American Statistical Association against the misuse of statistical significance as well as the proposals to abandon the use of p value and to reduce the significance threshold from 0.05 to 0.005. We highlight reasons for a conservative approach, as clinical research needs dichotomic answers to guide decision-making, in particular in the case of diagnostic imaging and interventional radiology. With a reduced p value threshold, the cost of research could increase while spontaneous research could be reduced. Secondary evidence from systematic reviews/meta-analyses, data sharing, and cost-effective analyses are better ways to mitigate the false discovery rate and lack of reproducibility associated with the use of the 0.05 threshold. Importantly, when reporting p values, authors should always provide the actual value, not only statements of "p < 0.05" or "p ≥ 0.05", because p values give a measure of the degree of data compatibility with the null hypothesis. Notably, radiomics and big data, fuelled by the application of artificial intelligence, involve hundreds/thousands of tested features similarly to other "omics" such as genomics, where a reduction in the significance threshold, based on well-known corrections for multiple testing, has been already adopted.
Collapse
Affiliation(s)
- Giovanni Di Leo
- Radiology Unit, IRCCS Policlinico San Donato, Via Morandi 30, 20097, San Donato Milanese, Italy.
| | - Francesco Sardanelli
- Radiology Unit, IRCCS Policlinico San Donato, Via Morandi 30, 20097, San Donato Milanese, Italy
- Dipartimento di Scienze Biomediche per la Salute, Università degli Studi di Milano, Via Morandi 30, 20097, San Donato Milanese, Italy
| |
Collapse
|
22
|
Emmert-Streib F, Dehmer M, Yli-Harja O. Ensuring Quality Standards and Reproducible Research for Data Analysis Services in Oncology: A Cooperative Service Model. Front Cell Dev Biol 2020; 7:349. [PMID: 31921859 PMCID: PMC6929679 DOI: 10.3389/fcell.2019.00349] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2019] [Accepted: 12/04/2019] [Indexed: 11/13/2022] Open
Abstract
Modern molecular high-throughput devices, e.g., next-generation sequencing, have transformed medical research. Resulting data sets are usually high-dimensional on a genomic-scale providing multi-factorial information from intertwined molecular and cellular activities of genes and their products. This genomics-revolution installed precision medicine offering breathtaking opportunities for patient's diagnosis and treatment. However, due to the speed of these developments the quality standards of the involved data analyses are lacking behind, as exemplified by the infamous Duke Saga. In this paper, we argue in favor of a two-stage cooperative serve model that couples data generation and data analysis in the most beneficial way from the perspective of a patient to ensure data analysis quality standards including reproducible research.
Collapse
Affiliation(s)
- Frank Emmert-Streib
- Predictive Society and Data Analytics Lab, Faculty of Information Technology and Communication Sciences, Tampere University, Tampere, Finland.,Institute of Biosciences and Medical Technology, Tampere, Finland
| | - Matthias Dehmer
- Steyr School of Management, University of Applied Sciences Upper Austria, Steyr, Austria.,Department of Mechatronics and Biomedical Computer Science, UMIT, Hall in Tyrol, Austria.,College of Artificial Intelligence, Nankai University, Tianjin, China
| | - Olli Yli-Harja
- Institute of Biosciences and Medical Technology, Tampere, Finland.,Institute for Systems Biology, Seattle, WA, United States
| |
Collapse
|
23
|
Rutter L, Moran Lauter AN, Graham MA, Cook D. Visualization methods for differential expression analysis. BMC Bioinformatics 2019; 20:458. [PMID: 31492109 PMCID: PMC6731617 DOI: 10.1186/s12859-019-2968-1] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2019] [Accepted: 06/25/2019] [Indexed: 12/30/2022] Open
Abstract
BACKGROUND Despite the availability of many ready-made testing software, reliable detection of differentially expressed genes in RNA-seq data is not a trivial task. Even though the data collection is considered high-throughput, data analysis has intricacies that require careful human attention. Researchers should use modern data analysis techniques that incorporate visual feedback to verify the appropriateness of their models. While some RNA-seq packages provide static visualization tools, their capabilities should be expanded and their meaningfulness should be explicitly demonstrated to users. RESULTS In this paper, we 1) introduce new interactive RNA-seq visualization tools, 2) compile a collection of examples that demonstrate to biologists why visualization should be an integral component of differential expression analysis. We use public RNA-seq datasets to show that our new visualization tools can detect normalization issues, differential expression designation problems, and common analysis errors. We also show that our new visualization tools can identify genes of interest in ways undetectable with models. Our R package "bigPint" includes the plotting tools introduced in this paper, many of which are unique additions to what is currently available. The "bigPint" website is located at https://lindsayrutter.github.io/bigPint and contains short vignette articles that introduce new users to our package, all written in reproducible code. CONCLUSIONS We emphasize that interactive graphics should be an indispensable component of modern RNA-seq analysis, which is currently not the case. This paper and its corresponding software aim to persuade 1) users to slightly modify their differential expression analyses by incorporating statistical graphics into their usual analysis pipelines, 2) developers to create additional complex and interactive plotting methods for RNA-seq data, possibly using lessons learned from our open-source codes. We hope our work will serve a small part in upgrading the RNA-seq analysis world into one that more wholistically extracts biological information using both models and visuals.
Collapse
Affiliation(s)
- Lindsay Rutter
- Bioinformatics and Computational Biology Program, Iowa State University, Ames, USA
| | | | - Michelle A. Graham
- USDA-ARS, Corn Insects and Crop Genetics Research Unit, Ames, USA
- Department of Agronomy, Iowa State University, Ames, USA
| | - Dianne Cook
- Econometrics and Business Statistics, Monash University, Clayton VIC, Australia
| |
Collapse
|
24
|
|
25
|
Reproducibility and replicability in zebrafish behavioral neuroscience research. Pharmacol Biochem Behav 2019; 178:30-38. [DOI: 10.1016/j.pbb.2018.02.005] [Citation(s) in RCA: 51] [Impact Index Per Article: 10.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/15/2017] [Revised: 02/06/2018] [Accepted: 02/22/2018] [Indexed: 12/26/2022]
|
26
|
Holmes DT, Buhr KA. Widespread Incorrect Implementation of the Hoffmann Method, the Correct Approach, and Modern Alternatives. Am J Clin Pathol 2019; 151:328-336. [PMID: 30475946 DOI: 10.1093/ajcp/aqy149] [Citation(s) in RCA: 31] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2023] Open
Abstract
Objectives The Hoffmann method is a procedure for reference interval estimation using routine clinical results. Many authors incorrectly prepare Hoffmann plots on a linear rather than normal probability scale. We explore the consequences. Methods This was investigated algebraically, by random number simulations (45 simulations, n = 100,000 each) and using clinical data sets. Strategies compared were: Hoffmann's method as originally and incorrectly implemented, Bhattacharya's method, and maximum likelihood (ML). All R source code and data sets are provided. Results As the proportion of healthy individuals approaches 1, the incorrect approach generates reference interval estimates of approximately μH ± 1.19 σH delineating the central 77% of the healthy subpopulation, not the central 95%. Inappropriately narrow reference interval estimates were seen on random simulations and clinical data sets. ML methods performed best. Conclusions The erroneous variant Hoffmann method should not be used. ML methods outperform others and are not restricted by Gaussian assumptions.
Collapse
Affiliation(s)
- Daniel T Holmes
- Department of Pathology and Laboratory Medicine, St Paul’s Hospital, Vancouver, Canada
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, Canada
| | - Kevin A Buhr
- Biostatistics and Medical Informatics, University of Wisconsin, Madison
| |
Collapse
|
27
|
Wright ML, Higgins M, Taylor JY, Hertzberg VS. NuRsing Research in the 21st Century: R You Ready? Biol Res Nurs 2018; 21:114-120. [PMID: 30384771 DOI: 10.1177/1099800418810514] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/11/2022]
Abstract
Nurse scientists are adept at translating findings from basic science into useful clinical- and community-based interventions to improve health. Over time, the focus of some nursing research has grown to include the assessment and evaluation of genomic and other output from high-throughput, or "omic," technologies as indicators related to health and disease. To date, the growth in the application of omics technologies in nursing research has included calls to increase attention to omics in nursing school curricula and educational training opportunities, such as the Summer Genetics Institute offered by the National Institute of Nursing Research. However, there has been scant attention paid in the nursing literature to the complexity of data analysis or issues of reproducibility related to omics studies. The goals of this article are to (1) familiarize nurse scientists with tools that encourage reproducibility in omics studies, with a focus on the free and open-source data processing and analysis pipeline, and (2) provide a baseline understanding of how these tools can be used to improve collaboration and cohesion among interdisciplinary research team members. Knowledge of these tools and skill in applying them will be important for communication across disciplines and imperative for the advancement of omics research in nursing.
Collapse
Affiliation(s)
| | - Melinda Higgins
- 2 Nell Hodgson Woodruff School of Nursing, Emory University, Atlanta, GA, USA
| | - Jacquelyn Y Taylor
- 3 Rory Meyers College of Nursing, New York University, Emory, Atlanta, USA
| | | |
Collapse
|
28
|
Kang J, Rancati T, Lee S, Oh JH, Kerns SL, Scott JG, Schwartz R, Kim S, Rosenstein BS. Machine Learning and Radiogenomics: Lessons Learned and Future Directions. Front Oncol 2018; 8:228. [PMID: 29977864 PMCID: PMC6021505 DOI: 10.3389/fonc.2018.00228] [Citation(s) in RCA: 42] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2018] [Accepted: 06/04/2018] [Indexed: 12/25/2022] Open
Abstract
Due to the rapid increase in the availability of patient data, there is significant interest in precision medicine that could facilitate the development of a personalized treatment plan for each patient on an individual basis. Radiation oncology is particularly suited for predictive machine learning (ML) models due to the enormous amount of diagnostic data used as input and therapeutic data generated as output. An emerging field in precision radiation oncology that can take advantage of ML approaches is radiogenomics, which is the study of the impact of genomic variations on the sensitivity of normal and tumor tissue to radiation. Currently, patients undergoing radiotherapy are treated using uniform dose constraints specific to the tumor and surrounding normal tissues. This is suboptimal in many ways. First, the dose that can be delivered to the target volume may be insufficient for control but is constrained by the surrounding normal tissue, as dose escalation can lead to significant morbidity and rare. Second, two patients with nearly identical dose distributions can have substantially different acute and late toxicities, resulting in lengthy treatment breaks and suboptimal control, or chronic morbidities leading to poor quality of life. Despite significant advances in radiogenomics, the magnitude of the genetic contribution to radiation response far exceeds our current understanding of individual risk variants. In the field of genomics, ML methods are being used to extract harder-to-detect knowledge, but these methods have yet to fully penetrate radiogenomics. Hence, the goal of this publication is to provide an overview of ML as it applies to radiogenomics. We begin with a brief history of radiogenomics and its relationship to precision medicine. We then introduce ML and compare it to statistical hypothesis testing to reflect on shared lessons and to avoid common pitfalls. Current ML approaches to genome-wide association studies are examined. The application of ML specifically to radiogenomics is next presented. We end with important lessons for the proper integration of ML into radiogenomics.
Collapse
Affiliation(s)
- John Kang
- Department of Radiation Oncology, University of Rochester Medical Center, Rochester, NY, United States
| | - Tiziana Rancati
- Prostate Cancer Program, Fondazione IRCCS Istituto Nazionale dei Tumori, Milan, Italy
| | - Sangkyu Lee
- Department of Medical Physics, Memorial Sloan Kettering Cancer Center, New York, NY, United States
| | - Jung Hun Oh
- Department of Medical Physics, Memorial Sloan Kettering Cancer Center, New York, NY, United States
| | - Sarah L. Kerns
- Department of Radiation Oncology, University of Rochester Medical Center, Rochester, NY, United States
| | - Jacob G. Scott
- Department of Translational Hematology and Oncology Research, Cleveland Clinic, Cleveland, OH, United States
- Department of Radiation Oncology, Cleveland Clinic, Cleveland, OH, United States
| | - Russell Schwartz
- Computational Biology Department, Carnegie Mellon School of Computer Science, Pittsburgh, PA, United States
- Department of Biological Sciences, Carnegie Mellon University, Pittsburgh, PA, United States
| | - Seyoung Kim
- Computational Biology Department, Carnegie Mellon School of Computer Science, Pittsburgh, PA, United States
| | - Barry S. Rosenstein
- Department of Radiation Oncology, Icahn School of Medicine at Mount Sinai, New York, NY, United States
- Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY, United States
| |
Collapse
|
29
|
Abstract
Big Data is increasingly prevalent in science and data analysis. We provide a short tutorial for adapting to these changes and making the necessary adjustments to the academic culture to keep Biostatistics truly impactful in scientific research.
Collapse
Affiliation(s)
- Ekaterina Smirnova
- Assistant Professor, Department of Mathematical Sciences, University of Montana, 32 Campus Dr, Missoula, MT 59812
| | - Andrada Ivanescu
- Assistant Professor, Department of Mathematical Sciences, Montclair University, 1 Normal Avenue Montclair, NJ 07043
| | - Jiawei Bai
- Assistant Scientist, Department of Biostatistics, Johns Hopkins University, 615 N. Wolfe St. Baltimore, MD 21205 USA
| | - Ciprian M Crainiceanu
- Professor, Department of Biostatistics, Johns Hopkins University, 615 N. Wolfe St. Baltimore, MD 21205 USA
| |
Collapse
|
30
|
Gerhards NM, Blomen VA, Mutlu M, Nieuwenhuis J, Howald D, Guyader C, Jonkers J, Brummelkamp TR, Rottenberg S. Haploid genetic screens identify genetic vulnerabilities to microtubule-targeting agents. Mol Oncol 2018; 12:953-971. [PMID: 29689640 PMCID: PMC5983209 DOI: 10.1002/1878-0261.12307] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2017] [Revised: 03/30/2018] [Accepted: 04/05/2018] [Indexed: 12/12/2022] Open
Abstract
The absence of biomarkers to accurately predict anticancer therapy response remains a major obstacle in clinical oncology. We applied a genome‐wide loss‐of‐function screening approach in human haploid cells to characterize genetic vulnerabilities to classical microtubule‐targeting agents. Using docetaxel and vinorelbine, two well‐established chemotherapeutic agents, we sought to identify genetic alterations sensitizing human HAP1 cells to these drugs. Despite the fact that both drugs act on microtubules, a set of distinct genes were identified whose disruption affects drug sensitivity. For docetaxel, this included a number of genes with a function in mitosis, while for vinorelbine we identified inactivation of FBXW7,RB1, and NF2, three frequently mutated tumor suppressor genes, as sensitizing factors. We validated these genes using independent knockout clones and confirmed FBXW7 as an important regulator of the mitotic spindle assembly. Upon FBXW7 depletion, vinorelbine treatment led to decreased survival of cells due to defective mitotic progression and subsequent mitotic catastrophe. We show that haploid insertional mutagenesis screens are a useful tool to study genetic vulnerabilities to classical chemotherapeutic drugs by identifying thus far unknown sensitivity factors. These results provide a rationale for investigating patient response to vinca alkaloid‐based anticancer treatment in relation to the mutational status of these three tumor suppressor genes, and could in the future lead to the establishment of novel predictive biomarkers or suggest new drug combinations based on molecular mechanisms of drug sensitivity.
Collapse
Affiliation(s)
- Nora M Gerhards
- Institute of Animal Pathology, Vetsuisse Faculty, University of Bern, Switzerland
| | - Vincent A Blomen
- Division of Biochemistry, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Merve Mutlu
- Institute of Animal Pathology, Vetsuisse Faculty, University of Bern, Switzerland
| | - Joppe Nieuwenhuis
- Division of Biochemistry, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Denise Howald
- Institute of Animal Pathology, Vetsuisse Faculty, University of Bern, Switzerland
| | - Charlotte Guyader
- Division of Molecular Oncology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Jos Jonkers
- Division of Molecular Pathology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Thijn R Brummelkamp
- Division of Biochemistry, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Sven Rottenberg
- Institute of Animal Pathology, Vetsuisse Faculty, University of Bern, Switzerland.,Division of Molecular Pathology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| |
Collapse
|
31
|
Abstract
Next-generation sequencing has made major strides in the past decade. Studies based on large sequencing data sets are growing in number, and public archives for raw sequencing data have been doubling in size every 18 months. Leveraging these data requires researchers to use large-scale computational resources. Cloud computing, a model whereby users rent computers and storage from large data centres, is a solution that is gaining traction in genomics research. Here, we describe how cloud computing is used in genomics for research and large-scale collaborations, and argue that its elasticity, reproducibility and privacy features make it ideally suited for the large-scale reanalysis of publicly available archived data, including privacy-protected data.
Collapse
Affiliation(s)
- Ben Langmead
- Department of Computer Science, Center for Computational Biology, Johns Hopkins University, Baltimore, MD, USA
| | - Abhinav Nellore
- Department of Biomedical Engineering, Department of Surgery, Computational Biology Program, Oregon Health and Science University, Portland, OR, USA
| |
Collapse
|
32
|
Biomarker Guidelines for High-Dimensional Genomic Studies in Transplantation: Adding Method to the Madness. Transplantation 2018; 101:457-463. [PMID: 28212255 DOI: 10.1097/tp.0000000000001622] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
33
|
Kafkafi N, Agassi J, Chesler EJ, Crabbe JC, Crusio WE, Eilam D, Gerlai R, Golani I, Gomez-Marin A, Heller R, Iraqi F, Jaljuli I, Karp NA, Morgan H, Nicholson G, Pfaff DW, Richter SH, Stark PB, Stiedl O, Stodden V, Tarantino LM, Tucci V, Valdar W, Williams RW, Würbel H, Benjamini Y. Reproducibility and replicability of rodent phenotyping in preclinical studies. Neurosci Biobehav Rev 2018; 87:218-232. [PMID: 29357292 PMCID: PMC6071910 DOI: 10.1016/j.neubiorev.2018.01.003] [Citation(s) in RCA: 112] [Impact Index Per Article: 18.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2016] [Revised: 12/13/2017] [Accepted: 01/11/2018] [Indexed: 12/15/2022]
Abstract
The scientific community is increasingly concerned with the proportion of
published “discoveries” that are not replicated in subsequent
studies. The field of rodent behavioral phenotyping was one of the first to
raise this concern, and to relate it to other methodological issues: the complex
interaction between genotype and environment; the definitions of behavioral
constructs; and the use of laboratory mice and rats as model species for
investigating human health and disease mechanisms. In January 2015, researchers
from various disciplines gathered at Tel Aviv University to discuss these
issues. The general consensus was that the issue is prevalent and of concern,
and should be addressed at the statistical, methodological and policy levels,
but is not so severe as to call into question the validity and the usefulness of
model organisms as a whole. Well-organized community efforts, coupled with
improved data and metadata sharing, have a key role in identifying specific
problems and promoting effective solutions. Replicability is closely related to
validity, may affect generalizability and translation of findings, and has
important ethical implications.
Collapse
Affiliation(s)
| | | | | | - John C Crabbe
- Oregon Health & Science University, and VA Portland Health Care System, United States
| | | | | | | | | | | | | | | | | | - Natasha A Karp
- Discovery Sciences, IMED Biotech Unit, AstraZeneca, Cambridge, UK
| | | | | | | | | | | | | | | | | | | | - William Valdar
- University of North Carolina at Chapel Hill, United States
| | | | | | | |
Collapse
|
34
|
Plesser HE. Reproducibility vs. Replicability: A Brief History of a Confused Terminology. Front Neuroinform 2018; 11:76. [PMID: 29403370 PMCID: PMC5778115 DOI: 10.3389/fninf.2017.00076] [Citation(s) in RCA: 139] [Impact Index Per Article: 23.2] [Reference Citation Analysis] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2017] [Accepted: 12/18/2017] [Indexed: 12/01/2022] Open
Affiliation(s)
- Hans E. Plesser
- Faculty of Science and Technology, Norwegian University of Life Sciences, Ås, Norway
- Institute for Neuroscience and Medicine (INM-6), Jülich Research Centre, Jülich, Germany
| |
Collapse
|
35
|
Shepherd BE, Blevins Peratikos M, Rebeiro PF, Duda SN, McGowan CC. A Pragmatic Approach for Reproducible Research With Sensitive Data. Am J Epidemiol 2017; 186:387-392. [PMID: 28830079 DOI: 10.1093/aje/kwx066] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2016] [Accepted: 02/24/2017] [Indexed: 11/13/2022] Open
Abstract
Reproducible research is important for assessing the integrity of findings and disseminating methods, but it requires making original study data sets publicly available. This requirement is difficult to meet in settings with sensitive data, which can mean that resulting studies are not reproducible. For studies in which data cannot be shared, we propose a pragmatic approach to make research quasi-reproducible. On a publicly available website without restriction, researchers should post 1) analysis code used in the published study, 2) simulated data, and 3) results obtained by applying the analysis code used in the published study to the simulated data. Although it is not a perfect solution, such an approach makes analyses transparent for critical evaluation and dissemination and is therefore a significant improvement over current practice.
Collapse
|
36
|
Alileche A, Hampikian G. The effect of Nullomer-derived peptides 9R, 9S1R and 124R on the NCI-60 panel and normal cell lines. BMC Cancer 2017; 17:533. [PMID: 28793867 PMCID: PMC5551024 DOI: 10.1186/s12885-017-3514-z] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2017] [Accepted: 07/28/2017] [Indexed: 12/29/2022] Open
Abstract
BACKGROUND Nullomer peptides are the smallest sequences absent from databases of natural proteins. We first began compiling a list of absent 5-amino acid strings in 2006 (1). We report here the effects of Nullomer-derived peptides 9R, 9S1R and 124R on the NCI-60 panel, derived from human cancers of 9 organs (kidney, ovary, skin melanoma, lung, brain, lung, colon, prostate and the hematopoietic system), and four normal cell lines (endothelial HUVEC, skin fibroblasts BJ, colon epithelial FHC and normal prostate RWPE-1). METHODS NCI-60 cancer cell panel and four normal cell lines were cultured in vitro in RPMI1640 supplemented with 10% Hyclone fetal bovine serum and exposed for 48 h to 5 μM, 25 μM and 50 μM of peptides 9R, 9S1R and 124R. Viability was assessed by CCK-8 assay. For peptide ATP depletion effects, one cell line representing each organ in the NCI-60 panel, and four normal cell lines were exposed to 50 μM of peptides 9R, 9S1R and 124R for 3 h. The ATP content was assessed in whole cells, and their supernatants. RESULTS Peptides 9S1R and 9R are respectively lethal to 95 and 81.6% of the 60 cancer cell lines tested. Control peptide 124R has no effect on the growth of these cells. Especially interesting the fact that peptides 9R and 9S1R are capable of killing drug-resistant and hormone-resistant cell lines, and even cancer stem cells. Peptides 9R and 9S1R have a broader activity spectrum than many cancer drugs in current use, can completely deplete cellular ATP within 3 h, and are less toxic to 3 of the 4 normal cell lines tested than they are to several cancers. CONCLUSIONS Nullomer peptides 9R and 9S1R have a large broad lethal effect on cancer cell lines derived from nine organs represented in the NCI-60 panel. This broad activity crosses many of the categorical divisions used in the general classification of cancers: solid vs liquid cancers, drug sensitive vs drug resistant, hormone sensitive vs hormone resistant, cytokine sensitive vs cytokine non sensitive, slow growing vs rapid growing, differentiated vs dedifferentiated cancers. Furthermore peptides 9R and 9S1R are lethal to cancer stem cells and breast canrcinosarcoma.
Collapse
Affiliation(s)
- Abdelkrim Alileche
- Biology Department Room SN-215, Boise State University, 1910 University Drive, Boise, ID 83725 USA
| | - Greg Hampikian
- Biology Department Room SN-215, Boise State University, 1910 University Drive, Boise, ID 83725 USA
| |
Collapse
|
37
|
DeMets DL, Fleming TR, Geller G, Ransohoff DF. Institutional Responsibility and the Flawed Genomic Biomarkers at Duke University: A Missed Opportunity for Transparency and Accountability. SCIENCE AND ENGINEERING ETHICS 2017; 23:1199-1205. [PMID: 27882502 DOI: 10.1007/s11948-016-9844-4] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/20/2016] [Accepted: 11/07/2016] [Indexed: 06/06/2023]
Abstract
When there have been substantial failures by institutional leadership in their oversight responsibility to protect research integrity, the public should demand that these be recognized and addressed by the institution itself, or the funding bodies. This commentary discusses a case of research failures in developing genomic predictors for cancer risk assessment and treatment at a leading university. In its review of this case, the Office of Research Integrity, an agency within the US Department of Health and Human Services, focused their report entirely on one individual faculty member and made no comment on the institution's responsibility and its failure to provide adequate oversight and investigation. These actions missed an important opportunity to emphasize the institution's critical responsibilities in oversight of research integrity and the importance of institutional transparency and accountability.
Collapse
Affiliation(s)
- David L DeMets
- Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison, 201 WARF, 610 Walnut Street, Madison, WI, 53726, USA.
| | - Thomas R Fleming
- Department of Biostatistics, University of Washington, Box 357232, 1959 NE Pacific Street, Seattle, WA, 98195, USA
| | - Gail Geller
- Department of Medicine, Johns Hopkins School of Medicine, 1809 Ashland Ave, Baltimore, MD, 21205, USA
| | - David F Ransohoff
- School of Medicine, University of North Carolina-Chapel Hill, 101 Manning Dr, Chapel Hill, NC, 27514, USA
| |
Collapse
|
38
|
Kanwal S, Khan FZ, Lonie A, Sinnott RO. Investigating reproducibility and tracking provenance - A genomic workflow case study. BMC Bioinformatics 2017; 18:337. [PMID: 28701218 PMCID: PMC5508699 DOI: 10.1186/s12859-017-1747-0] [Citation(s) in RCA: 46] [Impact Index Per Article: 6.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2017] [Accepted: 07/04/2017] [Indexed: 11/10/2022] Open
Abstract
Background Computational bioinformatics workflows are extensively used to analyse genomics data, with different approaches available to support implementation and execution of these workflows. Reproducibility is one of the core principles for any scientific workflow and remains a challenge, which is not fully addressed. This is due to incomplete understanding of reproducibility requirements and assumptions of workflow definition approaches. Provenance information should be tracked and used to capture all these requirements supporting reusability of existing workflows. Results We have implemented a complex but widely deployed bioinformatics workflow using three representative approaches to workflow definition and execution. Through implementation, we identified assumptions implicit in these approaches that ultimately produce insufficient documentation of workflow requirements resulting in failed execution of the workflow. This study proposes a set of recommendations that aims to mitigate these assumptions and guides the scientific community to accomplish reproducible science, hence addressing reproducibility crisis. Conclusions Reproducing, adapting or even repeating a bioinformatics workflow in any environment requires substantial technical knowledge of the workflow execution environment, resolving analysis assumptions and rigorous compliance with reproducibility requirements. Towards these goals, we propose conclusive recommendations that along with an explicit declaration of workflow specification would result in enhanced reproducibility of computational genomic analyses. Electronic supplementary material The online version of this article (doi:10.1186/s12859-017-1747-0) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Sehrish Kanwal
- Department of Computing and Information Systems, The University of Melbourne, Melbourne, VIC, 3010, Australia.
| | - Farah Zaib Khan
- Department of Computing and Information Systems, The University of Melbourne, Melbourne, VIC, 3010, Australia.
| | - Andrew Lonie
- Melbourne Bioinformatics, The University of Melbourne, Melbourne, VIC, 3010, Australia
| | - Richard O Sinnott
- Department of Computing and Information Systems, The University of Melbourne, Melbourne, VIC, 3010, Australia
| |
Collapse
|
39
|
Morris JS, Baladandayuthapani V. Statistical Contributions to Bioinformatics: Design, Modeling, Structure Learning, and Integration. STAT MODEL 2017; 17:245-289. [PMID: 29129969 PMCID: PMC5679480 DOI: 10.1177/1471082x17698255] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
The advent of high-throughput multi-platform genomics technologies providing whole-genome molecular summaries of biological samples has revolutionalized biomedical research. These technologiees yield highly structured big data, whose analysis poses significant quantitative challenges. The field of Bioinformatics has emerged to deal with these challenges, and is comprised of many quantitative and biological scientists working together to effectively process these data and extract the treasure trove of information they contain. Statisticians, with their deep understanding of variability and uncertainty quantification, play a key role in these efforts. In this article, we attempt to summarize some of the key contributions of statisticians to bioinformatics, focusing on four areas: (1) experimental design and reproducibility, (2) preprocessing and feature extraction, (3) unified modeling, and (4) structure learning and integration. In each of these areas, we highlight some key contributions and try to elucidate the key statistical principles underlying these methods and approaches. Our goals are to demonstrate major ways in which statisticians have contributed to bioinformatics, encourage statisticians to get involved early in methods development as new technologies emerge, and to stimulate future methodological work based on the statistical principles elucidated in this article and utilizing all availble information to uncover new biological insights.
Collapse
Affiliation(s)
- Jeffrey S Morris
- Department of Biostatistics, The University of Texas M.D. Anderson Cancer Center, Houston, Texas, USA
| | | |
Collapse
|
40
|
Zych K, Snoek BL, Elvin M, Rodriguez M, Van der Velde KJ, Arends D, Westra HJ, Swertz MA, Poulin G, Kammenga JE, Breitling R, Jansen RC, Li Y. reGenotyper: Detecting mislabeled samples in genetic data. PLoS One 2017; 12:e0171324. [PMID: 28192439 PMCID: PMC5305221 DOI: 10.1371/journal.pone.0171324] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2016] [Accepted: 01/19/2017] [Indexed: 12/11/2022] Open
Abstract
In high-throughput molecular profiling studies, genotype labels can be wrongly assigned at various experimental steps; the resulting mislabeled samples seriously reduce the power to detect the genetic basis of phenotypic variation. We have developed an approach to detect potential mislabeling, recover the “ideal” genotype and identify “best-matched” labels for mislabeled samples. On average, we identified 4% of samples as mislabeled in eight published datasets, highlighting the necessity of applying a “data cleaning” step before standard data analysis.
Collapse
Affiliation(s)
- Konrad Zych
- Groningen Bioinformatics Centre, University of Groningen, Groningen, The Netherlands
| | - Basten L. Snoek
- Laboratory of Nematology, Wageningen University, Wageningen, The Netherlands
| | - Mark Elvin
- Faculty of Life Sciences, University of Manchester, Manchester, United Kingdom
| | - Miriam Rodriguez
- Laboratory of Nematology, Wageningen University, Wageningen, The Netherlands
| | - K. Joeri Van der Velde
- Genomics Coordination Center, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands
| | - Danny Arends
- Groningen Bioinformatics Centre, University of Groningen, Groningen, The Netherlands
| | - Harm-Jan Westra
- Divisions of Genetics and Rheumatology, Department of Medicine, Brigham and Women’s Hospital and Harvard Medical School, Boston, Massachusetts, United States of America
- Partners Center for Personalized Genetic Medicine, Boston, Massachusetts, United States of America
- Program in Medical and Population Genetics, Broad Institute of MIT and Harvard, Cambridge, Massachusetts, United States of America
| | - Morris A. Swertz
- Genomics Coordination Center, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands
| | - Gino Poulin
- Faculty of Life Sciences, University of Manchester, Manchester, United Kingdom
| | - Jan E. Kammenga
- Laboratory of Nematology, Wageningen University, Wageningen, The Netherlands
| | - Rainer Breitling
- Manchester Institute of Biotechnology, Faculty of Life Sciences, University of Manchester, Manchester, United Kingdom
| | - Ritsert C. Jansen
- Groningen Bioinformatics Centre, University of Groningen, Groningen, The Netherlands
| | - Yang Li
- Groningen Bioinformatics Centre, University of Groningen, Groningen, The Netherlands
- University of Groningen, University Medical Center Groningen, Department of Genetics, Groningen, The Netherlands
- * E-mail:
| |
Collapse
|
41
|
Díez FJ, Yebra M, Bermejo I, Palacios-Alonso MA, Calleja MA, Luque M, Pérez-Martín J. Markov Influence Diagrams. Med Decis Making 2017; 37:183-195. [DOI: 10.1177/0272989x16685088] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Markov influence diagrams (MIDs) are a new type of probabilistic graphical model that extends influence diagrams in the same way that Markov decision trees extend decision trees. They have been designed to build state-transition models, mainly in medicine, and perform cost-effectiveness analyses. Using a causal graph that may contain several variables per cycle, MIDs can model various patient characteristics without multiplying the number of states; in particular, they can represent the history of the patient without using tunnel states. OpenMarkov, an open-source tool, allows the decision analyst to build and evaluate MIDs—including cost-effectiveness analysis and several types of deterministic and probabilistic sensitivity analysis—with a graphical user interface, without writing any code. This way, MIDs can be used to easily build and evaluate complex models whose implementation as spreadsheets or decision trees would be cumbersome or unfeasible in practice. Furthermore, many problems that previously required discrete event simulation can be solved with MIDs; i.e., within the paradigm of state-transition models, in which many health economists feel more comfortable.
Collapse
Affiliation(s)
- Francisco J. Díez
- Department Artificial Intelligence, UNED, Madrid, Spain (FJD, MA, ML, JP)
- Centre for Biomedical Technology, Technical University of Madrid, Spain (MY)
- School of Health and Related Research, University of Sheffield, UK (IB)
- Computer Science Department, National Institute for Astrophysics, Optics and Electronics, Tonantzintla, Puebla, Mexico (MAP)
| | - Mar Yebra
- Department Artificial Intelligence, UNED, Madrid, Spain (FJD, MA, ML, JP)
- Centre for Biomedical Technology, Technical University of Madrid, Spain (MY)
- School of Health and Related Research, University of Sheffield, UK (IB)
- Computer Science Department, National Institute for Astrophysics, Optics and Electronics, Tonantzintla, Puebla, Mexico (MAP)
| | - Iñigo Bermejo
- Department Artificial Intelligence, UNED, Madrid, Spain (FJD, MA, ML, JP)
- Centre for Biomedical Technology, Technical University of Madrid, Spain (MY)
- School of Health and Related Research, University of Sheffield, UK (IB)
- Computer Science Department, National Institute for Astrophysics, Optics and Electronics, Tonantzintla, Puebla, Mexico (MAP)
| | - Miguel A. Palacios-Alonso
- Department Artificial Intelligence, UNED, Madrid, Spain (FJD, MA, ML, JP)
- Centre for Biomedical Technology, Technical University of Madrid, Spain (MY)
- School of Health and Related Research, University of Sheffield, UK (IB)
- Computer Science Department, National Institute for Astrophysics, Optics and Electronics, Tonantzintla, Puebla, Mexico (MAP)
| | - Manuel Arias Calleja
- Department Artificial Intelligence, UNED, Madrid, Spain (FJD, MA, ML, JP)
- Centre for Biomedical Technology, Technical University of Madrid, Spain (MY)
- School of Health and Related Research, University of Sheffield, UK (IB)
- Computer Science Department, National Institute for Astrophysics, Optics and Electronics, Tonantzintla, Puebla, Mexico (MAP)
| | - Manuel Luque
- Department Artificial Intelligence, UNED, Madrid, Spain (FJD, MA, ML, JP)
- Centre for Biomedical Technology, Technical University of Madrid, Spain (MY)
- School of Health and Related Research, University of Sheffield, UK (IB)
- Computer Science Department, National Institute for Astrophysics, Optics and Electronics, Tonantzintla, Puebla, Mexico (MAP)
| | - Jorge Pérez-Martín
- Department Artificial Intelligence, UNED, Madrid, Spain (FJD, MA, ML, JP)
- Centre for Biomedical Technology, Technical University of Madrid, Spain (MY)
- School of Health and Related Research, University of Sheffield, UK (IB)
- Computer Science Department, National Institute for Astrophysics, Optics and Electronics, Tonantzintla, Puebla, Mexico (MAP)
| |
Collapse
|
42
|
Building a Personalized Cancer Treatment System. J Med Syst 2016; 41:28. [PMID: 28028763 DOI: 10.1007/s10916-016-0678-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2016] [Accepted: 12/11/2016] [Indexed: 10/20/2022]
Abstract
This paper reports the process by which a personalized cancer treatment system was built, following a user-centered approach. We give some background on personalized cancer treatment, the particular tumor chemosensitivity assay supported by the system, as well as some quality and legal issues related to such health systems. We describe how Contextual Design was applied when building the system. Contextual design is a user-centered design technique involving seven steps. We also provide some details about the system implementation. Finally, we explain how the Think-Aloud protocol and Heuristic Evaluation methods were used to evaluate the system and report its results. A qualitative assessment from the users perspective is also provided. Results from the heuristic evaluation indicate that only one of ten heuristics was missing from the system, while five were partially covered and four were fully covered.
Collapse
|
43
|
Juarez EF, Lau R, Friedman SH, Ghaffarizadeh A, Jonckheere E, Agus DB, Mumenthaler SM, Macklin P. Quantifying differences in cell line population dynamics using CellPD. BMC SYSTEMS BIOLOGY 2016; 10:92. [PMID: 27655224 PMCID: PMC5031291 DOI: 10.1186/s12918-016-0337-5] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/27/2016] [Accepted: 09/14/2016] [Indexed: 11/10/2022]
Abstract
BACKGROUND The increased availability of high-throughput datasets has revealed a need for reproducible and accessible analyses which can quantitatively relate molecular changes to phenotypic behavior. Existing tools for quantitative analysis generally require expert knowledge. RESULTS CellPD (cell phenotype digitizer) facilitates quantitative phenotype analysis, allowing users to fit mathematical models of cell population dynamics without specialized training. CellPD requires one input (a spreadsheet) and generates multiple outputs including parameter estimation reports, high-quality plots, and minable XML files. We validated CellPD's estimates by comparing it with a previously published tool (cellGrowth) and with Microsoft Excel's built-in functions. CellPD correctly estimates the net growth rate of cell cultures and is more robust to data sparsity than cellGrowth. When we tested CellPD's usability, biologists (without training in computational modeling) ran CellPD correctly on sample data within 30 min. To demonstrate CellPD's ability to aid in the analysis of high throughput data, we created a synthetic high content screening (HCS) data set, where a simulated cell line is exposed to two hypothetical drug compounds at several doses. CellPD correctly estimates the drug-dependent birth, death, and net growth rates. Furthermore, CellPD's estimates quantify and distinguish between the cytostatic and cytotoxic effects of both drugs-analyses that cannot readily be performed with spreadsheet software such as Microsoft Excel or without specialized computational expertise and programming environments. CONCLUSIONS CellPD is an open source tool that can be used by scientists (with or without a background in computational or mathematical modeling) to quantify key aspects of cell phenotypes (such as cell cycle and death parameters). Early applications of CellPD may include drug effect quantification, functional analysis of gene knockout experiments, data quality control, minable big data generation, and integration of biological data with computational models.
Collapse
Affiliation(s)
- Edwin F Juarez
- Lawrence J. Ellison Institute for Transformative Medicine, University of Southern California, Los Angeles, California, USA. .,Department of Electrical Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, California, USA.
| | - Roy Lau
- Lawrence J. Ellison Institute for Transformative Medicine, University of Southern California, Los Angeles, California, USA
| | - Samuel H Friedman
- Lawrence J. Ellison Institute for Transformative Medicine, University of Southern California, Los Angeles, California, USA
| | - Ahmadreza Ghaffarizadeh
- Lawrence J. Ellison Institute for Transformative Medicine, University of Southern California, Los Angeles, California, USA
| | - Edmond Jonckheere
- Department of Electrical Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, California, USA
| | - David B Agus
- Lawrence J. Ellison Institute for Transformative Medicine, University of Southern California, Los Angeles, California, USA
| | - Shannon M Mumenthaler
- Lawrence J. Ellison Institute for Transformative Medicine, University of Southern California, Los Angeles, California, USA
| | - Paul Macklin
- Lawrence J. Ellison Institute for Transformative Medicine, University of Southern California, Los Angeles, California, USA.
| |
Collapse
|
44
|
Emmert-Streib F, Dehmer M, Yli-Harja O. Against Dataism and for Data Sharing of Big Biomedical and Clinical Data with Research Parasites. Front Genet 2016; 7:154. [PMID: 27630666 PMCID: PMC5005320 DOI: 10.3389/fgene.2016.00154] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2016] [Accepted: 08/10/2016] [Indexed: 12/03/2022] Open
Affiliation(s)
- Frank Emmert-Streib
- Predictive Medicine and Analytics Lab, Department of Signal Processing, Tampere University of Technology Tampere, Finland
| | - Matthias Dehmer
- Department of Mechatronics and Biomedical Computer Science, UMITHall in Tyrol, Austria; College of Computer and Control Engineering, Nankai UniversityTianjin, China
| | - Olli Yli-Harja
- Computational Systems Biology, Department of Signal Processing, Tampere University of Technology Tampere, Finland
| |
Collapse
|
45
|
Patil P, Colantuoni E, Leek JT, Rosenblum M. Genomic and clinical predictors for improving estimator precision in randomized trials of breast cancer treatments. Contemp Clin Trials Commun 2016; 3:48-54. [PMID: 29736456 PMCID: PMC5935844 DOI: 10.1016/j.conctc.2016.03.001] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2015] [Revised: 12/07/2015] [Accepted: 03/21/2016] [Indexed: 12/30/2022] Open
Abstract
Background The hope that genomic biomarkers would dramatically and immediately improve care for common, complex diseases has been tempered by slow progress in their translation beyond bioinformatics. We propose a novel use of genomic information where the goal is to improve estimator precision in a randomized trial. We analyze the potential precision gains from the popular MammaPrint genomic signature and clinical variables in simulations of randomized trials analyzed using covariate adjustment. Methods We apply an estimator for the average treatment effect in the trial that adjusts for prognostic baseline variables to improve precision [1]. This precision gain can be translated directly into sample size reduction and corresponding cost savings. We conduct simulation studies based on resampling genomic and clinical data of breast cancer patients from four publicly available observational studies. Results Separate simulation studies were conducted based on each of the four data sets, with sample sizes ranging from 115 to 307. Adjusting only for clinical variables provided gains of −1%, 5%, 6%, 17%, compared to the unadjusted estimator. Adjusting only for the MammaPrint genomic signature provided gains of 2%, 4%, 4%, 5%. Simultaneously adjusting for clinical variables and the genomic signature provided gains of 2%, 6%, 7%, 16%. The differences between precision gains from adjusting for genomic plus clinical variables, versus only clinical variables, were −1%, 0%, 1%, 3%. Conclusions Adjusting only for clinical variables led to substantial precision gains (at least 5%) in three of the four data sets, with a 1% precision loss in the remaining data set. These gains were unchanged or increased when sample sizes were doubled in our simulations. The precision gains due to incorporating genomic information, beyond the gains from adjusting for clinical variables, were not substantial.
Collapse
Affiliation(s)
| | | | - Jeffrey T. Leek
- Corresponding authors. 615 N. Wolfe St., Baltimore, MD 21205, USA.
| | | |
Collapse
|
46
|
Kricka LJ. Quality assurance for multiplexed assays - how can it be achieved? Scand J Clin Lab Invest Suppl 2016; 245:S100-3. [PMID: 27454808 DOI: 10.1080/00365513.2016.1210335] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Abstract
Multiplexed assays are now a common form of analysis in routine clinical and research laboratories. Assuring the quality of this type of complex, massively-parallel testing poses challenges not encountered in the traditional single-plex assay. A range of quality assurance measures is implemented at different stages in a multiplex assay, beginning in the manufacturing process, and the ensuing analytical and data interpretation stages. This article explores quality issues and the quality assurance measures that have been devised for multiplex assays ranging from simple two-plex assays to the types of assays that involve simultaneous testing on millions of test sites on a single analytical device.
Collapse
Affiliation(s)
- Larry J Kricka
- a Department of Pathology and Laboratory Medicine , University of Pennsylvania Medical Center , Philadelphia , PA , USA
| |
Collapse
|
47
|
Abstract
When reporting research findings, scientists document the steps they followed so that others can verify and build upon the research. When those steps have been described in sufficient detail that others can retrace the steps and obtain similar results, the research is said to be reproducible. Computers play a vital role in many research disciplines and present both opportunities and challenges for reproducibility. Computers can be programmed to execute analysis tasks, and those programs can be repeated and shared with others. The deterministic nature of most computer programs means that the same analysis tasks, applied to the same data, will often produce the same outputs. However, in practice, computational findings often cannot be reproduced because of complexities in how software is packaged, installed, and executed-and because of limitations associated with how scientists document analysis steps. Many tools and techniques are available to help overcome these challenges; here we describe seven such strategies. With a broad scientific audience in mind, we describe the strengths and limitations of each approach, as well as the circumstances under which each might be applied. No single strategy is sufficient for every scenario; thus we emphasize that it is often useful to combine approaches.
Collapse
Affiliation(s)
- Stephen R Piccolo
- Department of Biology, Brigham Young University, Provo, UT, 84602, USA.
| | - Michael B Frampton
- Department of Computer Science, Brigham Young University, Provo, UT, USA
| |
Collapse
|
48
|
O'Neill K, Brinkman RR. Publishing code is essential for reproducible flow cytometry bioinformatics. Cytometry A 2016; 89:10-1. [DOI: 10.1002/cyto.a.22805] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2015] [Revised: 11/18/2015] [Accepted: 11/21/2015] [Indexed: 11/06/2022]
Affiliation(s)
- Kieran O'Neill
- Genome Sciences Centre, British Columbia Cancer Agency; Vancouver British Columbia Canada
- Department of Pathology, University of British Columbia; Vancouver British Columbia Canada
| | - Ryan R. Brinkman
- Terry Fox Laboratory, British Columbia Cancer Agency; Vancouver British Columbia Canada
- Department of Medical Genetics, University of British Columbia; Vancouver British Columbia Canada
| |
Collapse
|
49
|
Hill H, Pitt JH. Statistical Analysis of Numerical Preclinical Radiobiological Data. SCIENCEOPEN RESEARCH 2016. [DOI: 10.14293/s2199-1006.1.sor-stat.afhtwc.v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022] Open
Abstract
<p><strong>Background: </strong>Scientific fraud is an increasingly vexing problem. Many current programs for fraud detection focus on image manipulation, while techniques for detection based on anomalous patterns that may be discoverable in the underlying numerical data get much less attention, even though these techniques are often easy to apply. </p><p><strong>Methods: </strong>We applied statistical techniques in considering and comparing data sets from ten researchers in one laboratory and three outside investigators to determine whether anomalous patterns in data from a research teaching assistant (RTS) were likely to have occurred by chance. Rightmost digits of values in RTS data sets were not, as expected, uniform; equal pairs of terminal digits occurred at higher than expected frequency (> 10%); and, an unexpectedly large number of data triples commonly produced in such research included values near their means as an element. We applied standard statistical tests (chi-squared goodness of fit, binomial probabilities) to determine the likelihood of the first two anomalous patterns, and developed a new statistical model to test the third.</p><p><strong> Results: </strong>Application of the three tests to various data sets reported by RTS resulted in repeated rejection of the hypotheses (often at p-levels well below 0.001) that anomalous patterns in those data may have occurred by chance. Similar application to data sets from other investigators were entirely consistent with chance occurrence.</p><p><strong>Conclusions: </strong>This analysis emphasizes the importance of access to raw data that form the bases of publications, reports and grant applications in order to evaluate the correctness of the conclusions, and the importance of applying statistical methods to detect anomalous, especially potentially fabricated, numerical results. </p>
Collapse
|
50
|
Hofner B, Schmid M, Edler L. Reproducible research in statistics: A review and guidelines for theBiometrical Journal. Biom J 2015; 58:416-27. [DOI: 10.1002/bimj.201500156] [Citation(s) in RCA: 21] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2015] [Revised: 10/06/2015] [Accepted: 11/17/2015] [Indexed: 11/10/2022]
Affiliation(s)
- Benjamin Hofner
- Department of Medical Informatics, Biometry and Epidemiology; Friedrich-Alexander University; Erlangen-Nuremberg, Waldstraße 6 91054 Erlangen Germany
| | - Matthias Schmid
- Department of Medical Biometry, Informatics and Epidemiology; University of Bonn; Sigmund-Freud-Straße 25 53127 Bonn Germany
| | - Lutz Edler
- Division of Biostatistics-C060; German Cancer Research Center; Im Neuenheimer Feld 581 69120 Heidelberg Germany
| |
Collapse
|