1
|
Serio CD, Malgaroli A, Ferrari P, Kenett RS. The reproducibility of COVID-19 data analysis: paradoxes, pitfalls, and future challenges. PNAS NEXUS 2022; 1:pgac125. [PMID: 36741433 PMCID: PMC9896906 DOI: 10.1093/pnasnexus/pgac125] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/28/2022] [Accepted: 08/08/2022] [Indexed: 02/07/2023]
Abstract
In the midst of the COVID-19 experience, we learned an important scientific lesson: knowledge acquisition and information quality in medicine depends more on "data quality" rather than "data quantity." The large number of COVID-19 reports, published in a very short time, demonstrated that the most advanced statistical and computational tools cannot properly overcome the poor quality of acquired data. The main evidence for this observation comes from the poor reproducibility of results. Indeed, understanding the data generation process is fundamental when investigating scientific questions such as prevalence, immunity, transmissibility, and susceptibility. Most of COVID-19 studies are case reports based on "non probability" sampling and do not adhere to the general principles of controlled experimental designs. Such collected data suffers from many limitations when used to derive clinical conclusions. These include confounding factors, measurement errors and bias selection effects. Each of these elements represents a source of uncertainty, which is often ignored or assumed to provide an unbiased random contribution. Inference retrieved from large data in medicine is also affected by data protection policies that, while protecting patients' privacy, are likely to reduce consistently usefulness of big data in achieving fundamental goals such as effective and efficient data-integration. This limits the degree of generalizability of scientific studies and leads to paradoxical and conflicting conclusions. We provide such examples from assessing the role of risks factors. In conclusion, new paradigms and new designs schemes are needed in order to reach inferential conclusions that are meaningful and informative when dealing with data collected during emergencies like COVID-19.
Collapse
Affiliation(s)
- Clelia Di Serio
- Vita-Salute San Raffaele University, UniSR, Milan, Italy
- University Centre of Statistics in the Biomedical Sciences CUSSB, UniSR, Milan, Italy
- Biomedical Faculty, Università della Svizzera Italiana, Lugano, Switzerland
| | | | - Paolo Ferrari
- Biomedical Faculty, Università della Svizzera Italiana, Lugano, Switzerland
- Ente Ospedaliero Cantonale, Lugano, Switzerland
- Clinical School, University of New South Wales, Sydney, Australia
| | - Ron S Kenett
- KPA,Samuel Neaman Institute, Technion, Haifa, Israel
- University of Turin, Turin, Italy
| |
Collapse
|
2
|
Lázaro-Muñoz G, Sabatello M, Huckins L, Peay H, Degenhardt F, Meiser B, Lencz T, Soda T, Docherty A, Crepaz-Keay D, Austin J, Peterson RE, Davis LK. International Society of Psychiatric Genetics Ethics Committee: Issues facing us. Am J Med Genet B Neuropsychiatr Genet 2019; 180:543-554. [PMID: 31124312 PMCID: PMC6861601 DOI: 10.1002/ajmg.b.32736] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/03/2018] [Revised: 03/21/2019] [Accepted: 05/10/2019] [Indexed: 12/19/2022]
Abstract
Psychiatric genetics research is improving our understanding of the biological underpinnings of neurodiversity and mental illness. Using psychiatric genetics in ways that maximize benefits and minimize harms to individuals and society depends largely on how the ethical, legal, and social implications (ELSI) of psychiatric genetics are managed. The International Society of Psychiatric Genetics (ISPG) is the largest international organization dedicated to psychiatric genetics. Given its history, membership, and international reach, we believe the ISPG is well-equipped to contribute to the resolution of these ELSI challenges. As such, we recently created the ISPG Ethics Committee, an interdisciplinary group comprised of psychiatric genetics researchers, clinical geneticists, genetic counselors, mental health professionals, patients, patient advocates, bioethicists, and lawyers. This article highlights key ELSI challenges identified by the ISPG Ethics Committee to be of paramount importance for the ethical translation of psychiatric research into society in three contexts: research settings, clinical settings, and legal proceedings. For each of these arenas, we identify and discuss pressing psychiatric genetics ELSI dilemmas that merit attention and require action. The goal is to increase awareness about psychiatric genetics ELSI issues and encourage dialogue and action among stakeholders.
Collapse
Affiliation(s)
| | | | - Laura Huckins
- Icahn School of Medicine at Mount Sinai, New York, NY, USA 10029
| | - Holly Peay
- RTI International, Research Triangle Park, NC, USA 27709
| | | | - Bettina Meiser
- University of New South Wales, UNSW Sydney 2052, Australia
| | - Todd Lencz
- Hofstra University, Hempstead, NY, USA 11549
| | - Takahiro Soda
- University of North Carolina at Chapel Hill, NC, USA 27599
| | | | | | | | | | | |
Collapse
|
3
|
Dick DM, Barr P, Guy M, Nasim A, Scott D. Review: Genetic research on alcohol use outcomes in African American populations: A review of the literature, associated challenges, and implications. Am J Addict 2017; 26:486-493. [PMID: 28240821 PMCID: PMC5884102 DOI: 10.1111/ajad.12495] [Citation(s) in RCA: 28] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2016] [Revised: 11/18/2016] [Accepted: 12/18/2016] [Indexed: 11/26/2022] Open
Abstract
BACKGROUND AND OBJECTIVES There have been remarkable advances in understanding genetic influences on complex traits; however, individuals of African descent have been underrepresented in genetic research. METHODS We review the limitations of existing genetic research on alcohol phenotypes in African Americans (AA) including both twin and gene identification studies, possible reasons for underrepresentation of AAs in genetic research, the implications of the lack of racially diverse samples, and special considerations regarding conducting genetic research in AA populations. RESULTS There is a marked absence of large-scale AA twin studies so little is known about the genetic epidemiology of alcohol use and problems among AAs. Individuals of African descent have also been underrepresented in gene identification efforts; however, there have been recent efforts to enhance representation. It remains unknown the extent to which genetic variants associated with alcohol use outcomes in individuals of European and African descent will be shared. Efforts to increase representation must be accompanied by careful attention to the ethical, legal, and social implications of genetic research. This is particularly true for AAs due to the history of abuse by the biomedical community and the persistent racial discrimination targeting this population. CONCLUSIONS AND SCIENTIFIC SIGNIFICANCE Lack of representation in genetic studies limits our understanding of the etiological factors that contribute to substance use and psychiatric outcomes in populations of African descent and has the potential to further perpetuate health disparities. Involving individuals of diverse ancestry in discussions about genetic research will be critical to ensure that all populations benefit equally from genetic advances. (Am J Addict 2017;26:486-493).
Collapse
Affiliation(s)
- Danielle M Dick
- Department of Psychology, Virginia Commonwealth University, Richmond, Virginia
- Department of African American Studies, Virginia Commonwealth University, Richmond, Virginia
- Department of Human and Molecular Genetics, Virginia Commonwealth University, Richmond, Virginia
| | - Peter Barr
- Department of Psychology, Virginia Commonwealth University, Richmond, Virginia
- Department of African American Studies, Virginia Commonwealth University, Richmond, Virginia
| | - Mignonne Guy
- Department of African American Studies, Virginia Commonwealth University, Richmond, Virginia
| | - Aashir Nasim
- Department of African American Studies, Virginia Commonwealth University, Richmond, Virginia
| | - Denise Scott
- Department of Pediatrics and Human Genetics and Alcohol Research Center, Howard University College of Medicine, Washington, District of Columbia
| |
Collapse
|
4
|
Vos S, van Delden JJM, van Diest PJ, Bredenoord AL. Moral Duties of Genomics Researchers: Why Personalized Medicine Requires a Collective Approach. Trends Genet 2016; 33:118-128. [PMID: 28017398 DOI: 10.1016/j.tig.2016.11.006] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2016] [Revised: 11/21/2016] [Accepted: 11/28/2016] [Indexed: 12/30/2022]
Abstract
Advances in genome sequencing together with the introduction of personalized medicine offer promising new avenues for research and precision treatment, particularly in the field of oncology. At the same time, the convergence of genomics, bioinformatics, and the collection of human tissues and patient data creates novel moral duties for researchers. After all, unprecedented amounts of potentially sensitive information are being generated. Over time, traditional research ethics principles aimed at protecting individual participants have become supplemented with social obligations related to the interests of society and the research enterprise at large, illustrating that genomic medicine is also a social endeavor. In this review we provide a comprehensive assembly of moral duties that have been attributed to genomics researchers and offer suggestions for responsible advancement of personalized genomic cancer care.
Collapse
Affiliation(s)
- Shoko Vos
- Department of Pathology, University Medical Center Utrecht, Utrecht, The Netherlands.
| | - Johannes J M van Delden
- Department of Medical Humanities, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Paul J van Diest
- Department of Pathology, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Annelien L Bredenoord
- Department of Medical Humanities, University Medical Center Utrecht, Utrecht, The Netherlands
| |
Collapse
|
5
|
Abstract
Human biospecimens have played a crucial role in scientific and medical advances. Although the ethical and policy issues associated with biospecimen research have long been the subject of scholarly debate, the story of Henrietta Lacks, her family, and the creation of HeLa cells captured the attention of a much broader audience. The story has been a catalyst for policy change, including major regulatory changes proposed in the United States surrounding informed consent. These proposals are premised in part on public opinion data, necessitating a closer look at what such data tell us. The development of biospecimen policy should be informed by many considerations-one of which is public input, robustly gathered, on acceptable approaches that optimize shared interests, including access for all to the benefits of research. There is a need for consent approaches that are guided by realistic aspirations and a balanced view of autonomy within an expanded ethical framework.
Collapse
Affiliation(s)
- Laura M Beskow
- Program for Empirical Bioethics, Duke Clinical Research Institute, Duke University School of Medicine, Durham, North Carolina 27705;
- Department of Medicine, Duke University School of Medicine, Durham, North Carolina 27710
| |
Collapse
|
6
|
Federer LM, Lu YL, Joubert DJ, Welsh J, Brandys B. Biomedical Data Sharing and Reuse: Attitudes and Practices of Clinical and Scientific Research Staff. PLoS One 2015; 10:e0129506. [PMID: 26107811 PMCID: PMC4481309 DOI: 10.1371/journal.pone.0129506] [Citation(s) in RCA: 55] [Impact Index Per Article: 6.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2015] [Accepted: 05/08/2015] [Indexed: 01/01/2023] Open
Abstract
Background Significant efforts are underway within the biomedical research community to encourage sharing and reuse of research data in order to enhance research reproducibility and enable scientific discovery. While some technological challenges do exist, many of the barriers to sharing and reuse are social in nature, arising from researchers’ concerns about and attitudes toward sharing their data. In addition, clinical and basic science researchers face their own unique sets of challenges to sharing data within their communities. This study investigates these differences in experiences with and perceptions about sharing data, as well as barriers to sharing among clinical and basic science researchers. Methods Clinical and basic science researchers in the Intramural Research Program at the National Institutes of Health were surveyed about their attitudes toward and experiences with sharing and reusing research data. Of 190 respondents to the survey, the 135 respondents who identified themselves as clinical or basic science researchers were included in this analysis. Odds ratio and Fisher’s exact tests were the primary methods to examine potential relationships between variables. Worst-case scenario sensitivity tests were conducted when necessary. Results and Discussion While most respondents considered data sharing and reuse important to their work, they generally rated their expertise as low. Sharing data directly with other researchers was common, but most respondents did not have experience with uploading data to a repository. A number of significant differences exist between the attitudes and practices of clinical and basic science researchers, including their motivations for sharing, their reasons for not sharing, and the amount of work required to prepare their data. Conclusions Even within the scope of biomedical research, addressing the unique concerns of diverse research communities is important to encouraging researchers to share and reuse data. Efforts at promoting data sharing and reuse should be aimed at solving not only technological problems, but also addressing researchers’ concerns about sharing their data. Given the varied practices of individual researchers and research communities, standardizing data practices like data citation and repository upload could make sharing and reuse easier.
Collapse
Affiliation(s)
- Lisa M. Federer
- NIH Library, Division of Library Services, Office of Research Services, National Institutes of Health, Bethesda, Maryland, United States of America
- * E-mail:
| | - Ya-Ling Lu
- NIH Library, Division of Library Services, Office of Research Services, National Institutes of Health, Bethesda, Maryland, United States of America
| | - Douglas J. Joubert
- NIH Library, Division of Library Services, Office of Research Services, National Institutes of Health, Bethesda, Maryland, United States of America
| | - Judith Welsh
- NIH Library, Division of Library Services, Office of Research Services, National Institutes of Health, Bethesda, Maryland, United States of America
| | - Barbara Brandys
- NIH Library, Division of Library Services, Office of Research Services, National Institutes of Health, Bethesda, Maryland, United States of America
| |
Collapse
|
7
|
van Schaik TA, Kovalevskaya NV, Protopapas E, Wahid H, Nielsen FGG. The need to redefine genomic data sharing: A focus on data accessibility. Appl Transl Genom 2014; 3:100-4. [PMID: 27294022 PMCID: PMC4888834 DOI: 10.1016/j.atg.2014.09.013] [Citation(s) in RCA: 18] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Abstract
DNAdigest's mission is to investigate and address the issues hindering efficient and ethical genomic data sharing in the human genomics research community. We conducted contextual interviews with human genomics researchers in clinical, academic or industrial R&D settings about their experience with accessing and sharing human genomic data. The qualitative interviews were followed by an online survey which provided quantitative support for our findings. Here we present the generalised workflow for accessing human genomic data through both public and restricted-access repositories and discuss reported points of frustration and their possible improvements. We discuss how data discoverability and accessibility are lacking in current mechanisms and how these are the prerequisites for adoption of best practices in the research community. We summarise current initiatives related to genomic data discovery and present a new data discovery platform available at http://nucleobase.co.uk.
Collapse
Affiliation(s)
- Tempest A van Schaik
- DNAdigest, Future Business Centre, Kings Hedges Road, Cambridge CB4 2HY, United Kingdom
| | | | - Elena Protopapas
- DNAdigest, Future Business Centre, Kings Hedges Road, Cambridge CB4 2HY, United Kingdom
| | - Hamza Wahid
- DNAdigest, Future Business Centre, Kings Hedges Road, Cambridge CB4 2HY, United Kingdom
| | - Fiona G G Nielsen
- DNAdigest, Future Business Centre, Kings Hedges Road, Cambridge CB4 2HY, United Kingdom
| |
Collapse
|