1
|
Damen JA, Heus P, Lamberink HJ, Tijdink JK, Bouter L, Glasziou P, Moher D, Otte WM, Vinkers CH, Hooft L. Indicators of questionable research practices were identified in 163,129 randomized controlled trials. J Clin Epidemiol 2023; 154:23-32. [PMID: 36470577 DOI: 10.1016/j.jclinepi.2022.11.020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2022] [Revised: 11/17/2022] [Accepted: 11/29/2022] [Indexed: 12/03/2022]
Abstract
OBJECTIVES To explore indicators of the following questionable research practices (QRPs) in randomized controlled trials (RCTs): (1) risk of bias in four domains (random sequence generation, allocation concealment, blinding of participants and personnel, and blinding of outcome assessment); (2) modifications in primary outcomes that were registered in trial registration records (proxy for selective reporting bias); (3) ratio of the achieved to planned sample sizes; and (4) statistical discrepancy. STUDY DESIGN AND SETTING Full texts of all human RCTs published in PubMed in 1996-2017 were automatically identified and information was collected automatically. Potential indicators of QRPs included author-specific, publication-specific, and journal-specific characteristics. Beta, logistic, and linear regression models were used to identify associations between these potential indicators and QRPs. RESULTS We included 163,129 RCT publications. The median probability of bias assessed using Robot Reviewer software ranged between 43% and 63% for the four risk of bias domains. A more recent publication year, trial registration, mentioning of CONsolidated Standards Of Reporting Trials-checklist, and a higher journal impact factor were consistently associated with a lower risk of QRPs. CONCLUSION This comprehensive analysis provides an insight into indicators of QRPs. Researchers should be aware that certain characteristics of the author team and publication are associated with a higher risk of QRPs.
Collapse
Affiliation(s)
- Johanna A Damen
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands.
| | - Pauline Heus
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Herm J Lamberink
- Department of Child Neurology, UMC Utrecht Brain Center, University Medical Center Utrecht and Utrecht University, Utrecht, The Netherlands; Department of Neurology, Haaglanden Medical Center, Den Haag, The Netherlands
| | - Joeri K Tijdink
- Department of Ethics, Law and Humanities, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands; Department of Philosophy, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Lex Bouter
- Department of Epidemiology and Data Science, Amsterdam UMC, Amsterdam, The Netherlands; Department of Philosophy, Vrije Universiteit, Amsterdam, The Netherlands
| | - Paul Glasziou
- Institute for Evidence-Based Healthcare, Bond University, Gold Coast, Australia
| | - David Moher
- Centre for Journalology, Clinical Epidemiology Program, The Ottawa Hospital Research Institute, Ottawa, Canada; School of Epidemiology and Public Health, University of Ottawa, Ottawa, Canada
| | - Willem M Otte
- Department of Child Neurology, UMC Utrecht Brain Center, University Medical Center Utrecht and Utrecht University, Utrecht, The Netherlands; Biomedical MR Imaging and Spectroscopy group, Center for Image Sciences, University Medical Center Utrecht and Utrecht University, Utrecht, The Netherlands
| | - Christiaan H Vinkers
- Department of Psychiatry and Anatomy & Neurosciences, Amsterdam University Medical Center Location Vrije Universiteit Amsterdam, 1081 HV Amsterdam, The Netherlands; Amsterdam Public Health, Mental Health Program and Amsterdam Neuroscience, Mood, Anxiety, Psychosis, Sleep & Stress Program, Amsterdam, The Netherlands; Amsterdam Public Health (Mental Health Program) Research Institute, 1081 HV Amsterdam, The Netherlands; GGZ inGeest Mental Health Care, 1081 HJ Amsterdam, The Netherlands
| | - Lotty Hooft
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| |
Collapse
|
2
|
Bruckner T, Wieschowski S, Heider M, Deutsch S, Drude N, Tölch U, Bleich A, Tolba R, Strech D. Measurement challenges and causes of incomplete results reporting of biomedical animal studies: Results from an interview study. PLoS One 2022; 17:e0271976. [PMID: 35960759 PMCID: PMC9374215 DOI: 10.1371/journal.pone.0271976] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2022] [Accepted: 07/11/2022] [Indexed: 11/30/2022] Open
Abstract
Background Existing evidence indicates that a significant amount of biomedical research involving animals remains unpublished. At the same time, we lack standards for measuring the extent of results reporting in animal research. Publication rates may vary significantly depending on the level of measurement such as an entire animal study, individual experiments within a study, or the number of animals used. Methods Drawing on semi-structured interviews with 18 experts and qualitative content analysis, we investigated challenges and opportunities for the measurement of incomplete reporting of biomedical animal research with specific reference to the German situation. We further investigate causes of incomplete reporting. Results The in-depth expert interviews revealed several reasons for why incomplete reporting in animal research is difficult to measure at all levels under the current circumstances. While precise quantification based on regulatory approval documentation is feasible at the level of entire studies, measuring incomplete reporting at the more individual experiment and animal levels presents formidable challenges. Expert-interviews further identified six drivers of incomplete reporting of results in animal research. Four of these are well documented in other fields of research: a lack of incentives to report non-positive results, pressures to ‘deliver’ positive results, perceptions that some data do not add value, and commercial pressures. The fifth driver, reputational concerns, appears to be far more salient in animal research than in human clinical trials. The final driver, socio-political pressures, may be unique to the field. Discussion Stakeholders in animal research should collaborate to develop a clear conceptualisation of complete reporting in animal research, facilitate valid measurements of the phenomenon, and develop incentives and rewards to overcome the causes for incomplete reporting.
Collapse
Affiliation(s)
- Till Bruckner
- QUEST Center for Responsible Research, Berlin Institute of Health at Charité –Universitätsmedizin, Berlin, Germany
- Institute for Ethics, History and Philosophy of Medicine, Hannover Medical School, Hannover, Germany
| | - Susanne Wieschowski
- QUEST Center for Responsible Research, Berlin Institute of Health at Charité –Universitätsmedizin, Berlin, Germany
- Institute for Ethics, History and Philosophy of Medicine, Hannover Medical School, Hannover, Germany
| | - Miriam Heider
- Institute for Laboratory Animal Science, Hannover Medical School, Hannover, Germany
| | - Susanne Deutsch
- Institute for Laboratory Animal Science, RWTH Aachen University, Faculty of Medicine, Aachen, Germany
| | - Natascha Drude
- QUEST Center for Responsible Research, Berlin Institute of Health at Charité –Universitätsmedizin, Berlin, Germany
| | - Ulf Tölch
- QUEST Center for Responsible Research, Berlin Institute of Health at Charité –Universitätsmedizin, Berlin, Germany
| | - André Bleich
- Institute for Laboratory Animal Science, Hannover Medical School, Hannover, Germany
| | - René Tolba
- Institute for Laboratory Animal Science, RWTH Aachen University, Faculty of Medicine, Aachen, Germany
| | - Daniel Strech
- QUEST Center for Responsible Research, Berlin Institute of Health at Charité –Universitätsmedizin, Berlin, Germany
- Institute for Ethics, History and Philosophy of Medicine, Hannover Medical School, Hannover, Germany
- * E-mail:
| |
Collapse
|
3
|
Torensma B, Hisham M, Eldawlatly AA, Hany M. Differences Between the 2016 and 2022 Editions of the Enhanced Recovery After Bariatric Surgery (ERABS) Guidelines: Call to Action of FAIR Data and the Creation of a Global Consortium of Bariatric Care and Research. Obes Surg 2022; 32:2753-2763. [PMID: 35654929 PMCID: PMC9162377 DOI: 10.1007/s11695-022-06132-7] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2022] [Revised: 05/25/2022] [Accepted: 05/25/2022] [Indexed: 12/19/2022]
Abstract
In 2016, the Enhanced Recovery After Bariatric Surgery guidelines (G16) was published, and in 2022, an update to it was released (G22). Grading of recommendations, assessment, development, and evaluations (GRADE), emphasizing the level of evidence (LoE) of both the guidelines, was performed. An overview of methodology was also performed, considering the following questions: how can research be improved, what can be done in the future using data, and how to collaborate more? Both guidelines did not explain how the LoE conclusions were derived regarding the risk of bias. There is also potential for forming a global consortium that deals with bariatric research, which can serve as a repository for all relevant data. Ensuring that this data is FAIR (findability, accessibility, interoperability, reusability) compliant and using this data to formulate future guidelines will benefit clinicians and patients alike.
Collapse
Affiliation(s)
- Bart Torensma
- Clinical Epidemiologist, Leiden University Medical Center (LUMC), Albinusdreef 2, 2333 ZA, Leiden, The Netherlands.
| | - Mohamed Hisham
- Medical Research Institute, Alexandria University, Alexandria, Egypt
| | - Abdelazeem A Eldawlatly
- Head of the Department of Anaesthesiology, King Saudi University Hospital, Riyadh, Saudi Arabia
| | - Mohamed Hany
- Medical Research Institute, Alexandria University, Alexandria, Egypt.,Head of the Department of Surgery, Medical Research Institute, Alexandria University, Alexandria, Egypt
| |
Collapse
|
4
|
Trace amine-associated receptor 1 (TAAR1): Potential application in mood disorders: A systematic review. Neurosci Biobehav Rev 2021; 131:192-210. [PMID: 34537265 DOI: 10.1016/j.neubiorev.2021.09.020] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2021] [Revised: 09/07/2021] [Accepted: 09/12/2021] [Indexed: 12/29/2022]
Abstract
There is a need for innovation with respect to therapeutics in psychiatry. Available evidence indicates that the trace amine-associated receptor 1 (TAAR1) agonist SEP-363856 is promising, as it improves measures of cognitive and reward function in schizophrenia. Hedonic and cognitive impairments are transdiagnostic and constitute major burdens in mood disorders. Herein, we systematically review the behavioural and genetic literature documenting the role of TAAR1 in reward and cognitive function, and propose a mechanistic model of TAAR1's functions in the brain. Notably, TAAR1 activity confers antidepressant-like effects, enhances attention and response inhibition, and reduces compulsive reward seeking without impairing normal function. Further characterization of the responsible mechanisms suggests ion-homeostatic, metabolic, neurotrophic, and anti-inflammatory enhancements in the limbic system. Multiple lines of evidence establish the viability of TAAR1 as a biological target for the treatment of mood disorders. Furthermore, the evidence suggests a role for TAAR1 in reward and cognitive function, which is attributed to a cascade of events that are relevant to the cellular integrity and function of the central nervous system.
Collapse
|
5
|
Spybrook J, Maynard R, Anderson D. Study Registration for the Field of Prevention Science: Considering Options and Paths Forward. PREVENTION SCIENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR PREVENTION RESEARCH 2021; 23:764-773. [PMID: 34386938 DOI: 10.1007/s11121-021-01290-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/26/2021] [Indexed: 10/20/2022]
Abstract
The practice of prospectively registering the details of intervention studies in a public database or registry is gaining momentum across disciplines as a strategy for increasing the transparency, credibility, and accessibility of study findings. In this article, we consider five registries that may be relevant for registration of intervention studies in the field of prevention science: ClinicalTrials.gov, the American Economic Association Registry of Randomized Controlled Trials (AEA RCT Registry), the Open Science Framework Preregistration (OSF Preregistration), the Registry for International Development Impact Evaluations (RIDIE), and the Registry of Efficacy and Effectiveness Studies (REES). We examine the five registries in terms of substantive focus, study designs, and contents of registry entries. We consider two paths forward for prospective registration of intervention studies in the field of prevention science: Path A: register all studies in ClinicalTrials.gov and Path B: allow individual researchers to select the registry with the "best fit." Lastly, we consider how the field might begin to establish norms around registration.
Collapse
Affiliation(s)
- Jessaca Spybrook
- Western Michigan University, 1903 W. Michigan Avenue, Kalamazoo, MI, 49008-5283, USA.
| | - Rebecca Maynard
- University of Pennsylvania, 3700 Walnut Street, Philadelphia, PA, 19104, USA
| | - Dustin Anderson
- EVALCORP, 15615 Alton Parkway, Suite 450, Irvine, CA, 92618, USA
| |
Collapse
|
6
|
Schmidtke KA, Vlaev I, Kabbani S, Klauznicer H, Baasiri A, Osseiran A, El Rifai G, Fares H, Saleh N, Makki F. An exploratory randomised controlled trial evaluating text prompts in Lebanon to encourage health-seeking behaviour for hypertension. Int J Clin Pract 2021; 75:e13669. [PMID: 32772451 DOI: 10.1111/ijcp.13669] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/12/2020] [Accepted: 08/06/2020] [Indexed: 12/01/2022] Open
Abstract
AIMS OF THE STUDY The current study evaluates the effectiveness of an opportunistic mobile screening on the percentage of people who are aware of whether they may be hypertensive (in an observational study) and the effectiveness of reminder prompts on the percentage of people who seek further medical attention (in a randomised controlled trial). METHODS USED TO CONDUCT THE STUDY The screening of 1227 participants (529 female) was conducted during the registration period of the 2018 Beirut International Marathon in Lebanon. Next, 266 participants whose screening indicated hypertension (64 Female) were randomly allocated to a treatment group or a control group in a 1:1 fashion. The treatment group received a reminder prompt to seek further medical attention for their potential hypertension and the control group did not. The overt nature of the text message meant that participants in the treatment group could not be blinded to their group allocation. The primary outcome is participants' self-reports of whether they sought further medical attention. RESULTS OF THE STUDY For the opportunistic screening, a 25% prevalence rate and a 24% awareness rate of hypertension was indicated. A McNemar analysis suggested that the screening increased participant awareness (X2 (N = 1227) = 72.16, P < .001). For the randomised controlled trial, 219 participants provided follow-up data via a phone call (82% retention). A Chi-squared analysis suggested that the reminder prompt successfully encouraged more participants to seek further medical attention, 45.5% treatment group vs 28.0% control group (X2 (1, N = 219) = 7.19, P = .007, φ = 0.18). CONCLUSIONS DRAWN AND CLINICAL IMPLICATIONS Extra support in the form of a brief reminder message can increase the percentage of people who seek further medical attention after attending an opportunistic screening at a marathon event. The discussion reviews how the results align with previous research, strengths and limitations of the current study, and implications for future research and practice.
Collapse
Affiliation(s)
- K A Schmidtke
- Medical School, Warwick Medical School, University of Warwick, Coventry, UK
| | - I Vlaev
- Behavioural Science Group, Warwick Business School, University of Warwick, Coventry, UK
| | - S Kabbani
- Cardiology Department, Rafik Hariri University Hospital, Beirut, Lebanon
| | - H Klauznicer
- Supreme Committee for Delivery and Legacy, B4Development Foundation (formerly Qatar Behavioural Insights Unit), Doha, Qatar
| | | | | | | | - H Fares
- Nudge Lebanon, Beirut, Lebanon
| | - N Saleh
- Nudge Lebanon, Beirut, Lebanon
| | - F Makki
- Supreme Committee for Delivery and Legacy, B4Development Foundation (formerly Qatar Behavioural Insights Unit), Doha, Qatar
| |
Collapse
|
7
|
Bakker M, Veldkamp CLS, van Assen MALM, Crompvoets EAV, Ong HH, Nosek BA, Soderberg CK, Mellor D, Wicherts JM. Ensuring the quality and specificity of preregistrations. PLoS Biol 2020; 18:e3000937. [PMID: 33296358 PMCID: PMC7725296 DOI: 10.1371/journal.pbio.3000937] [Citation(s) in RCA: 29] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2020] [Accepted: 10/23/2020] [Indexed: 11/29/2022] Open
Abstract
Researchers face many, often seemingly arbitrary, choices in formulating hypotheses, designing protocols, collecting data, analyzing data, and reporting results. Opportunistic use of "researcher degrees of freedom" aimed at obtaining statistical significance increases the likelihood of obtaining and publishing false-positive results and overestimated effect sizes. Preregistration is a mechanism for reducing such degrees of freedom by specifying designs and analysis plans before observing the research outcomes. The effectiveness of preregistration may depend, in part, on whether the process facilitates sufficiently specific articulation of such plans. In this preregistered study, we compared 2 formats of preregistration available on the OSF: Standard Pre-Data Collection Registration and Prereg Challenge Registration (now called "OSF Preregistration," http://osf.io/prereg/). The Prereg Challenge format was a "structured" workflow with detailed instructions and an independent review to confirm completeness; the "Standard" format was "unstructured" with minimal direct guidance to give researchers flexibility for what to prespecify. Results of comparing random samples of 53 preregistrations from each format indicate that the "structured" format restricted the opportunistic use of researcher degrees of freedom better (Cliff's Delta = 0.49) than the "unstructured" format, but neither eliminated all researcher degrees of freedom. We also observed very low concordance among coders about the number of hypotheses (14%), indicating that they are often not clearly stated. We conclude that effective preregistration is challenging, and registration formats that provide effective guidance may improve the quality of research.
Collapse
Affiliation(s)
- Marjan Bakker
- Department of Methodology and Statistics, Tilburg University, Tilburg, the Netherlands
| | | | - Marcel A. L. M. van Assen
- Department of Methodology and Statistics, Tilburg University, Tilburg, the Netherlands
- Department of Sociology, Utrecht University, Utrecht, the Netherlands
| | - Elise A. V. Crompvoets
- Department of Methodology and Statistics, Tilburg University, Tilburg, the Netherlands
- Cito Institute for Educational Measurement, Arnhem, the Netherlands
| | - How Hwee Ong
- Department of Social Psychology, Tilburg University, Tilburg, the Netherlands
| | - Brian A. Nosek
- Center for Open Science, Charlottesville, Virginia, United States of America
- Department of Psychology, University of Virginia, Virginia, United States of America
| | | | - David Mellor
- Center for Open Science, Charlottesville, Virginia, United States of America
| | - Jelte M. Wicherts
- Department of Methodology and Statistics, Tilburg University, Tilburg, the Netherlands
| |
Collapse
|
8
|
Bernard R, Weissgerber T, Bobrov E, Winham S, Dirnagl U, Riedel N. fiddle: a tool to combat publication bias by getting research out of the file drawer and into the scientific community. Clin Sci (Lond) 2020; 134:2729-2739. [PMID: 33111948 PMCID: PMC7593522 DOI: 10.1042/cs20201125] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2020] [Revised: 09/30/2020] [Accepted: 10/02/2020] [Indexed: 01/10/2023]
Abstract
Statistically significant findings are more likely to be published than non-significant or null findings, leaving scientists and healthcare personnel to make decisions based on distorted scientific evidence. Continuously expanding ´file drawers' of unpublished data from well-designed experiments waste resources creates problems for researchers, the scientific community and the public. There is limited awareness of the negative impact that publication bias and selective reporting have on the scientific literature. Alternative publication formats have recently been introduced that make it easier to publish research that is difficult to publish in traditional peer reviewed journals. These include micropublications, data repositories, data journals, preprints, publishing platforms, and journals focusing on null or neutral results. While these alternative formats have the potential to reduce publication bias, many scientists are unaware that these formats exist and don't know how to use them. Our open source file drawer data liberation effort (fiddle) tool (RRID:SCR_017327 available at: http://s-quest.bihealth.org/fiddle/) is a match-making Shiny app designed to help biomedical researchers to identify the most appropriate publication format for their data. Users can search for a publication format that meets their needs, compare and contrast different publication formats, and find links to publishing platforms. This tool will assist scientists in getting otherwise inaccessible, hidden data out of the file drawer into the scientific community and literature. We briefly highlight essential details that should be included to ensure reporting quality, which will allow others to use and benefit from research published in these new formats.
Collapse
Affiliation(s)
- René Bernard
- NeuroCure Cluster of Excellence, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health, Berlin, Germany
- QUEST Center for Transforming Biomedical Research, Berlin Institute of Health (BIH), Berlin, Germany
| | - Tracey L. Weissgerber
- QUEST Center for Transforming Biomedical Research, Berlin Institute of Health (BIH), Berlin, Germany
| | - Evgeny Bobrov
- QUEST Center for Transforming Biomedical Research, Berlin Institute of Health (BIH), Berlin, Germany
| | - Stacey J. Winham
- Division of Biomedical Statistics and Informatics, Mayo Clinic Rochester, MN, U.S.A
| | - Ulrich Dirnagl
- NeuroCure Cluster of Excellence, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health, Berlin, Germany
- QUEST Center for Transforming Biomedical Research, Berlin Institute of Health (BIH), Berlin, Germany
| | - Nico Riedel
- QUEST Center for Transforming Biomedical Research, Berlin Institute of Health (BIH), Berlin, Germany
| |
Collapse
|
9
|
Coppens DG, Gardarsdottir H, van den Bogert CA, De Bruin ML, Leufkens HG, Hoekman J. Publication rates and reported results in a cohort of gene- and cell-based therapy trials. Regen Med 2020; 15:1215-1227. [PMID: 32103712 DOI: 10.2217/rme-2019-0066] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/08/2023] Open
Abstract
Aim: We investigated publication rates and reported results for gene- and cell-based therapy trials. Materials & methods: In a cohort of Institutional Review Board (IRB)-authorized trials during 2007-2017 in the Netherlands (n = 105), we examine publication rates and reported results in scientific papers and conference abstracts as well as associations with the occurrence of trial characteristics. Results: The publication rate for scientific papers was 27% and 17% for conference abstracts (median survival time: 1050 days). Academic hospitals published more in scientific papers whereas private sponsors published more in conference abstracts. Manufacturing protocols were underreported compared with clinical outcomes. Most publications reported positive results (78%). Conclusion: Publication rates are currently suboptimal indicating a need for enhanced knowledge sharing to stimulate gene- and cell-based therapy development.
Collapse
Affiliation(s)
- Delphi Gm Coppens
- Division of Pharmacoepidemiology & Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Utrecht, The Netherlands
| | - Helga Gardarsdottir
- Division of Pharmacoepidemiology & Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Utrecht, The Netherlands.,Department of Clinical Pharmacy, Division Laboratories, Pharmacy & Biomedical Genetics, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Cornelis A van den Bogert
- Division of Pharmacoepidemiology & Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Utrecht, The Netherlands
| | - Marie L De Bruin
- Division of Pharmacoepidemiology & Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Utrecht, The Netherlands.,Copenhagen Centre for Regulatory Science, University of Copenhagen, Copenhagen, Denmark
| | - Hubert Gm Leufkens
- Division of Pharmacoepidemiology & Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Utrecht, The Netherlands
| | - Jarno Hoekman
- Division of Pharmacoepidemiology & Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Utrecht, The Netherlands.,Innovation Studies Group, Faculty of Geosciences, Utrecht University, Utrecht, The Netherlands
| |
Collapse
|
10
|
Alnefeesi Y, Siegel A, Lui LMW, Teopiz KM, Ho RCM, Lee Y, Nasri F, Gill H, Lin K, Cao B, Rosenblat JD, McIntyre RS. Impact of SARS-CoV-2 Infection on Cognitive Function: A Systematic Review. Front Psychiatry 2020; 11:621773. [PMID: 33643083 PMCID: PMC7902710 DOI: 10.3389/fpsyt.2020.621773] [Citation(s) in RCA: 60] [Impact Index Per Article: 15.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/27/2020] [Accepted: 12/22/2020] [Indexed: 12/29/2022] Open
Abstract
The prevalence and etiology of COVID-19's impact on brain health and cognitive function is poorly characterized. With mounting reports of delirium, systemic inflammation, and evidence of neurotropism, a statement on cognitive impairment among COVID-19 cases is needed. A substantial literature has demonstrated that inflammation can severely disrupt brain function, suggesting an immune response, a cytokine storm, as a possible cause of neurocognitive impairments. In this light, the aim of the present study was to summarize the available knowledge of the impact of COVID-19 on cognition (i.e., herein, we broadly define cognition reflecting the reporting on this topic in the literature) during the acute and recovery phases of the disease, in hospitalized patients and outpatients with confirmed COVID-19 status. A systematic review of the literature identified six studies which document the prevalence of cognitive impairment, and one which quantifies deficits after recovery. Pooling the samples of the included studies (total sample n = 644) at three standards of quality produced conservative estimates of cognitive impairment ranging from 43.0 to 66.8% prevalence in hospitalized COVID-19 patients only, as no studies which report on outpatients met criteria for inclusion in the main synthesis. The most common impairment reported was delirium and frequent reports of elevated inflammatory markers suggest etiology. Other studies have demonstrated that the disease involves marked increases in IL-6, TNFα, and IL-1β; cytokines known to have a profound impact on working memory and attention. Impairment of these cognitive functions is a characteristic aspect of delirium, which suggests these cytokines as key mediators in the etiology of COVID-19 induced cognitive impairments. Researchers are encouraged to assay inflammatory markers to determine the potential role of inflammation in mediating the disturbance of cognitive function in individuals affected by COVID-19.
Collapse
Affiliation(s)
- Yazen Alnefeesi
- Mood Disorders Psychopharmacology Unit, University Health Network, Toronto, ON, Canada
| | - Ashley Siegel
- Mood Disorders Psychopharmacology Unit, University Health Network, Toronto, ON, Canada
| | - Leanna M W Lui
- Mood Disorders Psychopharmacology Unit, University Health Network, Toronto, ON, Canada
| | - Kayla M Teopiz
- Mood Disorders Psychopharmacology Unit, University Health Network, Toronto, ON, Canada
| | - Roger C M Ho
- Department of Psychological Medicine, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
| | - Yena Lee
- Mood Disorders Psychopharmacology Unit, University Health Network, Toronto, ON, Canada
| | - Flora Nasri
- Mood Disorders Psychopharmacology Unit, University Health Network, Toronto, ON, Canada
| | - Hartej Gill
- Mood Disorders Psychopharmacology Unit, University Health Network, Toronto, ON, Canada
| | - Kangguang Lin
- Department of Affective Disorder, the Affiliated Brain Hospital of Guangzhou Medical University (Guangzhou Huiai Hospital), Guangzhou Medical University, Guangzhou, China.,Laboratory of Emotion and Cognition, the Affiliated Brain Hospital of Guangzhou Medical University (Guangzhou Huiai Hospital), Guangzhou Medical University, Guangzhou, China
| | - Bing Cao
- Key Laboratory of Cognition and Personality, Faculty of Psychology, Ministry of Education, Southwest University, Chongqing, China
| | - Joshua D Rosenblat
- Mood Disorders Psychopharmacology Unit, University Health Network, Toronto, ON, Canada.,Department of Psychiatry, University of Toronto, Toronto, ON, Canada
| | - Roger S McIntyre
- Mood Disorders Psychopharmacology Unit, University Health Network, Toronto, ON, Canada.,Department of Psychiatry, University of Toronto, Toronto, ON, Canada.,Department of Pharmacology, University of Toronto, Toronto, ON, Canada.,Brain and Cognition Discovery Foundation, Toronto, ON, Canada
| |
Collapse
|
11
|
van der Steen JT, Ter Riet G, van den Bogert CA, Bouter LM. Causes of reporting bias: a theoretical framework. F1000Res 2019; 8:280. [PMID: 31497290 PMCID: PMC6713068 DOI: 10.12688/f1000research.18310.2] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 07/02/2019] [Indexed: 11/20/2022] Open
Abstract
Reporting of research findings is often selective. This threatens the validity of the published body of knowledge if the decision to report depends on the nature of the results. The evidence derived from studies on causes and mechanisms underlying selective reporting may help to avoid or reduce reporting bias. Such research should be guided by a theoretical framework of possible causal pathways that lead to reporting bias. We build upon a classification of determinants of selective reporting that we recently developed in a systematic review of the topic. The resulting theoretical framework features four clusters of causes. There are two clusters of necessary causes: (A) motivations (e.g. a preference for particular findings) and (B) means (e.g. a flexible study design). These two combined represent a sufficient cause for reporting bias to occur. The framework also features two clusters of component causes: (C) conflicts and balancing of interests referring to the individual or the team, and (D) pressures from science and society. The component causes may modify the effect of the necessary causes or may lead to reporting bias mediated through the necessary causes. Our theoretical framework is meant to inspire further research and to create awareness among researchers and end-users of research about reporting bias and its causes.
Collapse
Affiliation(s)
- Jenny T van der Steen
- Department of Public Health and Primary Care, Leiden University Medical Center, Hippocratespad 21, Gebouw 3, Leiden, 2300 RC Leiden, The Netherlands.,Department of Primary and Community Care, Radboud university medical center, Geert Grooteplein Noord 21, 6500 HB Nijmegen, The Netherlands
| | - Gerben Ter Riet
- ACHIEVE Centre for Applied Research, Amsterdam University of Applied Sciences, Tafelbergweg 51, Amsterdam, 1105 BD Amsterdam, The Netherlands.,Department of Cardiology, Amsterdam University Medical Center (location Meibergdreef), University of Amsterdam, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands
| | | | - Lex M Bouter
- Department of Epidemiology and Biostatistics, Amsterdam University Medical Centers, location VUmc, Van der Boechorststraat 7, 1081 BT Amsterdam, The Netherlands.,Department of Philosophy, Faculty of Humanities, Vrije Universiteit Amsterdam, De Boelelaan 1105, 1081 HV Amsterdam, The Netherlands
| |
Collapse
|
12
|
Zhao J, Wang C, Totton SC, Cullen JN, O’Connor AM. Reporting and analysis of repeated measurements in preclinical animals experiments. PLoS One 2019; 14:e0220879. [PMID: 31404099 PMCID: PMC6690515 DOI: 10.1371/journal.pone.0220879] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2018] [Accepted: 07/25/2019] [Indexed: 01/24/2023] Open
Abstract
A common feature of preclinical animal experiments is repeated measurement of the outcome, e.g., body weight measured in mice pups weekly for 20 weeks. Separate time point analysis or repeated measures analysis approaches can be used to analyze such data. Each approach requires assumptions about the underlying data and violations of these assumptions have implications for estimation of precision, and type I and type II error rates. Given the ethical responsibilities to maximize valid results obtained from animals used in research, our objective was to evaluate approaches to reporting repeated measures design used by investigators and to assess how assumptions about variation in the outcome over time impact type I and II error rates and precision of estimates. We assessed the reporting of repeated measures designs of 58 studies in preclinical animal experiments. We used simulation modelling to evaluate three approaches to statistical analysis of repeated measurement data. In particular, we assessed the impact of (a) repeated measure analysis assuming that the outcome had non-constant variation at all time points (heterogeneous variance) (b) repeated measure analysis assuming constant variation in the outcome (homogeneous variance), (c) separate ANOVA at individual time point in repeated measures designs. The evaluation of the three model fitting was based on comparing the p-values distributions, the type I and type II error rates and by implication, the shrinkage or inflation of standard error estimates from 1000 simulated dataset. Of 58 studies with repeated measures design, three provided a rationale for repeated measurement and 23 studies reported using a repeated-measures analysis approach. Of the 35 studies that did not use repeated-measures analysis, fourteen studies used only two time points to calculate weight change which potentially means collected data was not fully utilized. Other studies reported only select time points (n = 12) raising the issue of selective reporting. Simulation studies showed that an incorrect assumption about the variance structure resulted in modified error rates and precision estimates. The reporting of the validity of assumptions for repeated measurement data is very poor. The homogeneous variation assumption, which is often invalid for body weight measurements, should be confirmed prior to conducting the repeated-measures analysis using homogeneous covariance structure and adjusting the analysis using corrections or model specifications if this is not met.
Collapse
Affiliation(s)
- Jing Zhao
- Department of Statistics, college of Statistics, Iowa State University, Ames, Iowa, United States of America
| | - Chong Wang
- Department of Statistics, college of Statistics, Iowa State University, Ames, Iowa, United States of America
- Department of Veterinary Diagnostic and Production Animal Medicine, College of Veterinary Medicine, Iowa State University, Ames, Iowa, United States of America
| | | | - Jonah N. Cullen
- Department of Veterinary Diagnostic and Production Animal Medicine, College of Veterinary Medicine, Iowa State University, Ames, Iowa, United States of America
| | - Annette M. O’Connor
- Department of Veterinary Diagnostic and Production Animal Medicine, College of Veterinary Medicine, Iowa State University, Ames, Iowa, United States of America
| |
Collapse
|
13
|
Gini R, Fournie X, Dolk H, Kurz X, Verpillat P, Simondon F, Strassmann V, Apostolidis K, Goedecke T. The ENCePP Code of Conduct: A best practise for scientific independence and transparency in noninterventional postauthorisation studies. Pharmacoepidemiol Drug Saf 2019; 28:422-433. [PMID: 30838708 PMCID: PMC6594014 DOI: 10.1002/pds.4763] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2019] [Revised: 02/08/2019] [Accepted: 02/12/2019] [Indexed: 11/26/2022]
Abstract
PURPOSE The ENCePP Code of Conduct provides a framework for scientifically independent and transparent pharmacoepidemiological research. Despite becoming a landmark reference, practical implementation of key provisions was still limited. The fourth revision defines scientific independence and clarifies uncertainties on the applicability to postauthorisation safety studies requested by regulators. To separate the influence of the funder from the investigator's scientific responsibility, the Code now requires that the lead investigator is not employed by the funding institution. METHOD To assess how the revised Code fits the ecosystem of noninterventional pharmacoepidemiology research in Europe, we first mapped key recommendations of the revised Code against ISPE Good Pharmacoepidemiology Practices and the ADVANCE Code of Conduct. We surveyed stakeholders to understand perceptions on its value and practical applicability. Representatives from the different stakeholders' groups described their experience and expectations. RESULTS Unmet needs in pharmacoepidemiological research are fulfilled by providing unique guidance on roles and responsibilities to support scientific independence. The principles of scientific independence and transparency are well understood and reinforce trust in study results; however, around 70% of survey respondents still found some provisions difficult to apply. Representatives from stakeholders' groups found the new version promising, although limitations still exist. CONCLUSION By clarifying definitions and roles, the latest revision of the Code sets a new standard in the relationship between investigators and funders to support scientific independence of pharmacoepidemiological research. Disseminating and training on the provisions of the Code would help stakeholders to better understand its advantages and promote its adoption in noninterventional research.
Collapse
Affiliation(s)
- Rosa Gini
- Osservatorio di EpidemiologiaAgenzia regionale di sanità della ToscanaFlorenceItaly
| | - Xavier Fournie
- Global Medical AffairsICON Commercialisation & OutcomesLyonFrance
| | - Helen Dolk
- Faculty of Life and Health SciencesUniversity of Ulster at JordanstownJordanstownUK
| | - Xavier Kurz
- Pharmacovigilance and Epidemiology Department, Inspections, Human Medicines Pharmacovigilance and Committees DivisionEuropean Medicines AgencyAmsterdamThe Netherlands
| | | | - François Simondon
- Mother and Child Health Research Unit IRDUniversite Paris DescartesParisFrance
| | - Valerie Strassmann
- PharmacovigilanzBundesinstitut für Arzneimittel und Medizinprodukte (BfArM)BonnGermany
| | - Kathi Apostolidis
- Vice PresidentEuropean Cancer Patient Coalition (ECPC)BrusselsBelgium
| | - Thomas Goedecke
- Pharmacovigilance and Epidemiology Department, Inspections, Human Medicines Pharmacovigilance and Committees DivisionEuropean Medicines AgencyAmsterdamThe Netherlands
| |
Collapse
|