1
|
Siebert M, Gaba J, Renault A, Laviolle B, Locher C, Moher D, Naudet F. Data-sharing and re-analysis for main studies assessed by the European Medicines Agency-a cross-sectional study on European Public Assessment Reports. BMC Med 2022; 20:177. [PMID: 35590360 PMCID: PMC9119701 DOI: 10.1186/s12916-022-02377-2] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/09/2019] [Accepted: 04/13/2022] [Indexed: 01/13/2023] Open
Abstract
BACKGROUND Transparency and reproducibility are expected to be normative practices in clinical trials used for decision-making on marketing authorisations for new medicines. This registered report introduces a cross-sectional study aiming to assess inferential reproducibility for main trials assessed by the European Medicines Agency. METHODS Two researchers independently identified all studies on new medicines, biosimilars and orphan medicines given approval by the European Commission between January 2017 and December 2019, categorised as 'main studies' in the European Public Assessment Reports (EPARs). Sixty-two of these studies were randomly sampled. One researcher retrieved the individual patient data (IPD) for these studies and prepared a dossier for each study, containing the IPD, the protocol and information on the conduct of the study. A second researcher who had no access to study reports used the dossier to run an independent re-analysis of each trial. All results of these re-analyses were reported in terms of each study's conclusions, p-values, effect sizes and changes from the initial protocol. A team of two researchers not involved in the re-analysis compared results of the re-analyses with published results of the trial. RESULTS Two hundred ninety-two main studies in 173 EPARs were identified. Among the 62 studies randomly sampled, we received IPD for 10 trials. The median number of days between data request and data receipt was 253 [interquartile range 182-469]. For these ten trials, we identified 23 distinct primary outcomes for which the conclusions were reproduced in all re-analyses. Therefore, 10/62 trials (16% [95% confidence interval 8% to 28%]) were reproduced, as the 52 studies without available data were considered non-reproducible. There was no change from the original study protocol regarding the primary outcome in any of these ten studies. Spin was observed in the report of one study. CONCLUSIONS Despite their results supporting decisions that affect millions of people's health across the European Union, most main studies used in EPARs lack transparency and their results are not reproducible for external researchers. Re-analyses of the few trials with available data showed very good inferential reproducibility. TRIAL REGISTRATION https://osf.io/mcw3t/.
Collapse
Affiliation(s)
- Maximilian Siebert
- Univ Rennes, CHU Rennes, Inserm, CIC 1414 [(Centre d'Investigation Clinique de Rennes)], F-35000, Rennes, France.,Univ Rennes, CHU Rennes, Inserm, Irset (Institut de recherche en santé, environnement et travail) - UMR_S 1085, F-35000, Rennes, France
| | - Jeanne Gaba
- Univ Rennes, CHU Rennes, Inserm, CIC 1414 [(Centre d'Investigation Clinique de Rennes)], F-35000, Rennes, France.,Univ Rennes, CHU Rennes, Inserm, Irset (Institut de recherche en santé, environnement et travail) - UMR_S 1085, F-35000, Rennes, France
| | - Alain Renault
- Univ Rennes, CHU Rennes, Inserm, CIC 1414 [(Centre d'Investigation Clinique de Rennes)], F-35000, Rennes, France.,Univ Rennes, CHU Rennes, Inserm, Irset (Institut de recherche en santé, environnement et travail) - UMR_S 1085, F-35000, Rennes, France
| | - Bruno Laviolle
- Univ Rennes, CHU Rennes, Inserm, CIC 1414 [(Centre d'Investigation Clinique de Rennes)], F-35000, Rennes, France.,Univ Rennes, CHU Rennes, Inserm, Irset (Institut de recherche en santé, environnement et travail) - UMR_S 1085, F-35000, Rennes, France
| | - Clara Locher
- Univ Rennes, CHU Rennes, Inserm, CIC 1414 [(Centre d'Investigation Clinique de Rennes)], F-35000, Rennes, France.,Univ Rennes, CHU Rennes, Inserm, Irset (Institut de recherche en santé, environnement et travail) - UMR_S 1085, F-35000, Rennes, France
| | - David Moher
- Center for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
| | - Florian Naudet
- Univ Rennes, CHU Rennes, Inserm, CIC 1414 [(Centre d'Investigation Clinique de Rennes)], F-35000, Rennes, France. .,Univ Rennes, CHU Rennes, Inserm, Irset (Institut de recherche en santé, environnement et travail) - UMR_S 1085, F-35000, Rennes, France. .,Clinical Investigation Center (Inserm 1414) and Adult Psychiatry Department, Rennes University Hospital, Rennes, France.
| |
Collapse
|
2
|
AlRyalat SA, El Khatib O, Al-qawasmi O, Alkasrawi H, al Zu’bi R, Abu-Halaweh M, alkanash Y, Habash I. The National Heart, Lung, and Blood Institute data: analyzing published articles that used BioLINCC open access data. F1000Res 2021; 9:30. [PMID: 34621520 PMCID: PMC8447052 DOI: 10.12688/f1000research.21884.4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 08/16/2021] [Indexed: 11/20/2022] Open
Abstract
Background: Data sharing is now a mandatory prerequisite for several major funders and journals, where researchers are obligated to deposit the data resulting from their studies in an openly accessible repository. Biomedical open data are now widely available in almost all disciplines, where researchers can freely access and reuse these data in new studies. We aim to study the BioLINCC datasets, number of publications that used BioLINCC open access data, and the citations received by these publications. Methods: As of July 2019, there was a total of 194 datasets stored in BioLINCC repository and accessible through their portal. We requested the full list of publications that used these datasets from BioLINCC, and we also performed a supplementary PubMed search for other publications. We used Web of Science (WoS) to analyze the characteristics of publications and the citations they received, where WoS database index high quality articles. Results: 1,086 published articles used data from BioLINCC repository for 79 (40.72%) datasets, where 115 (59.28%) datasets did not have any publications associated with it. Of the total publications, 987 (90.88%) articles were WoS indexed. The number of publications has steadily increased since 2002 and peaked in 2018 with a total number of 138 publications on that year. The 987 open data publications (i.e., secondary publications) received a total of 34,181 citations up to 1
st October 2019. The average citation per item for the open data publications was 34.63. The total number of citations received by open data publications per year has increased from only 2 citations in 2002, peaking in 2018 with 2361 citations. Conclusion: Majority of BioLINCC datasets were not used in secondary publications. Despite that, the datasets used for secondary publications yielded publications in WoS indexed journals and are receiving an increasing number of citations.
Collapse
Affiliation(s)
- Saif Aldeen AlRyalat
- Department of Ophthalmology, University of Jordan Hospital, University of Jordan, Amman, 11942, Jordan
| | - Osama El Khatib
- Department of Internal Medicine, University of Jordan Hospital, University of Jordan, Amman, 11942, Jordan
| | - Ola Al-qawasmi
- Department of Internal Medicine, University of Jordan Hospital, University of Jordan, Amman, 11942, Jordan
| | - Hadeel Alkasrawi
- Department of Internal Medicine, University of Jordan Hospital, University of Jordan, Amman, 11942, Jordan
| | - Raneem al Zu’bi
- Department of Internal Medicine, University of Jordan Hospital, University of Jordan, Amman, 11942, Jordan
| | - Maram Abu-Halaweh
- Department of Internal Medicine, University of Jordan Hospital, University of Jordan, Amman, 11942, Jordan
| | - Yara alkanash
- Department of Internal Medicine, University of Jordan Hospital, University of Jordan, Amman, 11942, Jordan
| | - Ibrahim Habash
- Department of Forensic Medicine, University of Jordan Hospital, The University of Jordan, Amman, 11942, Jordan
| |
Collapse
|
3
|
Anthony N, Pellen C, Ohmann C, Moher D, Naudet F. Social media attention and citations of published outputs from re-use of clinical trial data: a matched comparison with articles published in the same journals. BMC Med Res Methodol 2021; 21:119. [PMID: 34092224 PMCID: PMC8182934 DOI: 10.1186/s12874-021-01311-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2021] [Accepted: 04/30/2021] [Indexed: 12/17/2022] Open
Abstract
BACKGROUND Data-sharing policies in randomized clinical trials (RCTs) should have an evaluation component. The main objective of this case-control study was to assess the impact of published re-uses of RCT data in terms of media attention (Altmetric) and citation rates. METHODS Re-uses of RCT data published up to December 2019 (cases) were searched for by two reviewers on 3 repositories (CSDR, YODA project, and Vivli) and matched to control papers published in the same journal. The Altmetric Attention Score (primary outcome), components of this score (e.g. mention of policy sources, media attention) and the total number of citations were compared between these two groups. RESULTS 89 re-uses were identified: 48 (53.9%) secondary analyses, 34 (38.2%) meta-analyses, 4 (4.5%) methodological analyses and 3 (3.4%) re-analyses. The median (interquartile range) Altmetric Attention Scores were 5.9 (1.3-22.2) for re-use and 2.8 (0.3-12.3) for controls (p = 0.14). No statistical difference was found on any of the components of in the Altmetric Attention Score. The median (interquartile range) numbers of citations were 3 (1-8) for reuses and 4 (1 - 11.5) for controls (p = 0.30). Only 6/89 re-uses (6.7%) were cited in a policy source. CONCLUSIONS Using all available re-uses of RCT data to date from major data repositories, we were not able to demonstrate that re-uses attracted more attention than a matched sample of studies published in the same journals. Small average differences are still possible, as the sample size was limited. However matching choices have some limitations so results should be interpreted very cautiously. Also, citations by policy sources for re-uses were rare. TRIAL REGISTRATION Registration: osf.io/fp62e.
Collapse
Affiliation(s)
- N. Anthony
- University Hospital of La Réunion, Saint-Denis, Reunion Island France
- Univ Rennes, CHU Rennes, Inserm, CIC 1414 [(Centre d’Investigation Clinique de Rennes)], F-35000 Rennes, France
| | - C. Pellen
- Univ Rennes, CHU Rennes, Inserm, CIC 1414 [(Centre d’Investigation Clinique de Rennes)], F-35000 Rennes, France
| | - C. Ohmann
- European Clinical Research Infrastructure Network, Düsseldorf, Germany
| | - D. Moher
- Ottawa Hospital Research Institute, Ottawa, Canada
| | - F. Naudet
- Univ Rennes, CHU Rennes, Inserm, CIC 1414 [(Centre d’Investigation Clinique de Rennes)], F-35000 Rennes, France
| |
Collapse
|
4
|
Ross JS, Waldstreicher J, Bamford S, Berlin JA, Childers K, Desai NR, Gamble G, Gross CP, Kuntz R, Lehman R, Lins P, Morris SA, Ritchie JD, Krumholz HM. Overview and experience of the YODA Project with clinical trial data sharing after 5 years. Sci Data 2018; 5:180268. [PMID: 30480665 PMCID: PMC6257043 DOI: 10.1038/sdata.2018.268] [Citation(s) in RCA: 43] [Impact Index Per Article: 7.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2018] [Accepted: 10/24/2018] [Indexed: 02/08/2023] Open
Abstract
The Yale University Open Data Access (YODA) Project has facilitated access to clinical trial data since 2013. The purpose of this article is to provide an overview of the Project, describe key decisions that were made when establishing data sharing policies, and suggest how our experience and the experiences of our first two data generator partners, Medtronic, Inc. and Johnson & Johnson, can be used to enhance other ongoing or future initiatives.
Collapse
Affiliation(s)
- Joseph S Ross
- Section of General Medicine, Department of Medicine, Yale School of Medicine, New Haven, CT, USA.,National Clinician Scholars Program, Yale School of Medicine, New Haven, CT, USA.,Department of Health Policy and Management, Yale School of Public Health, New Haven, CT, USA.,Center for Outcomes Research and Evaluation, Yale-New Haven Hospital, New Haven, CT, USA
| | | | - Stephen Bamford
- Janssen Pharmaceutical Companies of Johnson & Johnson, High Wycombe, UK
| | | | | | - Nihar R Desai
- Center for Outcomes Research and Evaluation, Yale-New Haven Hospital, New Haven, CT, USA.,Section of Cardiovascular Medicine, Department of Medicine, Yale School of Medicine, New Haven, CT, USA
| | - Ginger Gamble
- Center for Outcomes Research and Evaluation, Yale-New Haven Hospital, New Haven, CT, USA
| | - Cary P Gross
- Section of General Medicine, Department of Medicine, Yale School of Medicine, New Haven, CT, USA.,National Clinician Scholars Program, Yale School of Medicine, New Haven, CT, USA.,Center for Outcomes Research and Evaluation, Yale-New Haven Hospital, New Haven, CT, USA.,Cancer Outcomes, Public Policy, and Effectiveness Research (COPPER) Center and Yale Cancer Center, Yale School of Medicine, New Haven, CT, USA
| | | | | | - Peter Lins
- Johnson & Johnson, New Brunswick, NJ, USA
| | | | - Jessica D Ritchie
- Center for Outcomes Research and Evaluation, Yale-New Haven Hospital, New Haven, CT, USA
| | - Harlan M Krumholz
- National Clinician Scholars Program, Yale School of Medicine, New Haven, CT, USA.,Department of Health Policy and Management, Yale School of Public Health, New Haven, CT, USA.,Center for Outcomes Research and Evaluation, Yale-New Haven Hospital, New Haven, CT, USA.,Section of Cardiovascular Medicine, Department of Medicine, Yale School of Medicine, New Haven, CT, USA
| |
Collapse
|
5
|
Natale A, Stagg R, Zhang B. Different Population and End Point Definitions in Reproduction Analysis Based on Shared Data. JAMA Cardiol 2018; 3:893-894. [PMID: 29971327 DOI: 10.1001/jamacardio.2018.1788] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Affiliation(s)
- Andrea Natale
- Texas Cardiac Arrhythmia Institute at St David's Medical Center, Austin
| | | | | |
Collapse
|
6
|
Affiliation(s)
- Clyde W. Yancy
- Feinberg School of Medicine, Northwestern University, Chicago, Illinois
- Deputy Editor, JAMA Cardiology
| | - Robert A. Harrington
- Deputy Editor, JAMA Cardiology
- Department of Medicine, Stanford University, Stanford, California
| | - Robert O. Bonow
- Feinberg School of Medicine, Northwestern University, Chicago, Illinois
- Northwestern Memorial Hospital, Chicago, Illinois
- Editor, JAMA Cardiology
| |
Collapse
|
7
|
Gay HC, Baldridge AS, Huffman MD. Different Population and End Point Definitions in Reproduction Analysis Based on Shared Data-Reply. JAMA Cardiol 2018; 3:894. [PMID: 29971351 DOI: 10.1001/jamacardio.2018.1791] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Affiliation(s)
- Hawkins C Gay
- Department of Medicine, Northwestern University, Chicago, Illinois
| | - Abigail S Baldridge
- Department of Preventive Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois
| | - Mark D Huffman
- Department of Medicine, Northwestern University, Chicago, Illinois.,Department of Preventive Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois.,Associate Editor
| |
Collapse
|
8
|
Dhruva SS, Ross JS, Desai NR. Real-World Evidence: Promise and Peril For Medical Product Evaluation. P & T : A PEER-REVIEWED JOURNAL FOR FORMULARY MANAGEMENT 2018; 43:464-472. [PMID: 30100687 PMCID: PMC6065494] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
Randomized controlled trials have long been the gold standard for evidence in medical product evaluation, but there is growing support for the use of real-world evidence (RWE). The authors review the benefits and limitations of RWE and discuss the implications for P&T committees.
Collapse
|
9
|
McCarthy CP, Vaduganathan M. Navigating Data Sharing in Cardiology From a Trainee's Perspective. J Am Coll Cardiol 2018; 71:2075-2078. [PMID: 29724360 DOI: 10.1016/j.jacc.2018.03.477] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Affiliation(s)
- Cian P McCarthy
- Department of Medicine, Massachusetts General Hospital, Boston, Massachusetts
| | - Muthiah Vaduganathan
- Brigham and Women's Hospital Heart & Vascular Center and Harvard Medical School, Boston, Massachusetts.
| |
Collapse
|