1
|
Gouraud H, Wallach JD, Boussageon R, Ross JS, Naudet F. Vibration of effect in more than 16 000 pooled analyses of individual participant data from 12 randomised controlled trials comparing canagliflozin and placebo for type 2 diabetes mellitus: multiverse analysis. BMJ Med 2022; 1:e000154. [PMID: 36936564 PMCID: PMC9978683 DOI: 10.1136/bmjmed-2022-000154] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/04/2022] [Accepted: 08/23/2022] [Indexed: 11/04/2022]
Abstract
Objective To evaluate the impact of conducting all possible pooled analyses across different combinations of randomised controlled trials and endpoints. Design Multiverse analysis, consisting of numerous pooled analyses of individual participant data. Setting Individual patient data from 12 randomised controlled trials comparing canagliflozin treatment with placebo, shared on the Yale University Open Data Access project (https://yoda.yale.edu/) platform, up to 16 April 2021. Participants 15 094 people with type 2 diabetes mellitus. Main outcome measures Pooled analyses estimated changes in serum glycated haemoglobin (HbA1c), major adverse cardiovascular events, and serious adverse events at weeks 12, 18, 26, and 52. The distribution of effect estimates was calculated for all possible combinations, and the direction and magnitude of the first and 99th centiles of effect estimates were compared. Results Across 16 332 distinct pooled analyses comparing canagliflozin with placebo for changes in HbA1c, standardised effect estimates were in favour of canagliflozin treatment at both the first centile (-0.75%) and 99th centile (-0.48%); 15 994 (97.93%) analyses showed significant results (P<0.05) in favour of canagliflozin. For major adverse cardiovascular events, estimated hazard ratios were 0.20 at the first centile and 0.90 at the 99th centile; 2705 of 8144 analyses (33.21%) were significant, all of which were in favour of canagliflozin treatment. For serious adverse events, estimated hazard ratios were 0.59 at the first centile and 1.14 at the 99th centile; 5793 of 16 332 (35.47%) analyses were significant, with 5754 in favour of canagliflozin and 39 in favour of placebo. Conclusion Results from pooled analyses can be subject to vibration of effects and should be critically appraised, especially regarding the risk for selection and availability bias in individual participant data retrieved.
Collapse
Affiliation(s)
- Henri Gouraud
- Inserm, CIC 1414 (Centre d’Investigation Clinique de Rennes), Rennes 1 University, Rennes, France
- Inserm, Irset (Institut de recherche en santé, environnement et travail), Rennes 1 University, Rennes, France
| | - Joshua D Wallach
- Department of Environmental Health Sciences, Yale University School of Public Health, New Haven, CT, USA
| | - Rémy Boussageon
- UCBL, CNRS, UMR 5558, LBBE, EMET, University Claude Bernard Lyon 1, Villeurbanne, France
| | - Joseph S Ross
- Department of Internal Medicine, Yale School of Medicine, New Haven, CT, USA
| | - Florian Naudet
- Inserm, CIC 1414 (Centre d’Investigation Clinique de Rennes), Rennes 1 University, Rennes, France
- Inserm, Irset (Institut de recherche en santé, environnement et travail), Rennes 1 University, Rennes, France
- Institut Universitaire de France, Paris, France
| |
Collapse
|
2
|
Vazquez E, Gouraud H, Naudet F, Gross CP, Krumholz HM, Ross JS, Wallach JD. Characteristics of available studies and dissemination of research using major clinical data sharing platforms. Clin Trials 2021; 18:657-666. [PMID: 34407656 DOI: 10.1177/17407745211038524] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
BACKGROUND/AIMS Over the past decade, numerous data sharing platforms have been launched, providing access to de-identified individual patient-level data and supporting documentation. We evaluated the characteristics of prominent clinical data sharing platforms, including types of studies listed as available for request, data requests received, and rates of dissemination of research findings from data requests. METHODS We reviewed publicly available information listed on the websites of six prominent clinical data sharing platforms: Biological Specimen and Data Repository Information Coordinating Center, ClinicalStudyDataRequest.com, Project Data Sphere, Supporting Open Access to Researchers-Bristol Myers Squibb, Vivli, and the Yale Open Data Access Project. We recorded key platform characteristics, including listed studies and available supporting documentation, information on the number and status of data requests, and rates of dissemination of research findings from data requests (i.e. publications in a peer-reviewed journals, preprints, conference abstracts, or results reported on the platform's website). RESULTS The number of clinical studies listed as available for request varied among five data sharing platforms: Biological Specimen and Data Repository Information Coordinating Center (n = 219), ClinicalStudyDataRequest.com (n = 2,897), Project Data Sphere (n = 154), Vivli (n = 5426), and the Yale Open Data Access Project (n = 395); Supporting Open Access to Researchers did not provide a list of Bristol Myers Squibb studies available for request. Individual patient-level data were nearly always reported as being available for request, as opposed to only Clinical Study Reports (Biological Specimen and Data Repository Information Coordinating Center = 211/219 (96.3%); ClinicalStudyDataRequest.com = 2884/2897 (99.6%); Project Data Sphere = 154/154 (100.0%); and the Yale Open Data Access Project = 355/395 (89.9%)); Vivli did not provide downloadable study metadata. Of 1201 data requests listed on ClinicalStudyDataRequest.com, Supporting Open Access to Researchers-Bristol Myers Squibb, Vivli, and the Yale Open Data Access Project platforms, 586 requests (48.8%) were approved (i.e. data access granted). The majority were for secondary analyses and/or developing/validating methods (ClinicalStudyDataRequest.com = 262/313 (83.7%); Supporting Open Access to Researchers-Bristol Myers Squibb = 22/30 (73.3%); Vivli = 63/84 (75.0%); the Yale Open Data Access Project = 111/159 (69.8%)); four were for re-analyses or corroborations of previous research findings (ClinicalStudyDataRequest.com = 3/313 (1.0%) and the Yale Open Data Access Project = 1/159 (0.6%)). Ninety-five (16.1%) approved data requests had results disseminated via peer-reviewed publications (ClinicalStudyDataRequest.com = 61/313 (19.5%); Supporting Open Access to Researchers-Bristol Myers Squibb = 3/30 (10.0%); Vivli = 4/84 (4.8%); the Yale Open Data Access Project = 27/159 (17.0%)). Forty-two (6.8%) additional requests reported results through preprints, conference abstracts, or on the platform's website (ClinicalStudyDataRequest.com = 12/313 (3.8%); Supporting Open Access to Researchers-Bristol Myers Squibb = 3/30 (10.0%); Vivli = 2/84 (2.4%); Yale Open Data Access Project = 25/159 (15.7%)). CONCLUSION Across six prominent clinical data sharing platforms, information on studies and request metrics varied in availability and format. Most data requests focused on secondary analyses and approximately one-quarter of all approved requests publicly disseminated their results. To further promote the use of shared clinical data, platforms should increase transparency, consistently clarify the availability of the listed studies and supporting documentation, and ensure that research findings from data requests are disseminated.
Collapse
Affiliation(s)
| | - Henri Gouraud
- Centre Hospitalier Universitaire Rennes, Inserm, Centre d'Investigation Clinique de Rennes, Universite de Rennes, Rennes, France
| | - Florian Naudet
- Centre Hospitalier Universitaire Rennes, Inserm, Centre d'Investigation Clinique de Rennes, Universite de Rennes, Rennes, France
| | - Cary P Gross
- Section of General Internal Medicine, Yale School of Medicine, New Haven, CT, USA.,Cancer Outcomes, Public Policy, and Effectiveness Research (COPPER) Center, Yale University, New Haven, CT, USA.,Department of Chronic Disease Epidemiology, Yale School of Public Health, New Haven, CT, USA
| | - Harlan M Krumholz
- Section of Cardiovascular Medicine, Yale School of Medicine, New Haven, CT, USA.,Yale-New Haven Hospital Center for Outcomes Research and Evaluation, New Haven, CT, USA.,Department of Health Policy and Management, Yale School of Public Health, New Haven, CT, USA
| | - Joseph S Ross
- Section of General Internal Medicine, Yale School of Medicine, New Haven, CT, USA.,Yale-New Haven Hospital Center for Outcomes Research and Evaluation, New Haven, CT, USA.,Department of Health Policy and Management, Yale School of Public Health, New Haven, CT, USA
| | - Joshua D Wallach
- Department of Environmental Health Sciences, Yale School of Public Health, New Haven, CT, USA
| |
Collapse
|
3
|
Siebert M, Gaba JF, Caquelin L, Gouraud H, Dupuy A, Moher D, Naudet F. Data-sharing recommendations in biomedical journals and randomised controlled trials: an audit of journals following the ICMJE recommendations. BMJ Open 2020; 10:e038887. [PMID: 32474433 PMCID: PMC7264700 DOI: 10.1136/bmjopen-2020-038887] [Citation(s) in RCA: 19] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/27/2020] [Revised: 04/23/2020] [Accepted: 04/30/2020] [Indexed: 11/04/2022] Open
Abstract
OBJECTIVE To explore the implementation of the International Committee of Medical Journal Editors (ICMJE) data-sharing policy which came into force on 1 July 2018 by ICMJE-member journals and by ICMJE-affiliated journals declaring they follow the ICMJE recommendations. DESIGN A cross-sectional survey of data-sharing policies in 2018 on journal websites and in data-sharing statements in randomised controlled trials (RCTs). SETTING ICMJE website; PubMed/Medline. ELIGIBILITY CRITERIA ICMJE-member journals and 489 ICMJE-affiliated journals that published an RCT in 2018, had an accessible online website and were not considered as predatory journals according to Beall's list. One hundred RCTs for member journals and 100 RCTs for affiliated journals with a data-sharing policy, submitted after 1 July 2018. MAIN OUTCOME MEASURES The primary outcome for the policies was the existence of a data-sharing policy (explicit data-sharing policy, no data-sharing policy, policy merely referring to ICMJE recommendations) as reported on the journal website, especially in the instructions for authors. For RCTs, our primary outcome was the intention to share individual participant data set out in the data-sharing statement. RESULTS Eight (out of 14; 57%) member journals had an explicit data-sharing policy on their website (three were more stringent than the ICMJE requirements, one was less demanding and four were compliant), five (35%) additional journals stated that they followed the ICMJE requirements, and one (8%) had no policy online. In RCTs published in these journals, there were data-sharing statements in 98 out of 100, with expressed intention to share individual patient data reaching 77 out of 100 (77%; 95% CI 67% to 85%). One hundred and forty-five (out of 489) ICMJE-affiliated journals (30%; 26% to 34%) had an explicit data-sharing policy on their website (11 were more stringent than the ICMJE requirements, 85 were less demanding and 49 were compliant) and 276 (56%; 52% to 61%) merely referred to the ICMJE requirements. In RCTs published in affiliated journals with an explicit data-sharing policy, data-sharing statements were rare (25%), and expressed intentions to share data were found in 22% (15% to 32%). CONCLUSION The implementation of ICMJE data-sharing requirements in online journal policies was suboptimal for ICMJE-member journals and poor for ICMJE-affiliated journals. The implementation of the policy was good in member journals and of concern for affiliated journals. We suggest the conduct of continuous audits of medical journal data-sharing policies in the future. REGISTRATION The protocol was registered before the start of the research on the Open Science Framework (https://osf.io/n6whd/).
Collapse
Affiliation(s)
- Maximilian Siebert
- Univ Rennes, CHU Rennes, Inserm, CIC 1414 (Centre d'Investigation Clinique de Rennes), F-35000, Rennes, France
- Univ Rennes, CHU Rennes, REPERES (Recherche en Pharmaco-épidémiologie et Recours aux Soins), EA 7449, 35000, Rennes, France
| | - Jeanne Fabiola Gaba
- Univ Rennes, CHU Rennes, Inserm, CIC 1414 (Centre d'Investigation Clinique de Rennes), F-35000, Rennes, France
- Univ Rennes, CHU Rennes, REPERES (Recherche en Pharmaco-épidémiologie et Recours aux Soins), EA 7449, 35000, Rennes, France
| | - Laura Caquelin
- Univ Rennes, CHU Rennes, Inserm, CIC 1414 (Centre d'Investigation Clinique de Rennes), F-35000, Rennes, France
| | - Henri Gouraud
- Univ Rennes, CHU Rennes, Inserm, CIC 1414 (Centre d'Investigation Clinique de Rennes), F-35000, Rennes, France
| | - Alain Dupuy
- Univ Rennes, CHU Rennes, REPERES (Recherche en Pharmaco-épidémiologie et Recours aux Soins), EA 7449, 35000, Rennes, France
| | - David Moher
- Center for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Florian Naudet
- Univ Rennes, CHU Rennes, Inserm, CIC 1414 (Centre d'Investigation Clinique de Rennes), F-35000, Rennes, France
| |
Collapse
|