1
|
Meier N, Munch MW, Granholm A, Perner A, Hertz FB, Venkatesh B, Hammond NE, Li Q, De Bus L, De Waele J, Kauzonas E, Sjövall F, Møller MH, Helleberg M. Empirical carbapenems or piperacillin/tazobactam for infections in intensive care: An international retrospective cohort study. Acta Anaesthesiol Scand 2024; 68:821-829. [PMID: 38549422 DOI: 10.1111/aas.14419] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2024] [Revised: 03/14/2024] [Accepted: 03/14/2024] [Indexed: 06/20/2024]
Abstract
BACKGROUND Critically ill patients in intensive care units (ICU) are frequently administered broad-spectrum antibiotics (e.g., carbapenems or piperacillin/tazobactam) for suspected or confirmed infections. This retrospective cohort study aimed to describe the use of carbapenems and piperacillin/tazobactam in two international, prospectively collected datasets. METHODS We conducted a post hoc analysis of data from the "Adjunctive Glucocorticoid Therapy in Patients with Septic Shock" (ADRENAL) trial (n = 3713) and the "Antimicrobial de-escalation in the critically ill patient and assessment of clinical cure" (DIANA) study (n = 1488). The primary outcome was the proportion of patients receiving initial antibiotic treatment with carbapenems and piperacillin/tazobactam. Secondary outcomes included mortality, days alive and out of ICU and ICU length of stay at 28 days. RESULTS In the ADRENAL trial, carbapenems were used in 648 out of 3713 (17%), whereas piperacillin/tazobactam was used in 1804 out of 3713 (49%) participants. In the DIANA study, carbapenems were used in 380 out of 1480 (26%), while piperacillin/tazobactam was used in 433 out of 1488 (29%) participants. Mortality at 28 days was 23% for patients receiving carbapenems and 24% for those receiving piperacillin/tazobactam in ADRENAL and 23% and 19%, respectively, in DIANA. We noted variations in secondary outcomes; in DIANA, patients receiving carbapenems had a median of 13 days alive and out of ICU compared with 18 days among those receiving piperacillin/tazobactam. In ADRENAL, the median hospital length of stay was 27 days for patients receiving carbapenems and 21 days for those receiving piperacillin/tazobactam. CONCLUSIONS In this post hoc analysis of ICU patients with infections, we found widespread initial use of carbapenems and piperacillin/tazobactam in international ICUs, with the latter being more frequently used. Randomized clinical trials are needed to assess if the observed variations in outcomes may be drug-related effects or due to confounders.
Collapse
Affiliation(s)
- Nick Meier
- Department of Intensive Care, Copenhagen University Hospital - Rigshospitalet, Copenhagen, Denmark
| | - Marie Warrer Munch
- Department of Intensive Care, Copenhagen University Hospital - Rigshospitalet, Copenhagen, Denmark
| | - Anders Granholm
- Department of Intensive Care, Copenhagen University Hospital - Rigshospitalet, Copenhagen, Denmark
| | - Anders Perner
- Department of Intensive Care, Copenhagen University Hospital - Rigshospitalet, Copenhagen, Denmark
| | - Frederik Boëtius Hertz
- Department of Clinical Microbiology, Copenhagen University Hospital - Rigshospitalet, Copenhagen, Denmark
- Department of Immunology & Microbiology, University of Copenhagen, Copenhagen, Denmark
| | - Balasubramanian Venkatesh
- Critical Care Program, The George Institute for Global Health, University of New South Wales, Sydney, New South Wales, Australia
| | - Naomi E Hammond
- Critical Care Program, The George Institute for Global Health, University of New South Wales, Sydney, New South Wales, Australia
| | - Qiang Li
- Critical Care Program, The George Institute for Global Health, University of New South Wales, Sydney, New South Wales, Australia
| | - Liesbet De Bus
- Department of Intensive Care Medicine, Ghent University Hospital, Ghent, Belgium
- Department of Internal Medicine and Pediatrics, Faculty of Medicine and Health Sciences, Ghent University, Ghent, Belgium
| | - Jan De Waele
- Department of Intensive Care Medicine, Ghent University Hospital, Ghent, Belgium
- Department of Internal Medicine and Pediatrics, Faculty of Medicine and Health Sciences, Ghent University, Ghent, Belgium
| | - Evaldas Kauzonas
- Department of Intensive and Perioperative Care, Skåne University Hospital, Malmö, Sweden
| | - Fredrik Sjövall
- Department of Intensive and Perioperative Care, Skåne University Hospital, Malmö, Sweden
| | - Morten Hylander Møller
- Department of Intensive Care, Copenhagen University Hospital - Rigshospitalet, Copenhagen, Denmark
- Department of Clinical Medicine, Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Marie Helleberg
- Department of Clinical Medicine, Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark
- Department of Infectious Diseases, Copenhagen University Hospital - Rigshospitalet, Copenhagen, Denmark
- Centre of Excellence for Health, Immunity and Infections, Copenhagen University Hospital - Rigshospitalet, Copenhagen, Denmark
| |
Collapse
|
2
|
Buka RJ, Sutton DJ, Nicolson PL. Enoxaparin is not an oral factor Xa inhibitor. Response to Sutton et al. real-world clinical outcomes among US veterans with oral factor Xa inhibitor-related major bleeding treated with andexanet alfa or 4-factor prothrombin complex concentrate. J Thromb Thrombolysis 2024; 57:739-741. [PMID: 38358568 DOI: 10.1007/s11239-024-02948-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 01/04/2024] [Indexed: 02/16/2024]
Affiliation(s)
- Richard J Buka
- Institute of Cardiovascular Sciences, University of Birmingham, Birmingham, UK.
| | - David J Sutton
- Staffordshire Thrombosis and Anticoagulation Service, University Hospitals North Midlands NHS Trust, Stoke-on-Trent, UK
| | - Phillip Lr Nicolson
- Institute of Cardiovascular Sciences, University of Birmingham, Birmingham, UK
- Department of Clinical Haematology, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| |
Collapse
|
3
|
Heyard R, Held L, Schneeweiss S, Wang SV. Design differences and variation in results between randomised trials and non-randomised emulations: meta-analysis of RCT-DUPLICATE data. BMJ MEDICINE 2024; 3:e000709. [PMID: 38348308 PMCID: PMC10860009 DOI: 10.1136/bmjmed-2023-000709] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/17/2023] [Accepted: 12/27/2023] [Indexed: 02/15/2024]
Abstract
Objective To explore how design emulation and population differences relate to variation in results between randomised controlled trials (RCT) and non-randomised real world evidence (RWE) studies, based on the RCT-DUPLICATE initiative (Randomised, Controlled Trials Duplicated Using Prospective Longitudinal Insurance Claims: Applying Techniques of Epidemiology). Design Meta-analysis of RCT-DUPLICATE data. Data sources Trials included in RCT-DUPLICATE, a demonstration project that emulated 32 randomised controlled trials using three real world data sources: Optum Clinformatics Data Mart, 2004-19; IBM MarketScan, 2003-17; and subsets of Medicare parts A, B, and D, 2009-17. Eligibility criteria for selecting studies Trials where the primary analysis resulted in a hazard ratio; 29 RCT-RWE study pairs from RCT-DUPLICATE. Results Differences and variation in effect sizes between the results from randomised controlled trials and real world evidence studies were investigated. Most of the heterogeneity in effect estimates between the RCT-RWE study pairs in this sample could be explained by three emulation differences in the meta-regression model: treatment started in hospital (which does not appear in health insurance claims data), discontinuation of some baseline treatments at randomisation (which would have been an unusual care decision in clinical practice), and delayed onset of drug effects (which would be under-reported in real world clinical practice because of the relatively short persistence of the treatment). Adding the three emulation differences to the meta-regression reduced heterogeneity from 1.9 to almost 1 (absence of heterogeneity). Conclusions This analysis suggests that a substantial proportion of the observed variation between results from randomised controlled trials and real world evidence studies can be attributed to differences in design emulation.
Collapse
Affiliation(s)
- Rachel Heyard
- Center for Reproducible Science, Epidemiology, Biostatistics and Prevention Institute, University of Zurich, Zurich, Switzerland
| | - Leonhard Held
- Center for Reproducible Science, Epidemiology, Biostatistics and Prevention Institute, University of Zurich, Zurich, Switzerland
| | - Sebastian Schneeweiss
- Division of Pharmacoepidemiology, Brigham and Womems Hospital Harvard Medical School, Boston, Massachusetts, USA
| | - Shirley V Wang
- Division of Pharmacoepidemiology, Brigham and Womems Hospital Harvard Medical School, Boston, Massachusetts, USA
| |
Collapse
|
4
|
Wang W, Liu M, He Q, Wang M, Xu J, Li L, Li G, He L, Zou K, Sun X. Validation and impact of algorithms for identifying variables in observational studies of routinely collected data. J Clin Epidemiol 2024; 166:111232. [PMID: 38043830 DOI: 10.1016/j.jclinepi.2023.111232] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2023] [Revised: 11/23/2023] [Accepted: 11/28/2023] [Indexed: 12/05/2023]
Abstract
BACKGROUND AND OBJECTIVES Among observational studies of routinely collected health data (RCD) for exploring treatment effects, algorithms are used to identify study variables. However, the extent to which algorithms are reliable and impact the credibility of effect estimates is far from clear. This study aimed to investigate the validation of algorithms for identifying study variables from RCD, and examine the impact of alternative algorithms on treatment effects. METHODS We searched PubMed for observational studies published in 2018 that used RCD to explore drug treatment effects. Information regarding the reporting, validation, and interpretation of algorithms was extracted. We summarized the reporting and methodological characteristics of algorithms and validation. We also assessed the divergence in effect estimates given alternative algorithms by calculating the ratio of estimates of the primary vs. alternative analyses. RESULTS A total of 222 studies were included, of which 93 (41.9%) provided a complete list of algorithms for identifying participants, 36 (16.2%) for exposure, and 132 (59.5%) for outcomes, and 15 (6.8%) for all study variables including population, exposure, and outcomes. Fifty-nine (26.6%) studies stated that the algorithms were validated, and 54 (24.3%) studies reported methodological characteristics of 66 validations, among which 61 validations in 49 studies were from the cross-referenced validation studies. Of those 66 validations, 22 (33.3%) reported sensitivity and 16 (24.2%) reported specificity. A total of 63.6% of studies reporting sensitivity and 56.3% reporting specificity used test-result-based sampling, an approach that potentially biases effect estimates. Twenty-eight (12.6%) studies used alternative algorithms to identify study variables, and 24 reported the effects estimated by primary analyses and sensitivity analyses. Of these, 20% had differential effect estimates when using alternative algorithms for identifying population, 18.2% for identifying exposure, and 45.5% for classifying outcomes. Only 32 (14.4%) studies discussed how the algorithms may affect treatment estimates. CONCLUSION In observational studies of RCD, the algorithms for variable identification were not regularly validated, and-even if validated-the methodological approach and performance of the validation were often poor. More seriously, different algorithms may yield differential treatment effects, but their impact is often ignored by researchers. Strong efforts, including recommendations, are warranted to improve good practice.
Collapse
Affiliation(s)
- Wen Wang
- Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-based Medicine and Cochrane China Center, West China Hospital, Sichuan University, Chengdu 610041, China; NMPA Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu 610041, China; Sichuan Center of Technology Innovation for Real World Data, Chengdu 610041, China.
| | - Mei Liu
- Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-based Medicine and Cochrane China Center, West China Hospital, Sichuan University, Chengdu 610041, China; NMPA Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu 610041, China; Sichuan Center of Technology Innovation for Real World Data, Chengdu 610041, China
| | - Qiao He
- Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-based Medicine and Cochrane China Center, West China Hospital, Sichuan University, Chengdu 610041, China; NMPA Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu 610041, China; Sichuan Center of Technology Innovation for Real World Data, Chengdu 610041, China
| | - Mingqi Wang
- Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-based Medicine and Cochrane China Center, West China Hospital, Sichuan University, Chengdu 610041, China; NMPA Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu 610041, China; Sichuan Center of Technology Innovation for Real World Data, Chengdu 610041, China
| | - Jiayue Xu
- Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-based Medicine and Cochrane China Center, West China Hospital, Sichuan University, Chengdu 610041, China; NMPA Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu 610041, China; Sichuan Center of Technology Innovation for Real World Data, Chengdu 610041, China
| | - Ling Li
- Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-based Medicine and Cochrane China Center, West China Hospital, Sichuan University, Chengdu 610041, China; NMPA Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu 610041, China; Sichuan Center of Technology Innovation for Real World Data, Chengdu 610041, China
| | - Guowei Li
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario L8S 4L8, Canada; Center for Clinical Epidemiology and Methodology, Guangdong Second Provincial General Hospital, Guangzhou, Guangdong 510317, China; Biostatistics Unit, Research Institute at St. Joseph's Healthcare Hamilton, Hamilton, Ontario L8N 4A6, Canada
| | - Lin He
- Intelligence Library Center, West China Hospital, Sichuan University, Chengdu, Sichuan 610041, China
| | - Kang Zou
- Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-based Medicine and Cochrane China Center, West China Hospital, Sichuan University, Chengdu 610041, China; NMPA Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu 610041, China; Sichuan Center of Technology Innovation for Real World Data, Chengdu 610041, China
| | - Xin Sun
- Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-based Medicine and Cochrane China Center, West China Hospital, Sichuan University, Chengdu 610041, China; NMPA Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu 610041, China; Sichuan Center of Technology Innovation for Real World Data, Chengdu 610041, China.
| |
Collapse
|
5
|
Toews I, Anglemyer A, Nyirenda JL, Alsaid D, Balduzzi S, Grummich K, Schwingshackl L, Bero L. Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials: a meta-epidemiological study. Cochrane Database Syst Rev 2024; 1:MR000034. [PMID: 38174786 PMCID: PMC10765475 DOI: 10.1002/14651858.mr000034.pub3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/05/2024]
Abstract
BACKGROUND Researchers and decision-makers often use evidence from randomised controlled trials (RCTs) to determine the efficacy or effectiveness of a treatment or intervention. Studies with observational designs are often used to measure the effectiveness of an intervention in 'real world' scenarios. Numerous study designs and their modifications (including both randomised and observational designs) are used for comparative effectiveness research in an attempt to give an unbiased estimate of whether one treatment is more effective or safer than another for a particular population. An up-to-date systematic analysis is needed to identify differences in effect estimates from RCTs and observational studies. This updated review summarises the results of methodological reviews that compared the effect estimates of observational studies with RCTs from evidence syntheses that addressed the same health research question. OBJECTIVES To assess and compare synthesised effect estimates by study type, contrasting RCTs with observational studies. To explore factors that might explain differences in synthesised effect estimates from RCTs versus observational studies (e.g. heterogeneity, type of observational study design, type of intervention, and use of propensity score adjustment). To identify gaps in the existing research comparing effect estimates across different study types. SEARCH METHODS We searched MEDLINE, the Cochrane Database of Systematic Reviews, Web of Science databases, and Epistemonikos to May 2022. We checked references, conducted citation searches, and contacted review authors to identify additional reviews. SELECTION CRITERIA We included systematic methodological reviews that compared quantitative effect estimates measuring the efficacy or effectiveness of interventions tested in RCTs versus in observational studies. The included reviews compared RCTs to observational studies (including retrospective and prospective cohort, case-control and cross-sectional designs). Reviews were not eligible if they compared RCTs with studies that had used some form of concurrent allocation. DATA COLLECTION AND ANALYSIS Using results from observational studies as the reference group, we examined the relative summary effect estimates (risk ratios (RRs), odds ratios (ORs), hazard ratios (HRs), mean differences (MDs), and standardised mean differences (SMDs)) to evaluate whether there was a relatively larger or smaller effect in the ratio of odds ratios (ROR) or ratio of risk ratios (RRR), ratio of hazard ratios (RHR), and difference in (standardised) mean differences (D(S)MD). If an included review did not provide an estimate comparing results from RCTs with observational studies, we generated one by pooling the estimates for observational studies and RCTs, respectively. Across all reviews, we synthesised these ratios to produce a pooled ratio of ratios comparing effect estimates from RCTs with those from observational studies. In overviews of reviews, we estimated the ROR or RRR for each overview using observational studies as the reference category. We appraised the risk of bias in the included reviews (using nine criteria in total). To receive an overall low risk of bias rating, an included review needed: explicit criteria for study selection, a complete sample of studies, and to have controlled for study methodological differences and study heterogeneity. We assessed reviews/overviews not meeting these four criteria as having an overall high risk of bias. We assessed the certainty of the evidence, consisting of multiple evidence syntheses, with the GRADE approach. MAIN RESULTS We included 39 systematic reviews and eight overviews of reviews, for a total of 47. Thirty-four of these contributed data to our primary analysis. Based on the available data, we found that the reviews/overviews included 2869 RCTs involving 3,882,115 participants, and 3924 observational studies with 19,499,970 participants. We rated 11 reviews/overviews as having an overall low risk of bias, and 36 as having an unclear or high risk of bias. Our main concerns with the included reviews/overviews were that some did not assess the quality of their included studies, and some failed to account appropriately for differences between study designs - for example, they conducted aggregate analyses of all observational studies rather than separate analyses of cohort and case-control studies. When pooling RORs and RRRs, the ratio of ratios indicated no difference or a very small difference between the effect estimates from RCTs versus from observational studies (ratio of ratios 1.08, 95% confidence interval (CI) 1.01 to 1.15). We rated the certainty of the evidence as low. Twenty-three of 34 reviews reported effect estimates of RCTs and observational studies that were on average in agreement. In a number of subgroup analyses, small differences in the effect estimates were detected: - pharmaceutical interventions only (ratio of ratios 1.12, 95% CI 1.04 to 1.21); - RCTs and observational studies with substantial or high heterogeneity; that is, I2 ≥ 50% (ratio of ratios 1.11, 95% CI 1.04 to 1.18); - no use (ratio of ratios 1.07, 95% CI 1.03 to 1.11) or unclear use (ratio of ratios 1.13, 95% CI 1.03 to 1.25) of propensity score adjustment in observational studies; and - observational studies without further specification of the study design (ratio of ratios 1.06, 95% CI 0.96 to 1.18). We detected no clear difference in other subgroup analyses. AUTHORS' CONCLUSIONS We found no difference or a very small difference between effect estimates from RCTs and observational studies. These findings are largely consistent with findings from recently published research. Factors other than study design need to be considered when exploring reasons for a lack of agreement between results of RCTs and observational studies, such as differences in the population, intervention, comparator, and outcomes investigated in the respective studies. Our results underscore that it is important for review authors to consider not only study design, but the level of heterogeneity in meta-analyses of RCTs or observational studies. A better understanding is needed of how these factors might yield estimates reflective of true effectiveness.
Collapse
Affiliation(s)
- Ingrid Toews
- Institute for Evidence in Medicine (for Cochrane Germany Foundation), Medical Center - University of Freiburg, Freiburg, Germany
| | - Andrew Anglemyer
- Department of Preventive and Social Medicine, Dunedin School of Medicine, University of Otago, Dunedin, New Zealand
| | - John Lz Nyirenda
- Institute for Evidence in Medicine (for Cochrane Germany Foundation), Medical Center - University of Freiburg, Freiburg, Germany
| | - Dima Alsaid
- Institute for Evidence in Medicine (for Cochrane Germany Foundation), Medical Center - University of Freiburg, Freiburg, Germany
| | - Sara Balduzzi
- Biometrics Department, Antoni van Leeuwenhoek - Netherlands Cancer Institute, Amsterdam, Netherlands
| | - Kathrin Grummich
- Institute for Evidence in Medicine (for Cochrane Germany Foundation), Medical Center - University of Freiburg, Freiburg, Germany
| | - Lukas Schwingshackl
- Institute for Evidence in Medicine (for Cochrane Germany Foundation), Medical Center - University of Freiburg, Freiburg, Germany
| | - Lisa Bero
- Charles Perkins Centre and School of Pharmacy, Faculty of Medicine and Health, The University of Sydney, Camperdown, Sydney, Australia
| |
Collapse
|
6
|
Lee YS, Lee YJ, Ha IH. Real-world data analysis on effectiveness of integrative therapies: A practical guide to study design and data analysis using healthcare databases. Integr Med Res 2023; 12:101000. [PMID: 37953753 PMCID: PMC10637915 DOI: 10.1016/j.imr.2023.101000] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2023] [Revised: 10/17/2023] [Accepted: 10/18/2023] [Indexed: 11/14/2023] Open
Abstract
Real world data (RWD) is increasingly used to investigate health outcomes and treatment efficacy in the field of integrative medicine. Due to the fact that the majority of RWDs are not intended for research, their secondary use in research necessitates complex study designs to account for bias and confounding. To conduct a robust analysis of RWD in integrative medicine, a comprehensive study design process that reflects the characteristics of integrative therapies is necessary. In this paper, we present a guide for designing comparative effectiveness RWE research in integrative medicine. We discuss key factors to consider when selecting RWDs for research on integrative medicine. We provide practical steps for developing a research question, formulating the PICOT objectives (population, intervention, comparator, outcome, and time horizon), and selecting and defining covariates with a summary table. Specific study designs are depicted with corresponding diagrams. Finally, data analysis procedures are introduced. We hope this article clarifies the importance of RWE research design and related processes in order to improve the rigor of RWD studies in the field of integrative medicine research.
Collapse
Affiliation(s)
- Ye-Seul Lee
- Jaseng Spine and Joint Research Institute, Jaseng Medical Foundation, Seoul, Korea
| | - Yoon Jae Lee
- Jaseng Spine and Joint Research Institute, Jaseng Medical Foundation, Seoul, Korea
| | - In-Hyuk Ha
- Jaseng Spine and Joint Research Institute, Jaseng Medical Foundation, Seoul, Korea
| |
Collapse
|
7
|
Wang W, He Q, Xu J, Liu M, Wang M, Li Q, Zhang X, Huang Y, Zhang Y, Li L, Zou K, Li G, Lu K, Gao P, Chen F, Guo JJ, Yang M, Sun X. Reporting, handling, and interpretation of time-varying drug treatments in observational studies using routinely collected healthcare data. J Evid Based Med 2023; 16:495-504. [PMID: 38108104 DOI: 10.1111/jebm.12577] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/21/2023] [Accepted: 12/07/2023] [Indexed: 12/19/2023]
Abstract
BACKGROUND Time-varying drug treatments are common in studies using routinely collected health data (RCD) for assessing treatment effects. This study aimed to examine how these studies reported, handled, and interpreted time-varying drug treatments. METHODS A systematic search was conducted on PubMed from 2018 to 2020. Eligible studies were those used RCD to explore drug treatment effects. We summarized the reporting characteristics and methods employed for handling time-varying treatments. Logistic regressions were performed to investigate the association between study characteristics and the reporting of time-varying treatments. RESULTS Two hundred and fifty-six studies were included, and 225 (87.9%) studies involved time-varying treatments. Of these, 24 (10.7%) reported the proportion of time-varying treatments and 105 (46.7%) reported methods used to handle time-varying treatments. Multivariable logistic regression showed that medical studies, prespecified protocol, and involvement of methodologists were associated with a higher likelihood of reporting the methods applied to handle time-varying treatments. Among the 105 studies that reported methods, as-treated analyses were the most commonly used analysis sets, which were employed in 73.9%, 75.3% and 88.2% of studies that reported approaches for treatment discontinuation, treatment switching and treatment add-on. Among the 225 studies involved time-varying treatments, 27 (12.0%) acknowledged the potential bias introduced by treatment change, of which 14 (51.9%) suggested that potential biases may impact acceptance or rejection of the null hypothesis. CONCLUSIONS Among observational studies using RCD, the underreporting about the presence and methods for handling time-varying treatments was largely common. The potential biases due to time-varying treatments have frequently been disregarded. Collaborative endeavors are strongly needed to enhance the prevailing practices.
Collapse
Affiliation(s)
- Wen Wang
- Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-based Medicine Center and Cochrane China Center, West China Hospital, Sichuan University, Chengdu, Sichuan, China
- National Medical Products Administration Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu, Sichuan, China
- Sichuan Center of Technology Innovation for Real World Data, Chengdu, Sichuan, China
| | - Qiao He
- Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-based Medicine Center and Cochrane China Center, West China Hospital, Sichuan University, Chengdu, Sichuan, China
- National Medical Products Administration Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu, Sichuan, China
- Sichuan Center of Technology Innovation for Real World Data, Chengdu, Sichuan, China
| | - Jiayue Xu
- Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-based Medicine Center and Cochrane China Center, West China Hospital, Sichuan University, Chengdu, Sichuan, China
- National Medical Products Administration Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu, Sichuan, China
- Sichuan Center of Technology Innovation for Real World Data, Chengdu, Sichuan, China
| | - Mei Liu
- Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-based Medicine Center and Cochrane China Center, West China Hospital, Sichuan University, Chengdu, Sichuan, China
- National Medical Products Administration Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu, Sichuan, China
- Sichuan Center of Technology Innovation for Real World Data, Chengdu, Sichuan, China
| | - Mingqi Wang
- Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-based Medicine Center and Cochrane China Center, West China Hospital, Sichuan University, Chengdu, Sichuan, China
- National Medical Products Administration Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu, Sichuan, China
- Sichuan Center of Technology Innovation for Real World Data, Chengdu, Sichuan, China
| | - Qianrui Li
- Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-based Medicine Center and Cochrane China Center, West China Hospital, Sichuan University, Chengdu, Sichuan, China
- National Medical Products Administration Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu, Sichuan, China
- Sichuan Center of Technology Innovation for Real World Data, Chengdu, Sichuan, China
| | - Xia Zhang
- Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-based Medicine Center and Cochrane China Center, West China Hospital, Sichuan University, Chengdu, Sichuan, China
- National Medical Products Administration Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu, Sichuan, China
- Sichuan Center of Technology Innovation for Real World Data, Chengdu, Sichuan, China
| | - Yunxiang Huang
- Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-based Medicine Center and Cochrane China Center, West China Hospital, Sichuan University, Chengdu, Sichuan, China
- National Medical Products Administration Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu, Sichuan, China
- Sichuan Center of Technology Innovation for Real World Data, Chengdu, Sichuan, China
| | - Yuanjin Zhang
- Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-based Medicine Center and Cochrane China Center, West China Hospital, Sichuan University, Chengdu, Sichuan, China
- National Medical Products Administration Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu, Sichuan, China
- Sichuan Center of Technology Innovation for Real World Data, Chengdu, Sichuan, China
| | - Ling Li
- Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-based Medicine Center and Cochrane China Center, West China Hospital, Sichuan University, Chengdu, Sichuan, China
- National Medical Products Administration Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu, Sichuan, China
- Sichuan Center of Technology Innovation for Real World Data, Chengdu, Sichuan, China
| | - Kang Zou
- Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-based Medicine Center and Cochrane China Center, West China Hospital, Sichuan University, Chengdu, Sichuan, China
- National Medical Products Administration Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu, Sichuan, China
- Sichuan Center of Technology Innovation for Real World Data, Chengdu, Sichuan, China
| | - Guowei Li
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada
- Center for Clinical Epidemiology and Methodology, Guangdong Second Provincial General Hospital, Guangzhou, Guangdong, China
| | - Kevin Lu
- South Carolina College of Pharmacy, University of South Carolina Columbia, Columbia, South Carolina, USA
| | - Pei Gao
- Department of Epidemiology and Biostatistics, Peking University Health Science Center, Beijing, China
| | - Feng Chen
- Department of Biostatistics, Center for Global Health, School of Public Health, Nanjing Medical University, Nanjing, Jiangsu, China
| | - Jeff Jianfei Guo
- College of Pharmacy, University of Cincinnati, Cincinnati, Ohio, USA
| | - Min Yang
- Department of Epidemiology and Biostatistics, West China School of Public Health, Sichuan University, Chengdu, Sichuan, China
- Faculty of Health, Design and Art, Swinburne Technology University, Victory, Australia
| | - Xin Sun
- Institute of Integrated Traditional Chinese and Western Medicine, Chinese Evidence-based Medicine Center and Cochrane China Center, West China Hospital, Sichuan University, Chengdu, Sichuan, China
- National Medical Products Administration Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu, Sichuan, China
- Sichuan Center of Technology Innovation for Real World Data, Chengdu, Sichuan, China
| |
Collapse
|
8
|
Plöderl M, Amendola S, Hengartner MP. Observational studies of antidepressant use and suicide risk are selectively published in psychiatric journals. J Clin Epidemiol 2023; 162:10-18. [PMID: 37544615 DOI: 10.1016/j.jclinepi.2023.07.015] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Revised: 07/15/2023] [Accepted: 07/24/2023] [Indexed: 08/08/2023]
Abstract
OBJECTIVES To investigate if observational studies showing favorable results for antidepressants on suicidal behavior (reduced risk) are preferably and more easily published in psychiatric journals and cited more often compared to studies with unfavorable results (increased risk). STUDY DESIGN AND SETTING Prespecified secondary analysis, including 27 original studies selected through a systematic review of observational studies reporting associations between the use of newer antidepressant drugs and suicide risk. RESULTS Independent of study quality, studies reporting favorable results were more frequently published in psychiatric than nonpsychiatric journals and were more often conducted by lead authors with financial conflicts of interest (fCOI). Within psychiatric journals, lead authors with fCOI published in journals with a higher impact factor (IF) and ranking. Within psychiatric journals, favorability of results also correlated with citation frequency, IF, and journal ranking, but these associations became weaker and inconclusive after adjusting for study quality. Results for ease of publishing were inconclusive. CONCLUSION Studies reporting unfavorable results (increased suicide risk with antidepressant exposure) are less likely to be published in psychiatric journals. Lead authors with fCOI report more favorable results, and their studies are published in the most prestigious psychiatric journals. This may create a biased evidence base and an unbalanced dissemination and appraisal of findings within psychiatry.
Collapse
Affiliation(s)
- Martin Plöderl
- Department of Inpatient Psychotherapy and Crisis Intervention, University Clinic for Psychiatry, Psychotherapy, and Psychosomatics, Christian Doppler Clinic, Paracelsus Medical University, Salzburg, Austria
| | - Simone Amendola
- Department of Applied Psychology, Zurich University of Applied Sciences, Zurich, Switzerland
| | | |
Collapse
|
9
|
Andreasen AS, Wetterslev M, Sigurdsson MI, Bove J, Kjaergaard J, Aslam TN, Järvelä K, Poulsen M, De Geer L, Agarwal A, Kjaer MBN, Møller MH. New-onset atrial fibrillation in critically ill adult patients-an SSAI clinical practice guideline. Acta Anaesthesiol Scand 2023; 67:1110-1117. [PMID: 37289426 DOI: 10.1111/aas.14262] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2023] [Accepted: 04/15/2023] [Indexed: 06/09/2023]
Abstract
BACKGROUND Acute or new-onset atrial fibrillation (NOAF) is the most common cardiac arrhythmia in critically ill adult patients, and observational data suggests that NOAF is associated to adverse outcomes. METHODS We prepared this guideline according to the Grading of Recommendations Assessment, Development and Evaluation methodology. We posed the following clinical questions: (1) what is the better first-line pharmacological agent for the treatment of NOAF in critically ill adult patients?, (2) should we use direct current (DC) cardioversion in critically ill adult patients with NOAF and hemodynamic instability caused by atrial fibrillation?, (3) should we use anticoagulant therapy in critically ill adult patients with NOAF?, and (4) should critically ill adult patients with NOAF receive follow-up after discharge from hospital? We assessed patient-important outcomes, including mortality, thromboembolic events, and adverse events. Patients and relatives were part of the guideline panel. RESULTS The quantity and quality of evidence on the management of NOAF in critically ill adults was very limited, and we did not identify any relevant direct or indirect evidence from randomized clinical trials for the prespecified PICO questions. We were able to propose one weak recommendation against routine use of therapeutic dose anticoagulant therapy, and one best practice statement for routine follow-up by a cardiologist after hospital discharge. We were not able to propose any recommendations on the better first-line pharmacological agent or whether to use DC cardioversion in critically ill patients with hemodynamic instability induced by NOAF. An electronic version of this guideline in layered and interactive format is available in MAGIC: https://app.magicapp.org/#/guideline/7197. CONCLUSIONS The body of evidence on the management of NOAF in critically ill adults is very limited and not informed by direct evidence from randomized clinical trials. Practice variation appears considerable.
Collapse
Affiliation(s)
- Anne Sofie Andreasen
- Department of Intensive Care, Copenhagen University Hospital - Herlev, Herlev, Denmark
| | - Mik Wetterslev
- Department of Intensive Care, Copenhagen University Hospital - Rigshospitalet, Copenhagen, Denmark
| | - Martin Ingi Sigurdsson
- Division of Anesthesia and Intensive Care Medicine, Landspitali - The National University Hospital of Iceland, Reykjavik, Iceland
- Faculty of Medicine, University of Iceland, Reykjavik, Iceland
| | - Jeppe Bove
- Department of Anaesthesia and Intensive Care, Odense University Hospital, Odense, Denmark
| | - Jesper Kjaergaard
- Department of Cardiology, Copenhagen University Hospital - Rigshospitalet, Copenhagen, Denmark
| | - Tayyba Naz Aslam
- Department of Research and Development, Division of Emergencies and Critical Care, Oslo University Hospital, Oslo, Norway
- Institute of Clinical Medicine, University of Oslo, Oslo, Norway
| | - Kati Järvelä
- Heart Hospital, Tampere University Hospital, Tampere, Finland
| | - Mette Poulsen
- Department of Intensive Care, Aarhus University Hospital, Aarhus, Denmark
| | - Lina De Geer
- Department of Anaesthesiology and Intensive Care, and Department of Biomedical and Clinical Sciences, Linköping University, Linköping, Sweden
| | - Arnav Agarwal
- Division of General Internal Medicine, Department of Medicine, McMaster University, Hamilton, Ontario, Canada
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada
- MAGIC Evidence Ecosystem Foundation, Norway
| | | | - Morten Hylander Møller
- Department of Intensive Care, Copenhagen University Hospital - Rigshospitalet, Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark
| |
Collapse
|
10
|
Heyard R, Held L, Schneeweiss S, Wang SV. DESIGN DIFFERENCES EXPLAIN VARIATION IN RESULTS BETWEEN RANDOMIZED TRIALS AND THEIR NON-RANDOMIZED EMULATIONS. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.07.13.23292601. [PMID: 37502999 PMCID: PMC10370236 DOI: 10.1101/2023.07.13.23292601] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/29/2023]
Abstract
Objectives While randomized controlled trials (RCTs) are considered a standard for evidence on the efficacy of medical treatments, non-randomized real-world evidence (RWE) studies using data from health insurance claims or electronic health records can provide important complementary evidence. The use of RWE to inform decision-making has been questioned because of concerns regarding confounding in non-randomized studies and the use of secondary data. RCT-DUPLICATE was a demonstration project that emulated the design of 32 RCTs with non-randomized RWE studies. We sought to explore how emulation differences relate to variation in results between the RCT-RWE study pairs. Methods We include all RCT-RWE study pairs from RCT-DUPLICATE where the measure of effect was a hazard ratio and use exploratory meta-regression methods to explain differences and variation in the effect sizes between the results from the RCT and the RWE study. The considered explanatory variables are related to design and population differences. Results Most of the observed variation in effect estimates between RCT-RWE study pairs in this sample could be explained by three emulation differences in the meta-regression model: (i) in-hospital start of treatment (not observed in claims data), (ii) discontinuation of certain baseline therapies at randomization (not part of clinical practice), (iii) delayed onset of drug effects (missed by short medication persistence in clinical practice). Conclusions This analysis suggests that a substantial proportion of the observed variation between results from RCTs and RWE studies can be attributed to design emulation differences. (238 words).
Collapse
Affiliation(s)
- Rachel Heyard
- Center for Reproducible Science, Epidemiology, Biostatistics and Prevention Institute, University of Zurich, Hirschengraben 84, 8001 Zurich, Switzerland
| | - Leonhard Held
- Center for Reproducible Science, Epidemiology, Biostatistics and Prevention Institute, University of Zurich, Hirschengraben 84, 8001 Zurich, Switzerland
| | - Sebastian Schneeweiss
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School, 1620 Tremon St, Boston MA 02120
| | - Shirley V Wang
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School, 1620 Tremon St, Boston MA 02120
| |
Collapse
|
11
|
Wang SV, Schneeweiss S, Franklin JM, Desai RJ, Feldman W, Garry EM, Glynn RJ, Lin KJ, Paik J, Patorno E, Suissa S, D'Andrea E, Jawaid D, Lee H, Pawar A, Sreedhara SK, Tesfaye H, Bessette LG, Zabotka L, Lee SB, Gautam N, York C, Zakoul H, Concato J, Martin D, Paraoan D, Quinto K. Emulation of Randomized Clinical Trials With Nonrandomized Database Analyses: Results of 32 Clinical Trials. JAMA 2023; 329:1376-1385. [PMID: 37097356 PMCID: PMC10130954 DOI: 10.1001/jama.2023.4221] [Citation(s) in RCA: 97] [Impact Index Per Article: 97.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/06/2022] [Accepted: 03/04/2023] [Indexed: 04/26/2023]
Abstract
Importance Nonrandomized studies using insurance claims databases can be analyzed to produce real-world evidence on the effectiveness of medical products. Given the lack of baseline randomization and measurement issues, concerns exist about whether such studies produce unbiased treatment effect estimates. Objective To emulate the design of 30 completed and 2 ongoing randomized clinical trials (RCTs) of medications with database studies using observational analogues of the RCT design parameters (population, intervention, comparator, outcome, time [PICOT]) and to quantify agreement in RCT-database study pairs. Design, Setting, and Participants New-user cohort studies with propensity score matching using 3 US claims databases (Optum Clinformatics, MarketScan, and Medicare). Inclusion-exclusion criteria for each database study were prespecified to emulate the corresponding RCT. RCTs were explicitly selected based on feasibility, including power, key confounders, and end points more likely to be emulated with real-world data. All 32 protocols were registered on ClinicalTrials.gov before conducting analyses. Emulations were conducted from 2017 through 2022. Exposures Therapies for multiple clinical conditions were included. Main Outcomes and Measures Database study emulations focused on the primary outcome of the corresponding RCT. Findings of database studies were compared with RCTs using predefined metrics, including Pearson correlation coefficients and binary metrics based on statistical significance agreement, estimate agreement, and standardized difference. Results In these highly selected RCTs, the overall observed agreement between the RCT and the database emulation results was a Pearson correlation of 0.82 (95% CI, 0.64-0.91), with 75% meeting statistical significance, 66% estimate agreement, and 75% standardized difference agreement. In a post hoc analysis limited to 16 RCTs with closer emulation of trial design and measurements, concordance was higher (Pearson r, 0.93; 95% CI, 0.79-0.97; 94% meeting statistical significance, 88% estimate agreement, 88% standardized difference agreement). Weaker concordance occurred among 16 RCTs for which close emulation of certain design elements that define the research question (PICOT) with data from insurance claims was not possible (Pearson r, 0.53; 95% CI, 0.00-0.83; 56% meeting statistical significance, 50% estimate agreement, 69% standardized difference agreement). Conclusions and Relevance Real-world evidence studies can reach similar conclusions as RCTs when design and measurements can be closely emulated, but this may be difficult to achieve. Concordance in results varied depending on the agreement metric. Emulation differences, chance, and residual confounding can contribute to divergence in results and are difficult to disentangle.
Collapse
Affiliation(s)
- Shirley V Wang
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| | - Sebastian Schneeweiss
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| | - Jessica M Franklin
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
- Now with Optum, Boston, Massachusetts
| | - Rishi J Desai
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| | - William Feldman
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| | | | - Robert J Glynn
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| | - Kueiyu Joshua Lin
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| | - Julie Paik
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| | - Elisabetta Patorno
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| | | | - Elvira D'Andrea
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
- Now with AbbVie Inc, Washington, DC
| | - Dureshahwar Jawaid
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| | - Hemin Lee
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| | - Ajinkya Pawar
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| | - Sushama Kattinakere Sreedhara
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| | - Helen Tesfaye
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| | - Lily G Bessette
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| | - Luke Zabotka
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| | - Su Been Lee
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| | - Nileesa Gautam
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| | - Cassie York
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| | - Heidi Zakoul
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| | - John Concato
- Office of Medical Policy, US Food and Drug Administration, Silver Springs, Maryland
| | - David Martin
- Office of Medical Policy, US Food and Drug Administration, Silver Springs, Maryland
- Now with Moderna, Cambridge, Massachusetts
| | - Dianne Paraoan
- Office of Medical Policy, US Food and Drug Administration, Silver Springs, Maryland
| | - Kenneth Quinto
- Office of Medical Policy, US Food and Drug Administration, Silver Springs, Maryland
| |
Collapse
|
12
|
Johannesen K, Siverskog J, Henriksson M, Janzon M, Lindahl B, Grönqvist E. Implementation of Ticagrelor Reduced Mortality in Routine Clinical Care: Evidence From a Natural Experiment Including 109 995 Patients With Myocardial Infarction in Sweden. J Am Heart Assoc 2023; 12:e027384. [PMID: 36870964 PMCID: PMC10111469 DOI: 10.1161/jaha.122.027384] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/19/2022] [Accepted: 12/06/2022] [Indexed: 03/06/2023]
Abstract
Background Effectiveness estimates from observational studies on ticagrelor use in routine clinical care are conflicting, with some contrary to the results of the pivotal randomized controlled trial of ticagrelor in acute coronary syndrome. The aim of this study was to estimate the effect of implementing and using ticagrelor in routine clinical care in patients with myocardial infarction using a natural experimental approach. Methods and Results This is a retrospective cohort study including patients hospitalized for myocardial infarction in Sweden between 2009 and 2015. The study exploited differences in the timing and speed of ticagrelor implementation between treatment centers as a source of random treatment assignment. The effect of implementing and using ticagrelor was estimated based on the admitting center's likelihood of treating patients with ticagrelor, measured as the proportion of patients treated in the 90 days before patient admission. The main outcome was 12-month mortality. The study included 109 955 patients, of whom 30 773 were treated with ticagrelor. Being admitted to a treatment center with higher past ticagrelor use was associated with a reduction in 12-month mortality (2.5 percentage points for 100% versus 0% past use [95% CI, 0.2-4.8]). The results are in line with the findings from the ticagrelor pivotal trial. Conclusions Using a natural experiment, this study finds that the implementation and use of ticagrelor in routine clinical care has reduced 12-month mortality in patients admitted to the hospital with myocardial infarction in Sweden and supports the external validity of randomized evidence on ticagrelor effectiveness.
Collapse
Affiliation(s)
- Kasper Johannesen
- Centre for Medical Technology Assessment, Department of Health, Medicine and Caring SciencesLinköping UniversityLinköpingSweden
| | - Jonathan Siverskog
- Centre for Medical Technology Assessment, Department of Health, Medicine and Caring SciencesLinköping UniversityLinköpingSweden
- Centre for Health Economic ResearchUppsala UniversityUppsalaSweden
- Department of Medical SciencesUppsala UniversityUppsalaSweden
| | - Martin Henriksson
- Centre for Medical Technology Assessment, Department of Health, Medicine and Caring SciencesLinköping UniversityLinköpingSweden
| | - Magnus Janzon
- Department of CardiologyUniversity HospitalLinköpingSweden
- Division of Diagnostics and Specialist Medicine, Department of Health, Medicine and Caring SciencesLinköping UniversityLinköpingSweden
| | - Bertil Lindahl
- Centre for Health Economic ResearchUppsala UniversityUppsalaSweden
- Department of Medical SciencesUppsala UniversityUppsalaSweden
| | - Erik Grönqvist
- Centre for Health Economic ResearchUppsala UniversityUppsalaSweden
- Department of EconomicsUppsala UniversityUppsalaSweden
| |
Collapse
|
13
|
Evans SR, Hamasaki T. Weighing evidence: robustness vs quantity. J Natl Cancer Inst 2023; 115:1-3. [PMID: 36156151 PMCID: PMC9830481 DOI: 10.1093/jnci/djac186] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2022] [Accepted: 09/20/2022] [Indexed: 01/12/2023] Open
Affiliation(s)
- Scott R Evans
- The Biostatistics Center, Milken Institute School of Public Health, George Washington University, Rockville, MD, USA
- Department of Biostatistics and Bioinformatics, Milken Institute School of Public Health, George Washington University, Washington, DC, USA
| | - Toshimitsu Hamasaki
- The Biostatistics Center, Milken Institute School of Public Health, George Washington University, Rockville, MD, USA
- Department of Biostatistics and Bioinformatics, Milken Institute School of Public Health, George Washington University, Washington, DC, USA
| |
Collapse
|
14
|
Dai M, Furuya-Kanamori L, Syed A, Lin L, Wang Q. An empirical comparison of the harmful effects for randomized controlled trials and non-randomized studies of interventions. Front Pharmacol 2023; 14:1064567. [PMID: 37025494 PMCID: PMC10070801 DOI: 10.3389/fphar.2023.1064567] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2022] [Accepted: 02/28/2023] [Indexed: 04/08/2023] Open
Abstract
Introduction: Randomized controlled trials (RCTs) are the gold standard to evaluate the efficacy of interventions (e.g., drugs and vaccines), yet the sample size of RCTs is often limited for safety assessment. Non-randomized studies of interventions (NRSIs) had been proposed as an important alternative source for safety assessment. In this study, we aimed to investigate whether there is any difference between RCTs and NRSIs in the evaluation of adverse events. Methods: We used the dataset of systematic reviews with at least one meta-analysis including both RCTs and NRSIs and collected the 2 × 2 table information (i.e., numbers of cases and sample sizes in intervention and control groups) of each study in the meta-analysis. We matched RCTs and NRSIs by their sample sizes (ratio: 0.85/1 to 1/0.85) within a meta-analysis. We estimated the ratio of the odds ratios (RORs) of an NRSI against an RCT in each pair and used the inverse variance as the weight to combine the natural logarithm of ROR (lnROR). Results: We included systematic reviews with 178 meta analyses, from which we confirmed 119 pairs of RCTs and NRSIs. The pooled ROR of NRSIs compared to that of RCTs was estimated to be 0.96 (95% confidence interval: 0.87 and 1.07). Similar results were obtained with different sample size subgroups and treatment subgroups. With the increase in sample size, the difference in ROR between RCTs and NRSIs decreased, although not significantly. Discussion: There was no substantial difference in the effects between RCTs and NRSIs in safety assessment when they have similar sample sizes. Evidence from NRSIs might be considered a supplement to RCTs for safety assessment.
Collapse
Affiliation(s)
- Minhan Dai
- Mental Health Center, West China Hospital of Sichuan University, Chengdu, China
| | - Luis Furuya-Kanamori
- School of Public Health, Faculty of Medicine, The University of Queensland, Herston, QL, Australia
| | - Asma Syed
- Department of Population Medicine, College of Medicine, Qatar University, Doha, Qatar
| | - Lifeng Lin
- Department of Epidemiology and Biostatistics, University of Arizona, Tucson, AZ, United States
| | - Qiang Wang
- Mental Health Center, West China Hospital of Sichuan University, Chengdu, China
- *Correspondence: Qiang Wang,
| |
Collapse
|
15
|
Meyhoff TS, Wichmann S, Messmer AS. Editorial: Fluid overload in the critically ill. Front Med (Lausanne) 2023; 10:1166202. [PMID: 36936240 PMCID: PMC10020629 DOI: 10.3389/fmed.2023.1166202] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2023] [Accepted: 02/15/2023] [Indexed: 03/06/2023] Open
Affiliation(s)
- Tine S. Meyhoff
- Department of Intensive Care, Rigshospitalet, University of Copenhagen, Copenhagen, Denmark
- Collaboration for Research in Intensive Care (CRIC), Copenhagen, Denmark
| | - Sine Wichmann
- Collaboration for Research in Intensive Care (CRIC), Copenhagen, Denmark
- Department of Anaesthesia and Intensive Care, Nordsjællands Hospital, University of Copenhagen, Copenhagen, Denmark
| | - Anna S. Messmer
- Department of Intensive Care Medicine, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland
- *Correspondence: Anna S. Messmer
| |
Collapse
|
16
|
Grosman S, Scott IA. Quality of observational studies of clinical interventions: a meta-epidemiological review. BMC Med Res Methodol 2022; 22:313. [PMID: 36476329 PMCID: PMC9727931 DOI: 10.1186/s12874-022-01797-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2021] [Accepted: 10/06/2022] [Indexed: 12/12/2022] Open
Abstract
BACKGROUND This meta-epidemiological study aimed to assess methodological quality of a sample of contemporary non-randomised clinical studies of clinical interventions. METHODS This was a cross-sectional study of observational studies published between January 1, 2012 and December 31, 2018. Studies were identified in PubMed using search terms 'association', 'observational,' 'non-randomised' 'comparative effectiveness' within titles or abstracts. Each study was appraised against 35 quality criteria by two authors independently, with each criterion rated fully, partially or not satisfied. These quality criteria were grouped into 6 categories: justification for observational design (n = 2); minimisation of bias in study design and data collection (n = 11); use of appropriate methods to create comparable groups (n = 6); appropriate adjustment of observed effects (n = 5); validation of observed effects (n = 9); and authors interpretations (n = 2). RESULTS Of 50 unique studies, 49 (98%) were published in two US general medical journals. No study fully satisfied all applicable criteria; the mean (+/-SD) proportion of applicable criteria fully satisfied across all studies was 72% (+/- 10%). The categories of quality criteria demonstrating the lowest proportions of fully satisfied criteria were measures used to adjust observed effects (criteria 20, 23, 24) and validate observed effects (criteria 25, 27, 33). Criteria associated with ≤50% of full satisfaction across studies, where applicable, comprised: imputation methods to account for missing data (50%); justification for not performing an RCT (42%); interaction analyses in identifying independent prognostic factors potentially influencing intervention effects (42%); use of statistical correction to minimise type 1 error in multiple outcome analyses (33%); clinically significant effect sizes (30%); residual bias analyses for unmeasured or unknown confounders (14%); and falsification tests for residual confounding (8%). The proportions of fully satisfied criteria did not change over time. CONCLUSIONS Recently published observational studies fail to fully satisfy more than one in four quality criteria. Criteria that were not or only partially satisfied were identified which serve as remediable targets for researchers and journal editors.
Collapse
Affiliation(s)
- Sergei Grosman
- grid.412744.00000 0004 0380 2017Department of Internal Medicine and Clinical Epidemiology, Princess Alexandra Hospital, 199 Ipswich Road, Brisbane, Queensland 4102 Australia ,grid.413210.50000 0004 4669 2727Department of Medicine, Cairns Hospital, 165 The Esplanade, Cairns, Queensland 4870 Australia
| | - Ian A. Scott
- grid.412744.00000 0004 0380 2017Department of Internal Medicine and Clinical Epidemiology, Princess Alexandra Hospital, 199 Ipswich Road, Brisbane, Queensland 4102 Australia
| |
Collapse
|
17
|
The potential value of observational studies of elective surgical interventions using routinely collected data. Ann Epidemiol 2022; 76:13-19. [DOI: 10.1016/j.annepidem.2022.10.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2022] [Revised: 10/03/2022] [Accepted: 10/06/2022] [Indexed: 11/23/2022]
|
18
|
Dang Q, Sun Z, Wang Y, Wang L, Liu Z, Han X. Ferroptosis: a double-edged sword mediating immune tolerance of cancer. Cell Death Dis 2022; 13:925. [PMID: 36335094 PMCID: PMC9637147 DOI: 10.1038/s41419-022-05384-6] [Citation(s) in RCA: 27] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2022] [Revised: 10/24/2022] [Accepted: 10/26/2022] [Indexed: 11/08/2022]
Abstract
The term ferroptosis was put forward in 2012 and has been researched exponentially over the past few years. Ferroptosis is an unconventional pattern of iron-dependent programmed cell death, which belongs to a type of necrosis and is distinguished from apoptosis and autophagy. Actuated by iron-dependent phospholipid peroxidation, ferroptosis is modulated by various cellular metabolic and signaling pathways, including amino acid, lipid, iron, and mitochondrial metabolism. Notably, ferroptosis is associated with numerous diseases and plays a double-edged sword role. Particularly, metastasis-prone or highly-mutated tumor cells are sensitive to ferroptosis. Hence, inducing or prohibiting ferroptosis in tumor cells has vastly promising potential in treating drug-resistant cancers. Immunotolerant cancer cells are not sensitive to the traditional cell death pathway such as apoptosis and necroptosis, while ferroptosis plays a crucial role in mediating tumor and immune cells to antagonize immune tolerance, which has broad prospects in the clinical setting. Herein, we summarized the mechanisms and delineated the regulatory network of ferroptosis, emphasized its dual role in mediating immune tolerance, proposed its significant clinical benefits in the tumor immune microenvironment, and ultimately presented some provocative doubts. This review aims to provide practical guidelines and research directions for the clinical practice of ferroptosis in treating immune-resistant tumors.
Collapse
Affiliation(s)
- Qin Dang
- grid.412633.10000 0004 1799 0733Department of Interventional Radiology, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, Henan 450052 China ,grid.412633.10000 0004 1799 0733Department of Colorectal Surgery, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, Henan 450052 China
| | - Ziqi Sun
- grid.412633.10000 0004 1799 0733Department of Interventional Radiology, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, Henan 450052 China
| | - Yang Wang
- grid.412633.10000 0004 1799 0733Department of Interventional Radiology, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, Henan 450052 China
| | - Libo Wang
- grid.412633.10000 0004 1799 0733Department of Hepatobiliary and Pancreatic Surgery, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, Henan 450052 China
| | - Zaoqu Liu
- grid.412633.10000 0004 1799 0733Department of Interventional Radiology, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, Henan 450052 China
| | - Xinwei Han
- grid.412633.10000 0004 1799 0733Department of Interventional Radiology, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, Henan 450052 China
| |
Collapse
|
19
|
Zucatti KP, Teixeira PP, Wayerbacher LF, Piccoli GF, Correia PE, Fonseca NKO, Moresco KS, Guerra BA, Maduré MG, Farenzena LP, Frankenberg AD, Brietzke E, Halpern B, Franco O, Colpani V, Gerchman F. Long-term Effect of Lifestyle Interventions on the Cardiovascular and All-Cause Mortality of Subjects With Prediabetes and Type 2 Diabetes: A Systematic Review and Meta-analysis. Diabetes Care 2022; 45:2787-2795. [PMID: 36318674 DOI: 10.2337/dc22-0642] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/31/2022] [Accepted: 08/16/2022] [Indexed: 11/06/2022]
Abstract
BACKGROUND Lifestyle interventions improve the metabolic control of individuals with hyperglycemia. PURPOSE We aimed to determine the effect of lifestyle interventions on cardiovascular and all-cause mortality in this population. DATA SOURCES Searches were made through MEDLINE, Cochrane CENTRAL, Embase, and Web of Science (no date/language restriction, until 15 May 2022). STUDY SELECTION We included randomized clinical trials (RCTs) of subjects with prediabetes and type 2 diabetes, comparing intensive lifestyle interventions with usual care, with a minimum of 2 years of active intervention. DATA EXTRACTION Data from the 11 RCTs selected were extracted in duplicate. A frequentist and arm-based meta-analysis was performed with random-effects models to estimate relative risk (RR) for mortality, and heterogeneity was assessed through I2 metrics. A generalized linear mixed model (GLMM) was used to confirm the findings. DATA SYNTHESIS Lifestyle interventions were not superior to usual care in reducing cardiovascular (RR 0.99; 95% CI 0.79-1.23) or all-cause (RR 0.93; 95% CI 0.85-1.03) mortality. Subgroup, sensitivity, and meta-regression analyses showed no influence of type of intervention, mean follow-up, age, glycemic status, geographical location, risk of bias, or weight change. All of these results were confirmed with the GLMM. Most studies had a low risk of bias according to the RoB 2.0 tool and the certainty of evidence was moderate for both outcomes. LIMITATIONS Most studies had a low risk of bias according to the RoB 2.0 tool, and the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach resulted in moderate certainty of evidence for both outcomes. Differences in lifestyle programs and in usual care between the studies should be considered in the interpretation of our results. CONCLUSIONS Intensive lifestyle interventions implemented so far did not show superiority to usual care in reducing cardiovascular or all-cause mortality for subjects with prediabetes and type 2 diabetes.
Collapse
Affiliation(s)
- Kelly P Zucatti
- Division of Endocrinology and Metabolism, Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, RS, Brazil
- Postgraduate Program in Medical Sciences: Endocrinology, Department of Internal Medicine, Faculty of Medicine, Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, RS, Brazil
| | - Paula P Teixeira
- Division of Endocrinology and Metabolism, Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, RS, Brazil
- Postgraduate Program in Medical Sciences: Endocrinology, Department of Internal Medicine, Faculty of Medicine, Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, RS, Brazil
| | - Laura F Wayerbacher
- Faculdade de Medicina, Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, RS, Brazil
| | - Giovana F Piccoli
- Division of Endocrinology and Metabolism, Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, RS, Brazil
- Postgraduate Program in Medical Sciences: Endocrinology, Department of Internal Medicine, Faculty of Medicine, Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, RS, Brazil
| | - Poliana E Correia
- Division of Endocrinology and Metabolism, Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, RS, Brazil
- Postgraduate Program in Medical Sciences: Endocrinology, Department of Internal Medicine, Faculty of Medicine, Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, RS, Brazil
| | - Natasha K O Fonseca
- Postgraduate Program in Psychiatry and Behavioral Sciences, Faculdade de Medicina, Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, RS, Brazil
| | - Karla S Moresco
- Postgraduate Program in Medical Sciences: Endocrinology, Department of Internal Medicine, Faculty of Medicine, Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, RS, Brazil
| | - Bruno A Guerra
- Faculdade de Nutrição, Universidade Federal de Ciências da Saúde de Porto Alegre (UFCSPA), Porto Alegre, RS, Brazil
| | - Michelle G Maduré
- Faculdade de Nutrição, Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, RS, Brazil
| | - Laura P Farenzena
- Faculdade de Medicina, Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, RS, Brazil
| | - Anize D Frankenberg
- Postgraduate Program in Medical Sciences: Endocrinology, Department of Internal Medicine, Faculty of Medicine, Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, RS, Brazil
| | - Elisa Brietzke
- Department of Psychiatry, Queen's University School of Medicine, Kingston, Ontario, Canada
- Center of Neurosciences Studies (CND), Kingston, Ontario, Canada
| | - Bruno Halpern
- Obesity Group, Department of Endocrinology, Universidade de São Paulo, São Paulo, Brazil
- Weight Control Center, Hospital 9 de Julho, São Paulo, Brazil
| | - Oscar Franco
- Institute of Social and Preventive Medicine (ISPM), University of Bern, Bern, Switzerland
- Harvard T. H. Chan School of Public Health, Harvard University, Cambridge, MA
| | - Verônica Colpani
- Postgraduate Program in Medical Sciences: Endocrinology, Department of Internal Medicine, Faculty of Medicine, Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, RS, Brazil
| | - Fernando Gerchman
- Division of Endocrinology and Metabolism, Hospital de Clínicas de Porto Alegre (HCPA), Porto Alegre, RS, Brazil
- Postgraduate Program in Medical Sciences: Endocrinology, Department of Internal Medicine, Faculty of Medicine, Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, RS, Brazil
| |
Collapse
|
20
|
Dai W, Leng X, Wang J, Cheng J, Hu X, Ao Y. Quadriceps Tendon Autograft Versus Bone-Patellar Tendon-Bone and Hamstring Tendon Autografts for Anterior Cruciate Ligament Reconstruction: A Systematic Review and Meta-analysis. Am J Sports Med 2022; 50:3425-3439. [PMID: 34494906 DOI: 10.1177/03635465211030259] [Citation(s) in RCA: 45] [Impact Index Per Article: 22.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/31/2023]
Abstract
BACKGROUND The best type of autograft for anterior cruciate ligament (ACL) reconstruction remains debatable. HYPOTHESIS Compared with bone-patellar tendon-bone (BPTB) and hamstring tendon (HT) autografts, the quadriceps tendon (QT) autograft has comparable graft survival as well as clinical function and pain outcomes. STUDY DESIGN Meta-analysis; Level of evidence, 4. METHODS A systematic literature search was conducted in PubMed, Embase, Scopus, and the Cochrane Library to July 2020. Randomized controlled trials (RCTs) and observational studies reporting comparisons of QT versus BPTB or HT autografts for ACL reconstruction were included. All analyses were stratified according to study design: RCTs or observational studies. RESULTS A total of 24 studies were included: 7 RCTs and 17 observational studies. The 7 RCTs included 388 patients, and the 17 observational studies included 19,196 patients. No significant differences in graft failure (P = .36), the International Knee Documentation Committee (IKDC) subjective score (P = .39), or the side-to-side difference in stability (P = .60) were noted between QT and BPTB autografts. However, a significant reduction in donor site morbidity was noted in the QT group compared with the BPTB group (risk ratio [RR], 0.17 [95% CI, 0.09-0.33]; P < .001). No significant differences in graft failure (P = .57), the IKDC subjective score (P = .25), or the side-to-side stability difference (P = .98) were noted between QT and HT autografts. However, the QT autograft was associated with a significantly lower rate of donor site morbidity than the HT autograft (RR, 0.60 [95% CI, 0.39-0.93]; P = .02). A similar graft failure rate between the QT and control groups was observed after both early and late full weightbearing, after early and late full range of motion, and after using the QT autograft with a bone plug and all soft tissue QT grafts. However, a significantly lower rate of donor site morbidity was observed in the QT group compared with the control group after both early and late full weightbearing, after early and late full range of motion, and after using the QT autograft with a bone plug and all soft tissue QT grafts. No difference in effect estimates was seen between RCTs and observational studies. CONCLUSION The QT autograft had comparable graft survival, functional outcomes, and stability outcomes compared with BPTB and HT autografts. However, donor site morbidity was significantly lower with the QT autograft than with BPTB and HT autografts.
Collapse
Affiliation(s)
- Wenli Dai
- Institute of Sports Medicine, Beijing Key Laboratory of Sports Injuries, Peking University Third Hospital, Beijing, China
| | - Xi Leng
- Medical Imaging Center, The First Affiliated Hospital of Guangzhou University of Chinese Medicine, Guangzhou, China
| | - Jian Wang
- Department of Orthopedic Surgery, Nanfang Hospital, Southern Medical University, Guangzhou, China
| | - Jin Cheng
- Institute of Sports Medicine, Beijing Key Laboratory of Sports Injuries, Peking University Third Hospital, Beijing, China
| | - Xiaoqing Hu
- Institute of Sports Medicine, Beijing Key Laboratory of Sports Injuries, Peking University Third Hospital, Beijing, China
| | - Yingfang Ao
- Institute of Sports Medicine, Beijing Key Laboratory of Sports Injuries, Peking University Third Hospital, Beijing, China
| |
Collapse
|
21
|
Wang SV, Sreedhara SK, Schneeweiss S. Reproducibility of real-world evidence studies using clinical practice data to inform regulatory and coverage decisions. Nat Commun 2022; 13:5126. [PMID: 36045130 PMCID: PMC9430007 DOI: 10.1038/s41467-022-32310-3] [Citation(s) in RCA: 24] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2021] [Accepted: 07/26/2022] [Indexed: 11/26/2022] Open
Abstract
Studies that generate real-world evidence on the effects of medical products through analysis of digital data collected in clinical practice provide key insights for regulators, payers, and other healthcare decision-makers. Ensuring reproducibility of such findings is fundamental to effective evidence-based decision-making. We reproduce results for 150 studies published in peer-reviewed journals using the same healthcare databases as original investigators and evaluate the completeness of reporting for 250. Original and reproduction effect sizes were positively correlated (Pearson’s correlation = 0.85), a strong relationship with some room for improvement. The median and interquartile range for the relative magnitude of effect (e.g., hazard ratiooriginal/hazard ratioreproduction) is 1.0 [0.9, 1.1], range [0.3, 2.1]. While the majority of results are closely reproduced, a subset are not. The latter can be explained by incomplete reporting and updated data. Greater methodological transparency aligned with new guidance may further improve reproducibility and validity assessment, thus facilitating evidence-based decision-making. Study registration number: EUPAS19636. Analyses of real-world evidence from digital clinical practice data provide important insights for healthcare decision makers. Here, authors test reproducibility of 150 peer-reviewed studies, reporting strong reproducibility, which could be further improved through more complete reporting in future original studies
Collapse
Affiliation(s)
- Shirley V Wang
- Division of Pharmacoepidemiology and Pharmacoeconomics, Brigham and Women's Hospital, Boston, MA, USA. .,Department of Medicine, Harvard Medical School, Boston, MA, USA.
| | | | - Sebastian Schneeweiss
- Division of Pharmacoepidemiology and Pharmacoeconomics, Brigham and Women's Hospital, Boston, MA, USA.,Department of Medicine, Harvard Medical School, Boston, MA, USA
| | | |
Collapse
|
22
|
Saesen R, Kantidakis G, Marinus A, Lacombe D, Huys I. How do cancer clinicians perceive real-world data and the evidence derived therefrom? Findings from an international survey of the European Organisation for Research and Treatment of Cancer. Front Pharmacol 2022; 13:969778. [PMID: 36091761 PMCID: PMC9449152 DOI: 10.3389/fphar.2022.969778] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2022] [Accepted: 08/02/2022] [Indexed: 11/13/2022] Open
Abstract
Background: The role of real-world evidence (RWE) in the development of anticancer therapies has been gradually growing over time. Regulators, payers and health technology assessment agencies, spurred by the rise of the precision medicine model, are increasingly incorporating RWE into their decision-making regarding the authorization and reimbursement of novel antineoplastic treatments. However, it remains unclear how this trend is viewed by clinicians in the field. This study aimed to investigate the opinions of these stakeholders with respect to RWE and its suitability for informing regulatory, reimbursement-related and clinical decisions in oncology.Methods: An online survey was disseminated to clinicians belonging to the network of the European Organisation for Research and Treatment of Cancer between May and July 2021.Results: In total, 557 clinicians across 30 different countries participated in the survey, representing 13 distinct cancer domains. Despite seeing the methodological challenges associated with its interpretation as difficult to overcome, the respondents mostly (75.0%) perceived RWE positively, and believed such evidence could be relatively strong, depending on the designs and data sources of the studies from which it is produced. Few (4.6%) saw a future expansion of its influence on decision-makers as a negative evolution. Furthermore, nearly all (94.0%) participants were open to the idea of sharing anonymized or pseudonymized electronic health data of their patients with external parties for research purposes. Nevertheless, most clinicians (77.0%) still considered randomized controlled trials (RCTs) to be the gold standard for generating clinical evidence in oncology, and a plurality (49.2%) thought that RWE cannot fully address the knowledge gaps that remain after a new antitumor intervention has entered the market. Moreover, a majority of respondents (50.7%) expressed that they relied more heavily on RCT-derived evidence than on RWE for their own decision-making.Conclusion: While cancer clinicians have positive opinions about RWE and want to contribute to its generation, they also continue to hold RCTs in high regard as sources of actionable evidence.
Collapse
Affiliation(s)
- Robbe Saesen
- European Organisation for Research and Treatment of Cancer (EORTC), Brussels, Belgium
- Clinical Pharmacology and Pharmacotherapy Research Unit, Department of Pharmaceutical and Pharmacological Sciences, KU Leuven, Leuven, Belgium
- *Correspondence: Robbe Saesen,
| | - Georgios Kantidakis
- European Organisation for Research and Treatment of Cancer (EORTC), Brussels, Belgium
| | - Ann Marinus
- European Organisation for Research and Treatment of Cancer (EORTC), Brussels, Belgium
| | - Denis Lacombe
- European Organisation for Research and Treatment of Cancer (EORTC), Brussels, Belgium
| | - Isabelle Huys
- Clinical Pharmacology and Pharmacotherapy Research Unit, Department of Pharmaceutical and Pharmacological Sciences, KU Leuven, Leuven, Belgium
| |
Collapse
|
23
|
Pelzer F, Loef M, Martin DD, Baumgartner S. Cancer-related fatigue in patients treated with mistletoe extracts: a systematic review and meta-analysis. Support Care Cancer 2022; 30:6405-6418. [PMID: 35239008 PMCID: PMC9213316 DOI: 10.1007/s00520-022-06921-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2021] [Accepted: 02/15/2022] [Indexed: 11/07/2022]
Abstract
PURPOSE Cancer-related fatigue remains one of the most prevalent and distressing symptoms experienced by cancer patients. Effective treatments for cancer-related fatigue are needed. The objective of this meta-analysis is to determine the impact of mistletoe extracts as a pharmacological treatment for the management of cancer-related fatigue. METHODS We included randomized clinical trials (RCTs) and non-randomized studies of interventions (NRSIs) in cancer patients. Inclusion criteria were cancer-related fatigue severity or prevalence as an outcome and testing of mistletoe extracts compared to control groups. We searched Medline (EuropePMC), Embase, the Cochrane Central Register of Controlled Trials, Clinicaltrials.gov, and opengrey.org through October 2020. We assessed the risk of bias using the Cochrane risk of bias tools for RCTs and NRSIs and conducted a meta-analysis. RESULTS We performed one meta-analysis with 12 RCTs, including 1494 participants, and one meta-analysis with seven retrospective NRSIs, including 2668 participants. Heterogeneity between the studies was high in both meta-analyses. Most studies had a high risk of bias. A random-effects model showed for RCTs a standardized mean difference of -0.48 (95% confidence interval -0.82 to -0.14; p = 0.006) and for NRSIs an odds ratio of 0.36 (95% confidence interval 0.20 to 0.66; p = 0.0008). CONCLUSION Treatment with mistletoe extracts shows a moderate effect on cancer-related fatigue of similar size to physical activity. These results need to be confirmed by more placebo-controlled trials. Future trials should investigate different treatment durations and their effect on cancer-related fatigue in post-treatment cancer survivors. TRIAL REGISTRATION This meta-analysis has been registered under the PROSPERO registration number CRD42020191967 on October 7, 2020.
Collapse
Affiliation(s)
- Florian Pelzer
- Institute of Integrative Medicine, Witten/Herdecke University, Witten, Germany.
- Society for Cancer Research, Arlesheim, Switzerland.
| | | | - David D Martin
- Institute of Integrative Medicine, Witten/Herdecke University, Witten, Germany
| | - Stephan Baumgartner
- Institute of Integrative Medicine, Witten/Herdecke University, Witten, Germany
- Society for Cancer Research, Arlesheim, Switzerland
| |
Collapse
|
24
|
Wallach JD, Moneer O, Ross JS. Generating evidence during a pandemic: what's reliable? BMJ 2022; 377:o1229. [PMID: 35577370 DOI: 10.1136/bmj.o1229] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Affiliation(s)
- Joshua D Wallach
- Department of Environmental Health Sciences, Yale School of Public Health, New Haven, CT, USA
| | - Osman Moneer
- Yale University School of Medicine, New Haven, CT, USA
| | - Joseph S Ross
- Section of General Medicine and the National Clinician Scholars Program, Department of Internal Medicine, Yale School of Medicine; Center for Outcomes Research and Evaluation, Yale-New Haven Health System; Department of Health Policy and Management, Yale School of Public Health, New Haven, CT, USA
| |
Collapse
|
25
|
Moneer O, Daly G, Skydel JJ, Nyhan K, Lurie P, Ross JS, Wallach JD. Agreement of treatment effects from observational studies and randomized controlled trials evaluating hydroxychloroquine, lopinavir-ritonavir, or dexamethasone for covid-19: meta-epidemiological study. BMJ 2022; 377:e069400. [PMID: 35537738 PMCID: PMC9086409 DOI: 10.1136/bmj-2021-069400] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/29/2022]
Abstract
OBJECTIVE To systematically identify, match, and compare treatment effects and study demographics from individual or meta-analysed observational studies and randomized controlled trials (RCTs) evaluating the same covid-19 treatments, comparators, and outcomes. DESIGN Meta-epidemiological study. DATA SOURCES National Institutes of Health Covid-19 Treatment Guidelines, a living review and network meta-analysis published in The BMJ, a living systematic review with meta-analysis and trial sequential analysis in PLOS Medicine (The LIVING Project), and the Epistemonikos "Living OVerview of Evidence" (L·OVE) evidence database. ELIGIBILITY CRITERIA FOR SELECTION OF STUDIES RCTs in The BMJ's living review that directly compared any of the three most frequently studied therapeutic interventions for covid-19 across all data sources (that is, hydroxychloroquine, lopinavir-ritonavir, or dexamethasone) for any safety and efficacy outcomes. Observational studies that evaluated the same interventions, comparisons, and outcomes that were reported in The BMJ's living review. DATA EXTRACTION AND SYNTHESIS Safety and efficacy outcomes from observational studies were identified and treatment effects for dichotomous (odds ratios) or continuous (mean differences or ratios of means) outcomes were calculated and, when possible, meta-analyzed to match the treatment effects from individual RCTs or meta-analyses of RCTs reported in The BMJ's living review with the same interventions, comparisons, and outcomes (that is, matched pairs). The analysis compared the distribution of study demographics and the agreement between treatment effects from matched pairs. Matched pairs were in agreement if both observational and RCT treatment effects were significantly increasing or decreasing (P<0.05) or if both treatment effects were not significant (P≥0.05). RESULTS 17 new, independent meta-analyses of observational studies were conducted that compared hydroxychloroquine, lopinavir-ritonavir, or dexamethasone with an active or placebo comparator for any safety or efficacy outcomes in covid-19 treatment. These studies were matched and compared with 17 meta-analyses of RCTs reported in The BMJ's living review. 10 additional matched pairs with only one observational study and/or one RCT were identified. Across all 27 matched pairs, 22 had adequate reporting of demographical and clinical data for all individual studies. All 22 matched pairs had studies with overlapping distributions of sex, age, and disease severity. Overall, 21 (78%) of the 27 matched pairs had treatment effects that were in agreement. Among the 17 matched pairs consisting of meta-analyses of observational studies and meta-analyses of RCTs, 14 (82%) were in agreement; seven (70%) of the 10 matched pairs consisting of at least one observational study or one RCT were in agreement. The 18 matched pairs with treatment effects for dichotomous outcomes had a higher proportion of agreement (n=16, 89%) than did the nine matched pairs with treatment effects for continuous outcomes (n=5, 56%). CONCLUSIONS Meta-analyses of observational studies and RCTs evaluating treatments for covid-19 have summary treatment effects that are generally in agreement. Although our evaluation is limited to three covid-19 treatments, these findings suggest that meta-analyzed evidence from observational studies might complement, but should not replace, evidence collected from RCTs.
Collapse
Affiliation(s)
- Osman Moneer
- Yale School of Medicine, Yale University, New Haven, CT, USA
| | - Garrison Daly
- Center for Science in the Public Interest, Washington, DC, USA
| | | | - Kate Nyhan
- Harvey Cushing/John Hay Whitney Medical Library, Yale University, New Haven, CT, USA
- Department of Environmental Health Sciences, Yale School of Public Health, New Haven, CT, USA
| | - Peter Lurie
- Center for Science in the Public Interest, Washington, DC, USA
| | - Joseph S Ross
- Section of General Medicine and the National Clinician Scholars Program, Department of Internal Medicine, Yale School of Medicine, New Haven, CT, USA
- Center for Outcomes Research and Evaluation, Yale-New Haven Health System, New Haven, CT, USA
- Department of Health Policy and Management, Yale School of Public Health, New Haven, CT, USA
| | - Joshua D Wallach
- Department of Environmental Health Sciences, Yale School of Public Health, New Haven, CT, USA
| |
Collapse
|
26
|
Issoufaly I, Petit C, Guihard S, Eugène R, Jung L, Clavier JB, Servagi Vernat S, Bellefqih S, Verret B, Bonnet N, Deutsch É, Rivera S. Favorable safety profile of moderate hypofractionated over normofractionated radiotherapy in breast cancer patients: a multicentric prospective real-life data farming analysis. Radiat Oncol 2022; 17:80. [PMID: 35443729 PMCID: PMC9019802 DOI: 10.1186/s13014-022-02044-z] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2022] [Accepted: 03/29/2022] [Indexed: 11/10/2022] Open
Abstract
Background Moderately hypofractionated whole-breast radiotherapy (HFRT) has proven to be as safe and efficient as normofractionated radiotherapy (NFRT) in randomized trials resulting in major changes in clinical practice. Toxicity rates observed in selected clinical trial patients may differ from those observed in unselected patients with possible comorbidities and frailty in real-life. This study aimed to examine the influence of HFRT versus NFRT on acute toxicity and identify risks factors of dermatitis in real-life patients.
Materials and methods Prospective data from breast cancer patients, treated with locoregional radiotherapy were collected between November 2015 and February 2020 in 3 comprehensive cancer centers. Through a systematic data-farming strategy, acute toxicity evaluation forms (CTCAEv4.0) were prospectively completed and extracted electronically. The results from each center were then anonymously merged into a single database for analysis. A Chi-2 test was used to compare HFRT and NFRT. Furthermore, risk factors of dermatitis were identified in a sub-study (622 patients) by multivariate logistic regression analysis.
Results In total, 3518 T0-4 N0-3 mostly M0 (85.8%) breast cancer patients with a median age of 60.7 (24–96 years old) were analyzed. Acute grade 2–3 dermatitis, grade 1–3 breast oedema, and grade 1–2 hyperpigmentation were less frequent with HFRT versus NFRT: respectively 8.9% versus 35.1% (Chi-2 = 373.7; p < 0.001), 29.0% versus 37.0% (Chi-2 = 23.1; p < 0.001) and 27.0% versus 55.8% (Chi-2 = 279.2; p < 0.001). Fewer patients experienced pain with HFRT versus NFRT: 33.4% versus 53.7% respectively (Chi-2 = 137.1; p < 0.001). Factors such as high BMI (OR = 2.30 [95% CI, 1.28–4.26], p < 0.01), large breast size (OR = 1.88 [95% CI, 1.07–3.28], p < 0.01) and lumpectomy over mastectomy (OR = 0.52 [95% CI, 0.27–0.97], p < 0.05) were associated with greater risk factors of grade 2–3 dermatitis in multivariate analysis regardless of NFRT or HFRT. Conclusion The results of this study suggests that breast HFRT may be a better option even for patients with a high BMI or large breast size. Acute toxicity was low to mild, and lower with HFRT compared to NFRT. Results from real-life data were robust, and support the use of HFRT beyond randomized study populations. Long-term real-life data awaits further investigation.
Collapse
Affiliation(s)
- Irfane Issoufaly
- Department of Radiotherapy, Gustave Roussy, Université Paris-Saclay, 94805, Villejuif, France
| | - Claire Petit
- Department of Radiotherapy, Gustave Roussy, Université Paris-Saclay, 94805, Villejuif, France
| | | | | | - Loic Jung
- Radiotherapy, Paul Strauss, Strasbourg, France
| | | | | | | | - Benjamin Verret
- Department of Radiotherapy, Gustave Roussy, Université Paris-Saclay, 94805, Villejuif, France
| | - Naïma Bonnet
- Unicancer Radiation and Oncology Group, Paris, France
| | - Éric Deutsch
- Department of Radiotherapy, Gustave Roussy, Université Paris-Saclay, 94805, Villejuif, France.,UMR 1030, Molecular Radiotherapy and Therapeutic Innovation, INSERM, 94805, Villejuif, France
| | - Sofia Rivera
- Department of Radiotherapy, Gustave Roussy, Université Paris-Saclay, 94805, Villejuif, France. .,UMR 1030, Molecular Radiotherapy and Therapeutic Innovation, INSERM, 94805, Villejuif, France.
| |
Collapse
|
27
|
Affiliation(s)
- Scott R. Evans
- Biostatistics Center and the Department of Biostatistics and Bioinformatics, Milken Institute School of Public Health, George Washington University, Rockville, MD
| |
Collapse
|
28
|
Lee JJ, Price JC, Jackson WM, Whittington RA, Ioannidis JPA. COVID-19: A Catalyst for Transforming Randomized Trials. J Neurosurg Anesthesiol 2022; 34:107-112. [PMID: 34870631 DOI: 10.1097/ana.0000000000000804] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/27/2022]
Abstract
The coronavirus disease 2019 (COVID-19) pandemic incited a global clinical trial research agenda of unprecedented speed and high volume. This expedited research activity in a time of crisis produced both successes and failures that offer valuable learning opportunities for the scientific community to consider. Successes include the implementation of large adaptive and pragmatic trials as well as burgeoning efforts toward rapid data synthesis and open science principles. Conversely, notable failures include: (1) inadequate study design and execution; (2) data reversal, fraud, and retraction; and (3) research duplication and waste. Other challenges that became highlighted were the need to find unbiased designs for investigating complex, nonpharmaceutical interventions and the use of routinely collected data for outcomes assessment. This article discusses these issues juxtaposing the COVID-19 trials experience against trials in anesthesiology and other fields. These lessons may serve as a positive catalyst for transforming future clinical trial research.
Collapse
Affiliation(s)
- Jennifer J Lee
- Department of Anesthesiology, Columbia University Irving Medical Center, New York, NY
| | - Jerri C Price
- Department of Anesthesiology, Columbia University Irving Medical Center, New York, NY
| | - William M Jackson
- Department of Anesthesiology, Columbia University Irving Medical Center, New York, NY
| | - Robert A Whittington
- Department of Anesthesiology, Columbia University Irving Medical Center, New York, NY
| | - John P A Ioannidis
- Department of Medicine, Stanford Prevention Research Center
- Departments of Epidemiology and Population Health
- Biomedical Data Science
- Statistics, Stanford University, and Meta-Research Innovation Center at Stanford (METRICS), Stanford, CA
| |
Collapse
|
29
|
Katsanos K. The Gordian Knot of Paclitaxel Devices in the Lower Limbs. JACC Cardiovasc Interv 2021; 14:2610-2613. [PMID: 34887052 DOI: 10.1016/j.jcin.2021.09.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/30/2021] [Accepted: 09/06/2021] [Indexed: 11/17/2022]
|
30
|
Katsura M, Kuriyama A, Tada M, Tsujimoto Y, Luo Y, Yamamoto K, So R, Aga M, Matsushima K, Fukuma S, Furukawa TA. High variability in results and methodological quality among overlapping systematic reviews on the same topics in surgery: a meta-epidemiological study. Br J Surg 2021; 108:1521-1529. [PMID: 34791075 DOI: 10.1093/bjs/znab328] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2021] [Accepted: 08/27/2021] [Indexed: 01/04/2023]
Abstract
BACKGROUND Redundant publication of systematic reviews and meta-analyses (SRs/MAs) on the same topic presents an increasing burden for clinicians. The aim of this study was to describe variabilities in effect size and methodological quality of overlapping surgery-related SRs/MAs and to investigate factors associated with their postpublication citations. METHODS PubMed/MEDLINE was searched to identify SRs/MAs of RCTs on thoracoabdominal surgeries published in 2015. Previous SRs/MAs on the same topics published within the preceding 5 years (2011-2015) were identified and 5-year citation counts (through to 2020) were evaluated. Discrepancies in pooled effect sizes and their methodological quality using A Measurement Tool to Assess Systematic Reviews (AMSTAR) among overlapping SRs/MAs were assessed. The SR/MA-level factors associated with 5-year citation counts were explored, using a mixed-effects regression model with a random intercept for surgical topics. RESULTS A total of 57 surgery-related SRs/MAs (48 topics) published in 2015 were identified, and 146 SRs/MAs had overlapping publications on 29 topics (60.4 per cent of all topics) in the preceding 5 years. There was considerable variability in methodological quality of SRs/MAs and coverage probability for relevant RCTs, resulting in discrepant effect size estimates for the same topic. High quality (AMSTAR score 8-11) was independently associated with higher 5-year citation counts (coefficient = 32.82; 95 per cent c.i. 15.63 to 50.02; P < 0.001). CONCLUSION Overlapping SRs/MAs with high variability in results and methodological quality were common in surgery. A high-quality SR/MA score was an independent predictor of more frequent citations. Researchers and journal editors should concentrate their efforts on limiting publications to higher-quality reviews.
Collapse
Affiliation(s)
- Morihiro Katsura
- Human Health Sciences, Kyoto University Graduate School of Medicine, Kyoto, Japan.,Department of Surgery, Okinawa Chubu Hospital, Okinawa, Japan
| | - Akira Kuriyama
- Emergency and Critical Care Centre, Kurashiki Central Hospital, Okayama, Japan
| | - Masafumi Tada
- Department of Health Promotion and Human Behavior, Kyoto University Graduate School of Medicine/School of Public Health, Kyoto, Japan.,Department of Neurology, Emergency Medicine, Nagoya City University East Medical Centre, Nagoya, Japan
| | - Yasushi Tsujimoto
- Department of Health Promotion and Human Behavior, Kyoto University Graduate School of Medicine/School of Public Health, Kyoto, Japan.,Department of Nephrology and Dialysis, Kyoritsu Hospital, Hyogo, Japan
| | - Yan Luo
- Department of Health Promotion and Human Behavior, Kyoto University Graduate School of Medicine/School of Public Health, Kyoto, Japan
| | - Kazumichi Yamamoto
- Department of Health Promotion and Human Behavior, Kyoto University Graduate School of Medicine/School of Public Health, Kyoto, Japan.,Institute for Airway Disease, Hyogo, Japan
| | - Ryuhei So
- Department of Health Promotion and Human Behavior, Kyoto University Graduate School of Medicine/School of Public Health, Kyoto, Japan.,Okayama Psychiatric Medical Centre, Okayama, Japan
| | - Masaharu Aga
- Department of Respiratory Medicine, Yokohama Municipal Citizen's Hospital, Kanagawa, Japan
| | - Kazuhide Matsushima
- Division of Acute Care Surgery, University of Southern California, Los Angeles, California, USA
| | - Shingo Fukuma
- Human Health Sciences, Kyoto University Graduate School of Medicine, Kyoto, Japan
| | - Toshi A Furukawa
- Department of Health Promotion and Human Behavior, Kyoto University Graduate School of Medicine/School of Public Health, Kyoto, Japan
| |
Collapse
|
31
|
Scheres LJJ, Middeldorp S, Goddijn M. Antithrombotic therapy for live birth in women with recurrent pregnancy loss: do not confound available high-level evidence. Hum Reprod 2021; 37:190-191. [PMID: 34791264 DOI: 10.1093/humrep/deab247] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
Affiliation(s)
- Luuk J J Scheres
- Department of Internal Medicine & Radboud Institute of Health Sciences (RIHS), Radboud University Medical Center, Nijmegen, the Netherlands.,Department of Clinical Epidemiology, Leiden University Medical Center, Leiden, the Netherlands
| | - Saskia Middeldorp
- Department of Internal Medicine & Radboud Institute of Health Sciences (RIHS), Radboud University Medical Center, Nijmegen, the Netherlands
| | - Mariëtte Goddijn
- Center for Reproductive Medicine, Department of Obstetrics and Gynecology, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
| |
Collapse
|
32
|
van der Plas-Krijgsman WG, Giardiello D, Putter H, Steyerberg EW, Bastiaannet E, Stiggelbout AM, Mooijaart SP, Kroep JR, Portielje JEA, Liefers GJ, de Glas NA. Development and validation of the PORTRET tool to predict recurrence, overall survival, and other-cause mortality in older patients with breast cancer in the Netherlands: a population-based study. THE LANCET. HEALTHY LONGEVITY 2021; 2:e704-e711. [PMID: 36098027 DOI: 10.1016/s2666-7568(21)00229-4] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2021] [Revised: 09/03/2021] [Accepted: 09/06/2021] [Indexed: 12/24/2022] Open
Abstract
BACKGROUND Current prediction tools for breast cancer outcomes are not tailored to the older patient, in whom competing risk strongly influences treatment effects. We aimed to develop and validate a prediction tool for 5-year recurrence, overall mortality, and other-cause mortality for older patients (aged ≥65 years) with early invasive breast cancer and to estimate individualised expected benefits of adjuvant systemic treatment. METHODS We selected surgically treated patients with early invasive breast cancer (stage I-III) aged 65 years or older from the population-based FOCUS cohort in the Netherlands. We developed prediction models for 5-year recurrence, overall mortality, and other-cause mortality using cause-specific Cox proportional hazard models. External validation was performed in a Dutch Cancer registry cohort. Performance was evaluated with discrimination accuracy and calibration plots. FINDINGS We included 2744 female patients in the development cohort and 13631 female patients in the validation cohort. Median age was 74·8 years (range 65-98) in the development cohort and 76·0 years (70-101) in the validation cohort. 5-year follow-up was complete for more than 99% of all patients. We observed 343 and 1462 recurrences, and 831 and 3594 deaths, of which 586 and 2565 were without recurrence, in the development and validation cohort, respectively. The area under the receiver-operating-characteristic curve at 5 years in the external dataset was 0·76 (95% CI 0·75-0·76) for overall mortality, 0·76 (0·76-0·77) for recurrence, and 0·75 (0·74-0·75) for other-cause mortality. INTERPRETATION The PORTRET tool can accurately predict 5-year recurrence, overall mortality, and other-cause mortality in older patients with breast cancer. The tool can support shared decision making, especially since it provides individualised estimated benefits of adjuvant treatment. FUNDING Dutch Cancer Foundation and ZonMw.
Collapse
Affiliation(s)
| | - Daniele Giardiello
- Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, Netherlands; Division of Molecular Pathology, The Netherlands Cancer Institute-Antoni van Leeuwenhoek Hospital, Amsterdam, Netherlands; Eurac Research, Institute for Biomedicine, Bolzano, Italy
| | - Hein Putter
- Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, Netherlands
| | - Ewout W Steyerberg
- Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, Netherlands; Department of Public Health, Erasmus MC, Rotterdam, Netherlands
| | - Esther Bastiaannet
- Department of Medical Oncology, Leiden University Medical Center, Leiden, Netherlands; Department of Surgery, Leiden University Medical Center, Leiden, Netherlands
| | - Anne M Stiggelbout
- Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, Netherlands
| | - Simon P Mooijaart
- Department of Gerontology and Geriatrics, Leiden University Medical Center, Leiden, Netherlands
| | - Judith R Kroep
- Department of Medical Oncology, Leiden University Medical Center, Leiden, Netherlands
| | | | - Gerrit-Jan Liefers
- Department of Surgery, Leiden University Medical Center, Leiden, Netherlands.
| | - Nienke A de Glas
- Department of Medical Oncology, Leiden University Medical Center, Leiden, Netherlands
| |
Collapse
|
33
|
Zheng M, Wang C, Ali A, Shih YA, Xie Q, Guo C. Prevalence of periodontitis in people clinically diagnosed with diabetes mellitus: a meta-analysis of epidemiologic studies. Acta Diabetol 2021; 58:1307-1327. [PMID: 34028620 DOI: 10.1007/s00592-021-01738-2] [Citation(s) in RCA: 22] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/02/2021] [Accepted: 05/06/2021] [Indexed: 12/19/2022]
Abstract
AIMS The relationship between diabetes mellitus and periodontitis has long been discussed, but the conclusion of this relationship was controversial. We conducted this meta-analysis to explore the association between diabetes mellitus and the prevalence of periodontitis. METHODS PubMed, Embase, Web of Science, and the Cochrane Library were searched for studies reporting associations of periodontitis with diabetes. Gray literature was integrated from the Open-Gray database. Prevalence estimates and odds ratios were extracted by two independent reviewers and synthesized through meta-analyses. RESULTS Twenty-seven papers met the inclusion criteria involving 3092 diabetic patients and 23,494 controls. Overall prevalence of periodontitis was 67.8% in diabetics (combined types) and 35.5% in patients without diabetes (OR, 1.85; 95%CI, 1.61-2.11). Probing depth (MD 0.23; 95% CI, 0.17-0.29; I2 = 25%) and plaque index (MD, 0.20; 95% CI, 0.18-0.23; P = 0.59; I2 = 0%) severity scores were significantly greater among the diabetic group. CONCLUSIONS The prevalence and severity of periodontitis are greater in patients with diabetes than in non-diabetic populations.
Collapse
Affiliation(s)
- Mengwen Zheng
- Department of Stomatology, School of Medicine, The First Affiliated Hospital, Shihezi University, Shihezi, Xinjiang, China
| | - Cheng Wang
- Institute of Basic Medical Sciences, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Ali Ali
- Sheffield Teaching Hospitals NIHR Biomedical Research Centre, Glossop Rd, Sheffield, S10 2JF, UK
| | - Yi An Shih
- School of Nursing, Peking University, Beijing, China
| | - Qinqin Xie
- Department of Stomatology, The Second Affiliated Hospital, Xinjiang Medical University, Urumqi, Xinjiang, China
| | - Chao Guo
- Department of Stomatology, School of Medicine, The First Affiliated Hospital, Shihezi University, Shihezi, Xinjiang, China.
- The Affiliated Hospital of Qingdao University, Qingdao,Shangdong, China.
| |
Collapse
|
34
|
The PRolaCT studies - a study protocol for a combined randomised clinical trial and observational cohort study design in prolactinoma. Trials 2021; 22:653. [PMID: 34563236 PMCID: PMC8465768 DOI: 10.1186/s13063-021-05604-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2020] [Accepted: 09/05/2021] [Indexed: 11/30/2022] Open
Abstract
Background First-line treatment for prolactinomas is a medical treatment with dopamine agonists (DAs), which effectively control hyperprolactinaemia in most patients, although post-withdrawal remission rates are approximately 34%. Therefore, many patients require prolonged DA treatment, while side effects negatively impact health-related quality of life (HRQoL). Endoscopic transsphenoidal resection is reserved for patients with severe side effects, or with DA-resistant prolactinoma. Surgery has a good safety profile and high probability of remission and may thus deserve a more prominent place in prolactinoma treatment. The hypothesis for this study is that early or upfront surgical resection is superior to DA treatment both in terms of HRQoL and remission rate in patients with a non-invasive prolactinoma of limited size. Methods We present a combined randomised clinical trial and observational cohort study design, which comprises three unblinded randomised controlled trials (RCTs; PRolaCT-1, PRolaCT-2, PRolaCT-3), and an observational study arm (PRolaCT-O) that compare neurosurgical counselling, and potential subsequent endoscopic transsphenoidal adenoma resection, with current standard care. Patients with a non-invasive prolactinoma (< 25 mm) will be eligible for one of three RCTs based on the duration of pre-treatment with DAs: PRolaCT-1: newly diagnosed, treatment-naïve patients; PRolaCT-2: patients with limited duration of DA treatment (4–6 months); and PRolaCT-3: patients with persisting prolactinoma after DA treatment for > 2 years. PRolaCT-O will include patients who decline randomisation, due to e.g. a clear treatment preference. Primary outcomes are disease remission after 36 months and HRQoL after 12 months. Discussion Early or upfront surgical resection for patients with a limited-sized prolactinoma may be a reasonable alternative to the current standard practice of DA treatment, which we will investigate in three RCTs and an observational cohort study. Within the three RCTs, patients will be randomised between neurosurgical counselling and standard care. The observational study arm will recruit patients who refuse randomisation and have a pronounced treatment preference. PRolaCT will collect randomised and observational data, which may facilitate a more individually tailored practice of evidence-based medicine. Trial registration US National Library of Medicine registry (ClinicalTrials.gov) NCT04107480. Registered on 27 September 2019, registered retrospectively (by 2 months). Supplementary Information The online version contains supplementary material available at 10.1186/s13063-021-05604-y.
Collapse
|
35
|
Toews I, Bode S, Metzner G, Farin-Glattacker E, Meerpohl JJ. [Relevance of different study types in benefit assessment: Results from expert interviews]. ZEITSCHRIFT FUR EVIDENZ FORTBILDUNG UND QUALITAET IM GESUNDHEITSWESEN 2021; 167:25-32. [PMID: 34556460 DOI: 10.1016/j.zefq.2021.08.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/28/2021] [Revised: 08/03/2021] [Accepted: 08/09/2021] [Indexed: 10/20/2022]
Abstract
BACKGROUND Careful evaluation of new interventions is essential. Randomized controlled trials (RCTs) and systematic reviews of RCTs provide the most reliable evidence base in comparative evaluations of treatments and interventions. Observational study designs are found at the lower levels of the classical evidence hierarchy. In light of policy efforts to accelerate the availability of innovations, the use of evidence from non-randomized studies for reimbursement decisions is being discussed. The present study addresses the question of the value and feasibility of RCTs and observational studies for benefit assessment after an intervention has been introduced into practice. METHODS Experts from German-speaking countries were interviewed using qualitative, semi-structured interviews. Participants included experts in health economics, health services and health services research, health communication, ethics, and health policy. The interviews were conducted by one researcher and then analyzed using a deductive approach. Main categories and subcategories were extracted. RESULTS Twenty-six experts were invited to participate, and 15 agreed to be interviewed. The duration of the individual interviews varied between 35 and 80minutes. The interviewed experts considered it possible and necessary to conduct RCTs when an intervention is already available in care in order to demonstrate that an intervention is more effective than existing alternatives. Experts considered good study design, methodological knowledge of those conducting the study, infrastructure, and funding as underlying conditions. They emphasized that the benefit of an intervention cannot be conclusively clarified on the basis of observational studies and that RCTs can therefore only be dispensed with in exceptional cases. Therefore, observational studies do not represent an alternative to RCTs in benefit assessment in general. In the opinion of the experts, the requirements for observational studies included sensible criteria and quality assurance measures. As alternatives to RCTs, the experts named studies based on routine data or high-quality, prospective, comparative cohort studies. Individual experts were very critical of studies based on data from registries. DISCUSSION AND CONCLUSION The results of the interviews are underlined by findings from the scientific literature. For example, international scientists advocate for conducting RCTs instead of observational studies and for a reduction in hurdles and barriers for conducting RCTs. In international methodological standards for the preparation of health technology assessments, RCTs are also generally preferred for benefit assessment. Nevertheless, there is still a need for research about the optimal use of observational studies, alternative RCT designs and the use of data from routine care or registries, as policy makers, among others, believe that observational studies should be used for benefit assessment. The results show that experts still adhere to the established principles of clinical epidemiology. Observational studies should only be used as an alternative to RCTs in exceptional cases to examine whether one intervention is more effective than existing alternatives.
Collapse
Affiliation(s)
- Ingrid Toews
- Institut für Evidenz in der Medizin, Universitätsklinikum Freiburg, Medizinische Fakultät, Albert-Ludwigs-Universität Freiburg, Freiburg, Deutschland
| | - Stefanie Bode
- Institut für Evidenz in der Medizin, Universitätsklinikum Freiburg, Medizinische Fakultät, Albert-Ludwigs-Universität Freiburg, Freiburg, Deutschland
| | - Gloria Metzner
- Sektion Versorgungsforschung und Rehabilitationsforschung, Universitätsklinikum Freiburg, Medizinische Fakultät, Albert-Ludwigs-Universität Freiburg, Freiburg, Deutschland
| | - Erik Farin-Glattacker
- Sektion Versorgungsforschung und Rehabilitationsforschung, Universitätsklinikum Freiburg, Medizinische Fakultät, Albert-Ludwigs-Universität Freiburg, Freiburg, Deutschland
| | - Joerg J Meerpohl
- Institut für Evidenz in der Medizin, Universitätsklinikum Freiburg, Medizinische Fakultät, Albert-Ludwigs-Universität Freiburg, Freiburg, Deutschland; Cochrane Deutschland, Cochrane Deutschland Stiftung, Freiburg, Deutschland.
| |
Collapse
|
36
|
Ezzalfani M, Porcher R, Savignoni A, Delaloge S, Filleron T, Robain M, Pérol D. Addressing the issue of bias in observational studies: Using instrumental variables and a quasi-randomization trial in an ESME research project. PLoS One 2021; 16:e0255017. [PMID: 34525119 PMCID: PMC8443058 DOI: 10.1371/journal.pone.0255017] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/09/2020] [Accepted: 07/08/2021] [Indexed: 11/18/2022] Open
Abstract
Purpose Observational studies using routinely collected data are faced with a number of potential shortcomings that can bias their results. Many methods rely on controlling for measured and unmeasured confounders. In this work, we investigate the use of instrumental variables (IV) and quasi-trial analysis to control for unmeasured confounders in the context of a study based on the retrospective Epidemiological Strategy and Medical Economics (ESME) database, which compared overall survival (OS) with paclitaxel plus bevacizumab or paclitaxel alone as first-line treatment in patients with HER2-negative metastatic breast cancer (MBC). Patients and methods Causal interpretations and estimates can be made from observation data using IV and quasi-trial analysis. Quasi-trial analysis has the same conceptual basis as IV, however, instead of using IV in the analysis, a “superficial” or “pseudo” randomized trial is used in a Cox model. For instance, in a multicenter trial, instead of using the treatment variable, quasi-trial analysis can consider the treatment preference in each center, which can be informative, and then comparisons of results between centers or clinicians can be informative. Results In the original analysis, the OS adjusted for major factors was significantly longer with paclitaxel and bevacizumab than with paclitaxel alone. Using the center-treatment preference as an instrument yielded to concordant results. For the quasi-trial analysis, a Cox model was used, adjusted on all factors initially used. The results consolidate those obtained with a conventional multivariate Cox model. Conclusion Unmeasured confounding is a major concern in observational studies, and IV or quasi-trial analysis can be helpful to complement analysis of studies of this nature.
Collapse
Affiliation(s)
- Monia Ezzalfani
- Department of Biostatistics, Pôle de Biometrie, Institut Curie, Paris, France
- * E-mail:
| | - Raphaël Porcher
- Université de Paris, Centre of Research Epidemiology and Statistics, INSERM U1153, Paris, France
| | - Alexia Savignoni
- Department of Biostatistics, Pôle de Biometrie, Institut Curie, Paris, France
| | - Suzette Delaloge
- Department of Cancer Medicine, Institut Gustave Roussy, Villejuif, France
| | | | - Mathieu Robain
- Department of Research and Development, R&D Unicancer, Paris, France
| | - David Pérol
- Department of Biostatistics, DRCI, Centre Leon Berard, Lyon, France
| | | |
Collapse
|
37
|
Hemkens LG. [Benefit assessment of digital health applications-challenges and opportunities]. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz 2021; 64:1269-1277. [PMID: 34524477 PMCID: PMC8441956 DOI: 10.1007/s00103-021-03413-x] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2021] [Accepted: 08/20/2021] [Indexed: 01/31/2023]
Abstract
Digital health applications promise to improve patient health and medical care. This analysis provides a brief overview of evidence-based benefit assessment and the challenges to the underlying evidence as prerequisites for optimal patient-oriented decision making. Classical concepts in study design, recent developments, and innovative approaches are described with the aim of highlighting future areas of development in innovative study designs and strategic evaluation concepts for digital health applications. A special focus is on pragmatic study designs.Evidence-based benefit assessment has fundamental requirements and criteria regardless of the type of treatments evaluated. Reliable evidence is essential. Fast, efficient, reliable, and practice-relevant evaluation of digital health applications is not achieved by turning to nonrandomized trials, but rather by better pragmatic randomized trials. They are feasible and combine the characteristics of digital health applications, classical methodological concepts, and new approaches to study conduct. Routinely collected data, low-contact study conduct (remote trials, virtual trials), and digital biomarkers promote useful randomized real-world evidence as solid evidence base for digital health applications. Continuous learning evaluation with randomized designs embedded in routine care is key to sustainable and efficient benefit assessment of digital health applications and may be crucial for strategic improvement of healthcare.
Collapse
Affiliation(s)
- Lars G Hemkens
- Basel Institute for Clinical Epidemiology and Biostatistics (ceb), Department of Clinical Research, University Hospital Basel, Spitalstrasse 12, 4031, Basel, Schweiz. .,Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA. .,Meta-Research Innovation Center Berlin (METRIC-B), Berlin Institute of Health, Berlin, Deutschland.
| |
Collapse
|
38
|
Gilyarevsky SR. Approaches to Assessing the Quality of Observational Studies of Clinical Practice Based on the Big Data Analysis. RATIONAL PHARMACOTHERAPY IN CARDIOLOGY 2021. [DOI: 10.20996/1819-6446-2021-08-01] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2022] Open
Abstract
The article is devoted to the discussion of the problems of assessing the quality of observational studies in real clinical practice and determining their place in the hierarchy of evidence-based information. The concept of “big data” and the acceptability of using such a term to refer to large observational studies is being discussed. Data on the limitations of administrative and claims databases when performing observational studies to assess the effects of interventions are presented. The concept of confounding factors influencing the results of observational studies is discussed. Modern approaches to reducing the severity of bias in real-life clinical practice studies are presented. The criteria for assessing the quality of observational pharmacoepidemiological studies and the fundamental differences between such studies and randomized clinical trials are presented. The results of systematic reviews of real-life clinical trials to assess the effects of direct oral anticoagulants are discussed.
Collapse
|
39
|
Krogh J, Hjorthøj C, Kristensen SL, Selmer C, Haugaard SB. The effect of sodium-glucose transport protein 2 inhibitors on mortality and heart failure in randomized trials versus observational studies. Diabet Med 2021; 38:e14600. [PMID: 33991127 DOI: 10.1111/dme.14600] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/14/2020] [Accepted: 05/10/2021] [Indexed: 12/26/2022]
Abstract
AIM Randomized clinical trials (RCTs) allocating type 2 diabetes patients to treatment with sodium-glucose transport protein 2 (SGLT-2) inhibitors or placebo have found significant effects on the risk of heart failure and modest effects on mortality. In the wake of the first trials, a number of observational studies have been conducted, some of these reporting a mortality reduction of 50% compared to active comparators. In this review, we systematically assess and compare the results on all-cause mortality, cardiovascular mortality and heart failure hospitalization observed in RCTs with the results obtained in observational studies. METHOD We performed a systematic bibliographical search including cardiovascular outcome trials and observational studies assessing the effect of SGLT-2 inhibitors on mortality and heart failure. RESULTS Seven RCTs and 23 observational studies were included in the current review. The observed heterogeneity between study results for all-cause mortality (p-interaction < 0.001) and cardiovascular mortality (p-interaction < 0.001) was explained by study type, whereas this was not the case for heart failure (p-interaction = 0.18). CONCLUSION Methodological considerations such as the omission of important confounders, immortal-time bias and residual confounding such as unmeasured social economic inequality may be the cause of the inflated results observed in observational studies and that calls for caution when observational studies are used to guide treatment of patients with type 2 diabetes.
Collapse
Affiliation(s)
- Jesper Krogh
- Department of Endocrinology, Rigshospitalet University Hospital, Copenhagen, Denmark
| | - Carsten Hjorthøj
- Copenhagen Research Center for Mental Health-Core, Mental Health Centre Copenhagen, Copenhagen University Hospital, Copenhagen, Denmark
- Department of Public Health, Section of Epidemiology, University of Copenhagen, Copenhagen, Denmark
| | - Søren L Kristensen
- Department of Cardiology, Rigshospitalet University Hospital, Copenhagen, Denmark
| | - Christian Selmer
- Department of Endocrinology, Bispebjerg and Frederiksberg Hospital, Copenhagen University Hospital, Copenhagen, Denmark
| | - Steen B Haugaard
- Department of Endocrinology, Bispebjerg and Frederiksberg Hospital, Copenhagen University Hospital, Copenhagen, Denmark
| |
Collapse
|
40
|
Janiaud P, Agarwal A, Tzoulaki I, Theodoratou E, Tsilidis KK, Evangelou E, Ioannidis JPA. Validity of observational evidence on putative risk and protective factors: appraisal of 3744 meta-analyses on 57 topics. BMC Med 2021; 19:157. [PMID: 34225716 PMCID: PMC8259334 DOI: 10.1186/s12916-021-02020-6] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/17/2021] [Accepted: 05/28/2021] [Indexed: 12/12/2022] Open
Abstract
BACKGROUND The validity of observational studies and their meta-analyses is contested. Here, we aimed to appraise thousands of meta-analyses of observational studies using a pre-specified set of quantitative criteria that assess the significance, amount, consistency, and bias of the evidence. We also aimed to compare results from meta-analyses of observational studies against meta-analyses of randomized controlled trials (RCTs) and Mendelian randomization (MR) studies. METHODS We retrieved from PubMed (last update, November 19, 2020) umbrella reviews including meta-analyses of observational studies assessing putative risk or protective factors, regardless of the nature of the exposure and health outcome. We extracted information on 7 quantitative criteria that reflect the level of statistical support, the amount of data, the consistency across different studies, and hints pointing to potential bias. These criteria were level of statistical significance (pre-categorized according to 10-6, 0.001, and 0.05 p-value thresholds), sample size, statistical significance for the largest study, 95% prediction intervals, between-study heterogeneity, and the results of tests for small study effects and for excess significance. RESULTS 3744 associations (in 57 umbrella reviews) assessed by a median number of 7 (interquartile range 4 to 11) observational studies were eligible. Most associations were statistically significant at P < 0.05 (61.1%, 2289/3744). Only 2.6% of associations had P < 10-6, ≥1000 cases (or ≥20,000 participants for continuous factors), P < 0.05 in the largest study, 95% prediction interval excluding the null, and no large between-study heterogeneity, small study effects, or excess significance. Across the 57 topics, large heterogeneity was observed in the proportion of associations fulfilling various quantitative criteria. The quantitative criteria were mostly independent from one another. Across 62 associations assessed in both RCTs and in observational studies, 37.1% had effect estimates in opposite directions and 43.5% had effect estimates differing beyond chance in the two designs. Across 94 comparisons assessed in both MR and observational studies, such discrepancies occurred in 30.8% and 54.7%, respectively. CONCLUSIONS Acknowledging that no gold-standard exists to judge whether an observational association is genuine, statistically significant results are common in observational studies, but they are rarely convincing or corroborated by randomized evidence.
Collapse
Affiliation(s)
- Perrine Janiaud
- Meta-Research Innovation Center at Stanford (METRICS), Stanford, CA, 94305, USA.,Department of Clinical Research, University Hospital Basel, University of Basel, CH-4056, Basel, Switzerland
| | - Arnav Agarwal
- Department of Medicine, University of Toronto, 1 King's College Circle #3172, Toronto, ON, M5S 1A8, Canada
| | - Ioanna Tzoulaki
- Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, University Campus, 45110, Ioannina, Greece.,Department of Epidemiology and Biostatistics, School of Public Health, Imperial College London, London, W2 1PG, UK
| | - Evropi Theodoratou
- Centre for Global Health, The University of Edinburgh, Edinburgh, EH8 9AG, UK.,Cancer Research UK Edinburgh Centre, Institute of Genetics and Cancer, Western General Hospital, The University of Edinburgh, Edinburgh, EH4 2XU, UK
| | - Konstantinos K Tsilidis
- Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, University Campus, 45110, Ioannina, Greece.,Department of Epidemiology and Biostatistics, School of Public Health, Imperial College London, London, W2 1PG, UK
| | - Evangelos Evangelou
- Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, University Campus, 45110, Ioannina, Greece.,Department of Epidemiology and Biostatistics, School of Public Health, Imperial College London, London, W2 1PG, UK
| | - John P A Ioannidis
- Meta-Research Innovation Center at Stanford (METRICS), Stanford, CA, 94305, USA. .,Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, CA, 94305, USA. .,Stanford Prevention Research Center, Department of Medicine, Stanford University School of Medicine, Stanford, CA, 94305, USA. .,Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA, 94305, USA. .,Department of Statistics, Stanford University School of Humanities and Sciences, Stanford, CA, 94305, USA.
| |
Collapse
|
41
|
Rahmatian D, Tadrous M. Old dog with new tricks: An introduction to real-world evidence for pharmacists. Am J Health Syst Pharm 2021; 78:2277-2280. [PMID: 34153111 PMCID: PMC8344619 DOI: 10.1093/ajhp/zxab261] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2022] Open
Affiliation(s)
- Donna Rahmatian
- Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, ON, Canada, and St. Paul's Hospital, Vancouver, BC, Canada
| | - Mina Tadrous
- Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, ON, Canada, and Women's College Hospital Research Institute, Toronto, ON, Canada
| |
Collapse
|
42
|
Katsanos K, Kitrou P, Spiliopoulos S. The Rollercoaster of Paclitaxel in the Lower Limbs and Skeletons in the Closet: An Opinion Review. J Vasc Interv Radiol 2021; 32:785-791. [PMID: 33811999 DOI: 10.1016/j.jvir.2021.03.537] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2021] [Accepted: 03/24/2021] [Indexed: 02/08/2023] Open
Abstract
There have been concerns about the long-term risk of all-cause death with the use of paclitaxel-coated devices in the lower limbs. Results from a 2018 meta-analysis were corroborated by the US Food and Drug Administration and later confirmed by an individual patient data meta-analysis. However, population-based observational studies have produced contradictory results and often claimed a survival benefit with the use of paclitaxel. The recently published Swedish drug-elution trial in peripheral arterial disease did not confirm a significant mortality risk. In this review, the authors discuss the key elements of the identified mortality signal and stress important facts and figures that remain underrecognized and elusive. They also highlight the important types of epidemiological bias that pertain to the ongoing debate on paclitaxel.
Collapse
|
43
|
Janiaud P, Hemkens LG, Ioannidis JPA. Challenges and Lessons Learned From COVID-19 Trials: Should We Be Doing Clinical Trials Differently? Can J Cardiol 2021; 37:1353-1364. [PMID: 34077789 PMCID: PMC8164884 DOI: 10.1016/j.cjca.2021.05.009] [Citation(s) in RCA: 27] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2021] [Revised: 05/18/2021] [Accepted: 05/22/2021] [Indexed: 12/15/2022] Open
Abstract
The COVID-19 crisis led to a flurry of clinical trials activity. The COVID-evidence database shows 2814 COVID-19 randomized trials registered as of February 16, 2021. Most were small (only 18% have a planned sample size > 500) and the rare completed ones have not provided published results promptly (only 283 trial publications as of February 2021). Small randomized trials and observational, nonrandomized analyses have not had a successful track record and have generated misleading expectations. Different large trials on the same intervention have generally been far more efficient in producing timely and consistent evidence. The rapid generation of evidence and accelerated dissemination of results have led to new challenges for systematic reviews and meta-analyses (eg, rapid, living, and scoping reviews). Pressure to regulatory agencies has also mounted with massive emergency authorizations, but some of them have had to be revoked. Pandemic circumstances have disrupted the way trials are conducted; therefore, new methods have been developed and adopted more widely to facilitate recruitment, consent, and overall trial conduct. On the basis of the COVID-19 experience and its challenges, planning of several large, efficient trials, and wider use of adaptive designs might change the future of clinical research. Pragmatism, integration in clinical care, efficient administration, promotion of collaborative structures, and enhanced integration of existing data and facilities might be several of the legacies of COVID-19 on future randomized trials.
Collapse
Affiliation(s)
- Perrine Janiaud
- Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland
| | - Lars G Hemkens
- Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland; Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, California, USA; Meta-Research Innovation Center Berlin (METRIC-B), Berlin Institute of Health, Berlin, Germany
| | - John P A Ioannidis
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, California, USA; Meta-Research Innovation Center Berlin (METRIC-B), Berlin Institute of Health, Berlin, Germany; Department of Medicine, Stanford University School of Medicine, Stanford, California, USA; Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, California, USA; Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, California, USA; Department of Statistics, Stanford University School of Humanities and Sciences, Stanford, California, USA.
| |
Collapse
|
44
|
Piovani D, Pansieri C, Peyrin-Biroulet L, Danese S, Bonovas S. Confounding and bias in observational studies in inflammatory bowel disease: a meta-epidemiological study. Aliment Pharmacol Ther 2021; 53:712-721. [PMID: 33296517 DOI: 10.1111/apt.16222] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/13/2020] [Revised: 11/08/2020] [Accepted: 11/27/2020] [Indexed: 12/30/2022]
Abstract
BACKGROUND Observational research concerning inflammatory bowel disease (IBD) is highly susceptible to spurious findings because of confounding and bias. AIM To investigate how these issues were reported in this research field. METHODS We identified and appraised a random sample of 160 observational studies concerning IBD published in high-impact gastroenterology journals and the most respected specialty journals of the condition. We applied a standardised methodology to assess how confounding and bias were reported and discussed, and investigated the association between yearly citations and study characteristics using mixed-effect multivariable regression analysis. RESULTS The authors of 67 out of 160 articles (41.9%) mentioned confounding, and in 89 cases (55.6%) reported any bias. Although most authors applied strategies to minimise confounding or bias (n = 139; 86.9%) and acknowledged at least one unadjusted confounder (n = 116; 72.5%), a minority commented about whether the main findings could have been affected (n = 60; 37.5%). Very few authors (n = 7; 4.4%) called for caution in interpreting the results in the discussion. Reporting of confounding and bias was particularly lacking for case-control studies, those not using routinely collected data, those employing laboratory analyses as the primary method of assessment and studies investigating non-modifiable exposures. In adjusted analyses, mentioning or alluding to confounding was positively associated with yearly citations (P = 0.010), whereas calling for a cautious interpretation of the findings was not. CONCLUSIONS Reporting of confounding is inadequate and its acknowledgement is often neglected in interpreting high-impact observational research in IBD. These results encourage a more careful evaluation of the consequences of confounding and bias.
Collapse
Affiliation(s)
- Daniele Piovani
- Department of Biomedical Sciences, Humanitas University, Milan, Italy.,IBD Center, Humanitas Clinical and Research Center - IRCCS, Milan, Italy
| | - Claudia Pansieri
- Department of Biomedical Sciences, Humanitas University, Milan, Italy.,IBD Center, Humanitas Clinical and Research Center - IRCCS, Milan, Italy
| | - Laurent Peyrin-Biroulet
- Department of Gastroenterology, Nancy University Hospital, University of Lorraine, Vandoeuvre-lès-Nancy, France
| | - Silvio Danese
- Department of Biomedical Sciences, Humanitas University, Milan, Italy.,IBD Center, Humanitas Clinical and Research Center - IRCCS, Milan, Italy
| | - Stefanos Bonovas
- Department of Biomedical Sciences, Humanitas University, Milan, Italy.,IBD Center, Humanitas Clinical and Research Center - IRCCS, Milan, Italy
| |
Collapse
|
45
|
Mathes T, Rombey T, Kuss O, Pieper D. No inexplicable disagreements between real-world data-based nonrandomized controlled studies and randomized controlled trials were found. J Clin Epidemiol 2021; 133:1-13. [PMID: 33359322 DOI: 10.1016/j.jclinepi.2020.12.019] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2020] [Revised: 12/07/2020] [Accepted: 12/15/2020] [Indexed: 01/21/2023]
Abstract
OBJECTIVES We assessed disagreements between nonrandomized controlled studies based on real-world data (NRCS-RWDs) and randomized controlled trials (RCTs). STUDY DESIGN AND SETTING We systematically searched for studies that compared treatment effect estimates from NRCS-RWDs and RCTs on the same clinical question. We assessed the potential difference between NRCS-RWDs and RCTs related to internal and external validity. We calculated various meta-epidemiological measures to assess agreement. In case of disagreements, we tried to identify the probable causes of disagreements. RESULTS We included 12 studies comparing 15 treatment effect estimates of NRCS-RWDs and RCTs. There were many potential causes of disagreement. Ninety-five percent confidence intervals overlapped for 12 of 15 treatment effect estimates. Our analysis on predicted vs. observed overlap showed that there were no more disagreements than expected by chance. We observed only two substantial differences between the 15 treatment effect estimates. In both cases, we identified risk of bias in the NRCS-RWDs as the most probable cause of disagreement. CONCLUSION Our findings suggest that there are clinical questions where the difference in risk of bias between a well-conducted NRCS-RWD and an RCT is negligible. In our analysis, threats to external validity appeared to have no or only a weak impact on the disagreements of treatment effect estimates.
Collapse
Affiliation(s)
- Tim Mathes
- Institute for Research in Operative Medicine, Faculty of Health, School of Medicine, Witten/Herdecke University, 51067 Cologne, Germany.
| | - Tanja Rombey
- Institute for Research in Operative Medicine, Faculty of Health, School of Medicine, Witten/Herdecke University, 51067 Cologne, Germany
| | - Oliver Kuss
- Institute for Biometrics and Epidemiology, German Diabetes Center, Leibniz Institute for Diabetes Research, Heinrich Heine University Düsseldorf, Germany
| | - Dawid Pieper
- Institute for Research in Operative Medicine, Faculty of Health, School of Medicine, Witten/Herdecke University, 51067 Cologne, Germany
| |
Collapse
|
46
|
Franklin JM, Patorno E, Desai RJ, Glynn RJ, Martin D, Quinto K, Pawar A, Bessette LG, Lee H, Garry EM, Gautam N, Schneeweiss S. Emulating Randomized Clinical Trials With Nonrandomized Real-World Evidence Studies: First Results From the RCT DUPLICATE Initiative. Circulation 2020; 143:1002-1013. [PMID: 33327727 DOI: 10.1161/circulationaha.120.051718] [Citation(s) in RCA: 163] [Impact Index Per Article: 40.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
BACKGROUND Regulators are evaluating the use of noninterventional real-world evidence (RWE) studies to assess the effectiveness of medical products. The RCT DUPLICATE initiative (Randomized, Controlled Trials Duplicated Using Prospective Longitudinal Insurance Claims: Applying Techniques of Epidemiology) uses a structured process to design RWE studies emulating randomized, controlled trials (RCTs) and compare results. We report findings of the first 10 trial emulations, evaluating cardiovascular outcomes of antidiabetic or antiplatelet medications. METHODS We selected 3 active-controlled and 7 placebo-controlled RCTs for replication. Using patient-level claims data from US commercial and Medicare payers, we implemented inclusion and exclusion criteria, selected primary end points, and comparator populations to emulate those of each corresponding RCT. Within the trial-mimicking populations, we conducted propensity score matching to control for >120 preexposure confounders. All study measures were prospectively defined and protocols registered before hazard ratios and 95% CIs were computed. Success criteria for the primary analysis were prespecified for each replication. RESULTS Despite attempts to emulate RCT design as closely as possible, differences between the RCT and corresponding RWE study populations remained. The regulatory conclusions were equivalent in 6 of 10. The RWE emulations achieved a hazard ratio estimate that was within the 95% CI from the corresponding RCT in 8 of 10 studies. In 9 of 10, either the regulatory or estimate agreement success criteria were fulfilled. The largest differences in effect estimates were found for RCTs where second-generation sulfonylureas were used as a proxy for placebo regarding cardiovascular effects. Nine of 10 replications had a standardized difference between effect estimates of <2, which suggests differences within expected random variation. CONCLUSIONS Agreement between RCT and RWE findings varies depending on which agreement metric is used. Interim findings indicate that selection of active comparator therapies with similar indications and use patterns enhances the validity of RWE. Even in the context of active comparators, concordance between RCT and RWE findings is not guaranteed, partially because trials are not emulated exactly. More trial emulations are needed to understand how often and in what contexts RWE findings match RCTs. Registration: URL: https://www.clinicaltrials.gov; Unique identifiers: NCT03936049, NCT04215523, NCT04215536, NCT03936010, NCT03936036, NCT03936062, NCT03936023, NCT03648424, NCT04237935, NCT04237922.
Collapse
Affiliation(s)
- Jessica M Franklin
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, MA (J.M.F., E.P., R.J.D., R.J.G., A.P., L.G.B., H.L., N.G., S.S.)
| | - Elisabetta Patorno
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, MA (J.M.F., E.P., R.J.D., R.J.G., A.P., L.G.B., H.L., N.G., S.S.)
| | - Rishi J Desai
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, MA (J.M.F., E.P., R.J.D., R.J.G., A.P., L.G.B., H.L., N.G., S.S.)
| | - Robert J Glynn
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, MA (J.M.F., E.P., R.J.D., R.J.G., A.P., L.G.B., H.L., N.G., S.S.)
| | - David Martin
- Office of Medical Policy, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, MD (D.M., K.Q.)
| | - Kenneth Quinto
- Office of Medical Policy, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, MD (D.M., K.Q.)
| | - Ajinkya Pawar
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, MA (J.M.F., E.P., R.J.D., R.J.G., A.P., L.G.B., H.L., N.G., S.S.)
| | - Lily G Bessette
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, MA (J.M.F., E.P., R.J.D., R.J.G., A.P., L.G.B., H.L., N.G., S.S.)
| | - Hemin Lee
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, MA (J.M.F., E.P., R.J.D., R.J.G., A.P., L.G.B., H.L., N.G., S.S.)
| | | | - Nileesa Gautam
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, MA (J.M.F., E.P., R.J.D., R.J.G., A.P., L.G.B., H.L., N.G., S.S.)
| | - Sebastian Schneeweiss
- Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, MA (J.M.F., E.P., R.J.D., R.J.G., A.P., L.G.B., H.L., N.G., S.S.)
| |
Collapse
|
47
|
Ota HCU, Smith BG, Alamri A, Robertson FC, Marcus H, Hirst A, Broekman M, Hutchinson P, McCulloch P, Kolias A. The IDEAL framework in neurosurgery: a bibliometric analysis. Acta Neurochir (Wien) 2020; 162:2939-2947. [PMID: 32651707 PMCID: PMC7593304 DOI: 10.1007/s00701-020-04477-5] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2020] [Accepted: 06/29/2020] [Indexed: 12/15/2022]
Abstract
BACKGROUND The Idea, Development, Exploration, Assessment and Long-term study (IDEAL) framework was created to provide a structured way for assessing and evaluating novel surgical techniques and devices. OBJECTIVES The aim of this paper was to investigate the utilization of the IDEAL framework within neurosurgery, and to identify factors influencing implementation. METHODS A bibliometric analysis of the 7 key IDEAL papers on Scopus, PubMed, Embase, Web of Science, and Google Scholar databases (2009-2019) was performed. A second journal-specific search then identified additional papers citing the IDEAL framework. Publications identified were screened by two independent reviewers to select neurosurgery-specific articles. RESULTS The citation search identified 1336 articles. The journal search identified another 16 articles. Following deduplication and review, 51 relevant articles remained; 14 primary papers (27%) and 37 secondary papers (73%). Of the primary papers, 5 (36%) papers applied the IDEAL framework to their research correctly; two were aligned to the pre-IDEAL stage, one to the Idea and Development stages, and two to the Exploration stage. Of the secondary papers, 21 (57%) explicitly discussed the IDEAL framework. Eighteen (86%) of these were supportive of implementing the framework, while one was not, and two were neutral. CONCLUSION The adoption of the IDEAL framework in neurosurgery has been slow, particularly for early-stage neurosurgical techniques and inventions. However, the largely positive reviews in secondary literature suggest potential for increased use that may be achieved with education and publicity.
Collapse
Affiliation(s)
| | - Brandon G Smith
- Department of Clinical Neurosciences, University of Cambridge & Addenbrooke's Hospital, Cambridge, UK
| | - Alexander Alamri
- Department of Neurosurgery, The Royal London Hospital, London, UK
| | - Faith C Robertson
- Department. of Neurosurgery, Massachusetts General Hospital, Boston, MA, USA
| | - Hani Marcus
- The Victor Horsley Department of Neurosurgery, The National Hospital for Neurology and Neurosurgery, London, UK
| | - Allison Hirst
- IDEAL Collaboration, Nuffield Department of Surgical Sciences, University of Oxford and John Radcliffe Hospital, Oxford, UK
| | - Marike Broekman
- IDEAL Collaboration, Nuffield Department of Surgical Sciences, University of Oxford and John Radcliffe Hospital, Oxford, UK
- Department of Neurosurgery, Haaglanden Medical Center/Leiden University Medical Center, The Hague, Netherlands
| | - Peter Hutchinson
- Department of Clinical Neurosciences, University of Cambridge & Addenbrooke's Hospital, Cambridge, UK
| | - Peter McCulloch
- IDEAL Collaboration, Nuffield Department of Surgical Sciences, University of Oxford and John Radcliffe Hospital, Oxford, UK
| | - Angelos Kolias
- Department of Clinical Neurosciences, University of Cambridge & Addenbrooke's Hospital, Cambridge, UK.
- IDEAL Collaboration, Nuffield Department of Surgical Sciences, University of Oxford and John Radcliffe Hospital, Oxford, UK.
| |
Collapse
|
48
|
Unknown confounders did not bias the treatment effect when improving balance of known confounders in randomized trials. J Clin Epidemiol 2020; 126:9-16. [DOI: 10.1016/j.jclinepi.2020.06.012] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2019] [Revised: 04/02/2020] [Accepted: 06/10/2020] [Indexed: 11/18/2022]
|
49
|
Sheffield KM, Dreyer NA, Murray JF, Faries DE, Klopchin MN. Replication of randomized clinical trial results using real-world data: paving the way for effectiveness decisions. J Comp Eff Res 2020; 9:1043-1050. [PMID: 32914653 DOI: 10.2217/cer-2020-0161] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2022] Open
Abstract
The FDA is preparing guidance about using real-world evidence (RWE) to support decisions about product effectiveness. Several ongoing efforts aim to replicate randomized clinical trial (RCT) results using RWE with the intent of identifying circumstances and methods that provide valid evidence of drug effects. Lack of agreement may not be due to faulty methods but rather to the challenges with emulating RCTs, differences in healthcare settings and patient populations, differences in effect measures and data analysis, bias, and/or the efficacy-effectiveness gap. In fact, for some decisions, RWE may lead to better understanding of how treatments work in usual care settings than a more constrained view from RCTs. Efforts to reconcile the role and opportunities for generating complementary evidence from RWE and RCTs will advance regulatory science.
Collapse
Affiliation(s)
- Kristin M Sheffield
- Global Patient Outcomes & Real World Evidence, Eli Lilly & Company, Indianapolis, IN 46225, USA
| | - Nancy A Dreyer
- IQVIA Real-World & Analytic Solutions, Cambridge, MA 02139, USA
| | - James F Murray
- Global Patient Outcomes & Real World Evidence, Eli Lilly & Company, Indianapolis, IN 46225, USA
| | - Douglas E Faries
- Real World Analytics, Eli Lilly & Company, Indianapolis, IN 46225, USA
| | - Megan N Klopchin
- Global Patient Outcomes & Real World Evidence, Eli Lilly & Company, Indianapolis, IN 46225, USA
| |
Collapse
|
50
|
Eckles D, Bakshy E. Bias and High-Dimensional Adjustment in Observational Studies of Peer Effects. J Am Stat Assoc 2020. [DOI: 10.1080/01621459.2020.1796393] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/25/2022]
Affiliation(s)
- Dean Eckles
- Sloan School of Management, Massachusetts Institute of Technology, Cambridge, MA
- Institute for Data, Systems & Society, Massachusetts Institute of Technology, Cambridge, MA
| | | |
Collapse
|