1
|
Guyatt G, Iorio A, De Beer H, Owen A, Agoritsas T, Murad MH, Karthikeyan G, Cuello C, Prasad M, Kim K, Ali DS, Agarwal A, Hemkens LG, Yao L, Hultcrantz M, Rylance J, Chu DK, Vandvik PO, Djulbegovic B, Mustafa RA, Zeng L, Eachempati P, Rochwerg B, Prasad K, Montori VM, Brignardello-Petersen R. Core GRADE 5: rating certainty of evidence-assessing indirectness. BMJ 2025; 389:e083865. [PMID: 40393729 DOI: 10.1136/bmj-2024-083865] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 05/22/2025]
Abstract
This fifth article in a seven part series presents the Core GRADE (Grading of Recommendations Assessment, Development and Evaluation) approach to systematic reviews, clinical practice guidelines, and health technology assessments and addresses issues of indirect evidence. Guideline developers and health technology assessment practitioners must carefully specify the population, intervention, comparison, and outcome (PICO)—their target PICO—and consider the extent to which the best available evidence matches their target. When target and study PICOs differ substantially, studies provide indirect evidence and Core GRADE users may rate down the certainty of evidence as a result of this indirectness. Whether examining studies from a search for direct evidence or a deliberate search for indirect evidence, for each substantial difference between target and study PICO Core GRADE users must judge the likelihood that magnitude of effects will differ substantially. The greater the likelihood of substantial differences the more advisable rating down for indirectness.
Collapse
Affiliation(s)
- Gordon Guyatt
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
- Department of Medicine, McMaster University, Hamilton, ON, L8S 4L8, Canada
- MAGIC Evidence Ecosystem Foundation, Oslo, Norway
| | - Alfonso Iorio
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
- Department of Medicine, McMaster University, Hamilton, ON, L8S 4L8, Canada
| | - Hans De Beer
- Guide2Guidance, Lemelerberg 7, Utrecht, Netherlands
| | - Andrew Owen
- Department of Pharmacology and Therapeutics, Centre of Excellence in Long-acting Therapeutics (CELT), University of Liverpool, Liverpool, UK
| | - Thomas Agoritsas
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
- MAGIC Evidence Ecosystem Foundation, Oslo, Norway
- Division General Internal Medicine, University Hospitals of Geneva, Geneva, Switzerland
| | - M Hassan Murad
- Evidence-based Practice Center, Mayo Clinic, Rochester, MN, USA
| | - Ganesan Karthikeyan
- Translational Health Science Technology Institute, Faridabad, India
- All India Institute of Medical Sciences, New Delhi, India
| | - Carlos Cuello
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
| | - Manya Prasad
- Clinical Research and Epidemiology, Institute of Liver and Biliary Sciences, New Delhi, India
| | - Kevin Kim
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
- Population Health Research Institute, Hamilton, ON, Canada
| | - Dalal S Ali
- Divisions of Endocrinology and Metabolism, McMaster University, Hamilton, ON, Canada
| | - Arnav Agarwal
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
- Department of Medicine, McMaster University, Hamilton, ON, L8S 4L8, Canada
- MAGIC Evidence Ecosystem Foundation, Oslo, Norway
| | - Lars G Hemkens
- Pragmatic Evidence Lab, Research Center for Clinical Neuroimmunology and Neuroscience Basel (RC2NB), University Hospital Basel and University of Basel, Basel, Switzerland
- Department of Clinical Research, University Hospital Basel and University of Basel, Basel, Switzerland
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA
| | - Liang Yao
- Lee Kong Chian School of Medicine, Nanyang Technological University Singapore, Singapore
| | - Monica Hultcrantz
- HTA Region Stockholm, Centre for Health Economics, Informatics and Health Care Research (CHIS), Stockholm Health Care Services, Stockholm, Sweden
- Department of Learning, Informatics, Management and Ethics, Karolinska Institutet, Stockholm, Sweden
| | - Jamie Rylance
- Liverpool School of Tropical Medicine, Liverpool, UK
| | - Derek K Chu
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
- Department of Medicine, McMaster University, Hamilton, ON, L8S 4L8, Canada
| | - Per Olav Vandvik
- MAGIC Evidence Ecosystem Foundation, Oslo, Norway
- Institute of Health and Society, University of Oslo Faculty of Medicine, Oslo, Norway
| | - Benjamin Djulbegovic
- Division of Hematology/Oncology, Department of Medicine, Medical University of South Carolina, Charleston, SC, USA
| | - Reem A Mustafa
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
- Department of Medicine, University of Kansas Medical Center, Kansas City, MO, USA
| | - Linan Zeng
- Pharmacy Department/Evidence-based Pharmacy Centre/Children's Medicine Key Laboratory of Sichuan Province, West China Second University Hospital, Sichuan University, Chengdu, China
- Sichuan University and Key Laboratory of Birth Defects and Related Disease of Women and Children, Ministry of Education, Chengdu, China
- West China Biomedical Big Data Center, West China Hospital, Sichuan University, Chengdu, China
| | - Prashanti Eachempati
- MAGIC Evidence Ecosystem Foundation, Oslo, Norway
- Peninsula Dental School, University of Plymouth, Plymouth, UK
- Faculty of Dentistry, Manipal University College Malaysia, Malaysia
| | - Bram Rochwerg
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
- Department of Medicine, McMaster University, Hamilton, ON, L8S 4L8, Canada
| | - Kameshwar Prasad
- Department of Neurology, All India Institute of Medical Sciences, New Delhi, India
- Fortis CSR Foundation, New Delhi, India
| | - Victor M Montori
- Division of Endocrinology, Department of Medicine, Mayo Clinic, Rochester, MN, USA
- Knowledge and Evaluation Research Unit, Mayo Clinic, Rochester, MN, USA
| | | |
Collapse
|
2
|
Djulbegovic B, Hozo I, Iskander R, Parish AJ, Kimmelman J, Ioannidis JPA. There is no upper limit on the maximum effect that can be detected in randomized trials. J Clin Epidemiol 2025; 184:111828. [PMID: 40368222 DOI: 10.1016/j.jclinepi.2025.111828] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2025] [Revised: 04/26/2025] [Accepted: 05/06/2025] [Indexed: 05/16/2025]
Abstract
BACKGROUND AND OBJECTIVES Randomized controlled trials (RCTs) are commonly viewed as essential for detecting small treatment benefits, yet they also identify large ("dramatic") effects. Forecasting the likelihood of future large effects helps guide resource allocation for conducting clinical trials. METHODS We included consecutive cancer RCTs from 5 cohorts identified by funders or trial registries, minimizing publication bias. Between 1955 and 2018 (and published by 2022), 716 RCTs compared 984 experimental vs standard treatments in approximately 350,000 patients. We applied a generalized Pareto distribution (GPD) under Extreme Value Theory to predict future maximum treatment effects using data spanning 65 years. RESULTS The GPD's positive shape parameter implies no upper limit on maximum treatment effects. Historically, the largest observed effect had an odds ratio (OR) of 45 (95% CI: 2-1008). If current patterns hold, the largest effect over the next 50 years is projected at OR = 23 (95% CI: 13-106). We estimated 20% probability of detecting new treatments with OR >50 within the same time frame. Increasing the number of RCTs from about 20 to 40 or 60 per year would double or triple the likelihood of detecting breakthrough treatments with dramatic effects. CONCLUSION Our findings suggest there may be no absolute upper bound on discoverable treatment effects in cancer RCTs, although estimates will likely remain in the range observed between 1955 and 2022. Conducting more RCTs would boost the probability of identifying treatments with large effects, underscoring the importance of sustained or expanded trial activity to accelerate breakthrough discoveries. PLAIN LANGUAGE SUMMARY New treatments cannot be discovered without individuals volunteering to participate in clinical studies. Among all types of human clinical studies, RCTs are considered the most reliable method for evaluating new medical treatments and are especially effective at detecting small beneficial effects. This study also demonstrates that conducting more RCTs would accelerate the discovery of treatments with both small and large effects. By increasing public participation in RCTs, we can drive faster advances in therapeutics and influence policy decisions to allocate more resources toward the conduct of these trials.
Collapse
Affiliation(s)
- Benjamin Djulbegovic
- Division of Medical Hematology and Oncology, Department of Medicine, Medical University of South Carolina, Charleston, SC, USA.
| | - Iztok Hozo
- Department of Mathematics, Indiana University Northwest, Gary, IN, USA
| | - Renata Iskander
- Department of Equity, Ethics and Policy School of Population and Global Health, McGill University, 2001 McGill College Ave. / Rm 1155, Montreal, Quebec H3A 1G1, Canada
| | - Austin J Parish
- Department of Emergency Medicine, Lincoln Medical Center, Bronx, New York, NY, USA; Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA; Stanford Prevention Research Center, Department of Medicine, Stanford, CA, USA; Department of Epidemiology and Population Health, Stanford, CA, USA; Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA, USA; Department of Statistics, Stanford University School of Humanities and Sciences, Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA
| | - Jonathan Kimmelman
- Department of Equity, Ethics and Policy School of Population and Global Health, McGill University, 2001 McGill College Ave. / Rm 1155, Montreal, Quebec H3A 1G1, Canada
| | - John P A Ioannidis
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA; Stanford Prevention Research Center, Department of Medicine, Stanford, CA, USA; Department of Epidemiology and Population Health, Stanford, CA, USA; Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA, USA; Department of Statistics, Stanford University School of Humanities and Sciences, Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA
| |
Collapse
|
3
|
Pereira MMA, Dini C, Souza JGS, Barão VAR, de Avila ED. Industry support for dental implant research: A metatrend study of industry partnership in the development of new technologies. J Prosthet Dent 2024; 132:72-80. [PMID: 35811163 DOI: 10.1016/j.prosdent.2022.05.026] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2021] [Revised: 05/25/2022] [Accepted: 05/25/2022] [Indexed: 10/17/2022]
Abstract
STATEMENT OF PROBLEM Industry needs scientific knowledge to develop new products and services, and their financial support to dental implant researchers translates into commercial products. Therefore, identifying the relevant factors for a successful industry partnership is important. PURPOSE The purpose of this study was to provide a 20-year bibliometric overview of industry-sponsored studies in implant dentistry to identify possible factors involved in industry partnership motivations. MATERIAL AND METHODS A hand search of 6 of the most established journals in the implant dentistry field was performed for articles published in 1999, 2004, 2009, 2014, and 2019. Information regarding the continent of origin of the corresponding author, interinstitute collaboration, type and topic of research, and the h-index of the corresponding author was recorded for each included article. Bivariate and multivariate logistic regression was used to determine statistical relationship between industry support and exploratory factors (α=.05). RESULTS A 6% increase in the chance of industry investment was observed over the years (odds ratio [OD]=1.06; P<.001). Studies from North America (OD=4.87; P<.001) and Europe (OD=3.13; P<.001) were more likely to receive industry funding. Data also revealed a direct relationship between the increasing number of institutions involved in the study and the probability of industry funding (OD=1.21; P<.001). Animal studies (OD=2.26; P<.001) about surgical procedures and prosthodontic topics (OD=1.40; P=.044) stood out for having greater industry support. Researchers with an h-index between 31 and 40 were more likely to receive industry financial support (OD=2.46; P=.001). CONCLUSIONS Industry support for dental implant research was closely aligned with the continent of origin, interinstitute collaboration, type and topic of research, and the h-index of the corresponding author.
Collapse
Affiliation(s)
- Marta M A Pereira
- PhD student, Department of Dental Materials and Prosthodontics, School of Dentistry at Araraquara, Sao Paulo State University (UNESP), Araraquara, São Paulo, Brazil
| | - Caroline Dini
- PhD student, Department of Prosthodontics and Periodontology, Piracicaba Dental School, University of Campinas (UNICAMP), Piracicaba, São Paulo, Brazil
| | - João G S Souza
- Professor, Dental Research Division, Guarulhos University (UNG), Guarulhos, São Paulo, Brazil; Dental Science School (Faculdade de Ciências Odontológicas - FCO), Montes Claros, Minas Gerais, Brazil
| | - Valentim A R Barão
- Associate Professor, Department of Prosthodontics and Periodontology, Piracicaba Dental School, University of Campinas (UNICAMP), Piracicaba, São Paulo, Brazil
| | - Erica D de Avila
- Professor, Dental Research Division, Guarulhos University (UNG), Guarulhos, São Paulo, Brazil; Graduate Program Professor, Department of Dental Materials and Prosthodontics, School of Dentistry at Araraquara, São Paulo State University (UNESP), Araraquara, São Paulo, Brazil.
| |
Collapse
|
4
|
Cazzola M, Stolz D, Bafadhel M, Rogliani P. Understanding evidence from randomised controlled trials and meta-analyses: a comparative overview. Eur Respir J 2024; 63:2400694. [PMID: 38816037 DOI: 10.1183/13993003.00694-2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2024] [Accepted: 04/18/2024] [Indexed: 06/01/2024]
Affiliation(s)
- Mario Cazzola
- Unit of Respiratory Medicine, Department of Experimental Medicine, University of Rome 'Tor Vergata', Rome, Italy
| | - Daiana Stolz
- Department of Pneumology, University Medical Center Freiburg, University of Freiburg, Freiburg, Germany
| | - Mona Bafadhel
- King's Centre of Lung Health, School of Immunology and Microbial Sciences, Faculty of Life Sciences and Medicine, King's College London, London, UK
| | - Paola Rogliani
- Unit of Respiratory Medicine, Department of Experimental Medicine, University of Rome 'Tor Vergata', Rome, Italy
| |
Collapse
|
5
|
Dubin JA, Hameed D, Baksh N, Bains SS, Mont MA, Nace J, Delanois RE, Golladay G. Impact of Reporting Bias, Conflict of Interest, and Funding Sources on Quality of Orthopaedic Research. J Arthroplasty 2024; 39:1348-1352. [PMID: 37972663 DOI: 10.1016/j.arth.2023.11.017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/14/2023] [Revised: 11/06/2023] [Accepted: 11/10/2023] [Indexed: 11/19/2023] Open
Abstract
BACKGROUND Influence of factors like reporting outcomes, conflicts of interest, and funding sources on study outcomes, particularly positive outcomes in orthopedics, remains underexplored. As transparency of partnerships in orthopaedic surgery through conflicts of interest statements has increased over the years, there has been a lack of focus on the value of these partnerships in influencing study outcomes. We aimed to investigate the associations between reporting outcomes, conflicts of interest, and sources of funding on study outcomes. METHODS We reviewed articles published in 1 year in The Journal of Bone and Joint Surgery, The American Journal of Sports Medicine, and The Journal of Arthroplasty. The abstracts were examined for appropriate inclusion, while the authors' names, academic degrees, funding disclosures, and departmental and institutional affiliations were redacted. There were a total of 1,351 publications reviewed from January 1, 2021 to December 31, 2021. RESULTS A significant association was found between positive outcomes and reported conflicts of interest (75% versus 25%, P < .001). Likewise, conflicts of interest showed significant association with industry-sponsored studies (88% versus 12%, P < .001) and evidence level > II (72% versus 28%, P < .001). Industry-sponsored research accounted for the highest percentage of studies involving a conflict of interest (88%) and level I studies (12%). CONCLUSIONS Conflicts of interest are significantly associated with positive outcomes in orthopaedics. Sponsored studies were more inclined to have conflicts of interest and accounted for the majority of level I studies.
Collapse
Affiliation(s)
- Jeremy A Dubin
- Rubin Institute for Advanced Orthopedics Sinai Hospital of Baltimore, Baltimore, Maryland
| | - Daniel Hameed
- Rubin Institute for Advanced Orthopedics Sinai Hospital of Baltimore, Baltimore, Maryland
| | - Nayeem Baksh
- Department of Orthopaedic Surgery and Rehabilitation Medicine, Downstate Medical Center, State University of New York (SUNY), Brooklyn, New York
| | - Sandeep S Bains
- Rubin Institute for Advanced Orthopedics Sinai Hospital of Baltimore, Baltimore, Maryland
| | - Michael A Mont
- Rubin Institute for Advanced Orthopedics Sinai Hospital of Baltimore, Baltimore, Maryland
| | - James Nace
- Rubin Institute for Advanced Orthopedics Sinai Hospital of Baltimore, Baltimore, Maryland
| | - Ronald E Delanois
- Rubin Institute for Advanced Orthopedics Sinai Hospital of Baltimore, Baltimore, Maryland
| | - Gregory Golladay
- Department of Orthopaedic Surgery and Rehabilitation Medicine, Downstate Medical Center, State University of New York (SUNY), Brooklyn, New York; Department of Orthopaedic Surgery, Virginia Commonwealth University Health, Richmond, Virginia
| |
Collapse
|
6
|
Gonzalez-Del-Hoyo M, Mas-Llado C, Siquier-Padilla J, Blaya-Peña L, Coughlan JJ, Peral V, Rossello X. A systematic assessment of the characteristics of randomized controlled trials cited by acute coronary syndrome clinical practice guidelines. EUROPEAN HEART JOURNAL. QUALITY OF CARE & CLINICAL OUTCOMES 2024; 10:176-188. [PMID: 37296213 DOI: 10.1093/ehjqcco/qcad034] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/22/2023] [Revised: 05/23/2023] [Accepted: 06/08/2023] [Indexed: 06/12/2023]
Abstract
AIMS The aim of this study was to describe the methodological features of the randomized controlled trials (RCTs) cited in American and European clinical practice guidelines (CPGs) for ST elevation myocardial infarction (STEMI) and non-ST elevation acute coronary syndrome (NSTE-ACS). METHODS AND RESULTS Out of 2128 non-duplicated references cited in the 2013 and 2014 American College of Cardiology/American Heart Association and 2017 and 2020 European Society of Cardiology CPGs for STEMI and NSTE-ACS, we extracted data for 407 RCTs (19.1% of total references). The majority were multicenter studies (81.8%), evaluated pharmacological interventions (63.1%), had a 2-arm (82.6%), and superiority (90.4%) design. Most RCTs (60.2%) had an active comparator, and 46.2% were funded by industry. The median observed sample size was 1001 patients (84.2% of RCTs achieved ≥80% of the intended sample size). Most RCTs had a single primary outcome (90.9%), which was a composite in just over half (51.9%). Among the RCTs testing for superiority, 44.0% reported a P-value of ≥0.05 for the primary outcome and 61.9% observed a risk reduction of >15%. The observed treatment effect was lower-than-expected in 67.6% of RCTs, with 34.4% having at least a 20% lower-than-expected treatment effect. The calculated post hoc statistical power was ≥80% for 33.9% of cited RCTs. CONCLUSIONS This analysis demonstrates that RCTs cited by CPGs can still have significant methodological issues and limitations, highlighting that a better understanding of the methodological aspects of RCTs is crucial in order to formulate recommendations relevant to clinical practice.
Collapse
Affiliation(s)
- Maribel Gonzalez-Del-Hoyo
- Health Research Institute of the Balearic Islands (IdISBa), Carretera de Valldemossa, 79, 07120 Palma, Spain
- Cardiology Department, Hospital Universitari Son Espases, Carretera de Valldemossa, 79, 07120 Palma, Spain
| | - Caterina Mas-Llado
- Health Research Institute of the Balearic Islands (IdISBa), Carretera de Valldemossa, 79, 07120 Palma, Spain
- Cardiology Department, Hospital de Manacor, Carretera Manacor-Alcudia, 07500 Manacor, Spain
- Facultad de Medicina, Universitat de les Illes Balears (UIB), Carretera de Valldemossa, 79, 07120 Palma, Spain
| | - Joan Siquier-Padilla
- Cardiology Department, Hospital Universitari Son Espases, Carretera de Valldemossa, 79, 07120 Palma, Spain
| | - Laura Blaya-Peña
- Cardiology Department, Hospital Universitari Son Espases, Carretera de Valldemossa, 79, 07120 Palma, Spain
| | - J J Coughlan
- Cardiovascular Research Institute, Universtiy of Medicine and Health Sciences, Mater Private Network, D07 KWR1 Dublin, Ireland
| | - Vicente Peral
- Health Research Institute of the Balearic Islands (IdISBa), Carretera de Valldemossa, 79, 07120 Palma, Spain
- Cardiology Department, Hospital Universitari Son Espases, Carretera de Valldemossa, 79, 07120 Palma, Spain
- Facultad de Medicina, Universitat de les Illes Balears (UIB), Carretera de Valldemossa, 79, 07120 Palma, Spain
| | - Xavier Rossello
- Health Research Institute of the Balearic Islands (IdISBa), Carretera de Valldemossa, 79, 07120 Palma, Spain
- Cardiology Department, Hospital Universitari Son Espases, Carretera de Valldemossa, 79, 07120 Palma, Spain
- Facultad de Medicina, Universitat de les Illes Balears (UIB), Carretera de Valldemossa, 79, 07120 Palma, Spain
- Centro Nacional de Investigaciones Cardiovasculares (CNIC), Calle Melchor Fernández Almagro, 3, 28029 Madrid, Spain
| |
Collapse
|
7
|
Doernberg S, Truog R. Spheres of Morality: The Ethical Codes of the Medical Profession. THE AMERICAN JOURNAL OF BIOETHICS : AJOB 2023; 23:8-22. [PMID: 36607309 DOI: 10.1080/15265161.2022.2160514] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/17/2023]
Abstract
The medical profession contains five "spheres of morality": clinical care, clinical research, scientific knowledge, population health, and the market. These distinct sets of normative commitments require physicians to act in different ways depending on the ends of the activity in question. For example, a physician-scientist emphasizes patients' well-being in clinic, prioritizes the scientific method in lab, and seeks to maximize shareholder returns as a board member of a pharmaceutical firm. Physicians increasingly occupy multiple roles in healthcare and move between them frequently, creating the possibility of conflict between the ethical obligations of their various roles. This paper examines the entire moral landscape of medicine through the lens of role morality. It develops a novel framework that helps physicians recognize how their moral commitments depend on the nature and context of the situation, clarifies ethical conflicts that physicians face, and concludes with ideas for resolving these conflicts.
Collapse
|
8
|
Lyman GH, Kuderer NM. Perception, Cognition and Thought: Part III: Reasoning, Judgement and Decision-Making. Cancer Invest 2023; 41:699-703. [PMID: 37467515 DOI: 10.1080/07357907.2023.2238944] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Accepted: 07/18/2023] [Indexed: 07/21/2023]
Affiliation(s)
- Gary H Lyman
- Editor-in-Chief, Division of Public Health Sciences, Fred Hutchinson Cancer Center, Seattle, WA, USA
| | - Nicole M Kuderer
- Deputy Editor-in-Chief, Cancer Investigation, Advanced Cancer Research Group, Kirkland, WA, USA
| |
Collapse
|
9
|
Heydari A, Shakiba B, Moradi A, Soofian SE, Abian N, Heidari K, Maghsoudi R. Pharmaceutical industry funding and chemotherapy trials for prostate cancer: A systematic review. Cancer Treat Res Commun 2023; 36:100739. [PMID: 37419057 DOI: 10.1016/j.ctarc.2023.100739] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2023] [Revised: 06/16/2023] [Accepted: 06/24/2023] [Indexed: 07/09/2023]
Abstract
INTRODUCTION Clinical trials are increasingly supported by industries while previous studies have shown that industry-supported studies have more favorable results than studies with other sources of funding. In the present study, we investigated the association of industrial funding on the results of clinical trials regarding chemotherapy in prostate cancer. METHODS A systematic literature search was performed in the Cochrane Library, MEDLINE, and EMBASE to identify clinical trials comparing chemotherapy with treatments such as hormone therapy, surgery, radiotherapy, and placebo in patients with metastatic or non-metastatic prostate cancer. Data were extracted by two reviewers on the financial resources and the positive or negative results of chemotherapy in each study. The quality of articles was evaluated and compared based on Cochrane Critical Appraisal Tool. The trials were divided into two groups; industry funded and those not funded by industry. Association of industry funding and positive outcome was presented as odds ratio. RESULTS In this study, out of the 91 studies, 80.2% were funded by pharmaceutical companies and 19.8% were funded by government agencies. The end result of 61.6% of the studies funded by pharmaceutical companies was an increase in survival due to chemotherapy, whereas only 27.8% of the studies sponsored by government agencies reported positive results (P-value=0.010). In fact, industry-funded trials more often presented statistically significant positive results for survival (OR: 4.17; CI, 1.34-12.99). In general, there was no significant difference in the degree of bias between the two groups. CONCLUSION According to this study, despite of the similar quality of studies funded by pharmaceutical companies and government agencies, positive results were more common in studies related to pharmaceutical companies. Therefore, this point should be taken into account when making a decision on the best treatment approach.
Collapse
Affiliation(s)
- Amirreza Heydari
- School of medicine, Iran university of medical sciences, Tehran, Iran
| | - Behnam Shakiba
- Department of Urology, Firoozgar Hospital, School of Medicine, Iran University of Medical Sciences; Firoozgar Clinical Research Development Center, Iran University of Medical Sciences.
| | - Asaad Moradi
- Department of Urology, Firoozgar Hospital, School of Medicine, Iran University of Medical Sciences
| | - Saeed Esmaeil Soofian
- Hasheminejad Kidney Center, School of medicine, Iran University of Medical Sciences, Tehran, Iran
| | - Nasrollah Abian
- Azar Hospital, Golestan University of Medical Sciences and Health Servicess, Gorgan, Iran
| | - Kazem Heidari
- Clinical Trial Center, Tehran University of Medical Sciences, Tehran, Iran
| | - Robab Maghsoudi
- Department of Urology, Firoozgar Hospital, School of Medicine, Iran University of Medical Sciences
| |
Collapse
|
10
|
Perneger T, Gayet-Ageron A. Evidence of Lack of Treatment Efficacy Derived From Statistically Nonsignificant Results of Randomized Clinical Trials. JAMA 2023; 329:2050-2056. [PMID: 37338877 PMCID: PMC10282886 DOI: 10.1001/jama.2023.8549] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/14/2022] [Accepted: 05/01/2023] [Indexed: 06/21/2023]
Abstract
Importance Many randomized clinical trials yield statistically nonsignificant results. Such results are difficult to interpret within the dominant statistical framework. Objective To estimate the strength of evidence in favor of the null hypothesis of no effect vs the prespecified effectiveness hypothesis among nonsignificant primary outcome results of randomized clinical trials by application of the likelihood ratio. Design, Setting, and Participants Cross-sectional study of statistically nonsignificant results for primary outcomes of randomized clinical trials published in 6 leading general medical journals in 2021. Outcome measures The likelihood ratio for the null hypothesis of no effect vs the effectiveness hypothesis stated in the trial protocol (alternate hypothesis). The likelihood ratio quantifies the support that the data provide to one hypothesis vs the other. Results In 130 articles that reported 169 statistically nonsignificant results for primary outcomes, 15 results (8.9%) favored the alternate hypothesis (likelihood ratio, <1), and 154 (91.1%) favored the null hypothesis of no effect (likelihood ratio, >1). For 117 (69.2%), the likelihood ratio exceeded 10; for 88 (52.1%), it exceeded 100; and for 50 (29.6%), it exceeded 1000. Likelihood ratios were only weakly correlated with P values (Spearman r, 0.16; P = .045). Conclusions A large proportion of statistically nonsignificant primary outcome results of randomized clinical trials provided strong support for the hypothesis of no effect vs the alternate hypothesis of clinical efficacy stated a priori. Reporting the likelihood ratio may improve the interpretation of clinical trials, particularly when observed differences in the primary outcome are statistically nonsignificant.
Collapse
Affiliation(s)
- Thomas Perneger
- Division of Clinical Epidemiology, Geneva University Hospitals, and Faculty of Medicine, University of Geneva, Geneva, Switzerland
| | - Angèle Gayet-Ageron
- Division of Clinical Epidemiology, Geneva University Hospitals, and Faculty of Medicine, University of Geneva, Geneva, Switzerland
| |
Collapse
|
11
|
Alawi SA, Grigorescu O, Bota O, Dragu A. [The Impact of Industrial Sponsorship on Research Findings in Randomised Clinical Trials in Plastic and Aesthetic Surgery: a Cross- Section Analysis of the Past 12 Years]. HANDCHIR MIKROCHIR P 2023; 55:132-139. [PMID: 36758580 DOI: 10.1055/a-1996-1894] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/11/2023] Open
Abstract
BACKGROUND Industry funding of research studies can cause the results in the field of plastic, reconstructive and aesthetic surgery to be biased towards the sponsors. This paper aims to review randomised clinical trials (RCTs) published by leading international journals in plastic, reconstructive and aesthetic surgery with respect to characteristics and possible industry-friendly conclusions in favour of those funding the study. MATERIAL AND METHODS Characteristic data and funding of RCTs published by leading international plastic surgery journals from January 2010 to January 2022 were evaluated. The studies were categorised based on their thematic focus and the results in light of the existing funding. RESULTS A total of 381 RCTs were included in the analysis, 68.5% of which were from a university (n=261). The larger proportion of studies did not disclose any information about the research funding. While 26% (n=98) of the studies analysed were funded by companies, 9.7% (n=37) were funded by universities/the government (p<0.05, 95% confidence interval). The overall focus of private clinics was aesthetics (n=32). Generally, the topic aesthetics (n=153) was funded by industry in 28 of 100 cases (28% industry/private sponsors compared with 9% university/government). Regarding reconstruction/burns/hand surgery, 9.2% of studies were funded by companies/private sponsors. Funding for research by university-based institutions was industry-sponsored in 25% (n=64) of cases. Regarding industry-sponsored studies, 73% (n=71) of the results were product-friendly in their conclusion (p<0.05, 95% confidence interval) and placed their thematic focus on therapeutic drug/product application (n=50, 75%). CONCLUSION In terms of funding, industry and private sponsors take a leading role both in reconstruction and aesthetics. Nonetheless, the majority of the evaluated studies were not financed by the private sector. In cases where the research was funded by private companies, the results were significantly in favour of the product. Aesthetics as a focus is currently more frequently funded by industry than reconstruction, burn, and hand surgery.
Collapse
Affiliation(s)
- Seyed Arash Alawi
- UniversitätsCentrum für Orthopädie, Unfall- & Plastische Chirurgie Abteilung für Plastische und Handchirurgie Universitätsklinikum Carl Gustav Carus, TU Dresden, Dresden, Germany
| | - Oana Grigorescu
- UniversitätsCentrum für Orthopädie, Unfall- & Plastische Chirurgie Abteilung für Plastische und Handchirurgie Universitätsklinikum Carl Gustav Carus, TU Dresden, Dresden, Germany
| | - Olimpiu Bota
- UniversitätsCentrum für Orthopädie, Unfall- & Plastische Chirurgie Abteilung für Plastische und Handchirurgie Universitätsklinikum Carl Gustav Carus, TU Dresden, Dresden, Germany
| | - Adrian Dragu
- UniversitätsCentrum für Orthopädie, Unfall- & Plastische Chirurgie Abteilung für Plastische und Handchirurgie Universitätsklinikum Carl Gustav Carus, TU Dresden, Dresden, Germany
| |
Collapse
|
12
|
Rubagumya F, Mutebi M, Manirakiza A, Abdihamid O, Mushonga M, Vanderpuye V, Hammad N, Booth CM. Pharmaceutical industry relationships with oncologists in sub-Saharan Africa. Lancet Oncol 2023; 24:e96-e101. [PMID: 36725154 DOI: 10.1016/s1470-2045(22)00639-8] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2022] [Revised: 09/29/2022] [Accepted: 10/03/2022] [Indexed: 02/03/2023]
Abstract
Health-care systems in sub-Saharan Africa are considered to be new markets for pharmaceutical companies. This perception is particularly relevant within oncology, as the pharmaceutical industry has changed strategic priorities in the past 10 years to focus on cancer. Since the 1930s, pharmaceutical companies have used advertisements, sample drugs, gifts, paid speaking engagements, advisory boards, and trips to conferences to influence clinical practice and policy. A large amount of literature describes the commonness of these practices and their effects on the behaviour of doctors. However, these data come almost exclusively from high-income countries. Industry-doctor relationships are increasingly common in sub-Saharan Africa and other low-income and middle-income countries. Although there are undoubtedly risks of industry engagement in low-income and middle-income countries, many programmes with educational, research, and clinical value would not occur in these countries without industry support. Thus, what is known about these relationships in high-income countries will not necessarily apply in low-income and middle-income countries. There is a need for widespread discussion about industry-oncologist interactions across the African continent and context-specific data to understand the potential risks and benefits of these relationships.
Collapse
Affiliation(s)
- Fidel Rubagumya
- Department of Oncology, Rwanda Military Hospital, Kigali, Rwanda; Division of Cancer Care and Epidemiology, Queen's Cancer Research Institute, and Department of Public Health Sciences, Queen's University, Kingston, ON, Canada.
| | - Miriam Mutebi
- Department of Surgery, Aga Khan University, Nairobi, Kenya
| | | | - Omar Abdihamid
- Garissa Cancer Center, Garissa County Referral Hospital, Garissa, Kenya
| | | | - Verna Vanderpuye
- Department of Oncology, Korle Bu Teaching Hospital, Accra, Ghana
| | - Nazik Hammad
- Department of Oncology, Queen's University, Kingston, ON, Canada
| | - Christopher M Booth
- Division of Cancer Care and Epidemiology, Queen's Cancer Research Institute, and Department of Public Health Sciences, Queen's University, Kingston, ON, Canada
| |
Collapse
|
13
|
Wilson MG, Palmer E, Asselbergs FW, Harris SK. Integrated rapid-cycle comparative effectiveness trials using flexible point of care randomisation in electronic health record systems. J Biomed Inform 2023; 137:104273. [PMID: 36535604 DOI: 10.1016/j.jbi.2022.104273] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2022] [Revised: 10/13/2022] [Accepted: 12/12/2022] [Indexed: 12/23/2022]
Abstract
Whilst the Randomised Controlled Trial remains the gold standard for deriving robust causal estimates of treatment efficacy, too often a traditional design proves prohibitively expensive or cumbersome when it comes to assessing questions regarding the comparative effectiveness of routinely used treatments. As a result, patients experience variation in practice as clinicians lack the evidence needed to personalise treatments effectively. This variation may be classified as unwarranted, where existing evidence is ignored, or legitimate where in the absence of evidence, clinicians rely on experience, expert opinion, and inferred principles from basic science to make decisions. We argue that within the right ethical and technological framework, legitimate variation can be transformed into a mechanism for evidence generation and learning. Learning Health Systems which harness existing variation in practice, represent a novel approach for generating evidence from everyday clinical practice. The development of these systems has gained traction due to the increased availability of modern Electronic Health Record Systems. However, despite their promise, overcoming hurdles to successfully integrating clinical trials within Learning Health Systems has proven challenging. This article describes the origins of integrated clinical trials and explores two main barriers to their further implementation - how best to obtain informed consent from patients to participate in routine comparative effectiveness research, and how to automate and integrate randomisation into a clinical workflow. Having described these barriers, we present a potential solution in the form of a research pipeline using a novel form of flexible point-of-care randomisation to allow clinicians and patients to participate in studies where there is clinical equipoise.
Collapse
Affiliation(s)
- Matthew G Wilson
- Institute of Health Informatics, Faculty of Population Health Sciences, University College London, UK.
| | - Edward Palmer
- Bloomsbury Institute of Intensive Care Medicine, University College London, UK; Whittington Hospital NHS Trust, UK
| | - Folkert W Asselbergs
- Institute of Cardiovascular Science and Institute of Health Informatics, Faculty of Population Health Sciences, University College London, UK; Department of Cardiology, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, the Netherlands
| | - Steve K Harris
- Institute of Health Informatics, Faculty of Population Health Sciences, University College London, UK; Critical Care Department, University College London Hospital, UK
| |
Collapse
|
14
|
Fadoul Y, Haddad C, Habib J, Zoghbi M. Pharmaceutical brochures in Lebanon: do they meet WHO recommendations? BMC PRIMARY CARE 2022; 23:314. [PMID: 36474170 PMCID: PMC9724251 DOI: 10.1186/s12875-022-01930-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/27/2022] [Accepted: 11/28/2022] [Indexed: 06/17/2023]
Abstract
BACKGROUND Drug promoting brochures can influence physicians prescription patterns. The validity of the info presented in these brochures is of major importance. World Health Organization (WHO) issued criteria to guarantee validity, equity and ethical presentation of data in medical brochures. This study aims to evaluate the quality and the validity of information presented in the pharmaceutical brochures distributed among family physicians in Lebanon. METHODS Pharmaceutical brochures were randomly collected at the family medicine center in Hôtel Dieu de France hospital in Beirut - Lebanon. These brochures were evaluated in reference to the WHO ethical criteria for medicinal drug promotion and on guidelines for quality assurance of the graphs, references, texts and illustrations. RESULTS Among the 60 brochures collected, only 4 fulfilled all the WHO ethical criteria, and 24 presented less than half the required criteria. Information concerning the drug safety are the least mentioned. Only 11.8% of the presented graphs are based on studies of high methodological level. Half of the brochures presented necessary information to identify studies references which are not always retrievable. Texts present mainly brand names instead of generic names and emphasize on information reflecting the drug efficacy. CONCLUSION The pharmaceutical brochures in this study presented incomplete or invalid information. Prescribing physicians should be aware of the claims found in the brochures distributed by pharmaceutical companies and should be familiar with the principles of the evidence-based medicine to be able to critically appraise the validity of the reference studies and avoid the pitfalls in graphs reading.
Collapse
Affiliation(s)
- Yara Fadoul
- Family Medicine Department, Saint Joseph University, Beirut, Lebanon
| | - Chadia Haddad
- Psychiatric Hospital of the Cross, Jal Eddib, Lebanon
- INSPECT-LB (Institut National de Santé Publique, d'Épidémiologie Clinique et de Toxicologie-Liban), Beirut, Lebanon
- School of Health Sciences, Modern University for Business and Science, Beirut, Lebanon
| | - Jad Habib
- Family Medicine Department, Saint Joseph University, Beirut, Lebanon
| | - Marouan Zoghbi
- Family Medicine Department, Saint Joseph University, Beirut, Lebanon.
- Psychiatric Hospital of the Cross, Jal Eddib, Lebanon.
| |
Collapse
|
15
|
Anderson TN, Kearse LE, Shi R, Kaba A, Schmiederer IS, Huffman EM, Ritter EM, Korndorffer JR. Surgical endoscopy education research: how are we doing? Surg Endosc 2022; 36:8403-8407. [PMID: 35194666 DOI: 10.1007/s00464-022-09104-1] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2021] [Accepted: 02/07/2022] [Indexed: 01/06/2023]
Abstract
BACKGROUND Surgical endoscopy (SE), the official journal of the Society of American Gastrointestinal and Endoscopic Surgeons and the European Association for Endoscopic Surgery, is an important source of new evidence pertaining to surgical education in the field. However, qualitative deficiencies in medical education research have prompted medical education leaders to advocate for increased methodological rigor. The purpose of this study is to review the quality of education-focused research published through SE. METHODS A PubMed search examining all SE articles categorized as education-related research from 2010 to 2019 was conducted; studies not meeting inclusion criteria were excluded. Remaining publications were independently reviewed, classified, and scored by 7 raters using the medical education research study quality instrument (MERSQI). Intraclass correlation was calculated and data were examined with descriptive statistics. RESULTS A total of 227 studies met inclusion criteria. There was no significant difference in number of publications by year (average 25.88 [SD 5.6]); 60% were conducted outside of the United States, and 47% (n = 106) were funded. The average MERSQI was 12.5 (SD 2). Most studies used two-group non-random (42%, n = 96) or post/cross-sectional designs (29%, n = 65). Thirty-six (16%) were randomized controlled trials. Multi-institutional studies comprised 24% (n = 54). Of the manuscripts, 96% (n = 217) reported at least one measure of validity evidence and 28% (n = 67) described three levels of validity evidence. Studies primarily reported changes in skills or knowledge (45%, n = 103) or satisfaction or general facts (44%, n = 99), while patient-related outcomes encompassed 3% (n = 6) of studies. ICC between raters was 0.93 (CI 0.90-0.93, p < 0.001). CONCLUSIONS Based on publications to date, this journal's peer review process appears to facilitate the dissemination of education-related studies of moderate to good quality. However, there were uncovered deficits, ranging from validity evidence to study designs and level of outcomes. This journal's breadth of viewership offers a potential venue to advance education-related research.
Collapse
Affiliation(s)
- Tiffany N Anderson
- Department of Surgery, University of Florida College of Medicine, 1600 SW Archer Road, P.O. Box 100286, Gainesville, FL, 32610, USA.
| | - LaDonna E Kearse
- Department of Surgery, Stanford University School of Medicine, Stanford, CA, USA
| | - Robert Shi
- Department of Surgery, Stanford University School of Medicine, Stanford, CA, USA
| | - Aboubacar Kaba
- Department of Urology, David Geffen School of Medicine at the University of California, Los Angeles (UCLA), Los Angeles, CA, USA
| | | | - Elizabeth M Huffman
- Department of Surgery, Indiana University School of Medicine, Indianapolis, IN, USA
| | - E M Ritter
- Department of Surgery, Indiana University School of Medicine, Indianapolis, IN, USA
| | - James R Korndorffer
- Department of Surgery, Stanford University School of Medicine, Stanford, CA, USA
| |
Collapse
|
16
|
Graham SS, Karnes MS, Jensen JT, Sharma N, Barbour JB, Majdik ZP, Rousseau JF. Evidence for stratified conflicts of interest policies in research contexts: a methodological review. BMJ Open 2022; 12:e063501. [PMID: 36123074 PMCID: PMC9486359 DOI: 10.1136/bmjopen-2022-063501] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
OBJECTIVES The purpose of this study was to conduct a methodological review of research on the effects of conflicts of interest (COIs) in research contexts. DESIGN Methodological review. DATA SOURCES Ovid. ELIGIBILITY CRITERIA Studies published between 1986 and 2021 conducting quantitative assessments of relationships between industry funding or COI and four target outcomes: positive study results, methodological biases, reporting quality and results-conclusions concordance. DATA EXTRACTION AND SYNTHESIS We assessed key facets of study design: our primary analysis identified whether studies stratified industry funding or COI variables by magnitude (ie, number of COI or disbursement amount), type (employment, travel fees, speaking fees) or if they assessed dichotomous variables (ie, conflict present or absent). Secondary analyses focused on target outcomes and available effects measures. RESULTS Of the 167 articles included in this study, a substantial majority (98.2%) evaluated the effects of industry sponsorship. None evaluated associations between funding magnitude and outcomes of interest. Seven studies (4.3%) stratified industry funding based on the mechanism of disbursement or funder relationship to product (manufacturer or competitor). A fifth of the articles (19.8%) assessed the effects of author COI on target outcomes. None evaluated COI magnitude, and three studies (9.1%) stratified COI by disbursement type and/or reporting practices. Participation of an industry-employed author showed the most consistent effect on favourability of results across studies. CONCLUSIONS Substantial evidence demonstrates that industry funding and COI can bias biomedical research. Evidence-based policies are essential for mitigating the risks associated with COI. Although most policies stratify guidelines for managing COI, differentiating COIs based on the type of relationship or monetary value, this review shows that the available research has generally not been designed to assess the differential risks of COI types or magnitudes. Targeted research is necessary to establish an evidence base that can effectively inform policy to manage COI.
Collapse
Affiliation(s)
- S Scott Graham
- Department of Rhetoric & Writing, University of Texas at Austin, Austin, Texas, USA
| | - Martha S Karnes
- Department of English, The University of Texas at Austin, Austin, Texas, USA
| | - Jared T Jensen
- Department of Communication Studies, The Unviersity of Texas at Austin, Austin, TX, USA
| | - Nandini Sharma
- Department of Communication Studies, The Unviersity of Texas at Austin, Austin, TX, USA
| | - Joshua B Barbour
- Department of Communication Studies, The Unviersity of Texas at Austin, Austin, TX, USA
| | - Zoltan P Majdik
- Department of Communication, North Dakota State University, Fargo, North Dakota, USA
| | - Justin F Rousseau
- Department of Population Health and Neurology, The University of Texas at Austin Dell Medical School, Austin, Texas, USA
| |
Collapse
|
17
|
Fogarty AE, Conger A, Burnham T. Systematic reviews: Not always a pain. INTERVENTIONAL PAIN MEDICINE 2022; 1:100128. [PMID: 39239125 PMCID: PMC11373034 DOI: 10.1016/j.inpm.2022.100128] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/16/2022] [Accepted: 06/27/2022] [Indexed: 09/07/2024]
Abstract
Systematic reviews analyze the evidence surrounding a specific intervention within a population. High quality systematic reviews can help clinicians and policymakers accurately understand a treatment intervention. This article outlines the basic principles of systematic review development, including assembling a research team, defining the research question, publishing a protocol, designing and executing the search, study selection, extracting the data, assessing risk of bias, synthesizing the data and conducting a certainty assessment. In addition, we will address common pitfalls and highlight special considerations for the field of interventional pain medicine. Understanding systematic review methodology will help investigators improve their primary research and in turn, better primary literature will improve the value of high quality reviews.
Collapse
Affiliation(s)
- Alexandra E Fogarty
- Department of Orthopedic Surgery, Division of Physical Medicine & Rehabilitation, Washington University School of Medicine, St. Louis, MO, USA
| | - Aaron Conger
- Department of Physical Medicine and Rehabilitation, University of Utah School of Medicine, Salt Lake City, UT, USA
| | - Taylor Burnham
- Department of Physical Medicine and Rehabilitation, University of Utah School of Medicine, Salt Lake City, UT, USA
| |
Collapse
|
18
|
Vaugon E, Mircescu A, Caya C, Yao M, Gore G, Dendukuri N, Papenburg J. Diagnostic accuracy of rapid one-step PCR assays for detection of herpes Simplex virus -1 and -2 in cerebrospinal fluid: A systematic Review and meta-analysis. Clin Microbiol Infect 2022; 28:1547-1557. [PMID: 35718347 DOI: 10.1016/j.cmi.2022.06.004] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2021] [Revised: 04/26/2022] [Accepted: 06/03/2022] [Indexed: 11/18/2022]
Abstract
BACKGROUND Rapid and accurate diagnosis of HSV-1 and -2 (HSV1/2) in cerebrospinal fluid (CSF) is important for patient management. OBJECTIVES Summarize the diagnostic accuracy of commercial rapid sample-to-answer PCR assays (results in <90 minutes, without a separate nucleic acid extraction step) for HSV1/2 detection in CSF. DATA SOURCES Four databases (MEDLINE, EMBASE, Scopus and CENTRAL) and five conference abstract datasets from January 2012 to March 2022. STUDY ELIGIBILITY CRITERIA Diagnostic accuracy studies of FilmArray Meningitis-Encephalitis Panel™ and Simplexa™ HSV 1&2 Direct Kit compared to a PCR reference standard were included. Eligible studies provided sufficient data for the construction of a standard diagnostic accuracy two-by-two table. PARTICIPANTS Patients with suspected meningitis and/or encephalitis. ASSESSMENT OF RISK OF BIAS Two investigators independently extracted data, rated risk of bias and assessed quality using QUADAS-2. METHODS Accuracy estimates were pooled using Bayesian random effects models. RESULTS Thirty-one studies were included (27 FilmArray; 4 Simplexa), comprising 9,924 samples, with 95 HSV-1 and 247 HSV-2 infections. Pooled FilmArray sensitivities were 84.3% (95% credible interval 72.3%-93.0%) and 92.9% (95%CrI, 82.0%-98.5%) for HSV-1 and HSV-2, respectively; specificities were 99.8% (95%CrI, 99.6%-99.9%) and 99.9% (95%CrI, 99.9%-100%). Pooled Simplexa sensitivities were 97.1% (95%CrI, 88.1%-99.6%) and 97.9% (95%CrI, 89.6%-99.9%), respectively; specificities were 98.9% (95%CrI, 96.8%-99.7%) and 98.9% (95%CrI, 97.1%-99.7%). Pooled FilmArray sensitivities favored industry-sponsored studies by 10.0 and 13.0 percentage points for HSV-1 and HSV-2, respectively. Incomplete reporting frequently led to unclear risk of bias. Several FilmArray studies did not fully report true negative data leading to their exclusion. CONCLUSION Our results suggest Simplexa is accurate for HSV1/2 detection in CSF. Moderate FilmArray sensitivity for HSV-1 suggests additional testing and/or repeat CSF sampling is required for suspected HSV encephalitis when the HSV-1 result is negative. Low prevalence of HSV-1 infections limited summary estimates' precision. Underreporting of covariates limited exploration of heterogeneity.
Collapse
Affiliation(s)
- Esther Vaugon
- Division of Paediatric Infectious Diseases, Department of Paediatrics, Montreal Children's Hospital, McGill University Health Centre, Montreal, Quebec, Canada
| | | | - Chelsea Caya
- Research Institute of the McGill University Health Centre, McGill University Health Centre, Montreal, Quebec, Canada
| | - Mandy Yao
- Research Institute of the McGill University Health Centre, McGill University Health Centre, Montreal, Quebec, Canada
| | - Genevieve Gore
- Schulich Library of Physical Sciences, Life Sciences, and Engineering, McGill University Montreal, Quebec, Canada
| | - Nandini Dendukuri
- Research Institute of the McGill University Health Centre, McGill University Health Centre, Montreal, Quebec, Canada; Department of Epidemiology, Biostatistics and Occupational Health, Faculty of Medicine and Health Sciences, McGill University, Montreal, Quebec, Canada
| | - Jesse Papenburg
- Division of Paediatric Infectious Diseases, Department of Paediatrics, Montreal Children's Hospital, McGill University Health Centre, Montreal, Quebec, Canada; Research Institute of the McGill University Health Centre, McGill University Health Centre, Montreal, Quebec, Canada; Department of Epidemiology, Biostatistics and Occupational Health, Faculty of Medicine and Health Sciences, McGill University, Montreal, Quebec, Canada; Division of Microbiology, Department of Clinical Laboratory Medicine, McGill University Health Centre, Montreal, Quebec, Canada.
| |
Collapse
|
19
|
Siebert M, Gaba J, Renault A, Laviolle B, Locher C, Moher D, Naudet F. Data-sharing and re-analysis for main studies assessed by the European Medicines Agency-a cross-sectional study on European Public Assessment Reports. BMC Med 2022; 20:177. [PMID: 35590360 PMCID: PMC9119701 DOI: 10.1186/s12916-022-02377-2] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/09/2019] [Accepted: 04/13/2022] [Indexed: 01/13/2023] Open
Abstract
BACKGROUND Transparency and reproducibility are expected to be normative practices in clinical trials used for decision-making on marketing authorisations for new medicines. This registered report introduces a cross-sectional study aiming to assess inferential reproducibility for main trials assessed by the European Medicines Agency. METHODS Two researchers independently identified all studies on new medicines, biosimilars and orphan medicines given approval by the European Commission between January 2017 and December 2019, categorised as 'main studies' in the European Public Assessment Reports (EPARs). Sixty-two of these studies were randomly sampled. One researcher retrieved the individual patient data (IPD) for these studies and prepared a dossier for each study, containing the IPD, the protocol and information on the conduct of the study. A second researcher who had no access to study reports used the dossier to run an independent re-analysis of each trial. All results of these re-analyses were reported in terms of each study's conclusions, p-values, effect sizes and changes from the initial protocol. A team of two researchers not involved in the re-analysis compared results of the re-analyses with published results of the trial. RESULTS Two hundred ninety-two main studies in 173 EPARs were identified. Among the 62 studies randomly sampled, we received IPD for 10 trials. The median number of days between data request and data receipt was 253 [interquartile range 182-469]. For these ten trials, we identified 23 distinct primary outcomes for which the conclusions were reproduced in all re-analyses. Therefore, 10/62 trials (16% [95% confidence interval 8% to 28%]) were reproduced, as the 52 studies without available data were considered non-reproducible. There was no change from the original study protocol regarding the primary outcome in any of these ten studies. Spin was observed in the report of one study. CONCLUSIONS Despite their results supporting decisions that affect millions of people's health across the European Union, most main studies used in EPARs lack transparency and their results are not reproducible for external researchers. Re-analyses of the few trials with available data showed very good inferential reproducibility. TRIAL REGISTRATION https://osf.io/mcw3t/.
Collapse
Affiliation(s)
- Maximilian Siebert
- Univ Rennes, CHU Rennes, Inserm, CIC 1414 [(Centre d'Investigation Clinique de Rennes)], F-35000, Rennes, France.,Univ Rennes, CHU Rennes, Inserm, Irset (Institut de recherche en santé, environnement et travail) - UMR_S 1085, F-35000, Rennes, France
| | - Jeanne Gaba
- Univ Rennes, CHU Rennes, Inserm, CIC 1414 [(Centre d'Investigation Clinique de Rennes)], F-35000, Rennes, France.,Univ Rennes, CHU Rennes, Inserm, Irset (Institut de recherche en santé, environnement et travail) - UMR_S 1085, F-35000, Rennes, France
| | - Alain Renault
- Univ Rennes, CHU Rennes, Inserm, CIC 1414 [(Centre d'Investigation Clinique de Rennes)], F-35000, Rennes, France.,Univ Rennes, CHU Rennes, Inserm, Irset (Institut de recherche en santé, environnement et travail) - UMR_S 1085, F-35000, Rennes, France
| | - Bruno Laviolle
- Univ Rennes, CHU Rennes, Inserm, CIC 1414 [(Centre d'Investigation Clinique de Rennes)], F-35000, Rennes, France.,Univ Rennes, CHU Rennes, Inserm, Irset (Institut de recherche en santé, environnement et travail) - UMR_S 1085, F-35000, Rennes, France
| | - Clara Locher
- Univ Rennes, CHU Rennes, Inserm, CIC 1414 [(Centre d'Investigation Clinique de Rennes)], F-35000, Rennes, France.,Univ Rennes, CHU Rennes, Inserm, Irset (Institut de recherche en santé, environnement et travail) - UMR_S 1085, F-35000, Rennes, France
| | - David Moher
- Center for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
| | - Florian Naudet
- Univ Rennes, CHU Rennes, Inserm, CIC 1414 [(Centre d'Investigation Clinique de Rennes)], F-35000, Rennes, France. .,Univ Rennes, CHU Rennes, Inserm, Irset (Institut de recherche en santé, environnement et travail) - UMR_S 1085, F-35000, Rennes, France. .,Clinical Investigation Center (Inserm 1414) and Adult Psychiatry Department, Rennes University Hospital, Rennes, France.
| |
Collapse
|
20
|
Dhillon J, Khan T, Siddiqui B, Torgerson T, Ottwell R, Johnson AL, Skinner M, Buchanan P, Hartwell M, Vassar M. Analysis of Systematic Reviews in Clinical Practice Guidelines for Head and Neck Cancer. Laryngoscope 2022; 132:1976-1983. [DOI: 10.1002/lary.30051] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2021] [Revised: 12/17/2021] [Accepted: 01/25/2022] [Indexed: 11/11/2022]
Affiliation(s)
- Jaydeep Dhillon
- Rocky Vista University College of Osteopathic Medicine Parker Colorado U.S.A
| | - Taimoor Khan
- Office of Medical Student Research Oklahoma State University Center for Health Sciences Tulsa Oklahoma U.S.A
| | - Bilal Siddiqui
- Office of Medical Student Research Oklahoma State University Center for Health Sciences Tulsa Oklahoma U.S.A
| | - Trevor Torgerson
- Office of Medical Student Research Oklahoma State University Center for Health Sciences Tulsa Oklahoma U.S.A
| | - Ryan Ottwell
- Office of Medical Student Research Oklahoma State University Center for Health Sciences Tulsa Oklahoma U.S.A
- Department of Internal Medicine University of Oklahoma, School of Community Medicine Tulsa Oklahoma U.S.A
| | - Austin L. Johnson
- Office of Medical Student Research Oklahoma State University Center for Health Sciences Tulsa Oklahoma U.S.A
| | - Mason Skinner
- Office of Medical Student Research Oklahoma State University Center for Health Sciences Tulsa Oklahoma U.S.A
- Department of Otolaryngology—Head and Neck Surgery Oklahoma State University Medical Center Tulsa Oklahoma U.S.A
| | - Patrick Buchanan
- Office of Medical Student Research Oklahoma State University Center for Health Sciences Tulsa Oklahoma U.S.A
- Ascension Medical Group St. John ENT and Head and Neck Surgery Ascension St. John Tulsa Oklahoma U.S.A
| | - Micah Hartwell
- Office of Medical Student Research Oklahoma State University Center for Health Sciences Tulsa Oklahoma U.S.A
| | - Matt Vassar
- Office of Medical Student Research Oklahoma State University Center for Health Sciences Tulsa Oklahoma U.S.A
| |
Collapse
|
21
|
IDENTIFICATION OF THRESHOLD FOR LARGE (DRAMATIC) EFFECTS THAT WOULD OBVIATE RANDOMIZED TRIALS IS NOT POSSIBLE. J Clin Epidemiol 2022; 145:101-111. [PMID: 35091046 PMCID: PMC9232885 DOI: 10.1016/j.jclinepi.2022.01.016] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2021] [Revised: 01/15/2022] [Accepted: 01/20/2022] [Indexed: 01/14/2023]
Abstract
OBJECTIVE To analyze distribution of "dramatic", large treatment effects. STUDY DESIGN & SETTING Pareto distribution modeling of previously reported cohorts of 3,486 randomized trials (RCTs) that enrolled 1,532,459 patients and 730 non-randomized studies (NRS) enrolling 1,650,658 patients. RESULTS We calculated the Pareto α parameter, which determines the tail of the distribution for various starting points of distribution [odds ratiomin (ORmin)]. In default analysis using all data at ORmin ≥1, Pareto distribution fit well to the treatment effects of RCTs favoring the new treatments (P = 0.21, Kolmogorov-Smirnov test) with best α = 2.32. For NRS, Pareto fit for ORmin ≥2 with best α = 1.91. For RCTs, theoretical 99th percentile OR was 32.7. The actual 99th percentile OR was 25; which converted into relative risk (RR) = 7.1. The maximum observed effect size was OR = 121 (RR = 11.45). For NRS, theoretical 99th percentile was OR = 315. The actual 99th percentile OR was 294 (RR = 13). The maximum observed effect size was OR = 1473 (RR = 66). CONCLUSIONS The effects sizes observed in RCTs and NRS considerably overlap. Large effects are rare and there is no clear threshold for dramatic effects that would obviate future RCTs.
Collapse
|
22
|
Characteristics of completed clinical trials for spinal cord injury from 1999 to 2020. J Clin Neurosci 2021; 94:114-119. [PMID: 34863424 DOI: 10.1016/j.jocn.2021.10.002] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2021] [Accepted: 10/03/2021] [Indexed: 11/20/2022]
Abstract
Numerous clinical trials are being conducted in the field of spinal cord injury (SCI). These trials are typically registered on ClinicalTrials.gov. The objective of this study was to identify the characteristics of the completed SCI trials and characterize the potential factors associated with publication. ClinicalTrials.gov database was queried for all the completed trials on patients with SCI. Baseline characteristics of the completed trials were assessed. The publication status of these trials was identified using PubMed or Google Scholar. The secondary and primary outcomes reported in the publication were then compared to the outcomes registered in ClinicalTrial.gov. Multivariable logistic regression analysis was performed to determine the characteristics associated with publication status and time to publication. A total of 457 of 1,061 trials on SCI were completed. Of those, 60% were ultimately published. Trials that had received funding from sources besides the NIH, private industries, or the federal government were more likely to remain unpublished. The median time to publish was three years, with larger trials taking a longer time. The median sample size for completed trials was 30. Assessment of mismatch rates in primary outcomes of published data to registered outcomes was 8.9%. In SCI trials, outcomes registered on ClinicalTrial.gov often matched published results. Additionally, sample size and funding source play a significant role in the publication rate of these trials. Published data represents a reliable source for clinicians, researchers, and patients; efforts to curb publication bias and reporting bias are paramount for implementing evidence-based practices and ensure proper scientific conduct.
Collapse
|
23
|
Employees as a vector of strategic intent: an examination of corporate, social and environmental strategic intent recognition by employees. JOURNAL OF STRATEGY AND MANAGEMENT 2021. [DOI: 10.1108/jsma-04-2021-0101] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Purpose
One of the hallmarks of strategizing is having a clearly articulated vision and mission for the organization. It has been suggested that this provides a compass bearing for the organization's strategy, helps in motivation, commitment and retention of employees, serves as a guide to internal sensemaking and decision-making, has a potential performance effect, helps establish the identity of the organization and positions its desired reputation. The compass bearing role is important because it guides the selection of the goals and strategic orientation of the organization which in turn shapes its overall strategy and much of its internal decision making. The inspirational role is important because it helps to motivate and engage employees and other stakeholders.
Design/methodology/approach
This study provides a more rigorous indication as to whether employees can, in the first instance, recognize and distinguish their corporate and environmental strategy from that of their competitors within their own industry and random other companies from other industries. This first issue addresses, to a degree, if and why, such strategic communiqués are effective inside a range of different organizations. Secondly, the authors examine whether there are any specific individual level effects that could explain variations in these responses. Finally, the authors examine the extent to which the recognition rates the authors observe, relate to how employees are rewarded through appraisals, promotions and salary increases. This helps in the authors’ understanding of the role of hard incentives versus soft motivations. The authors’ approach to assessing employee knowledge of their organization's strategy is unique. Rather than survey employees about their knowledge, the authors use a matching study and a discrete choice measurement model to assess if they can recognize their organization's strategy from those of their competitors and some other randomly selected organizations. This approach allows us to mitigate social desirability and common method biases and directly estimate the underlying behavioral model being used to assess their organization's strategy.
Findings
Overall, the authors found that few employees could correctly identify their corporate strategy statements. In the case of corporate strategy statements, the authors find that, on average, only 29 percent of employees could correctly match their company to its publicly espoused corporate strategy. When the authors look at the environmental sustainability strategy of the firm, this is worse overall, with individuals doing no better than random on average. When the authors look at company training and communication practices across the realm of different strategies, the authors see a number of factors leading to the general results. First, most of the authors’ respondents could not recall a significant effort being given to communication and training by their employer. Indeed, most communication/training is simply related to having documentation/brochures available. Second, respondents indicated that more effort is put into communicating corporate strategy to employees in a more systematic manner than communication about environmental/corporate social responsible (CSR) strategy. Third, the authors see that individuals are evaluated more on and give more weight to, evaluations relating to their ability to meet individual/group financial and market performance metrics (targets) and work as a team than their involvement in environmental and social responsibility programs. Finally, the employees studied seemed to be more confident in understanding the corporate strategy. When asked to put their corporate strategy into words – a task the authors asked respondents to do after the matching phase of the study – 40% of participants did so for the corporate strategy but only 14% did so for the environmental strategy and seven percent for the CSR strategy.
Practical implications
The primary implication of the study is that the values-mission-strategy logic of strategic motivation seems to have limited validity and with respect to the view that employees are a vector of corporate strategy. It is hard to argue that employees can be a vector for something they cannot recall or even distinguish between.
Originality/value
The study is unique in terms of (1) asking the very simple question of whether employees internalize their company's strategies and (2) in the methodological approach to examine employee knowledge and informativeness.
Collapse
|
24
|
The methodological quality was low and conclusions discordant for meta-analyses comparing proximal humerus fracture treatments: a meta-epidemiological study. J Clin Epidemiol 2021; 142:100-109. [PMID: 34718123 DOI: 10.1016/j.jclinepi.2021.10.014] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2021] [Revised: 10/08/2021] [Accepted: 10/19/2021] [Indexed: 12/15/2022]
Abstract
OBJECTIVE To investigate the association between methodological quality and reported conclusions of meta-analyses comparing operative with non-operative treatments for proximal humerus fractures. STUDY DESIGN AND SETTING Cross-sectional meta-epidemiological study. We searched EMBASE, PubMed, The Cochrane Library, and Web of Science for systematic reviews with meta-analyses comparing non-operative with operative treatments for proximal humerus fractures. Methodological quality was assessed using AMSTAR2 and the reported conclusions were scored for three outcome domains (functional outcome, quality of life, and harm) on a scale from 1 to 6. The Mann-Whitney and Kruskal-Wallis tests were used to investigate the association between methodological quality and reported conclusions. RESULTS We included 21 systematic reviews: 19 pairwise meta-analyses and 2 network meta-analyses, although there are only 8 published randomized controlled trials. Most (n = 18) of the meta-analyses were rated as critically low quality, while the remaining 1 was rated as high quality. The conclusions were discordant for all three outcome domains, even for meta-analyses reporting similar inclusion criteria. We could not perform most of the statistical tests due to the predominantly critically low quality. CONCLUSION The methodological quality was so predominantly critically low that it was not possible to evaluate the association between methodological quality and reported conclusions.
Collapse
|
25
|
Fundytus A, Wells JC, Sharma S, Hopman WM, Del Paggio JC, Gyawali B, Mukherji D, Hammad N, Pramesh CS, Aggarwal A, Sullivan R, Booth CM. Industry Funding of Oncology Randomised Controlled Trials: Implications for Design, Results and Interpretation. Clin Oncol (R Coll Radiol) 2021; 34:28-35. [PMID: 34479769 DOI: 10.1016/j.clon.2021.08.003] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2021] [Revised: 06/24/2021] [Accepted: 08/02/2021] [Indexed: 11/19/2022]
Abstract
AIMS Most randomised controlled trials (RCTs) in oncology are now funded by the pharmaceutical industry. We explore the extent to which RCT design, results and interpretation differ between industry-funded and non-industry-funded RCTs. MATERIALS AND METHODS In this cross-sectional analysis, a structured literature search was used to identify all oncology RCTs published globally during 2014-2017. Industry funding was identified based on explicit statements in the publication. Descriptive statistics were used to compare elements of trial methodology and output between industry- and non-industry-funded RCTs. RESULTS The study sample included 694 RCTs; 71% were funded by industry. Industry-funded trials were more likely to test systemic therapy (97% versus 62%; P < 0.001), palliative-intent therapy (71% versus 41%; P < 0.001) and study breast cancer (20% versus 12%; P < 0.001). Industry-funded trials were larger (median sample size 474 versus 375; P < 0.001) and more likely to meet their primary end point (49% versus 41%; P < 0.001). Among positive trials, there were no differences in the magnitude of benefit between industry- and non-industry-funded RCTs. Trials funded by industry were published in journals that had a significantly higher median impact factor (21, interquartile range 7, 28) than non-industry-funded trials (impact factor 12, interquartile range 5, 24; P = 0.005); this persisted when adjusted for whether a trial was positive or negative. CONCLUSIONS The vast majority of oncology RCTs are now funded by industry. Industry-funded trials are larger, more likely to be positive, predominantly test systemic therapies in the palliative setting and are published in higher impact journals than trials without industry support.
Collapse
Affiliation(s)
- A Fundytus
- Division of Cancer Care and Epidemiology, Queen's University Cancer Research Institute, Kingston, Ontario, Canada; Department of Oncology, Queen's University, Kingston, Ontario, Canada
| | - J C Wells
- Division of Cancer Care and Epidemiology, Queen's University Cancer Research Institute, Kingston, Ontario, Canada; Department of Oncology, Queen's University, Kingston, Ontario, Canada
| | - S Sharma
- Division of Cancer Care and Epidemiology, Queen's University Cancer Research Institute, Kingston, Ontario, Canada
| | - W M Hopman
- Department of Public Health Sciences, Queen's University, Kingston, Ontario, Canada
| | - J C Del Paggio
- Department of Oncology, Northern Ontario School of Medicine, Thunder Bay, Ontario, Canada
| | - B Gyawali
- Division of Cancer Care and Epidemiology, Queen's University Cancer Research Institute, Kingston, Ontario, Canada; Department of Oncology, Queen's University, Kingston, Ontario, Canada; Department of Public Health Sciences, Queen's University, Kingston, Ontario, Canada
| | - D Mukherji
- American University of Beirut Medical Center, Beirut, Lebanon
| | - N Hammad
- Department of Oncology, Queen's University, Kingston, Ontario, Canada
| | - C S Pramesh
- Tata Memorial Centre, Homi Bhabha National Institute, Mumbai, India
| | - A Aggarwal
- Institute of Cancer Policy, King's College London, London, UK; London School of Hygiene and Tropical Medicine, London, UK
| | - R Sullivan
- Institute of Cancer Policy, King's College London, London, UK
| | - C M Booth
- Division of Cancer Care and Epidemiology, Queen's University Cancer Research Institute, Kingston, Ontario, Canada; Department of Oncology, Queen's University, Kingston, Ontario, Canada; Department of Public Health Sciences, Queen's University, Kingston, Ontario, Canada.
| |
Collapse
|
26
|
Robinson NB, Fremes S, Hameed I, Rahouma M, Weidenmann V, Demetres M, Morsi M, Soletti G, Di Franco A, Zenati MA, Raja SG, Moher D, Bakaeen F, Chikwe J, Bhatt DL, Kurlansky P, Girardi LN, Gaudino M. Characteristics of Randomized Clinical Trials in Surgery From 2008 to 2020: A Systematic Review. JAMA Netw Open 2021; 4:e2114494. [PMID: 34190996 PMCID: PMC8246313 DOI: 10.1001/jamanetworkopen.2021.14494] [Citation(s) in RCA: 54] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/16/2022] Open
Abstract
IMPORTANCE Randomized clinical trials (RCTs) provide the highest level of evidence to evaluate 2 or more surgical interventions. Surgical RCTs, however, face unique challenges in design and implementation. OBJECTIVE To evaluate the design, conduct, and reporting of contemporary surgical RCTs. EVIDENCE REVIEW A literature search performed in the 2 journals with the highest impact factor in general medicine as well as 6 key surgical specialties was conducted to identify RCTs published between 2008 and 2020. All RCTs describing a surgical intervention in both experimental and control arms were included. The quality of included data was assessed by establishing an a priori protocol containing all the details to extract. Trial characteristics, fragility index, risk of bias (Cochrane Risk of Bias 2 Tool), pragmatism (Pragmatic Explanatory Continuum Indicator Summary 2 [PRECIS-2]), and reporting bias were assessed. FINDINGS A total of 388 trials were identified. Of them, 242 (62.4%) were registered; discrepancies with the published protocol were identified in 81 (33.5%). Most trials used superiority design (329 [84.8%]), and intention-to-treat as primary analysis (221 [56.9%]) and were designed to detect a large treatment effect (50.0%; interquartile range [IQR], 24.7%-63.3%). Only 123 trials (31.7%) used major clinical events as the primary outcome. Most trials (303 [78.1%]) did not control for surgeon experience; only 17 trials (4.4%) assessed the quality of the intervention. The median sample size was 122 patients (IQR, 70-245 patients). The median follow-up was 24 months (IQR, 12.0-32.0 months). Most trials (211 [54.4%]) had some concern of bias and 91 (23.5%) had high risk of bias. The mean (SD) PRECIS-2 score was 3.52 (0.65) and increased significantly over the study period. Most trials (212 [54.6%]) reported a neutral result; reporting bias was identified in 109 of 211 (51.7%). The median fragility index was 3.0 (IQR, 1.0-6.0). Multiplicity was detected in 175 trials (45.1%), and only 35 (20.0%) adjusted for multiple comparisons. CONCLUSIONS AND RELEVANCE In this systematic review, the size of contemporary surgical trials was small and the focus was on minor clinical events. Trial registration remained suboptimal and discrepancies with the published protocol and reporting bias were frequent. Few trials controlled for surgeon experience or assessed the quality of the intervention.
Collapse
Affiliation(s)
- N. Bryce Robinson
- Department of Cardiothoracic Surgery, Weill Cornell Medicine, New York, New York
| | - Stephen Fremes
- Schulich Heart Centre, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, ON, Canada
| | - Irbaz Hameed
- Department of Cardiothoracic Surgery, Weill Cornell Medicine, New York, New York
- Division of Cardiothoracic Surgery, Yale University School of Medicine, New Haven, Connecticut
| | - Mohamed Rahouma
- Department of Cardiothoracic Surgery, Weill Cornell Medicine, New York, New York
| | - Viola Weidenmann
- Department of Cardiothoracic Surgery, Weill Cornell Medicine, New York, New York
| | - Michelle Demetres
- Samuel J. Wood Library and C.V. Starr Biomedical Information Centre, Weill Cornell Medicine, New York, New York
| | - Mahmoud Morsi
- Department of Cardiothoracic Surgery, Weill Cornell Medicine, New York, New York
| | - Giovanni Soletti
- Department of Cardiothoracic Surgery, Weill Cornell Medicine, New York, New York
| | - Antonino Di Franco
- Department of Cardiothoracic Surgery, Weill Cornell Medicine, New York, New York
| | - Marco A. Zenati
- BHS Department of Cardiothoracic Surgery, West Roxbury, Massachusetts
| | - Shahzad G. Raja
- Department of Cardiac Surgery, Harefield Hospital, London, United Kingdom
| | - David Moher
- Ottawa Methods Centre, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Faisal Bakaeen
- Department of Thoracic Surgery, Heart and Vascular Institute, Cleveland Clinic, Cleveland, Ohio
- Department of Cardiovascular Surgery, Heart and Vascular Institute, Cleveland Clinic, Cleveland, Ohio
| | - Joanna Chikwe
- Department of Cardiac Surgery, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, California
| | - Deepak L. Bhatt
- Division of Cardiovascular Medicine, Brigham and Women’s Hospital, Boston, Massachusetts
| | - Paul Kurlansky
- Department of Surgery, Columbia University Medical Center, New York, New York
| | - Leonard N. Girardi
- Department of Cardiothoracic Surgery, Weill Cornell Medicine, New York, New York
| | - Mario Gaudino
- Department of Cardiothoracic Surgery, Weill Cornell Medicine, New York, New York
| |
Collapse
|
27
|
Affiliation(s)
- Eue-Keun Choi
- Department of Internal Medicine, Seoul National University Hospital, Seoul, Republic of Korea
| | - So-Ryoung Lee
- Department of Internal Medicine, Seoul National University Hospital, Seoul, Republic of Korea
| |
Collapse
|
28
|
Del Paggio JC, Berry JS, Hopman WM, Eisenhauer EA, Prasad V, Gyawali B, Booth CM. Evolution of the Randomized Clinical Trial in the Era of Precision Oncology. JAMA Oncol 2021; 7:728-734. [PMID: 33764385 PMCID: PMC7995135 DOI: 10.1001/jamaoncol.2021.0379] [Citation(s) in RCA: 133] [Impact Index Per Article: 33.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2020] [Accepted: 02/10/2021] [Indexed: 01/10/2023]
Abstract
IMPORTANCE The randomized clinical trial (RCT) in oncology has evolved since its widespread adoption in the 1970s. In recent years, concerns have emerged regarding the use of putative surrogate end points, such as progression-free survival (PFS), and marginal effect sizes. OBJECTIVE To describe contemporary trends in oncology RCTs and compare these findings with earlier eras of RCT design and output. DESIGN, SETTING, AND PARTICIPANTS Retrospective cohort study of systemic therapy RCTs in breast, colorectal, and non-small cell lung cancer published in 7 major journals between 2010 and 2020. This strategy replicates prior work and allows for comparison of trends with RCTs published between 1995 to 2004 and 2005 to 2009. MAIN OUTCOMES AND MEASURES Data on RCT design, funding, results, and reporting were extracted from the published RCT report. Findings from the current period (2010-2020) were compared with data from RCTs published from 1995 to 2004 and 2005 to 2009. Descriptive and bivariate statistics were used to analyze temporal trends. RESULTS The cohort included 298 RCTs (132 [44%] breast, 111 [37%] non-small cell lung cancer, 55 [19%] colorectal cancer). Experimental treatment included molecular inhibitor (171 of 298 [57%]), cytotoxic (83 of 298 [28%]), hormone (15 of 298 [5%]), and immune (24 of 298 [8%]) therapies. Sixty-nine percent (206 of 298) of RCTs were of palliative intent. The most common primary end point is now PFS; this has increased substantially over time (from 0% [0 of 167] to 18% [25 of 137] to 42% [125 of 298]; P < .001). Of 298 RCTs, 265 (89%) are now funded by industry (previously 95 of 167 [57%] and 107 of 137 [78%]; P < .001). Fifty-eight percent (173 of 298) of trials met their primary end point. Among positive trials, median improvement in overall survival and PFS was 3.4 and 2.9 months, respectively. More than one-third (117 of 298 [39%]) of reports used a professional medical writer; this increased substantially during the study period (from 3 of 27 [11%] in 2010 to 12 of 18 [67%] in 2020; P < .001). CONCLUSIONS AND RELEVANCE This cohort study suggests that contemporary oncology RCTs now largely measure putative surrogate end points and are almost exclusively funded by the pharmaceutical industry. The increasing role of medical writers warrants attention. To demonstrate that new cancer treatments are high value, the oncology community needs to consider the extent to which study end points and target effect size provide meaningful benefit to patients.
Collapse
Affiliation(s)
- Joseph C. Del Paggio
- Department of Medical Oncology, Thunder Bay Regional Health Sciences Centre and Northern Ontario School of Medicine, Thunder Bay, Canada
| | - John S. Berry
- Division of Cancer Care and Epidemiology, Queen’s Cancer Research Institute, Queen’s University, Kingston, Canada
| | - Wilma M. Hopman
- Kingston General Health Research Institute, Kingston, Canada
- Department of Public Health Sciences, Queen’s University, Kingston, Canada
| | | | - Vinay Prasad
- Department of Epidemiology and Biostatistics, University of California at San Francisco
| | - Bishal Gyawali
- Division of Cancer Care and Epidemiology, Queen’s Cancer Research Institute, Queen’s University, Kingston, Canada
- Department of Public Health Sciences, Queen’s University, Kingston, Canada
- Department of Oncology, Queen’s University, Kingston, Canada
| | - Christopher M. Booth
- Division of Cancer Care and Epidemiology, Queen’s Cancer Research Institute, Queen’s University, Kingston, Canada
- Department of Public Health Sciences, Queen’s University, Kingston, Canada
- Department of Oncology, Queen’s University, Kingston, Canada
| |
Collapse
|
29
|
Desai A, Kuderer NM, Lyman GH. Aligning Cancer Clinical Trials With Cancer Burden: Need for Greater Global Leadership, Resources, and Vision. JAMA Oncol 2021; 7:357-358. [PMID: 33507227 DOI: 10.1001/jamaoncol.2020.7293] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022]
Affiliation(s)
- Aakash Desai
- Division of Medical Oncology, Department of Medicine, Mayo Clinic, Rochester, Minnesota.,Division of Hematology, Department of Medicine, Mayo Clinic, Rochester, Minnesota
| | | | - Gary H Lyman
- Divisions of Public Health Sciences and Clinical Research, Fred Hutchinson Cancer Research Center, Seattle, Washington.,Division of Oncology, Department of Medicine, University of Washington, Seattle
| |
Collapse
|
30
|
The influence of industry sponsorship and conflict of interest on results and conclusions of systematic reviews regarding treatment of knee osteoarthritis. OSTEOARTHRITIS AND CARTILAGE OPEN 2021; 3:100142. [DOI: 10.1016/j.ocarto.2021.100142] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2020] [Revised: 01/31/2021] [Accepted: 02/04/2021] [Indexed: 11/23/2022] Open
|
31
|
Hussain N, Brull R, Sheehy BT, Kushelev M, Essandoh MK, Abdallah FW. The mornings after-periarticular liposomal bupivacaine infiltration does not improve analgesic outcomes beyond 24 hours following total knee arthroplasty: a systematic review and meta-analysis. Reg Anesth Pain Med 2021; 46:61-72. [PMID: 33106282 DOI: 10.1136/rapm-2020-101995] [Citation(s) in RCA: 26] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2020] [Revised: 09/14/2020] [Accepted: 09/15/2020] [Indexed: 12/19/2022]
Abstract
Periarticular local infiltration analgesia (LIA) is integral to multimodal analgesia following total knee arthroplasty (TKA); however, the duration of analgesia using traditional long-acting local anesthetics is often insufficient. LIA with slow-release liposomal bupivacaine may provide extended analgesia, but evidence of efficacy beyond the first 24 hours is conflicting. This meta-analysis compares the effects of periarticular liposomal and plain bupivacaine LIA on day 2 analgesic outcomes post-TKA. Trials comparing liposomal and plain bupivacaine LIA for TKA were sought. The two coprimary outcomes were (1) cumulative oral morphine equivalent consumption and (2) difference in area under the curve (AUC) of pooled rest pain scores on day 2 (24-48 hours) post-TKA. We also evaluated pain and analgesic consumption on day 3 (48-72 hours), functional recovery, length of hospital stay, patient satisfaction; and opioid-related side effects. Data were pooled using random-effects modeling. Seventeen trials (1836 patients) were analyzed. Comparing liposomal versus plain bupivacaine LIA for TKA failed to detect differences in morphine consumption and pain AUC on day 2 postoperatively, with mean differences of 0.54 mg (95% CI -5.09 to 6.18) and 0.08 cm/hour (95% CI -0.19 to 0.35), respectively (high-quality evidence). Secondary outcome analysis did not uncover any additional analgesic, functional or safety advantages to liposomal bupivacaine on postoperative day 2 or 3. Results indicate that liposomal and plain bupivacaine LIAs are not different for extended postoperative analgesic outcomes, including pain control, opioid consumption, as well as functional and safety outcomes on days 2 and 3 post-TKA. High-quality evidence does not support using liposomal bupivacaine LIA for TKA.
Collapse
Affiliation(s)
- Nasir Hussain
- Anesthesiology, Ohio State University Wexner Medical Center, Columbus, Ohio, USA
| | - Richard Brull
- Anesthesiology, Toronto Western Hospital, Toronto, Ontario, Canada
| | - Brendan T Sheehy
- Anesthesiology, Ohio State University Wexner Medical Center, Columbus, Ohio, USA
| | - Michael Kushelev
- Anesthesiology, Ohio State University Wexner Medical Center, Columbus, Ohio, USA
| | - Michael K Essandoh
- Anesthesiology, Ohio State University Wexner Medical Center, Columbus, Ohio, USA
| | - Faraj W Abdallah
- Anesthesiology and Pain Medicine, University of Toronto and University of Ottawa, Ottawa, Ontario, Canada
| |
Collapse
|
32
|
Mozgunov P, Jaki T. An information theoretic approach for selecting arms in clinical trials. J R Stat Soc Series B Stat Methodol 2020. [DOI: 10.1111/rssb.12391] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Affiliation(s)
| | - Thomas Jaki
- Lancaster University and University of Cambridge UK
| |
Collapse
|
33
|
Kostis JB, Dobrzynski JM. Limitations of Randomized Clinical Trials. Am J Cardiol 2020; 129:109-115. [PMID: 32560898 DOI: 10.1016/j.amjcard.2020.05.011] [Citation(s) in RCA: 27] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/27/2020] [Revised: 04/27/2020] [Accepted: 05/05/2020] [Indexed: 01/31/2023]
Abstract
Randomized clinical trials are essential for determining the efficacy of interventions, but have limitations. The types of limitations discussed in this review may be grouped in 11 categories including incorrect statistical inference, low internal or external validity, misinterpretation of the difference between frequentist and Bayesian statistical approaches, publication bias, that healthy persons with a given condition participate in clinical trials although they are not representative of the population as a whole, the rather short duration (3 to 5 years) that does not give correct estimates of the lifetime effects of the interventions or the legacy effect when participants who receive active therapy derive residual benefit after the end of the study when all participants receive active medication and the tension between the generalizability of the evidence versus the reliability of the findings of different types of clinical trials and the difficulty in applying the findings of randomized clinical trials to individual patients. These limitations are described and illustrated by examples and figures from the literature. In conclusion, this review will be useful to clinical trialists, clinical trial participants and regulatory agents.
Collapse
Affiliation(s)
- John B Kostis
- Cardiovascular Institute, Rutgers Robert Wood Johnson Medical School, New Brunswick, New Jersey.
| | - Jeanne M Dobrzynski
- Cardiovascular Institute, Rutgers Robert Wood Johnson Medical School, New Brunswick, New Jersey
| |
Collapse
|
34
|
Sturmberg JP. From probability to believability. J Eval Clin Pract 2020; 26:1081-1086. [PMID: 31140703 DOI: 10.1111/jep.13186] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/28/2019] [Accepted: 05/05/2019] [Indexed: 12/26/2022]
Affiliation(s)
- Joachim P Sturmberg
- A/Prof of General Practice, School of Medicine and Public Health, University of Newcastle, Callaghan, New South Wales, 2308, Australia.,International Society for Systems and Complexity Sciences for Health
| |
Collapse
|
35
|
Gaudino M, Hameed I, Rahouma M, Khan FM, Tam DY, Biondi-Zoccai G, Demetres M, Charlson ME, Ruel M, Crea F, Falk V, Girardi LN, Fremes S, Chikwe J. Characteristics of Contemporary Randomized Clinical Trials and Their Association With the Trial Funding Source in Invasive Cardiovascular Interventions. JAMA Intern Med 2020; 180:993-1001. [PMID: 32478821 PMCID: PMC7265124 DOI: 10.1001/jamainternmed.2020.1670] [Citation(s) in RCA: 35] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/13/2020] [Accepted: 04/14/2020] [Indexed: 02/05/2023]
Abstract
IMPORTANCE Changes in evidence-based practice and guideline recommendations depend on high-quality randomized clinical trials (RCTs). Commercial device and pharmaceutical manufacturers are frequently involved in the funding, design, conduct, and reporting of trials, the implications of which have not been recently analyzed. OBJECTIVE To evaluate the design, outcomes, and reporting of contemporary randomized clinical trials of invasive cardiovascular interventions and their association with the funding source. DESIGN, SETTING, AND PARTICIPANTS This cross-sectional study analyzed published RCTs between January 1, 2008, to May 31, 2019. The trials included those involving coronary, vascular and structural interventional cardiology, and vascular and cardiac surgical procedures. MAIN OUTCOMES AND MEASURES We assessed (1) trial characteristics, (2) finding of a statistically significant difference in the primary end point favoring the experimental intervention, (3) reporting of implied treatment advantage in trials without significant differences in primary end point, (4) existence of major discrepancies between registered and published primary outcomes, (5) number of patients whose outcomes would need to switch from a nonevent to an event to convert a significant difference in primary end point to nonsignificant, and (6) association with funding source. RESULTS Of the 216 RCTs analyzed, 115 (53.2%) reported having commercial sponsorship. Most trials had 80% power to detect an estimated treatment effect of 30%, and 128 trials (59.3%) used composite primary end points. The median (interquartile range [IQR]) sample size was 502 (204-1702) patients, and the median (IQR) follow-up duration was 12 (1.0-14.4) months. Overall, 123 trials (57.0%) reported a statistically significant difference in the primary outcome favoring the experimental intervention; reporting strategies that implied an advantage were identified in 55 (65.5%) of 84 trials that reported nonsignificant differences. Commercial sponsorship was associated with a statistically significantly greater likelihood of favorable outcomes reporting (exponent of regression coefficient β, 2.80; 95% CI, 1.09-7.18; P = .03) and with the reporting of findings that are inconsistent with the trial results. Discrepancies between the registered and published primary outcomes were found in 82 trials (38.0%), without differences in trial sponsorship. A median (IQR) number of 5 (2.8-12.5) patients experiencing a different outcome would have change statistically significant results to nonsignificant. Commercial sponsorship was associated with a greater number of patients (exponent of regression coefficient β, 1.29; 95% CI, 1.00-1.66; P = .04). CONCLUSIONS AND RELEVANCE These results suggest that contemporary RCTs of invasive cardiovascular interventions are relatively small and fragile, have short follow-up, and have limited power to detect large treatment effects. Commercial support appeared to be associated with differences in trial design, results, and reporting.
Collapse
Affiliation(s)
- Mario Gaudino
- Department of Cardiothoracic Surgery, Weill Cornell Medicine, New York, New York
| | - Irbaz Hameed
- Department of Cardiothoracic Surgery, Weill Cornell Medicine, New York, New York
| | - Mohamed Rahouma
- Department of Cardiothoracic Surgery, Weill Cornell Medicine, New York, New York
| | - Faiza M. Khan
- Department of Cardiothoracic Surgery, Weill Cornell Medicine, New York, New York
| | - Derrick Y. Tam
- Schulich Heart Centre, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Ontario, Canada
| | - Giuseppe Biondi-Zoccai
- Department of Medico-Surgical Sciences and Biotechnologies, Sapienza University of Rome, Rome, Italy
- IRCCS Neuromed, Pozzilli, Italy
| | - Michelle Demetres
- Samuel J. Wood Library and C. V. Starr Biomedical Information Centre, Weill Cornell Medicine, New York, New York
| | - Mary E. Charlson
- Division of General Internal Medicine, Weill Cornell Medical College, New York, New York
| | - Marc Ruel
- Division of Cardiac Surgery, University of Ottawa Heart Institute, Ottawa, Ontario, Canada
| | - Filippo Crea
- Fondazione Policlinico Universitario A. Gemelli IRCCS, Rome, Italy
- Department of Cardiovascular and Thoracic Sciences, Università Cattolica de Sacro Cuore, Roma, Italy
| | - Volkmar Falk
- Department of Cardiovascular Surgery, Charite, Berlin, Germany
- Department of Cardiothoracic and Vascular Surgery, German Heart Center Berlin, Berlin, Germany
- German Center for Cardiovascular Research, Berlin, Germany
- Department of Health Science and Technology, Swiss Federal Institute of Technology, Zurich, Switzerland
| | - Leonard N. Girardi
- Department of Cardiothoracic Surgery, Weill Cornell Medicine, New York, New York
| | - Stephen Fremes
- Schulich Heart Centre, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Ontario, Canada
| | - Joanna Chikwe
- Department of Cardiac Surgery, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, California
| |
Collapse
|
36
|
de Souza Gutierres B, Aguiar PN, Dourado BB, Alves AL, Matsas S, Simões AR, Del Giglio A. Evidence Strength of Pharmaceutical Industry-Funded Clinical Trials in Metastatic NSCLC: A Comparison With Other Sources of Funding. J Thorac Oncol 2020; 15:1170-1176. [PMID: 32217131 DOI: 10.1016/j.jtho.2020.03.004] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2019] [Revised: 01/15/2020] [Accepted: 03/01/2020] [Indexed: 10/24/2022]
Abstract
INTRODUCTION Clinical trials are expensive and often require funding from the pharmaceutical industry (PI). We aimed to compare studies funded by the PI with those funded by other sources in terms of costs, reported results, and strength of evidence. METHODS We searched PubMed for clinical trial reports on metastatic NSCLC published between 2012 and 2017. We divided all the studies into two groups: studies funded by the PI and those funded by other sources. The primary end point was to compare the evidence strength of each group. The secondary end points were to compare the number of patients included, the number and costs of innovative drugs studied, whether there was preferential reporting of positive results in the experimental arm, and the risk of bias. RESULTS We found 3004 studies, and of these, we analyzed 477 studies (275 sponsored by the PI and 202 funded by other sources). A total of 85,328 patients overall were included (64,434 in studies sponsored by the PI and 20,894 in studies with other funding sources; p < 0.001). The studies funded by the PI had stronger evidence (p < 0.001), evaluated more innovative therapies (72% versus 36%; p < 0.001), and resulted in a higher proportion of open-access manuscripts (63% versus 47%; p < 0.001). There was no considerable difference regarding the reporting of experimental arm superiority or the risk of bias between the two groups. CONCLUSIONS Compared with studies from other sources of funding, those funded by the PI in the lung cancer field collected stronger evidence, assessed more expensive and innovative therapies, and seemed to equally emphasize positive and negative results.
Collapse
Affiliation(s)
| | - Pedro N Aguiar
- Clinical Oncology Department, Faculdade de Medicina do ABC, Santo André, Brazil; Américas Centro de Oncologia Integrado, São Paulo, Brazil.
| | | | - Alanda L Alves
- Nursing College, Universidade Paulista, São Paulo, Brazil
| | - Silvio Matsas
- Medical College, Faculdade de Medicina da Santa Casa de Misericórdia de São Paulo, São Paulo, Brazil
| | - Anderson R Simões
- Medical College, Faculdade de Medicina da Santa Casa de Misericórdia de São Paulo, São Paulo, Brazil
| | - Auro Del Giglio
- Clinical Oncology Department, Faculdade de Medicina do ABC, Santo André, Brazil
| |
Collapse
|
37
|
Bryce Robinson N, Naik A, Hameed I, Ruan Y, Rahouma M, Weidenmann V, Zenati MA, Bhatt DL, Girardi LN, Kurlansky P, Raja SG, Moher D, Fremes S, Chikwe J, Gaudino M. Characteristics, results, and reporting of contemporary surgical trials: A systematic review and analysis. Int J Surg Protoc 2020; 21:1-4. [PMID: 32258836 PMCID: PMC7125342 DOI: 10.1016/j.isjp.2020.03.002] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2020] [Revised: 03/09/2020] [Accepted: 03/10/2020] [Indexed: 11/18/2022] Open
Abstract
Introduction The complexities and risks inherent to the field of surgery and surgical interventions present unique challenges to the design and analysis of surgical randomized controlled trials (RCT). Prior studies have investigated the practical and methodologic challenges posed by surgical RCTs. To date, however, a comprehensive analysis of the contemporary literature across multiple surgical subspecialties does not exist. In this descriptive analysis, we set out to characterize surgical RCTs over the past 10 years across six major surgical specialties. Methods and analysis A literature search by a medical librarian will be performed to identify all surgical randomized clinical trials published between January 2009 and December 2019 in the two journals with the highest impact factor for six surgical specialties as well as two large general medicine journals. Two reviewers will independently screen the citations retrieved from the literature search and extract data according to a previously described protocol via a pre-defined data collection form. Categorical variables will be reported as counts and percentages. Following assessment of normality, continuous variables will be reported as mean (standard deviation) or median (inter-quartile range). Based on normality of data, independent t-test or the Mann-Whitney U test will be used to compare continuous variables and chi-square and Fisher's exact tests to compare categorical variables. Comparisons across multiple sets will be performed using ANOVA or Kruskak-Wallis tests. Two-sided significance testing will be used and a p-value <0.05 will be considered significant without adjustment for multiple testing. All analyses will be performed using SPSS version 24 and R within RStudio. PROSPERO (ID number: 162797). Ethics and dissemination There are no ethical concerns directly pertinent to this systematic review. The retrieved data will be made available upon request. The study will be written in English and submitted for publication in a peer-reviewed journal.
Collapse
Affiliation(s)
- N. Bryce Robinson
- Department of Cardiothoracic Surgery, Weill Cornell Medicine, New York, NY, USA
| | - Ajita Naik
- Department of Cardiothoracic Surgery, Weill Cornell Medicine, New York, NY, USA
| | - Irbaz Hameed
- Department of Cardiothoracic Surgery, Weill Cornell Medicine, New York, NY, USA
| | - Yongle Ruan
- Department of Cardiothoracic Surgery, Weill Cornell Medicine, New York, NY, USA
| | - Mohamed Rahouma
- Department of Cardiothoracic Surgery, Weill Cornell Medicine, New York, NY, USA
| | - Viola Weidenmann
- Department of Cardiothoracic Surgery, Weill Cornell Medicine, New York, NY, USA
| | - Marco A. Zenati
- BHS Department of Cardiothoracic Surgery, West Roxbury, MA, USA
| | - Deepak L. Bhatt
- Division of Cardiovascular Medicine, Brigham and Women’s Hospital, Boston, MA, USA
| | - Leonard N. Girardi
- Department of Cardiothoracic Surgery, Weill Cornell Medicine, New York, NY, USA
| | - Paul Kurlansky
- Department of Surgery, Columbia University Medical Center, New York, NY, USA
| | - Shahzad G. Raja
- Department of Cardiac Surgery, Harefield Hospital, London, UK
| | - David Moher
- Ottawa Methods Centre, Ottawa Hospital Research Institute, Ottawa, ON, Canada
| | - Stephen Fremes
- Schulich Heart Centre, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, ON, Canada
| | - Joanna Chikwe
- Department of Cardiac Surgery, Smidt Heart Institute, Cedars-Siani Medical Center, Los Angeles, CA, USA
| | - Mario Gaudino
- Department of Cardiothoracic Surgery, Weill Cornell Medicine, New York, NY, USA
- Corresponding author at: Department of Cardiothoracic Surgery, Weill Cornell Medicine, 525 E 68th St, New York, NY 10065, USA.
| |
Collapse
|
38
|
Rubinstein SM, Sigworth EA, Etemad S, Martin RL, Chen Q, Warner JL. Indication of Measures of Uncertainty for Statistical Significance in Abstracts of Published Oncology Trials: A Systematic Review and Meta-analysis. JAMA Netw Open 2019; 2:e1917530. [PMID: 31834396 PMCID: PMC6991218 DOI: 10.1001/jamanetworkopen.2019.17530] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/02/2022] Open
Abstract
IMPORTANCE There is growing consensus that reliance on P values, particularly a cutoff level of .05 for statistical significance, is a factor in the challenges in scientific reproducibility. Despite this consensus, publications describing clinical trial results with P values near .05 anecdotally use declarative statements that do not express uncertainty. OBJECTIVES To quantify uncertainty expression in abstracts describing the results of cancer randomized clinical trials (RCTs) with P values between .01 and .10 and examine whether trial features are associated with uncertainty expression. DATA SOURCES A total of 5777 prospective trials indexed on HemOnc.org, as of September 15, 2019. STUDY SELECTION Two-arm RCTs with a superiority end point with P values between .01 and .10. DATA EXTRACTION AND SYNTHESIS Abstracts were evaluated based on an uncertainty expression algorithm. Ordinal logistic regression modeling with multiple imputation was performed to identify whether characteristics of study design, results, trial authors, and context P values were normalized by dividing by prespecified α value. MAIN OUTCOMES AND MEASURES Uncertainty expression in abstracts as determined by the algorithm and its association with trial and publication characteristics. RESULTS Of 5777 trials screened, 556 met analysis criteria. Of these, 222 trials (39.9%) did not express uncertainty, 161 trials (29.0%) expressed some uncertainty, and 173 trials (31.1%) expressed full uncertainty. In ordinal logistic regression with multiple imputation, trial features with statistically significant associations with uncertainty expression included later year of publication (odds ratio [OR], 1.70; 95% CI, 1.24-2.32; P < .001), normalized P value (OR, 1.36; 95% CI, 1.11-1.67; P = .003), noncooperative group studies (OR, 1.72; 95% CI, 1.12-2.63; P = .01), and reporting an end point other than overall survival (OR, 1.41; 95% CI, 1.01-1.96; P = .047). Funding source, number of authors, journal impact tier, author nationality, study of unapproved drugs, abstract word count, whether the marginal end point was a primary or coprimary end point, and effect size (in subgroup analysis) did not have statistically significant associations with uncertainty expression. CONCLUSIONS AND RELEVANCE Published oncology articles with marginally significant results may often incompletely convey uncertainty. Although it appears that more uncertainty is expressed in recent abstracts, full uncertainty expression remains uncommon, and seemingly is less common when reporting overall survival, results with P values lower than α levels, and cooperative group studies.
Collapse
Affiliation(s)
- Samuel M. Rubinstein
- Division of Hematology and Oncology, Department of Medicine, Vanderbilt University, Nashville, Tennessee
| | | | - Shervin Etemad
- Vanderbilt University School of Medicine, Nashville, Tennessee
| | - Richard L. Martin
- Division of Hematology and Oncology, Department of Medicine, Vanderbilt University, Nashville, Tennessee
| | - Qingxia Chen
- Department of Biostatistics, Vanderbilt University, Nashville, Tennessee
- Department of Biomedical Informatics, Vanderbilt University, Nashville, Tennessee
| | - Jeremy L. Warner
- Division of Hematology and Oncology, Department of Medicine, Vanderbilt University, Nashville, Tennessee
- Department of Biomedical Informatics, Vanderbilt University, Nashville, Tennessee
- Vanderbilt-Ingram Cancer Center, Vanderbilt University Medical Center, Nashville, Tennessee
| |
Collapse
|
39
|
Goshtasbi K, Abouzari M, Abiri A, Yasaka T, Sahyouni R, Bitner B, Tajudeen BA, Kuan EC. Efficacy of steroid-eluting stents in management of chronic rhinosinusitis after endoscopic sinus surgery: updated meta-analysis. Int Forum Allergy Rhinol 2019; 9:1443-1450. [PMID: 31539461 PMCID: PMC6901756 DOI: 10.1002/alr.22443] [Citation(s) in RCA: 28] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2019] [Revised: 08/27/2019] [Accepted: 08/29/2019] [Indexed: 11/09/2022]
Abstract
BACKGROUND Recently, there has been mounting evidence suggesting the efficacy of steroid-eluting stents (SES) for management of chronic rhinosinusitis after endoscopic sinus surgery (ESS). This meta-analysis serves to evaluate the efficacy of SES in improving postoperative outcomes after ESS. METHODS A systematic literature search was performed of PubMed for articles published between 1985 and 2018. The outcome variables were reported at, on average, 30 days postintervention. RESULTS Seven of the 76 published studies, all of which were industry-sponsored, were included for a collective cohort of 444 SES and 444 control sinuses. In patients who received SES vs controls, collective odds ratios (ORs) for postoperative need for intervention, surgery, and oral steroid were 0.45 (95% confidence interval [CI], 0.33-0.62; p < 0.001), 0.30 (95% CI, 0.18-0.52; p < 0.001), and 0.58 (95% CI, 0.40-0.84; p = 0.004), respectively. In addition, collective ORs for frontal sinus ostia (FSO) patency, moderate-to-severe adhesion/scarring, and increase in polyp score were 2.53 (95% CI, 1.61-3.97; p < 0.001), 0.28 (95% CI, 0.13-0.59; p < 0.001), and 0.42 (95% CI, 0.25-0.74; p = 0.002), respectively. Collective mean differences for FSO/ethmoid inflammation and FSO diameter were -10.86 mm (p < 0.001) and +1.34 mm (p < 0.001), respectively. CONCLUSION Aggregate evidence suggests that SES can improve ESS outcomes by reducing rates of postoperative intervention and recurrent polyposis and inflammation, while promoting FSO patency. All included and analyzed studies were industry-sponsored and ruling-out publication bias was not possible. Future independent and nonsponsored studies to further evaluate SES's long-term efficacy are warranted.
Collapse
Affiliation(s)
- Khodayar Goshtasbi
- Department of Otolaryngology–Head and Neck Surgery, University of California, Irvine, USA
| | - Mehdi Abouzari
- Department of Otolaryngology–Head and Neck Surgery, University of California, Irvine, USA
| | - Arash Abiri
- Department of Otolaryngology–Head and Neck Surgery, University of California, Irvine, USA
| | - Tyler Yasaka
- Department of Otolaryngology–Head and Neck Surgery, University of California, Irvine, USA
| | - Ronald Sahyouni
- Department of Otolaryngology–Head and Neck Surgery, University of California, Irvine, USA
| | - Benjamin Bitner
- Department of Otolaryngology–Head and Neck Surgery, University of California, Irvine, USA
| | - Bobby A. Tajudeen
- Department of Otolaryngology–Head and Neck Surgery, Rush University Medical Center, Chicago, IL, USA
| | - Edward C. Kuan
- Department of Otolaryngology–Head and Neck Surgery, University of California, Irvine, USA
| |
Collapse
|
40
|
van Ravenzwaaij D, Ioannidis JPA. True and false positive rates for different criteria of evaluating statistical evidence from clinical trials. BMC Med Res Methodol 2019; 19:218. [PMID: 31775644 PMCID: PMC6882054 DOI: 10.1186/s12874-019-0865-y] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2019] [Accepted: 11/08/2019] [Indexed: 01/25/2023] Open
Abstract
Background Until recently a typical rule that has often been used for the endorsement of new medications by the Food and Drug Administration has been the existence of at least two statistically significant clinical trials favoring the new medication. This rule has consequences for the true positive (endorsement of an effective treatment) and false positive rates (endorsement of an ineffective treatment). Methods In this paper, we compare true positive and false positive rates for different evaluation criteria through simulations that rely on (1) conventional p-values; (2) confidence intervals based on meta-analyses assuming fixed or random effects; and (3) Bayes factors. We varied threshold levels for statistical evidence, thresholds for what constitutes a clinically meaningful treatment effect, and number of trials conducted. Results Our results show that Bayes factors, meta-analytic confidence intervals, and p-values often have similar performance. Bayes factors may perform better when the number of trials conducted is high and when trials have small sample sizes and clinically meaningful effects are not small, particularly in fields where the number of non-zero effects is relatively large. Conclusions Thinking about realistic effect sizes in conjunction with desirable levels of statistical evidence, as well as quantifying statistical evidence with Bayes factors may help improve decision-making in some circumstances.
Collapse
Affiliation(s)
- Don van Ravenzwaaij
- Department of Psychology, University of Groningen, Grote Kruisstraat 2/1, Heymans Building, room 169, 9712 TS, Groningen, The Netherlands.
| | - John P A Ioannidis
- Departments of Medicine, of Health Research and Policy, and of Statistics and Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, USA
| |
Collapse
|
41
|
Liang F, Zhu J, Mo M, Zhou CM, Jia HX, Xie L, Zheng Y, Zhang S. Role of industry funders in oncology RCTs published in high-impact journals and its association with trial conclusions and time to publication. Ann Oncol 2019; 29:2129-2134. [PMID: 30084933 DOI: 10.1093/annonc/mdy305] [Citation(s) in RCA: 21] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/04/2023] Open
Abstract
Background Previous studies have shown that industry funded trials are associated with pro-industry conclusions and publication bias. Less is known about the role of industry funders and their influence on trial conclusions and time to publication. Methods We identified all industry funded RCTs published in six high-impact clinical journals between 2014 and 2016 to estimate the prevalence of the role of industry funders in trial design, data collection, data analyses, data interpretation and manuscript writing. Ordinal logistic regression was used to assess the association between the role of industry funders and trial conclusions, which was classified on a five-point scale. Cox proportional-hazards were used to examine the effect of role of funder on time to publication. Results Of the 255 eligible RCTs, industry funders had a role in trial design in 179 (70.2%) trials, data collection in 160 (62.7%) trials, data analyses in 173 (67.8%) trials, data interpretation in 135 (52.9%) trials and manuscript writing in 168 (65.9%) trials. Trials with any role of industry funders had 3.6 times (95% CI 2.0-6.6) higher odds of having positive conclusions compared with those without role of industry funders. In trials with any role of industry funders, positive trials were published more rapidly than negative trials (hazard ratio = 4.3; 95% CI 2.7-6.7, P < 0.001), while for trials without role of industry funders, there was no association (hazard ratio = 1.07; 95% CI 0.57-1.99, P = 0.84). Conclusion The involvement of industry funders is common in all stages of clinical trials and was associated with more positive conclusions and more rapid publication of RCTs with positive results.
Collapse
Affiliation(s)
- F Liang
- Clinical Statistic Center, Shanghai Cancer Center and Shanghai Medical College, Fudan University, Shanghai, China.
| | - J Zhu
- Department of Radiation, Shanghai Cancer Center and Shanghai Medical College, Fudan University, Shanghai, China
| | - M Mo
- Clinical Statistic Center, Shanghai Cancer Center and Shanghai Medical College, Fudan University, Shanghai, China
| | - C M Zhou
- Clinical Statistic Center, Shanghai Cancer Center and Shanghai Medical College, Fudan University, Shanghai, China
| | - H X Jia
- Clinical Statistic Center, Shanghai Cancer Center and Shanghai Medical College, Fudan University, Shanghai, China
| | - L Xie
- Clinical Statistic Center, Shanghai Cancer Center and Shanghai Medical College, Fudan University, Shanghai, China
| | - Y Zheng
- Clinical Statistic Center, Shanghai Cancer Center and Shanghai Medical College, Fudan University, Shanghai, China
| | - S Zhang
- Medical Oncology, Shanghai Cancer Center and Shanghai Medical College, Fudan University, Shanghai, China
| |
Collapse
|
42
|
Kang JS. Ethics and Industry Interactions: Impact on Specialty Training, Clinical Practice, and Research. Rheum Dis Clin North Am 2019; 46:119-133. [PMID: 31757280 DOI: 10.1016/j.rdc.2019.09.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
Abstract
Physicians in training and their mentors must be cognizant of ethical concerns related to industry interactions. Mentors perceived to have conflicts of interest or to be engaging in misconduct can unconsciously and profoundly affect the learning and academic environment by implying certain values and expectations. Despite increased awareness of ethical concerns related to industry interactions in clinical practice and research, there remains a need for interventions to prevent ethical transgressions. Ethics education is essential and a move in the right direction, but it alone is likely inadequate in preventing unethical behavior. Education should be supplemented with ethical environments at institutions.
Collapse
Affiliation(s)
- Jane S Kang
- Division of Rheumatology, Columbia University Medical Center, 630 West 168th Street, P&S 3-450, New York, NY 10032, USA.
| |
Collapse
|
43
|
de Melo-Martín I. The commercialization of the biomedical sciences: (mis)understanding bias. HISTORY AND PHILOSOPHY OF THE LIFE SCIENCES 2019; 41:34. [PMID: 31485872 DOI: 10.1007/s40656-019-0274-x] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/20/2019] [Accepted: 08/21/2019] [Indexed: 06/10/2023]
Abstract
The growing commercialization of scientific research has raised important concerns about industry bias. According to some evidence, so-called industry bias can affect the integrity of the science as well as the direction of the research agenda. I argue that conceptualizing industry's influence in scientific research in terms of bias is unhelpful. Insofar as industry sponsorship negatively affects the integrity of the research, it does so through biasing mechanisms that can affect any research independently of the source of funding. Talk about industry bias thus offers no insight into the particular epistemic shortcomings at stake. If the concern is with the negative effects that industry funding can have on the research agenda, conceptualizing this influence as bias obscures the ways in which such impact is problematic and limits our ability to offer solutions that can successfully address the concerns raised by the growing role of private funding in science.
Collapse
|
44
|
Hashemipour MA, Pourmonajemzadeh S, Zoghitavana S, Navabi N. Relationship Between Declarations of Conflict of Interests and Reporting Positive Outcomes in Iranian Dental Journals. SCIENCE AND ENGINEERING ETHICS 2019; 25:1057-1067. [PMID: 29441446 DOI: 10.1007/s11948-018-0022-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/05/2017] [Accepted: 01/23/2018] [Indexed: 06/08/2023]
Abstract
Conflict of interests is a situation when someone is in need of other people's trust on one the hand and has personal or general interests on the other hand, resulting in conflict with the given responsibility. In this research work, an attempt was made to find the relation between declarations of conflict of interests and reporting positive outcomes in the dental journals in Iran (2000-2016). In this analytical/cross-sectional study, first Health and Biomedical Information was searched and all the Persian and English dental journals published in Iran were collected. Then, all the papers published in the journals from December 2000 to December 2016 were collected and categorized in terms of the year of publication, author or authors' affiliations, Persian and English journal, type of the substance or the drug used (including the manufacturing company), declarations of conflict of interests and the positive or negative conclusion of the report. Data were analyzed with the Fisher's exact test and Chi squared test, using the program SPSS 18. In numerical analysis, the significance was set at P < 0.05. Seventeen dental journals in Persian and English were analyzed: 10 in English and 7 in Persian. Reviewing these studies showed that of 1021 articles in Persian, in 128 cases there was no mention of a declaration of conflict of interests and in 11 cases, the COI had been stated. In addition, from 1220 articles in English, in 825 cases there was no mention of declarations of conflict of interests and in 45 cases, the declarations of COI had been mentioned. There was no significant relation between the COI and 'no' COI and the reporting of positive outcomes in papers in Iranian dental journals in terms of the journal type, year of publication and the journals' guarantee form (P = 0.25, P = 0.41 and P = 0.09). A total of 83% of studies with declarations of COI had one positive outcome, with a significant relationship in this field; however, in 73% of studies with no COI, there was one positive outcome, too. In general, the society expects that doctors would not consider any incentives except the health of the patients in the efforts made by them. The severity of the consequences of COI is of higher value when the patients' health is endangered due to it. In addition, COI might change the attitude and approach of other doctors and peers.
Collapse
Affiliation(s)
- Maryam Alsadat Hashemipour
- Kerman Oral and Dental Diseases Research Center, Kerman University of Medical Sciences, Kerman, Iran.
- Department of Oral Medicine, Dental School, Kerman University of Medical Sciences, Kerman, Iran.
| | | | | | - Nader Navabi
- Kerman Oral and Dental Diseases Research Center, Kerman University of Medical Sciences, Kerman, Iran
- Department of Oral Medicine, Dental School, Kerman University of Medical Sciences, Kerman, Iran
| |
Collapse
|
45
|
Domecq JP, Prutsky G, Elraiyah T, Wang Z, Mauck KF, Brito JP, Undavalli C, Sundaresh V, Prokop LJ, Montori VM, Murad MH. Medications affecting the biochemical conversion to type 2 diabetes: A systematic review and meta-analysis. J Clin Endocrinol Metab 2019; 104:3986-3995. [PMID: 31365088 DOI: 10.1210/jc.2019-01269] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/04/2019] [Accepted: 06/05/2019] [Indexed: 02/13/2023]
Abstract
BACKGROUND The extent to which some pharmacological interventions reduce or increase the risk of biochemical conversion to T2DM in at-risk individuals is unclear. METHODS We searched MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews and Scopus through August 24, 2017, for randomized controlled trials evaluating the effect of drugs suspected to modify the risk of biochemical conversion to T2DM. RESULTS We included 43 trials with 192,156 subjects (mean age 60 years; 56% men; mean BMI 30.4 kg/m2). Alpha-glucosidase inhibitors, angiotensin-converting enzyme inhibitors, angiotensin II receptor blockers, metformin, orlistat, phentermine-topiramate and pioglitazone significantly reduced the risk of biochemical conversion to T2DM, whereas statins and nateglinide increased the risk. There was insufficient direct evidence regarding the effects of sulfonylureas, glucagon-like peptide-1 receptor agonists, dipeptidyl peptidase-4 inhibitors and sodium-glucose cotransporter-2 inhibitors. Most trials were brief and evaluated this outcome during treatment without a withdrawal or washout period. CONCLUSIONS Several drugs modify the risk of biochemical conversation to T2DM, although whether this effect is persistent and clinically relevant is unclear. Future studies need to focus on cardiovascular disease prevention, mortality and patient-important outcomes instead of biochemical conversion to T2DM.
Collapse
Affiliation(s)
| | | | - Tarig Elraiyah
- Evidence Based Practice Center, Mayo Clinic, Rochester, MN
- Division of Nephrology, Department of Medicine, Drexel University College of Medicine, Philadelphia, PA
| | - Zhen Wang
- Evidence Based Practice Center, Mayo Clinic, Rochester, MN
| | - Karen F Mauck
- Division of General Internal Medicine, Mayo Clinic, Rochester, MN
| | - Juan Pablo Brito
- Knowledge and Evaluation Research Unit, Mayo Clinic, Rochester, MN
- Division of Endocrinology, Mayo Clinic, Rochester, MN
| | | | | | - Larry J Prokop
- Evidence Based Practice Center, Mayo Clinic, Rochester, MN
| | - Victor M Montori
- Evidence Based Practice Center, Mayo Clinic, Rochester, MN
- Knowledge and Evaluation Research Unit, Mayo Clinic, Rochester, MN
- Division of Endocrinology, Mayo Clinic, Rochester, MN
| | | |
Collapse
|
46
|
Khan NA, Nguyen CL, Khawar T, Spencer H, Torralba KD. Association of author’s financial conflict of interest with characteristics and outcome of rheumatoid arthritis randomized controlled trials. Rheumatology (Oxford) 2019; 58:776-785. [DOI: 10.1093/rheumatology/key368] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/01/2023] Open
Abstract
Abstract
Objective
To examine the prevalence, types and temporal trends of reported financial conflicts of interest (FCOIs) among authors of drug therapy randomized controlled trials (RCTs) for RA and their association with study outcomes.
Methods
We identified original, non–phase 1, parallel-group, drug therapy RA RCTs published in the years 2002–03, 2006–07, and 2010–11. Two investigators independently obtained trial characteristics data. Authors’ FCOIs were classified as honoraria/consultation fees receipt, employee status, research grant, and stock ownership. Multivariable logistic regression was performed to identify whether FCOIs were independently associated with study outcome.
Results
A total of 146 eligible RCTs were identified. Of these, 83 (58.4%) RCTs had at least one author with an FCOI [employee status: 63 (43.2%), honoraria/consultation fees receipt: 49 (33.6%), research grant: 30 (20.5%), and stock ownership: 28 (19.2%)]. A remarkable temporal increase in reporting of honoraria/consultation fees receipt, research grant, and stock ownership was seen. The reporting of any FCOI itself was not associated with positive outcome [50/73 (68.5%) with author FCOI vs 36/52 (69.2%) without author FCOI, P = 0.93]. However, honoraria/consulting fees receipt was independently associated with increased likelihood of a positive outcome [adjusted odds ratio (95% CI) of 3.24 (1.06–9.88)]. In general, trials with FCOIs were significantly more likely to be multicentre, have larger enrolment, use biologic or a small molecule as the experimental intervention, and have better reporting of some methodological quality measures.
Conclusion
FCOI reporting in RA drug RCT authors is common and temporally increasing. Receipt of honoraria/consulting fees was independently associated with a positive study outcome.
Collapse
Affiliation(s)
- Nasim Ahmed Khan
- Division of Rheumatology, University of Arkansas for Medical Sciences, Little Rock, AR, USA
- Central Arkansas Veterans Healthcare System, Little Rock, AR, USA
| | - Chau L Nguyen
- Division of Rheumatology, Loma Linda University, Loma Linda, CA, USA
| | - Talha Khawar
- Division of Rheumatology, Loma Linda University, Loma Linda, CA, USA
| | - Horace Spencer
- Department of Biostatistics, University of Arkansas for Medical Sciences, Little Rock, AR, USA
| | - Karina D Torralba
- Division of Rheumatology, Loma Linda University, Loma Linda, CA, USA
| |
Collapse
|
47
|
Abou-Setta AM, Rabbani R, Lix LM, Turgeon AF, Houston BL, Fergusson DA, Zarychanski R. Can authorship bias be detected in meta-analysis? Can J Anaesth 2019; 66:287-292. [PMID: 30725345 DOI: 10.1007/s12630-018-01268-6] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2018] [Revised: 09/12/2018] [Accepted: 10/10/2018] [Indexed: 11/26/2022] Open
Abstract
PURPOSE Statistical approaches have been developed to detect bias in individual trials, but guidance on how to detect systematic differences at a meta-analytical level is lacking. In this paper, we elucidate whether author bias can be detected in a cohort of randomized trials included in a meta-analysis. METHODS We utilized mortality data from 35 trials (10,880 patients) included in our previously published meta-analysis. First, we linked each author with their trial (or trials). Then we calculated author-specific odds ratios using univariate cross table methods. Finally, we tested the effect of authorship by comparing each author's estimated odds ratio with all other pooled estimated odds ratios using meta-regression. RESULTS The median number of investigators named as authors on the primary trial reports was six (interquartile range: 5-8, range: 2-32). The results showed that the slope of author effect for mortality ranged from - 1.35 to 0.71. We identified only one author team showing a marginally significant effect (- 0.39; 95% confidence interval, - 0.78 to 0.00). This author team has a history of retractions due to data manipulations and ethical violations. CONCLUSION When combining trial-level data to produce a pooled effect estimate, investigators must consider sources of potential bias. Our results suggest that systematic errors can be detected using meta-regression, although further research is needed to examine the sensitivity of this model. Systematic reviewers will benefit from the availability of methods to guard against the dissemination of results with the potential to mislead decision-making.
Collapse
Affiliation(s)
- Ahmed M Abou-Setta
- George and Fay Yee Centre for Healthcare Innovation, University of Manitoba/Winnipeg Regional Health Authority, Chown Building, 367-753 McDermot Ave, Winnipeg, MB, R3A 1R9, Canada.
- Department of Community Health Sciences, Max Rady College of Medicine, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, MB, Canada.
| | - Rasheda Rabbani
- George and Fay Yee Centre for Healthcare Innovation, University of Manitoba/Winnipeg Regional Health Authority, Chown Building, 367-753 McDermot Ave, Winnipeg, MB, R3A 1R9, Canada
- Department of Community Health Sciences, Max Rady College of Medicine, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, MB, Canada
| | - Lisa M Lix
- George and Fay Yee Centre for Healthcare Innovation, University of Manitoba/Winnipeg Regional Health Authority, Chown Building, 367-753 McDermot Ave, Winnipeg, MB, R3A 1R9, Canada
- Department of Community Health Sciences, Max Rady College of Medicine, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, MB, Canada
| | - Alexis F Turgeon
- Division of Critical Care Medicine, Department of Anesthesiology and Critical Care Medicine, Centre de recherche CHU de Québec - Université Laval, Population Health and Optimal Health Practice Research Unit, Université Laval, Québec City, QC, Canada
| | - Brett L Houston
- Department of Internal Medicine, Max Rady College of Medicine, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, MB, Canada
| | - Dean A Fergusson
- Clinical Epidemiology Program, Department of Medicine, Ottawa Hospital Research Institute, University of Ottawa, Ottawa, ON, Canada
| | - Ryan Zarychanski
- George and Fay Yee Centre for Healthcare Innovation, University of Manitoba/Winnipeg Regional Health Authority, Chown Building, 367-753 McDermot Ave, Winnipeg, MB, R3A 1R9, Canada
- Department of Community Health Sciences, Max Rady College of Medicine, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, MB, Canada
- Department of Internal Medicine, Max Rady College of Medicine, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, MB, Canada
| |
Collapse
|
48
|
Wilson B, Burnett P, Moher D, Altman DG, Al-Shahi Salman R. Completeness of reporting of randomised controlled trials including people with transient ischaemic attack or stroke: A systematic review. Eur Stroke J 2018; 3:337-346. [PMID: 31236481 PMCID: PMC6571510 DOI: 10.1177/2396987318782783] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2018] [Accepted: 05/10/2018] [Indexed: 12/01/2022] Open
Abstract
PURPOSE To assess the adherence of stroke randomised controlled trials to Consolidated Standards Of Reporting Trials reporting guidelines and investigate the factors that are associated with completeness of reporting. METHOD We took a random sample from the Cochrane Stroke Group's Trial Register of transient ischaemic attack or stroke randomised controlled trials, published in English in 1997-2016 inclusive. Two reviewers assessed the published report of the final primary results of stroke randomised controlled trials with a 10-point truncated Consolidated Standards Of Reporting Trials reporting checklist to investigate adherence over time, univariable associations and independent associations with total Consolidated Standards Of Reporting Trials reporting score in a multiple linear regression model. FINDINGS In this random sample of 177 stroke randomised controlled trials, the mean score on the truncated Consolidated Standards Of Reporting Trials checklist was 5.8 (SD 2.2); reporting improved from 1997-2000 (4.9 SD 2.0) to 2001-2009 (5.8 SD 2.1) and to 2010-2016 (6.8 SD 2.1). A higher Consolidated Standards Of Reporting Trials score was independently associated with publication during epochs following a revision of Consolidated Standards Of Reporting Trials reporting guidelines (p < 0.001), journal endorsement of the Consolidated Standards Of Reporting Trials reporting guideline at the time of randomised controlled trial publication (p < 0.001) and modified journal impact factor using median citation distribution (p = 0.012). DISCUSSION Stroke randomised controlled trial reporting to Consolidated Standards Of Reporting Trials standards has improved over time, but could be better. CONCLUSION Journal endorsement and enforcement of Consolidated Standards Of Reporting Trials reporting guidelines could further improve the reporting of stroke randomised controlled trials.Systematic review registration: Registered with PROSPERO (CRD42017072193).
Collapse
Affiliation(s)
- Blair Wilson
- Medical School, University of Edinburgh, Edinburgh, UK
| | - Peter Burnett
- Edinburgh Royal Infirmary,
NHS
Lothian, Edinburgh, UK
| | - David Moher
- Centre for Journalology, Clinical Epidemiology Program, Ottawa
Hospital Research Institute, Ottawa, Canada
| | - Douglas G Altman
- Centre for Statistics in Medicine, University of Oxford, Oxford,
UK
| | | |
Collapse
|
49
|
Kempf E, de Beyer JA, Cook J, Holmes J, Mohammed S, Nguyên TL, Simera I, Trivella M, Altman DG, Hopewell S, Moons KGM, Porcher R, Reitsma JB, Sauerbrei W, Collins GS. Overinterpretation and misreporting of prognostic factor studies in oncology: a systematic review. Br J Cancer 2018; 119:1288-1296. [PMID: 30353050 PMCID: PMC6251031 DOI: 10.1038/s41416-018-0305-5] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2018] [Revised: 09/21/2018] [Accepted: 09/24/2018] [Indexed: 02/07/2023] Open
Abstract
BACKGROUND Cancer prognostic biomarkers have shown disappointing clinical applicability. The objective of this study was to classify and estimate how study results are overinterpreted and misreported in prognostic factor studies in oncology. METHODS This systematic review focused on 17 oncology journals with an impact factor above 7. PubMed was searched for primary clinical studies published in 2015, evaluating prognostic factors. We developed a classification system, focusing on three domains: misleading reporting (selective, incomplete reporting, misreporting), misleading interpretation (unreliable statistical analysis, spin) and misleading extrapolation of the results (claiming irrelevant clinical applicability, ignoring uncertainty). RESULTS Our search identified 10,844 articles. The 98 studies included investigated a median of two prognostic factors (Q1-Q3, 1-7). The prognostic factors' effects were selectively and incompletely reported in 35/98 and 24/98 full texts, respectively. Twenty-nine articles used linguistic spin in the form of strong statements. Linguistic spin rejecting non-significant results was found in 34 full-text results and 15 abstract results sections. One in five articles had discussion and/or abstract conclusions that were inconsistent with the study findings. Sixteen reports had discrepancies between their full-text and abstract conclusions. CONCLUSIONS Our study provides evidence of frequent overinterpretation of findings of prognostic factor assessment in high-impact medical oncology journals.
Collapse
Affiliation(s)
- Emmanuelle Kempf
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Botnar Research Centre, University of Oxford, Oxford, UK
- Department of Medical Oncology, Henri Mondor and Albert Chenevier Teaching Hospital, APHP, Créteil, France
| | - Jennifer A de Beyer
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Botnar Research Centre, University of Oxford, Oxford, UK
| | - Jonathan Cook
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Botnar Research Centre, University of Oxford, Oxford, UK
| | - Jane Holmes
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Botnar Research Centre, University of Oxford, Oxford, UK
| | - Seid Mohammed
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Botnar Research Centre, University of Oxford, Oxford, UK
| | - Tri-Long Nguyên
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Botnar Research Centre, University of Oxford, Oxford, UK
- Laboratory UPRES EA2415, Biostatistics, Epidemiology, Clinical Research and Health Economics, University of Montpellier, Montpellier, France
| | - Iveta Simera
- Nuffield Department of Medicine, Centre for Tropical Medicine and Global Health, University of Oxford, Oxford, UK
| | - Marialena Trivella
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Botnar Research Centre, University of Oxford, Oxford, UK
| | - Douglas G Altman
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Botnar Research Centre, University of Oxford, Oxford, UK
| | - Sally Hopewell
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Botnar Research Centre, University of Oxford, Oxford, UK
| | - Karel G M Moons
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
- Cochrane Netherlands, UMC Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Raphael Porcher
- Department of Epidemiology, Hôtel Dieu Teaching Hospital, APHP, Paris, France
| | - Johannes B Reitsma
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
- Cochrane Netherlands, UMC Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Willi Sauerbrei
- Institute for Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
| | - Gary S Collins
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Botnar Research Centre, University of Oxford, Oxford, UK.
- NIHR Oxford Biomedical Research Centre, John Radcliffe Hospital, Oxford, OX3 9DU, UK.
| |
Collapse
|
50
|
Mercuri M, Gafni A. The evolution of GRADE (part 1): Is there a theoretical and/or empirical basis for the GRADE framework? J Eval Clin Pract 2018; 24:1203-1210. [PMID: 30009394 DOI: 10.1111/jep.12998] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/25/2018] [Accepted: 06/27/2018] [Indexed: 11/27/2022]
Abstract
RATIONALE, AIMS, AND OBJECTIVES The Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) framework has been presented as the best method available for developing clinical recommendations. GRADE has undergone a series of modifications. Here, we present the first part of a three article series examining the evolution of GRADE. Our purpose is to explore if (and if so, how) GRADE provides: (1) a justification (ie, theoretical and/or empirical) for why the criteria/components under consideration in the system are included (and other factors excluded), as well as why some criteria/components where added/modified in the evolution process, (2) clear and functional (ie, how to operationalize them) definitions of the included criteria/components, and (3) instruction and justification for how all the criteria/components are to be integrated when determining a recommendation. In part 1 of the series, we examine the first two versions of GRADE. METHODS Narrative review. RESULTS The justification scheme that sustains GRADE is not articulated in the first two versions of the framework. Why some criteria/components were included, and others excluded, is not justified theoretically nor is empirical support provided to suggest that the framework as presented includes that which is needed to produce valid recommendations. The first two versions of GRADE show a lack of clear instruction on how to operationalize the criteria for assessing the quality of evidence and the components for making a recommendation (including how to integrate the criteria/components at each step), which leaves substantial room for judgement on the part of the user of GRADE for guideline development. CONCLUSIONS This article revealed an absence of a justification (theoretical and/or empirical) to support important aspects of the GRADE framework, as well as a lack of clear instruction on how to operationalize the criteria and components in the framework. These issues limit one's ability to scientifically assess the appropriateness of GRADE for determining clinical recommendations.
Collapse
Affiliation(s)
- Mathew Mercuri
- Department of Medicine, Division of Emergency Medicine, McMaster University, Hamilton, ON, Canada.,Institute for the History and Philosophy of Science and Technology, University of Toronto, Toronto, ON, Canada.,African Centre for Epistemology and Philosophy of Science, University of Johannesburg, Auckland Park, South Africa
| | - Amiram Gafni
- Centre for Health Economics and Policy Analysis, Department of Health Research Methods, Evaluation and Impact (formerly, Clinical Epidemiology and Biostatistics), McMaster University, Hamilton, ON, Canada
| |
Collapse
|