1
|
Puljak L, Bala MM, Zając J, Meštrović T, Buttigieg S, Yanakoulia M, Briel M, Lunny C, Lesniak W, Poklepović Peričić T, Alonso-Coello P, Clarke M, Djulbegovic B, Gartlehner G, Giannakou K, Glenny AM, Glenton C, Guyatt G, Hemkens LG, Ioannidis JPA, Jaeschke R, Juhl Jørgensen K, Martins-Pfeifer CC, Marušić A, Mbuagbaw L, Meneses Echavez JF, Moher D, Nussbaumer-Streit B, Page MJ, Pérez-Gaxiola G, Robinson KA, Salanti G, Saldanha IJ, Savović J, Thomas J, Tricco AC, Tugwell P, van Hoof J, Pieper D. Methods proposed for monitoring the implementation of evidence-based research: a cross-sectional study. J Clin Epidemiol 2024; 168:111247. [PMID: 38185190 DOI: 10.1016/j.jclinepi.2024.111247] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2023] [Revised: 12/22/2023] [Accepted: 01/03/2024] [Indexed: 01/09/2024]
Abstract
OBJECTIVES Evidence-based research (EBR) is the systematic and transparent use of prior research to inform a new study so that it answers questions that matter in a valid, efficient, and accessible manner. This study surveyed experts about existing (e.g., citation analysis) and new methods for monitoring EBR and collected ideas about implementing these methods. STUDY DESIGN AND SETTING We conducted a cross-sectional study via an online survey between November 2022 and March 2023. Participants were experts from the fields of evidence synthesis and research methodology in health research. Open-ended questions were coded by recurring themes; descriptive statistics were used for quantitative questions. RESULTS Twenty-eight expert participants suggested that citation analysis should be supplemented with content evaluation (not just what is cited but also in which context), content expert involvement, and assessment of the quality of cited systematic reviews. They also suggested that citation analysis could be facilitated with automation tools. They emphasized that EBR monitoring should be conducted by ethics committees and funding bodies before the research starts. Challenges identified for EBR implementation monitoring were resource constraints and clarity on responsibility for EBR monitoring. CONCLUSION Ideas proposed in this study for monitoring the implementation of EBR can be used to refine methods and define responsibility but should be further explored in terms of feasibility and acceptability. Different methods may be needed to determine if the use of EBR is improving over time.
Collapse
Affiliation(s)
- Livia Puljak
- Center for Evidence-Based Medicine and Healthcare, Catholic University of Croatia, Zagreb, Croatia.
| | - Małgorzata M Bala
- Systematic Reviews Unit, Department of Hygiene and Dietetics, Chair of Epidemiology and Preventive Medicine, Jagiellonian University Medical College, Kraków, Poland
| | - Joanna Zając
- Department of Hygiene and Dietetics, Chair of Epidemiology and Preventive Medicine, Jagiellonian University Medical College, Kraków, Poland
| | | | - Sandra Buttigieg
- Department of Health Systems Management and Leadership, Faculty of Health Sciences, University of Malta, Msida, Malta
| | - Mary Yanakoulia
- Department of Nutrition and Dietetics, Harokopio University, Athens, Greece
| | - Matthias Briel
- Division of Clinical Epidemiology, Department of Clinical Research, University Hospital Basel, Basel, Switzerland; University of Basel, Basel, Switzerland
| | - Carole Lunny
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, Toronto, Ontario, Canada; Cochrane Hypertension Review Group, The Therapeutics Initiative, University of British Columbia, Vancouver, British Columbia, Canada
| | | | - Tina Poklepović Peričić
- Department of Prosthodontics, Study of Dental Medicine, University of Split School of Medicine, Split, Croatia
| | - Pablo Alonso-Coello
- Institut de Recerca Sant Pau (IR SANT PAU), Iberoamerican Cochrane Center, Barcelona, Spain; Centro de Investigación Biomédica en Red de Epidemiología y Salud Pública (CIBERESP), Madrid, Spain
| | - Mike Clarke
- Northern Ireland Clinical Trials Unit (NICTU), Belfast, UK; Centre for Public Health, Queen's University Belfast, Belfast, UK
| | - Benjamin Djulbegovic
- Division of Hematology/Oncology, Department of Medicine, Medical University of South Carolina, Charleston, South Carolina, USA
| | - Gerald Gartlehner
- RTI International, The RTI International-University of North Carolina Evidence-Based Practice Center, Durham, NC, USA; Department for Evidence-Based Medicine and Evaluation, University for Continuing Education Krems, Krems, Austria
| | - Konstantinos Giannakou
- Department of Health Sciences, School of Sciences, European University Cyprus, Nicosia, Cyprus
| | - Anne-Marie Glenny
- Division of Dentistry, Cochrane Oral Health, School of Medical Sciences, Faculty of Biology, Medicine and Health, University of Manchester, Manchester, UK
| | - Claire Glenton
- Western Norway University of Applied Sciences, Bergen, Norway
| | - Gordon Guyatt
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada
| | - Lars G Hemkens
- Research Center for Clinical Neuroimmunology and Neuroscience Basel (RC2NB), University Hospital Basel, University of Basel, Basel, Switzerland; Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland; Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA; Meta-Research Innovation Center Berlin (METRIC-B), Berlin Institute of Health, Berlin, Germany
| | - John P A Ioannidis
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA; Department of Medicine, Stanford University, Stanford, CA, USA; Department of Epidemiology and Population Health, Stanford University, Stanford, CA, USA; Department of Biomedical Data Science, Stanford University, Stanford, CA, USA; Department of Statistics, Stanford University, Stanford, CA, USA
| | - Roman Jaeschke
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Department of Medicine, McMaster University, Hamilton, Ontario, Canada
| | - Karsten Juhl Jørgensen
- Department of Clinical Research, Cochrane Denmark and Centre for Evidence-Based Medicine Odense, University of Southern Denmark, Odense, Denmark
| | | | - Ana Marušić
- Department of Research in Biomedicine and Health, Center for Evidence-based Medicine, University of Split School of Medicine, Split, Croatia
| | - Lawrence Mbuagbaw
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada; Biostatistics Unit, Father Sean O'Sullivan Research Centre, St Joseph's Healthcare Hamilton, Hamilton, Ontario, Canada
| | - Jose Francisco Meneses Echavez
- Facultad de Cultura Física, Deporte y Recreación, Universidad Santo Tomás, Bogotá, Colombia; Division of Health Services at Norwegian Institute of Public Health, Bergen, Norway
| | - David Moher
- Ottawa Hospital Research Institute and University of Ottawa, Ottawa, Canada
| | - Barbara Nussbaumer-Streit
- Department for Evidence-Based Medicine and Evaluation, University for Continuing Education Krems, Krems, Austria
| | - Matthew J Page
- Methods in Evidence Synthesis Unit, School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia
| | | | - Karen A Robinson
- Western Norway University of Applied Sciences, Bergen, Norway; Division of General Internal Medicine, Department of Medicine, School of Medicine, Johns Hopkins University, Baltimore, MD, USA
| | - Georgia Salanti
- Institute of Social and Preventive Medicine University of Bern, Switzerland
| | - Ian J Saldanha
- Department of Epidemiology, Center for Clinical Trials and Evidence Synthesis, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
| | - Jelena Savović
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK; NIHR Applied Research Collaboration West (ARC West) at University Hospitals Bristol and Weston NHS Foundation Trust, Bristol, UK
| | - James Thomas
- EPPI-Centre, UCL Social Research Institute, University College London, London, UK
| | - Andrea C Tricco
- Li Ka Shing Knowledge Institute of St. Michael's Hospital, Unity Health Toronto, Toronto, Canada; Epidemiology Division and the Institute of Health Management, Policy, and Evaluation, Dalla Lana School of Public Health University of Toronto, Toronto, Canada; Queen's Collaboration for Health Care Quality Joanna Briggs Institute Centre of Excellence, Queen's University, Kingston, Canada
| | - Peter Tugwell
- Department of Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada; Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Joost van Hoof
- The Hague University of Applied Sciences, Faculty Social Work & Education, The Hague, The Netherlands; Institute of Spatial Management, Faculty of Spatial Management and Landscape Architecture, Wrocław University of Environmental and Life Sciences, Wrocław, Poland
| | - Dawid Pieper
- Faculty of Health Sciences Brandenburg, Brandenburg Medical School Theodor Fontane, Institute for Health Services and Health System Research, Rüdersdorf, Germany; Center for Health Services Research, Brandenburg Medical School Theodor Fontane, Rüdersdorf, Germany
| |
Collapse
|
2
|
Siemens W, Bantle G, Mahler S, Nothacker J, Stadelmaier J, Bitzer EM, Schmucker C, Meerpohl JJ. Clinical and methodological implications for research elements in systematic reviews on COVID-19 treatment were often unstructured and under-reported: a metaresearch study. J Clin Epidemiol 2024; 166:111236. [PMID: 38072174 DOI: 10.1016/j.jclinepi.2023.111236] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2023] [Revised: 10/31/2023] [Accepted: 12/04/2023] [Indexed: 01/09/2024]
Abstract
OBJECTIVES Numerous systematic reviews (SRs) have been published in the first months of the COVID-19 pandemic and clinical trials were designed rapidly highlighting the importance of informative implications for research (IfRs) sections in SRs. IfR is one item of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2020 checklist and the Cochrane Handbook suggests considering population, intervention, control, outcome (PICO) and Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) domains when developing IfR. We aimed (1) to assess whether SRs on COVID-19 treatments included any IfR statements and, for SRs with an IfR statement, (2) to examine which elements informed the IfR statement. STUDY DESIGN AND SETTING We conducted a metaresearch study based on SRs on COVID-19 treatment identified in the Living OVerview of the Evidence COVID-19 database in May 2021 as part of another research project (CRD42021240423). We defined an IfR statement as at least one sentence that contained at least one bit of information that could be informative for planning future research. We extracted any IfR statements anywhere in the SRs on predefined IfR variables, in particular PICO elements, study design, and concepts underlying GRADE domains. Three authors extracted data independently after piloting the data extraction form. We resolved discrepancies in weekly discussions to ensure a high-quality data extraction. RESULTS We included 326 SRs, of which 284 SRs (87.1%) stated IfR. Of these 284 SRs, 201 (70.8%) reported using Preferred Reporting Items for Systematic Reviews and Meta-Analyses and 66 (23.2%) using GRADE. IfR statements (n = 284) addressing PICO were unstructured and commonly reported 'population' (n = 195, 68.7%), 'intervention' (n = 242, 85.2%), and 'outcome' (n = 127, 44.7%) but not 'control' (n = 29, 10.2%). Concepts underlying GRADE domains were infrequently reported in IfR statements of SRs (n = 284): 'risk of bias' (n = 14, 4.9%), 'imprecision' (n = 8, 2.8%), 'inconsistency' (n = 7, 2.5%), 'publication bias' (n = 3, 1.1%), and 'indirectness' (n = 1, 0.4%). Additional IfR elements mentioned in IfR were 'better reporting' of future studies (n = 17, 6.0%) and 'standardization of procedures in clinical trials' (n = 12, 4.2%). CONCLUSION Almost 90% of SRs on COVID-19 treatments reported IfR. IfR statements addressing PICO were unstructured across SRs and concepts underlying GRADE were rarely reported to inform IfR. Further work is needed to assess generalizability beyond COVID-19 and to define more precisely which IfR elements should be considered, and how they should be reported in SRs of interventions. Until then, considering PICO elements and concepts underlying GRADE to derive IfR seems to be a sensible starting point.
Collapse
Affiliation(s)
- Waldemar Siemens
- Institute for Evidence in Medicine, Medical Center-University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany; Cochrane Germany, Cochrane Germany Foundation, Freiburg, Germany.
| | - Gina Bantle
- Institute for Evidence in Medicine, Medical Center-University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Sonja Mahler
- Institute for Evidence in Medicine, Medical Center-University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Julia Nothacker
- Department of General Practice and Primary Care, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Julia Stadelmaier
- Institute for Evidence in Medicine, Medical Center-University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Eva Maria Bitzer
- University of Education Freiburg, Public Health and Health Education, Freiburg, Germany
| | - Christine Schmucker
- Institute for Evidence in Medicine, Medical Center-University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Jörg J Meerpohl
- Institute for Evidence in Medicine, Medical Center-University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany; Cochrane Germany, Cochrane Germany Foundation, Freiburg, Germany
| |
Collapse
|
3
|
Sewell KA, Schellinger J, Bloss JE. Effect of PRISMA 2009 on reporting quality in systematic reviews and meta-analyses in high-impact dental medicine journals between 1993-2018. PLoS One 2023; 18:e0295864. [PMID: 38096136 PMCID: PMC10721095 DOI: 10.1371/journal.pone.0295864] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2023] [Accepted: 11/30/2023] [Indexed: 12/17/2023] Open
Abstract
INTRODUCTION The PRISMA guidelines were published in 2009 to address inadequate reporting of key methodological details in systematic reviews and meta-analyses (SRs/MAs). This study sought to assess the impact of PRISMA on the quality of reporting in the full text of dental medicine journals. METHODS This study assessed the impact of PRISMA (2009) on thirteen methodological details in SRs/MAs published in the highest-impact dental medicine journals between 1993-2009 (n = 211) and 2012-2018 (n = 618). The study further examined the rate of described use of PRISMA in the abstract or full text of included studies published post- PRISMA and the impact of described use of PRISMA on level of reporting. This study also examined potential effects of inclusion of PRISMA in Instructions for Authors, along with study team characteristics. RESULTS The number of items reported in SRs/MAs increased following the publication of PRISMA (pre-PRISMA: M = 7.83, SD = 3.267; post-PRISMA: M = 10.55, SD = 1.4). Post-PRISMA, authors rarely mention PRISMA in abstracts (8.9%) and describe the use of PRISMA in the full text in 59.87% of SRs/MAs. The described use of PRISMA within the full text indicates that its intent (guidance for reporting) is not well understood, with over a third of SRs/MAs (35.6%) describing PRISMA as guiding the conduct of the review. However, any described use of PRISMA was associated with improved reporting. Among author team characteristics examined, only author team size had a positive relationship with improved reporting. CONCLUSION Following the 2009 publication of PRISMA, the level of reporting of key methodological details improved for systematic reviews/meta-analyses published in the highest-impact dental medicine journals. The positive relationship between reference to PRISMA in the full text and level of reporting provides further evidence of the impact of PRISMA on improving transparent reporting in dental medicine SRs/MAs.
Collapse
Affiliation(s)
- Kerry A. Sewell
- William E. Laupus Health Sciences Library, East Carolina University, Greenville, North Carolina, United States of America
| | - Jana Schellinger
- Center for Evidence-Based Policy, Oregon Health & Science University, Portland, Oregon, United States of America
| | - Jamie E. Bloss
- William E. Laupus Health Sciences Library, East Carolina University, Greenville, North Carolina, United States of America
| |
Collapse
|
4
|
Nørgaard B, Briel M, Chrysostomou S, Ristic Medic D, Buttigieg SC, Kiisk E, Puljak L, Bala M, Pericic TP, Lesniak W, Zając J, Lund H, Pieper D. A systematic review of meta-research studies finds substantial methodological heterogeneity in citation analyses to monitor evidence-based research. J Clin Epidemiol 2022; 150:126-141. [DOI: 10.1016/j.jclinepi.2022.06.021] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2022] [Revised: 06/21/2022] [Accepted: 06/29/2022] [Indexed: 10/17/2022]
|
5
|
Nørgaard B, Draborg E, Andreasen J, Juhl CB, Yost J, Brunnhuber K, Robinson KA, Lund H. Systematic Reviews are Rarely Used to Inform Study Design - a Systematic Review and Meta-analysis. J Clin Epidemiol 2022; 145:1-13. [PMID: 35045317 DOI: 10.1016/j.jclinepi.2022.01.007] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2021] [Revised: 12/28/2021] [Accepted: 01/13/2022] [Indexed: 12/30/2022]
Abstract
OBJECTIVE Our aim was to identify and synthesize the results from meta-research studies to determine whether and how authors of original studies in clinical health research use systematic reviews when designing new studies. STUDY DESIGN AND SETTING For this systematic review, we searched MEDLINE (OVID), Embase (OVID) and the Cochrane Methodology Register. We included meta-research studies and primary outcome was the percentage of original studies using systematic reviews to design their study. Risk of bias was assessed using an ad hoc created list of ten items. The results are presented both as a narrative synthesis and a meta-analysis. RESULTS Sixteen studies were included. The use of a systematic review to inform the design of new clinical studies varied between 0% and 73%, with a mean percentage of 17%. The number of components of the design in which information from previous systematic reviews was used varied from three to eleven. CONCLUSION Clinical health research is characterized by variability regarding the extent to which systematic reviews are used to guide the design. An evidence-based research (EBR) approach towards research design when new clinical health studies are designed is necessary to decrease potential research redundancy and increase end-user value.
Collapse
Affiliation(s)
- Birgitte Nørgaard
- Department of Public Health, University of Southern Denmark, Odense, Denmark.
| | - Eva Draborg
- Department of Public Health, University of Southern Denmark, Odense, Denmark
| | - Jane Andreasen
- Department of Physiotherapy and Occupational Therapy, Aalborg University Hospital, Denmark and Public Health and Epidemiology Group, Department of Health, Science and Technology, Aalborg University, Aalborg, Denmark
| | - Carsten Bogh Juhl
- Department of Sports Science and Clinical Biomechanics, University of Southern Denmark and Department of Physiotherapy and Occupational Therapy, University of Copenhagen Herlev and Gentofte, Denmark
| | - Jennifer Yost
- M. Louise Fitzpatrick College of Nursing, Villanova University, Philadelphia, Pennsylvania, USA
| | | | | | - Hans Lund
- Department of Evidence-Based Practice, Western Norway University of Applied Sciences, Bergen, Norway
| |
Collapse
|
6
|
Gosling CJ, Cartigny A, Mellier BC, Solanes A, Radua J, Delorme R. Efficacy of psychosocial interventions for Autism spectrum disorder: an umbrella review. Mol Psychiatry 2022; 27:3647-3656. [PMID: 35790873 PMCID: PMC9708596 DOI: 10.1038/s41380-022-01670-z] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/18/2021] [Revised: 05/31/2022] [Accepted: 06/09/2022] [Indexed: 02/08/2023]
Abstract
INTRODUCTION The wide range of psychosocial interventions designed to assist people with Autism Spectrum Disorder (ASD) makes it challenging to compile and hierarchize the scientific evidence that supports the efficacy of these interventions. Thus, we performed an umbrella review of published meta-analyses of controlled clinical trials that investigated the efficacy of psychosocial interventions on both core and related ASD symptoms. METHODS Each meta-analysis that was identified was re-estimated using a random-effects model with a restricted maximum likelihood estimator. The methodological quality of included meta-analyses was critically appraised and the credibility of the evidence was assessed algorithmically according to criteria adapted for the purpose of this study. RESULTS We identified a total of 128 meta-analyses derived from 44 reports. More than half of the non-overlapping meta-analyses were nominally statistically significant and/or displayed a moderate-to-large pooled effect size that favored the psychosocial interventions. The assessment of the credibility of evidence pointed out that the efficacy of early intensive behavioral interventions, developmental interventions, naturalistic developmental behavioral interventions, and parent-mediated interventions was supported by suggestive evidence on at least one outcome in preschool children. Possible outcomes included social communication deficits, global cognitive abilities, and adaptive behaviors. Results also revealed highly suggestive indications that parent-mediated interventions improved disruptive behaviors in early school-aged children. The efficacy of social skills groups was supported by suggestive evidence for improving social communication deficits and overall ASD symptoms in school-aged children and adolescents. Only four meta-analyses had a statistically significant pooled effect size in a sensitivity analysis restricted to randomized controlled trials at low risk of detection bias. DISCUSSION This umbrella review confirmed that several psychosocial interventions show promise for improving symptoms related to ASD at different stages of life. However, additional well-designed randomized controlled trials are still required to produce a clearer picture of the efficacy of these interventions. To facilitate the dissemination of scientific knowledge about psychosocial interventions for individuals with ASD, we built an open-access and interactive website that shares the information collected and the results generated during this umbrella review. PRE-REGISTRATION PROSPERO ID CRD42020212630.
Collapse
Affiliation(s)
- Corentin J. Gosling
- Paris Nanterre University, DysCo Laboratory, F-92000 Nanterre, France ,grid.508487.60000 0004 7885 7602Université de Paris, Laboratoire de Psychopathologie et Processus de Santé, F-92100 Boulogne-Billancourt, France ,grid.5491.90000 0004 1936 9297Centre for Innovation in Mental Health (CIMH), School of Psychology, Faculty of Environmental and Life Sciences, University of Southampton, Southampton, UK
| | - Ariane Cartigny
- grid.508487.60000 0004 7885 7602Université de Paris, Laboratoire de Psychopathologie et Processus de Santé, F-92100 Boulogne-Billancourt, France ,grid.413235.20000 0004 1937 0589Department of Child and Adolescent Psychiatry, Robert Debré Hospital, APHP, Paris, France
| | | | - Aleix Solanes
- grid.10403.360000000091771775Imaging of Mood- and Anxiety-Related Disorders (IMARD) Group, Institut d’Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), CIBERSAM, Barcelona, Spain
| | - Joaquim Radua
- grid.10403.360000000091771775Imaging of Mood- and Anxiety-Related Disorders (IMARD) Group, Institut d’Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), CIBERSAM, Barcelona, Spain ,grid.13097.3c0000 0001 2322 6764Department of Psychosis Studies, Institute of Psychiatry, Psychology, and Neuroscience, King’s College London, London, UK ,grid.4714.60000 0004 1937 0626Department of Clinical Neuroscience, Centre for Psychiatric Research and Education, Karolinska Institutet, Stockholm, Sweden
| | - Richard Delorme
- grid.413235.20000 0004 1937 0589Department of Child and Adolescent Psychiatry, Robert Debré Hospital, APHP, Paris, France ,grid.428999.70000 0001 2353 6535Human Genetics and Cognitive Functions, Institut Pasteur, Paris, France
| |
Collapse
|
7
|
Stellrecht E, Theis-Mahon NR, Schvaneveldt N. ROLE OF LIBRARIANS AND INFORMATION PROFESSIONALS IN IDENTIFYING dPROS IN THE EVIDENCE. J Evid Based Dent Pract 2022; 22:101649. [DOI: 10.1016/j.jebdp.2021.101649] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2021] [Revised: 10/05/2021] [Accepted: 10/06/2021] [Indexed: 10/20/2022]
|
8
|
Patel K, Cobourne MT, Pandis N, Seehra J. Are orthodontic randomised controlled trials justified with a citation of an appropriate systematic review? Prog Orthod 2021; 22:48. [PMID: 34918200 PMCID: PMC8677858 DOI: 10.1186/s40510-021-00395-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2021] [Accepted: 12/06/2021] [Indexed: 01/06/2023] Open
Abstract
Background A systematic review of the evidence should be undertaken to support the justification for undertaking a clinical trial. The aim of this study was to examine whether reports of orthodontic Randomised Clinical Trials (RCTs) cite prior systematic reviews (SR) to explain the rationale or justification of the trial. Study characteristics that predicated the citation of SR in the RCT report were also explored. Material and methods Orthodontic RCTs published between 1st January 2010 to 31st December 2020 in seven orthodontic journals were identified. All titles and abstracts were screened independently by two authors. Descriptive statistics and associations were assessed for the study characteristics. Logistic regression was used to identify predicators of SR inclusion in the trial report. Results 301 RCTs fulfilling the eligibility criteria were assessed. 220 SRs were available of which 74.5% (N = 164) were cited, and 24.5% (N = 56) were not included but were available in the literature within 12 months of trial commencement. When a SR was not included in the introduction or no SR was available within 12 months of trial commencement, interventional studies were commonly cited. The continent of the corresponding author predicated the possibility of inclusion of a SR in the introduction (OR 0.36; 95% CI 0.18–0.71; p = 0.003). Conclusions A quarter of orthodontic RCTs (24.5%) included in this study did not cite a SR in the introduction section to justify the rationale of the trial when a relevant SR was available. To reduce research waste and optimal usage of resources, researchers should identify or conduct a systematic review of the evidence to support the rationale and justification of the trial.
Collapse
Affiliation(s)
- Kishan Patel
- Department of Orthodontics, Faculty of Dentistry, Oral and Craniofacial Sciences, King's College London, Kings College Hospital NHS Foundation Trust, Denmark Hill, London, SE5 9RS, UK
| | - Martyn T Cobourne
- Centre for Craniofacial Development and Regeneration, Faculty of Dentistry, Oral and Craniofacial Sciences, King's College London, Guy's Hospital, Guy's and St Thomas NHS Foundation Trust, London, SE1 9RT, UK
| | - Nikolaos Pandis
- Department of Orthodontics and Dentofacial Orthopedics, Dental School/Medical Faculty, University of Bern, Bern, Switzerland
| | - Jadbinder Seehra
- Centre for Craniofacial Development and Regeneration, Faculty of Dentistry, Oral and Craniofacial Sciences, King's College London, Guy's Hospital, Guy's and St Thomas NHS Foundation Trust, London, SE1 9RT, UK.
| |
Collapse
|
9
|
Clayton GL, Elliott D, Higgins JPT, Jones HE. Use of external evidence for design and Bayesian analysis of clinical trials: a qualitative study of trialists' views. Trials 2021; 22:789. [PMID: 34749778 PMCID: PMC8577005 DOI: 10.1186/s13063-021-05759-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2021] [Accepted: 10/25/2021] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Evidence from previous studies is often used relatively informally in the design of clinical trials: for example, a systematic review to indicate whether a gap in the current evidence base justifies a new trial. External evidence can be used more formally in both trial design and analysis, by explicitly incorporating a synthesis of it in a Bayesian framework. However, it is unclear how common this is in practice or the extent to which it is considered controversial. In this qualitative study, we explored attitudes towards, and experiences of, trialists in incorporating synthesised external evidence through the Bayesian design or analysis of a trial. METHODS Semi-structured interviews were conducted with 16 trialists: 13 statisticians and three clinicians. Participants were recruited across several universities and trials units in the United Kingdom using snowball and purposeful sampling. Data were analysed using thematic analysis and techniques of constant comparison. RESULTS Trialists used existing evidence in many ways in trial design, for example, to justify a gap in the evidence base and inform parameters in sample size calculations. However, no one in our sample reported using such evidence in a Bayesian framework. Participants tended to equate Bayesian analysis with the incorporation of prior information on the intervention effect and were less aware of the potential to incorporate data on other parameters. When introduced to the concepts, many trialists felt they could be making more use of existing data to inform the design and analysis of a trial in particular scenarios. For example, some felt existing data could be used more formally to inform background adverse event rates, rather than relying on clinical opinion as to whether there are potential safety concerns. However, several barriers to implementing these methods in practice were identified, including concerns about the relevance of external data, acceptability of Bayesian methods, lack of confidence in Bayesian methods and software, and practical issues, such as difficulties accessing relevant data. CONCLUSIONS Despite trialists recognising that more formal use of external evidence could be advantageous over current approaches in some areas and useful as sensitivity analyses, there are still barriers to such use in practice.
Collapse
Affiliation(s)
- Gemma L Clayton
- Department of Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK.
| | - Daisy Elliott
- Department of Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
- Bristol Centre for Surgical Research, Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| | - Julian P T Higgins
- Department of Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
- NIHR Applied Research Collaboration West (ARC West) at University Hospitals Bristol and Weston NHS Foundation Trust, Bristol, UK
| | - Hayley E Jones
- Department of Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| |
Collapse
|
10
|
McLennan S, Nussbaumer-Streit B, Hemkens LG, Briel M. Barriers and Facilitating Factors for Conducting Systematic Evidence Assessments in Academic Clinical Trials. JAMA Netw Open 2021; 4:e2136577. [PMID: 34846522 PMCID: PMC8634056 DOI: 10.1001/jamanetworkopen.2021.36577] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/19/2022] Open
Abstract
IMPORTANCE A systematic assessment of existing research should justify the conduct and inform the design of new clinical research but is often lacking. There is little research on the barriers to and factors facilitating systematic evidence assessments. OBJECTIVE To examine the practices and attitudes of Swiss stakeholders and international funders regarding conducting systematic evidence assessments in academic clinical trials. DESIGN, SETTING, AND PARTICIPANTS In this qualitative study, individual semistructured qualitative interviews were conducted between February and August 2020 with 48 Swiss stakeholder groups (27 primary investigators, 9 funders and sponsors, 6 clinical trial support organizations, and 6 ethics committee members) and between January and March 2021 with 9 international funders of clinical trials from North America and Europe with a reputation for requiring systematic evidence synthesis in applications for academic clinical trials. MAIN OUTCOMES AND MEASURES The main outcomes were practices and attitudes of Swiss stakeholders and international funders regarding conducting systematic evidence assessments in academic clinical trials. Interviews were analyzed using conventional content analysis. RESULTS Of the 57 participants, 40 (70.2%) were male. Participants universally acknowledged that a comprehensive understanding of the previous evidence is important but reported wide variation regarding how this should be achieved. Participants reported that the conduct of formal systematic reviews was currently not expected before most clinical trials, but most international funders reported expecting a systematic search for the existing evidence. Whereas time and resources were reported by all participants as barriers to conducting systematic reviews, the Swiss research ecosystem was reported not to be as supportive of a systematic approach compared with international settings. CONCLUSIONS AND RELEVANCE In this qualitative study, Swiss stakeholders and international funders generally agreed that new clinical trials should be justified by a systematic evidence assessment but that barriers on individual, organizational, and political levels kept them from implementing it. More explicit requirements from funders appear to be needed to clarify the required level of comprehensiveness in summarizing existing evidence for different types of clinical trials.
Collapse
Affiliation(s)
- Stuart McLennan
- Department of Clinical Research, Basel Institute for Clinical Epidemiology and Biostatistics, University of Basel and University Hospital Basel, Basel, Switzerland
- Institute of History and Ethics in Medicine, TUM School of Medicine, Technical University of Munich, Munich, Germany
| | - Barbara Nussbaumer-Streit
- Cochrane Austria, Department for Evidence-based Medicine and Evaluation, Danube University Krems, Krems, Austria
| | - Lars G. Hemkens
- Department of Clinical Research, Basel Institute for Clinical Epidemiology and Biostatistics, University of Basel and University Hospital Basel, Basel, Switzerland
- Meta-Research Innovation Center at Stanford, Stanford University, Stanford, California
- Meta-Research Innovation Center Berlin, Berlin Institute of Health, Berlin, Germany
| | - Matthias Briel
- Department of Clinical Research, Basel Institute for Clinical Epidemiology and Biostatistics, University of Basel and University Hospital Basel, Basel, Switzerland
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada
| |
Collapse
|
11
|
Wong CHL, Wong W, Lin WL, Au DKY, Wu JCY, Leung TH, Wu IXY, Chung VCH. Prioritizing Chinese medicine clinical research questions in cancer palliative care from patient and caregiver perspectives. Health Expect 2021; 24:1487-1497. [PMID: 34107142 PMCID: PMC8369121 DOI: 10.1111/hex.13289] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2021] [Revised: 04/14/2021] [Accepted: 05/12/2021] [Indexed: 01/06/2023] Open
Abstract
BACKGROUND Chinese medicine (CM) modalities, including acupuncture and Chinese herbal medicine (CHM), are popular palliative interventions among patients with cancer, but further clinical research is required to assess their effectiveness and safety. OBJECTIVE To prioritize top ten important CM clinical research questions from patients with cancer, cancer survivors and caregivers' perspectives via a face-to-face prioritization workshop in Hong Kong. METHODS A list of 25 CM clinical research questions for cancer palliative care, which were identified from existing systematic reviews (SRs) and overview of SRs, was presented to 17 participants (patients with cancer [n = 5], cancer survivors [n = 6] and caregivers [n = 6]). The participants were then invited to establish consensus on prioritizing top ten research questions. RESULTS Among the top ten priorities, five (50%) focused on acupuncture and related therapies, while five (50%) were on CHM. The three most important research priorities were (i) manual acupuncture plus opioids for relieving pain; (ii) CHM for improving quality of life among patients receiving chemotherapy; and (iii) concurrent use of CHM plus loperamide for reducing stomatitis. CONCLUSION The top ten participant-endorsed CM clinical research priorities for cancer palliative care can guide local researchers on future direction. They can also inform local research funders on patient-centred allocation of limited funding. Under limited research funding, the most important co-prioritized research question from professional and patient perspectives may be addressed first. PATIENT OR PUBLIC CONTRIBUTION Patients with cancer, cancer survivors and caregivers participated in conduct of the study to prioritize CM clinical research questions.
Collapse
Affiliation(s)
- Charlene H. L. Wong
- Department of Medicine and TherapeuticsThe Chinese University of Hong KongShatinHong Kong
- Jockey Club School of Public Health and Primary CareThe Chinese University of Hong KongShatinHong Kong
| | - Wendy Wong
- National Institution of TCM Constitution and Preventive MedicineBeijing University of Chinese MedicineBeijingChina
- Yat Hei Hong Kong Company LimitedCentralHong Kong
| | - Wai Ling Lin
- Hong Kong Institute of Integrative MedicineThe Chinese University of Hong KongShatinHong Kong
| | - David K. Y. Au
- Hong Kong Institute of Integrative MedicineThe Chinese University of Hong KongShatinHong Kong
| | - Justin C. Y. Wu
- Department of Medicine and TherapeuticsThe Chinese University of Hong KongShatinHong Kong
| | - Ting Hung Leung
- Jockey Club School of Public Health and Primary CareThe Chinese University of Hong KongShatinHong Kong
- School of Chinese MedicineThe Chinese University of Hong KongShatinHong Kong
| | - Irene X. Y. Wu
- Department of Epidemiology and Health StatisticsXiangya School of Public HealthCentral South UniversityChangshaChina
| | - Vincent C. H. Chung
- Jockey Club School of Public Health and Primary CareThe Chinese University of Hong KongShatinHong Kong
- School of Chinese MedicineThe Chinese University of Hong KongShatinHong Kong
| |
Collapse
|
12
|
Seehra J, Stonehouse-Smith D, Pandis N. Assessment of early exaggerated treatment effects in orthodontic interventions using cumulative meta-analysis. Eur J Orthod 2021; 43:601-605. [PMID: 34184029 PMCID: PMC8633600 DOI: 10.1093/ejo/cjab042] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/04/2022]
Abstract
Background The reported initial strong treatment effects reported in early trials that are refuted in subsequent future studies assessing the same interventions have been attributed to novelty bias. The aim of this study was to determine whether there is any evidence of novelty bias in the reported treatment effects of orthodontics interventions. Materials and methods Relevant orthodontic systematic review (SRs) topics containing at least one meta-analysis on either binary or continuous outcomes with a minimum of three trials considered important areas in the field of orthodontic practice were identified. SR, meta-analysis, and primary study-level characteristics were extracted. Descriptive statistics were calculated at the SRs, meta-analysis, and at the individual study level. All SR and trial-level data were imported into the statistical software and all meta-analyses were replicated using the cumulative random-effects meta-analysis approach. Changes in the size and direction of the estimates between the first trial and the cumulative effect over time were recorded. Results Forty-seven meta-analyses were included. The total number of primary studies included within these meta-analyses was 408 (N = 408). Overall, the final effect size estimate decreased in 29 (61.7%, N = 29/47) cumulative meta-analyses whilst it increased in the remaining 18 (38.3%, N = 18/47). No association between the level of risk of bias and the cumulative absolute effect size was evident (OR 1.00; 95% CI: 0.98, 1.03; P = 0.717) after adjusting for year of the primary study (P = 0.22). Conclusions Clinicians should be wary of the results of trials reporting the effectiveness of new interventions as there is a possibility that the reported effect size will be often exaggerated.
Collapse
Affiliation(s)
- Jadbinder Seehra
- Department of Orthodontics, Faculty of Dentistry, Oral & Craniofacial Sciences, King's College London, UK
| | - Daniel Stonehouse-Smith
- Department of Orthodontics, Faculty of Dentistry, Oral & Craniofacial Sciences, King's College London, UK
| | - Nikolaos Pandis
- Department of Orthodontics and Dentofacial Orthopedics, Dental School/Medical Faculty, University of Bern, Switzerland
| |
Collapse
|
13
|
Seehra J, Liu C, Pandis N. Citation of prior systematic reviews in reports of randomized controlled trials published in dental speciality journals. J Dent 2021; 109:103658. [PMID: 33836249 DOI: 10.1016/j.jdent.2021.103658] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2021] [Revised: 03/27/2021] [Accepted: 04/02/2021] [Indexed: 11/18/2022] Open
Abstract
OBJECTIVES To assess the extent to which reports of dental Randomised Clinical Trials (RCTs) cite prior systematic reviews (SR) to explain the rationale or justification of the trial. Study characteristics that predicated the citation of SR in the RCT report were explored. METHODS An electronic database search was undertaken to identify dental RCTs published between 1st January 2014 and 31st December 2019. All titles and abstracts were screened independently by two authors. Descriptive statistics and associations were calculated for the study characteristics. Logistic regression was used to identify predicators of SR inclusion in the trial report. RESULTS 682 RCTs were analysed. 312 SRs were available of which 62.5 % were cited and 37.5 % were not included but were available in the literature within 12 months of trial commencement. An association between inclusion of SR and trial registration (P = 0.046) was detected. For the inclusion of a SR, authors based in Asia or other had lower odds than those based in Europe (OR: 0.53; 95 % CI:0.34,0.82; p = 0.005). Every unit increase in journal impact factor increased the odds of SR inclusion (OR: 1.23; 95 %: 1.06, 1.43; p = 0.006). CONCLUSIONS A relatively high proportion of dental RCTs (37.5 %) did not cite a SR in the introduction section to justify the rationale of the trial when a relevant SR was available. Trials conducted by a corresponding author based in Europe and published in journals with an increasing impact factor were also more likely to cite a SR. CLINICAL SIGNIFICANCE Further progress is required to minimise research waste and ensure resources are channelled towards clinically useful trials which have an appropriate rationale and justification.
Collapse
Affiliation(s)
- Jadbinder Seehra
- Department of Orthodontics, Faculty of Dentistry, Oral and Craniofacial Sciences, King's College London, United Kingdom.
| | - Catherine Liu
- Department of Oral and Maxillofacial Surgery, Queen's Hospital, Barking, Romford, United Kingdom
| | - Nikolaos Pandis
- Department of Orthodontics and Dentofacial Orthopedics, Dental School/Medical Faculty, University of Bern, Switzerland
| |
Collapse
|
14
|
Sankaran SP, Sonis S. Network meta-analysis from a pairwise meta-analysis design: to assess the comparative effectiveness of oral care interventions in preventing ventilator-associated pneumonia in critically ill patients. Clin Oral Investig 2021; 25:2439-2447. [PMID: 33537946 DOI: 10.1007/s00784-021-03802-1] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2020] [Accepted: 01/19/2021] [Indexed: 11/25/2022]
Abstract
OBJECTIVE In this research, we assessed the usefulness of network meta-analysis (NMA), in creating a hierarchy to define the most effective oral care intervention for the prevention and management of ventilation-associated pneumonia (VAP). MATERIALS AND METHODS We applied NMA to a previously published robust pairwise meta-analysis. Statistical analyses were based on comparing rates of total VAP events between intervention groups and placebo-usual care groups. We synthesized a netgraph, reported the ranking order of the interventions, and summarized output by a forest plot with a reference treatment placebo/usual care. RESULTS The results of this NMA are from the low and high risk of bias studies, and hence, we strongly recommend not to use findings of this NMA for clinical treatment needs, but based on results of the NMA, we highly recommend for future clinical trials. With our inclusion and exclusion criteria for the NMA, we extracted 25 studies (4473 subjects). The NMA included 16 treatments, 29 pairwise comparisons, and 15 designs. Based on results of NMA frequentist-ranking P scores, tooth brushing (P fixed-0.94, P random-0.89), tooth brushing with povidone-iodine (P fixed-0.90, P random-0.88), and furacillin (P fixed-0.88, P random-0.84) were the best three interventions for preventing VAP. CONCLUSIONS Any conclusion drawn from this NMA should be taken with caution and recommend future clinical trials with the results. CLINICAL RELEVANCE NMA appeared to be an effective platform from which multiple interventions reported in disparate clinical trials could be compared to derive a hierarchical assessment of efficacy in VAP intervention.
Collapse
Affiliation(s)
- Satheeshkumar P Sankaran
- Harvard Medical School, Boston, 02115, MA, USA.
- Department of Oral Oncology, Roswell Park Comprehensive Cancer Center, Buffalo, 14263, NY, USA.
| | - Stephen Sonis
- Brigham and Women's Hospital and the Harvard School of Dental Medicine, Boston, 02115, MA, USA
| |
Collapse
|
15
|
Kim D, Hasford J. Redundant trials can be prevented, if the EU clinical trial regulation is applied duly. BMC Med Ethics 2020; 21:107. [PMID: 33115456 PMCID: PMC7592564 DOI: 10.1186/s12910-020-00536-9] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2019] [Accepted: 09/15/2020] [Indexed: 12/19/2022] Open
Abstract
The problem of wasteful clinical trials has been debated relentlessly in the medical community. To a significant extent, it is attributed to redundant trials - studies that are carried out to address questions, which can be answered satisfactorily on the basis of existing knowledge and accessible evidence from prior research. This article presents the first evaluation of the potential of the EU Clinical Trials Regulation 536/2014, which entered into force in 2014 but is expected to become applicable at the end of 2021, to prevent such trials. Having reviewed provisions related to the trial authorisation, we propose how certain regulatory requirements for the assessment of trial applications can and should be interpreted and applied by national research ethics committees and other relevant authorities in order to avoid redundant trials and, most importantly, preclude the unnecessary recruitment of trial participants and their unjustified exposure to health risks.
Collapse
Affiliation(s)
- Daria Kim
- Research Fellow, Max Planck Institute for Innovation and Competition, Marstallplatz 1, 81545 Munich, Germany
| | - Joerg Hasford
- Ludwig-Maximilians-University of Munich, The Institute for Medical Information Processing, Biometry, and Epidemiology, and Chairman of the Permanent Working Party of Research Ethics Committees in Germany, Scharnitzerstaße 7, 82166 Gräfelfing, Germany
| |
Collapse
|
16
|
Lund H, Juhl CB, Nørgaard B, Draborg E, Henriksen M, Andreasen J, Christensen R, Nasser M, Ciliska D, Clarke M, Tugwell P, Martin J, Blaine C, Brunnhuber K, Robinson KA. Evidence-Based Research Series-Paper 2 : Using an Evidence-Based Research approach before a new study is conducted to ensure value. J Clin Epidemiol 2020; 129:158-166. [PMID: 32987159 DOI: 10.1016/j.jclinepi.2020.07.019] [Citation(s) in RCA: 29] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2020] [Accepted: 07/20/2020] [Indexed: 12/13/2022]
Abstract
BACKGROUND AND OBJECTIVES There is considerable actual and potential waste in research. The aim of this article is to describe how using an evidence-based research approach before conducting a study helps to ensure that the new study truly adds value. STUDY DESIGN AND SETTING Evidence-based research is the use of prior research in a systematic and transparent way to inform a new study so that it is answering questions that matter in a valid, efficient, and accessible manner. In this second article of the evidence-based research series, we describe how to apply an evidence-based research approach before starting a new study. RESULTS Before a new study is performed, researchers need to provide a solid justification for it using the available scientific knowledge as well as the perspectives of end users. The key method for both is to conduct a systematic review of earlier relevant studies. CONCLUSION Describing the ideal process illuminates the challenges and opportunities offered through the suggested evidence-based research approach. A systematic and transparent approach is needed to provide justification for and to optimally design a relevant and necessary new study.
Collapse
Affiliation(s)
- Hans Lund
- Section for Evidence-based Practice, Western Norway University of Applied Sciences, Bergen, Norway.
| | - Carsten B Juhl
- Department of Sport Science and Clinical Biomechanics, University of Southern Denmark, Odense, Denmark; Department of Physiotherapy and Occupational Therapy, University Hospital of Copenhagen, Herlev & Gentofte, Denmark
| | - Birgitte Nørgaard
- Department of Public Health, University of Southern Denmark, Odense, Denmark
| | - Eva Draborg
- Department of Public Health, University of Southern Denmark, Odense, Denmark
| | - Marius Henriksen
- The Parker Institute, Bispebjerg and Frederiksberg Hospital, University of Copenhagen, Copenhagen, Denmark
| | - Jane Andreasen
- Department of Health, Science and Technology, Public Health and Epidemiology Group, Aalborg University, Alborg, Denmark; Department of Physical and Occupational Therapy, Aalborg University Hospital, Aalborg, Denmark
| | - Robin Christensen
- Musculoskeletal Statistics Unit, The Parker Institute, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark; Department of Clinical Research, Research Unit of Rheumatology, University of Southern Denmark, Odense University Hospital, Denmark
| | - Mona Nasser
- Peninsula Dental School, Plymouth University, Plymouth, England, UK
| | - Donna Ciliska
- Section for Evidence-based Practice, Western Norway University of Applied Sciences, Bergen, Norway; School of Nursing, McMaster University, Hamilton, Canada
| | - Mike Clarke
- Northern Ireland Methodology Hub, Queen's University Belfast, Northern Ireland
| | - Peter Tugwell
- Department of Medicine, University of Ottawa, Ottawa, Canada
| | - Janet Martin
- MEDICI Centre, Schulich School of Medicine & Dentistry, Western University, London, Ontario, Canada; Departments of Anesthesia & Biostatistics and Epidemiology & Biostatistics, Western University, London, Canada
| | | | - Klara Brunnhuber
- Digital Content Services, Data Platform Operations, Elsevier, London UK
| | | |
Collapse
|
17
|
Robinson KA, Brunnhuber K, Ciliska D, Juhl CB, Christensen R, Lund H. Evidence-Based Research Series-Paper 1: What Evidence-Based Research is and why is it important? J Clin Epidemiol 2020; 129:151-157. [PMID: 32979491 DOI: 10.1016/j.jclinepi.2020.07.020] [Citation(s) in RCA: 40] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2020] [Accepted: 07/20/2020] [Indexed: 01/28/2023]
Abstract
OBJECTIVES There is considerable actual and potential waste in research. Evidence-based research ensures worthwhile and valuable research. The aim of this series, which this article introduces, is to describe the evidence-based research approach. STUDY DESIGN AND SETTING In this first article of a three-article series, we introduce the evidence-based research approach. Evidence-based research is the use of prior research in a systematic and transparent way to inform a new study so that it is answering questions that matter in a valid, efficient, and accessible manner. RESULTS We describe evidence-based research and provide an overview of the approach of systematically and transparently using previous research before starting a new study to justify and design the new study (article #2 in series) and-on study completion-place its results in the context with what is already known (article #3 in series). CONCLUSION This series introduces evidence-based research as an approach to minimize unnecessary and irrelevant clinical health research that is unscientific, wasteful, and unethical.
Collapse
Affiliation(s)
- Karen A Robinson
- Johns Hopkins Evidence-based Practice Center, Division of General Internal Medicine, Department of Medicine, Johns Hopkins University, Baltimore, MD, USA
| | - Klara Brunnhuber
- Digital Content Services, Operations, Elsevier Ltd., 125 London Wall, London, EC2Y 5AS, UK
| | - Donna Ciliska
- School of Nursing, McMaster University, Health Sciences Centre, Room 2J20, 1280 Main Street West, Hamilton, Ontario, Canada, L8S 4K1; Section for Evidence-Based Practice, Western Norway University of Applied Sciences, Inndalsveien 28, Bergen, P.O.Box 7030 N-5020 Bergen, Norway
| | - Carsten Bogh Juhl
- Department of Sport Science and Clinical Biomechanics, University of Southern Denmark, Campusvej 55, 5230, Odense M, Denmark; Department of Physiotherapy and Occupational Therapy, University Hospital of Copenhagen, Herlev & Gentofte, Kildegaardsvej 28, 2900, Hellerup, Denmark
| | - Robin Christensen
- Musculoskeletal Statistics Unit, the Parker Institute, Bispebjerg and Frederiksberg Hospital, Copenhagen, Nordre Fasanvej 57, 2000, Copenhagen F, Denmark; Department of Clinical Research, Research Unit of Rheumatology, University of Southern Denmark, Odense University Hospital, Denmark
| | - Hans Lund
- Section for Evidence-Based Practice, Western Norway University of Applied Sciences, Inndalsveien 28, Bergen, P.O.Box 7030 N-5020 Bergen, Norway.
| | | |
Collapse
|
18
|
Castillo-Gonzalez DA, Dorsey-Trevino EG, Gonzalez-Gonzalez JG, Garcia-Leal M, Bautista-Orduño KG, Raygoza K, Gionfriddo MR, Ospina NMS, Rodriguez-Gutierrez R. A deeper analysis in thyroid research: A meta-epidemiological study of the American Thyroid Association clinical guidelines. PLoS One 2020; 15:e0234297. [PMID: 32520949 PMCID: PMC7286515 DOI: 10.1371/journal.pone.0234297] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2019] [Accepted: 05/24/2020] [Indexed: 01/02/2023] Open
Abstract
Background The American Thyroid Association (ATA) uses the GRADE or the American College of Physicians (ACP) system to develop recommendations. Recommendations based on low-quality evidence should spur for the conduction of clinical studies, if feasible. The extent to which recommendations by the ATA based on low-quality of evidence are being actively researched remains unknown. Methods Clinical guidelines produced by the ATA using the GRADE or the ACP system to classify evidence were deemed eligible. Reviewers, in duplicate and independently, extracted therapeutic recommendations based on low-quality evidence, whereas recommendations with higher quality of evidence, aimed at diagnosis, or best practice statements were excluded. Eligible recommendations based on low-quality evidence were deconstructed to their components using the PICO format. We then searched on clinicaltrials.gov to identify ongoing research. Trials were deemed eligible if they addressed the PICO question with at least one of the intended outcomes. Results A total of 543 recommendations were retrieved, of which 305 (56%) were based on low-quality of evidence and only 90 were deemed eligible. Of these, we found that 33 (37%) recommendations were actively being researched in 53 clinical trials. Most of the trials were randomized and funded by non-profit organizations. Many clinical trials studied thyroid nodules and differentiated thyroid cancer (26/53; 49%), whereas few studied were aimed at anaplastic thyroid cancer (2/53; 4%). Conclusion One out of three of gaps in evidence, identified as low quality during the development of ATA guidelines, are currently actively researched. This finding calls for the need to develop a better research infrastructure and funding to support thyroid research.
Collapse
Affiliation(s)
- Dalia A. Castillo-Gonzalez
- Endocrinology Division, Department of Internal Medicine, University Hospital “Dr. José E. González”, Universidad Autónoma de Nuevo León, Monterrey, México
- Plataforma INVEST Medicina UANL–KER Unit Mayo Clinic (KER Unit México), Universidad Autónoma de Nuevo León, Monterrey, México
| | - Edgar G. Dorsey-Trevino
- Endocrinology Division, Department of Internal Medicine, University Hospital “Dr. José E. González”, Universidad Autónoma de Nuevo León, Monterrey, México
- Plataforma INVEST Medicina UANL–KER Unit Mayo Clinic (KER Unit México), Universidad Autónoma de Nuevo León, Monterrey, México
| | - Jose G. Gonzalez-Gonzalez
- Endocrinology Division, Department of Internal Medicine, University Hospital “Dr. José E. González”, Universidad Autónoma de Nuevo León, Monterrey, México
- Plataforma INVEST Medicina UANL–KER Unit Mayo Clinic (KER Unit México), Universidad Autónoma de Nuevo León, Monterrey, México
- Research Unit, University Hospital “Dr. José E. González”, Universidad Autónoma de Nuevo León, Monterrey, México
| | - Mariana Garcia-Leal
- Endocrinology Division, Department of Internal Medicine, University Hospital “Dr. José E. González”, Universidad Autónoma de Nuevo León, Monterrey, México
| | - Karen G. Bautista-Orduño
- Endocrinology Division, Department of Internal Medicine, University Hospital “Dr. José E. González”, Universidad Autónoma de Nuevo León, Monterrey, México
- Plataforma INVEST Medicina UANL–KER Unit Mayo Clinic (KER Unit México), Universidad Autónoma de Nuevo León, Monterrey, México
| | - Karina Raygoza
- Plataforma INVEST Medicina UANL–KER Unit Mayo Clinic (KER Unit México), Universidad Autónoma de Nuevo León, Monterrey, México
| | - Michael R. Gionfriddo
- Center for Pharmacy Innovation and Outcomes, Geisinger Health System, Forty Fort, PA, United States of America
| | - Naykky M. Singh Ospina
- Knowledge and Evaluation Research Unit in Endocrinology, Mayo Clinic, Rochester, MN, United States of America
- Division of Endocrinology, Department of Medicine, University of Florida, Gainesville, FL, United States of America
| | - Rene Rodriguez-Gutierrez
- Endocrinology Division, Department of Internal Medicine, University Hospital “Dr. José E. González”, Universidad Autónoma de Nuevo León, Monterrey, México
- Plataforma INVEST Medicina UANL–KER Unit Mayo Clinic (KER Unit México), Universidad Autónoma de Nuevo León, Monterrey, México
- Knowledge and Evaluation Research Unit in Endocrinology, Mayo Clinic, Rochester, MN, United States of America
- Division of Endocrinology, Diabetes, Metabolism, and Nutrition, Department of Medicine, Mayo Clinic, Rochester, MN, United States of America
- * E-mail:
| |
Collapse
|
19
|
Yu Y, Yan X, Song F, Yao C, Xia J. A reproducibility probability-based bias-adjustment approach on the specification of non-inferiority margin using historical data. J Biopharm Stat 2019; 29:990-1002. [PMID: 31215834 DOI: 10.1080/10543406.2019.1632879] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
Abstract
The effect of reference treatment over placebo, known as M1, is essential in the development of non-inferiority margin. We proposed a M1 adjustment approach to reduce the selection bias for collected data of historical trials. A quantitative illustration of selection bias of historical data is also defined. Simulation study shows that the proposed approaches would significantly reduce the bias when the proportion of positive studies in historical data is noticeably larger than the power of studies include in historical data. When historical data are constituted by only positive studies, the performance of the proposed method is also appreciable. However, when the proportion of positive studies is close to the power of studies included or the number of studies included is too small, the performance of the proposed approach may not be reliable. A real-data application is also presented. The proposed bias-adjustment approach is a reasonable method to reduce the over-estimate of effect size in the specification of non-inferiority margin. It could also be applied in most non-inferiority margin specification methods or be cooperate used with other bias-adjustment approaches.
Collapse
Affiliation(s)
- Yongpei Yu
- Department of Health Statistics, College of Military Preventive Medicine, the Fourth Military Medical University, Xi'an, Shaanxi, China.,Peking University Clinical Research Institute, Peking University Health Science Center, Beijing, China
| | - Xiaoyan Yan
- Peking University Clinical Research Institute, Peking University Health Science Center, Beijing, China
| | - Fuyu Song
- Center for Food and Drug Inspection of CFDA, Beijing, China
| | - Chen Yao
- Peking University Clinical Research Institute, Peking University Health Science Center, Beijing, China
| | - Jielai Xia
- Department of Health Statistics, College of Military Preventive Medicine, the Fourth Military Medical University, Xi'an, Shaanxi, China
| |
Collapse
|
20
|
Nikolakopoulou A, Trelle S, Sutton AJ, Egger M, Salanti G. Synthesizing existing evidence to design future trials: survey of methodologists from European institutions. Trials 2019; 20:334. [PMID: 31174597 PMCID: PMC6555919 DOI: 10.1186/s13063-019-3449-6] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2018] [Accepted: 05/13/2019] [Indexed: 12/26/2022] Open
Abstract
Background ‘Conditional trial design’ is a framework for efficiently planning new clinical trials based on a network of relevant existing trials. The framework considers whether new trials are required and how the existing evidence can be used to answer the research question and plan future research. The potential of this approach has not been fully realized. Methods We conducted an online survey among trial statisticians, methodologists, and users of evidence synthesis research using referral sampling to capture opinions about the conditional trial design framework and current practices among clinical researchers. The questions included in the survey were related to the decision of whether a meta-analysis answers the research question, the optimal way to synthesize available evidence, which relates to the acceptability of network meta-analysis, and the use of evidence synthesis in the planning of new studies. Results In total, 76 researchers completed the survey. Two out of three survey participants (65%) were willing to possibly or definitely consider using evidence synthesis to design a future clinical trial and around half of the participants would give priority to such a trial design. The median rating of the frequency of using such a trial design was 0.41 on a scale from 0 (never) to 1 (always). Major barriers to adopting conditional trial design include the current regulatory paradigm and the policies of funding agencies and sponsors. Conclusions Participants reported moderate interest in using evidence synthesis methods in the design of future trials. They indicated that a major paradigm shift is required before the use of network meta-analysis is regularly employed in the design of trials. Electronic supplementary material The online version of this article (10.1186/s13063-019-3449-6) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Adriani Nikolakopoulou
- Institute of Social and Preventive Medicine (ISPM), University of Bern, Bern, Switzerland.
| | - Sven Trelle
- CTU Bern, University of Bern, Bern, Switzerland
| | - Alex J Sutton
- Department of Health Sciences, College of Medicine, Biological Sciences and Psychology, University of Leicester, Leicester, UK
| | - Matthias Egger
- Institute of Social and Preventive Medicine (ISPM), University of Bern, Bern, Switzerland
| | - Georgia Salanti
- Institute of Social and Preventive Medicine (ISPM), University of Bern, Bern, Switzerland
| |
Collapse
|
21
|
Jones HE, Ades AE, Sutton AJ, Welton NJ. Use of a random effects meta-analysis in the design and analysis of a new clinical trial. Stat Med 2018; 37:4665-4679. [PMID: 30187505 PMCID: PMC6484819 DOI: 10.1002/sim.7948] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2016] [Revised: 06/29/2018] [Accepted: 07/28/2018] [Indexed: 01/08/2023]
Abstract
In designing a randomized controlled trial, it has been argued that trialists should consider existing evidence about the likely intervention effect. One approach is to form a prior distribution for the intervention effect based on a meta‐analysis of previous studies and then power the trial on its ability to affect the posterior distribution in a Bayesian analysis. Alternatively, methods have been proposed to calculate the power of the trial to influence the “pooled” estimate in an updated meta‐analysis. These two approaches can give very different results if the existing evidence is heterogeneous, summarised using a random effects meta‐analysis. We argue that the random effects mean will rarely represent the trialist's target parameter, and so, it will rarely be appropriate to power a trial based on its impact upon the random effects mean. Furthermore, the random effects mean will not generally provide an appropriate prior distribution. More appropriate alternatives include the predictive distribution and shrinkage estimate for the most similar study. Consideration of the impact of the trial on the entire random effects distribution might sometimes be appropriate. We describe how beliefs about likely sources of heterogeneity have implications for how the previous evidence should be used and can have a profound impact on the expected power of the new trial. We conclude that the likely causes of heterogeneity among existing studies need careful consideration. In the absence of explanations for heterogeneity, we suggest using the predictive distribution from the meta‐analysis as the basis for a prior distribution for the intervention effect.
Collapse
Affiliation(s)
- Hayley E Jones
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| | - A E Ades
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| | - Alex J Sutton
- Department of Health Sciences, University of Leicester, Leicester, UK
| | - Nicky J Welton
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| |
Collapse
|
22
|
Salanti G, Nikolakopoulou A, Sutton AJ, Reichenbach S, Trelle S, Naci H, Egger M. Planning a future randomized clinical trial based on a network of relevant past trials. Trials 2018; 19:365. [PMID: 29996869 PMCID: PMC6042258 DOI: 10.1186/s13063-018-2740-2] [Citation(s) in RCA: 26] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2017] [Accepted: 06/12/2018] [Indexed: 11/16/2022] Open
Abstract
Background The important role of network meta-analysis of randomized clinical trials in health technology assessment and guideline development is increasingly recognized. This approach has the potential to obtain conclusive results earlier than with new standalone trials or conventional, pairwise meta-analyses. Methods Network meta-analyses can also be used to plan future trials. We introduce a four-step framework that aims to identify the optimal design for a new trial that will update the existing evidence while minimizing the required sample size. The new trial designed within this framework does not need to include all competing interventions and comparisons of interest and can contribute direct and indirect evidence to the updated network meta-analysis. We present the method by virtually planning a new trial to compare biologics in rheumatoid arthritis and a new trial to compare two drugs for relapsing-remitting multiple sclerosis. Results A trial design based on updating the evidence from a network meta-analysis of relevant previous trials may require a considerably smaller sample size to reach the same conclusion compared with a trial designed and analyzed in isolation. Challenges of the approach include the complexity of the methodology and the need for a coherent network meta-analysis of previous trials with little heterogeneity. Conclusions When used judiciously, conditional trial design could significantly reduce the required resources for a new study and prevent experimentation with an unnecessarily large number of participants. Electronic supplementary material The online version of this article (10.1186/s13063-018-2740-2) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Georgia Salanti
- Institute of Social and Preventive Medicine (ISPM), University of Bern, Bern, Switzerland.
| | - Adriani Nikolakopoulou
- Institute of Social and Preventive Medicine (ISPM), University of Bern, Bern, Switzerland
| | - Alex J Sutton
- Department of Health Sciences, College of Medicine, Biological Sciences and Psychology, University of Leicester, Leicester, UK
| | - Stephan Reichenbach
- Institute of Social and Preventive Medicine (ISPM), University of Bern, Bern, Switzerland.,Department of Rheumatology, Immunology and Allergiology, University Hospital, University of Bern, Bern, Switzerland
| | - Sven Trelle
- CTU Bern, University of Bern, Bern, Switzerland
| | - Huseyin Naci
- LSE Health, Department of Health Policy, London School of Economics and Political Science, London, UK
| | - Matthias Egger
- Institute of Social and Preventive Medicine (ISPM), University of Bern, Bern, Switzerland
| |
Collapse
|
23
|
Nikolakopoulou A, Mavridis D, Furukawa TA, Cipriani A, Tricco AC, Straus SE, Siontis GCM, Egger M, Salanti G. Living network meta-analysis compared with pairwise meta-analysis in comparative effectiveness research: empirical study. BMJ 2018; 360:k585. [PMID: 29490922 PMCID: PMC5829520 DOI: 10.1136/bmj.k585] [Citation(s) in RCA: 56] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 01/22/2018] [Indexed: 12/01/2022]
Abstract
OBJECTIVE To examine whether the continuous updating of networks of prospectively planned randomised controlled trials (RCTs) ("living" network meta-analysis) provides strong evidence against the null hypothesis in comparative effectiveness of medical interventions earlier than the updating of conventional, pairwise meta-analysis. DESIGN Empirical study of the accumulating evidence about the comparative effectiveness of clinical interventions. DATA SOURCES Database of network meta-analyses of RCTs identified through searches of Medline, Embase, and the Cochrane Database of Systematic Reviews until 14 April 2015. ELIGIBILITY CRITERIA FOR STUDY SELECTION Network meta-analyses published after January 2012 that compared at least five treatments and included at least 20 RCTs. Clinical experts were asked to identify in each network the treatment comparison of greatest clinical interest. Comparisons were excluded for which direct and indirect evidence disagreed, based on side, or node, splitting test (P<0.10). OUTCOMES AND ANALYSIS Cumulative pairwise and network meta-analyses were performed for each selected comparison. Monitoring boundaries of statistical significance were constructed and the evidence against the null hypothesis was considered to be strong when the monitoring boundaries were crossed. A significance level was defined as α=5%, power of 90% (β=10%), and an anticipated treatment effect to detect equal to the final estimate from the network meta-analysis. The frequency and time to strong evidence was compared against the null hypothesis between pairwise and network meta-analyses. RESULTS 49 comparisons of interest from 44 networks were included; most (n=39, 80%) were between active drugs, mainly from the specialties of cardiology, endocrinology, psychiatry, and rheumatology. 29 comparisons were informed by both direct and indirect evidence (59%), 13 by indirect evidence (27%), and 7 by direct evidence (14%). Both network and pairwise meta-analysis provided strong evidence against the null hypothesis for seven comparisons, but for an additional 10 comparisons only network meta-analysis provided strong evidence against the null hypothesis (P=0.002). The median time to strong evidence against the null hypothesis was 19 years with living network meta-analysis and 23 years with living pairwise meta-analysis (hazard ratio 2.78, 95% confidence interval 1.00 to 7.72, P=0.05). Studies directly comparing the treatments of interest continued to be published for eight comparisons after strong evidence had become evident in network meta-analysis. CONCLUSIONS In comparative effectiveness research, prospectively planned living network meta-analyses produced strong evidence against the null hypothesis more often and earlier than conventional, pairwise meta-analyses.
Collapse
Affiliation(s)
- Adriani Nikolakopoulou
- Institute of Social and Preventive Medicine (ISPM), University of Bern, Bern, Switzerland
| | - Dimitris Mavridis
- Department of Primary Education, University of Ioannina, Ioannina, Greece
- Centre de Recherche Épidémiologie et Statistique Sorbonne Paris Cité, Inserm/Université Paris Descartes, Paris, France
| | - Toshi A Furukawa
- Departments of Health Promotion and Human Behavior and of Clinical Epidemiology, Kyoto University Graduate School of Medicine/School of Public Health, Kyoto, Japan
| | - Andrea Cipriani
- Department of Psychiatry, University of Oxford, Warneford Hospital, Oxford, UK
- Oxford Health NHS Foundation Trust, Warneford Hospital, Oxford, UK
| | - Andrea C Tricco
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St Michael's Hospital, Toronto, Ontario, Canada
- Epidemiology Division, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada
| | - Sharon E Straus
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St Michael's Hospital, Toronto, Ontario, Canada
- Department of Medicine, Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
| | | | - Matthias Egger
- Institute of Social and Preventive Medicine (ISPM), University of Bern, Bern, Switzerland
| | - Georgia Salanti
- Institute of Social and Preventive Medicine (ISPM), University of Bern, Bern, Switzerland
| |
Collapse
|
24
|
Raftery J, Hanney S, Greenhalgh T, Glover M, Blatch-Jones A. Models and applications for measuring the impact of health research: update of a systematic review for the Health Technology Assessment programme. Health Technol Assess 2018; 20:1-254. [PMID: 27767013 DOI: 10.3310/hta20760] [Citation(s) in RCA: 46] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022] Open
Abstract
BACKGROUND This report reviews approaches and tools for measuring the impact of research programmes, building on, and extending, a 2007 review. OBJECTIVES (1) To identify the range of theoretical models and empirical approaches for measuring the impact of health research programmes; (2) to develop a taxonomy of models and approaches; (3) to summarise the evidence on the application and use of these models; and (4) to evaluate the different options for the Health Technology Assessment (HTA) programme. DATA SOURCES We searched databases including Ovid MEDLINE, EMBASE, Cumulative Index to Nursing and Allied Health Literature and The Cochrane Library from January 2005 to August 2014. REVIEW METHODS This narrative systematic literature review comprised an update, extension and analysis/discussion. We systematically searched eight databases, supplemented by personal knowledge, in August 2014 through to March 2015. RESULTS The literature on impact assessment has much expanded. The Payback Framework, with adaptations, remains the most widely used approach. It draws on different philosophical traditions, enhancing an underlying logic model with an interpretative case study element and attention to context. Besides the logic model, other ideal type approaches included constructionist, realist, critical and performative. Most models in practice drew pragmatically on elements of several ideal types. Monetisation of impact, an increasingly popular approach, shows a high return from research but relies heavily on assumptions about the extent to which health gains depend on research. Despite usually requiring systematic reviews before funding trials, the HTA programme does not routinely examine the impact of those trials on subsequent systematic reviews. The York/Patient-Centered Outcomes Research Institute and the Grading of Recommendations Assessment, Development and Evaluation toolkits provide ways of assessing such impact, but need to be evaluated. The literature, as reviewed here, provides very few instances of a randomised trial playing a major role in stopping the use of a new technology. The few trials funded by the HTA programme that may have played such a role were outliers. DISCUSSION The findings of this review support the continued use of the Payback Framework by the HTA programme. Changes in the structure of the NHS, the development of NHS England and changes in the National Institute for Health and Care Excellence's remit pose new challenges for identifying and meeting current and future research needs. Future assessments of the impact of the HTA programme will have to take account of wider changes, especially as the Research Excellence Framework (REF), which assesses the quality of universities' research, seems likely to continue to rely on case studies to measure impact. The HTA programme should consider how the format and selection of case studies might be improved to aid more systematic assessment. The selection of case studies, such as in the REF, but also more generally, tends to be biased towards high-impact rather than low-impact stories. Experience for other industries indicate that much can be learnt from the latter. The adoption of researchfish® (researchfish Ltd, Cambridge, UK) by most major UK research funders has implications for future assessments of impact. Although the routine capture of indexed research publications has merit, the degree to which researchfish will succeed in collecting other, non-indexed outputs and activities remains to be established. LIMITATIONS There were limitations in how far we could address challenges that faced us as we extended the focus beyond that of the 2007 review, and well beyond a narrow focus just on the HTA programme. CONCLUSIONS Research funders can benefit from continuing to monitor and evaluate the impacts of the studies they fund. They should also review the contribution of case studies and expand work on linking trials to meta-analyses and to guidelines. FUNDING The National Institute for Health Research HTA programme.
Collapse
Affiliation(s)
- James Raftery
- Primary Care and Population Sciences, Faculty of Medicine, University of Southampton, Southampton General Hospital, Southampton, UK
| | - Steve Hanney
- Health Economics Research Group (HERG), Institute of Environment, Health and Societies, Brunel University London, London, UK
| | - Trish Greenhalgh
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
| | - Matthew Glover
- Health Economics Research Group (HERG), Institute of Environment, Health and Societies, Brunel University London, London, UK
| | - Amanda Blatch-Jones
- Wessex Institute, Faculty of Medicine, University of Southampton, Southampton, UK
| |
Collapse
|
25
|
|
26
|
Williamson PR, Altman DG, Bagley H, Barnes KL, Blazeby JM, Brookes ST, Clarke M, Gargon E, Gorst S, Harman N, Kirkham JJ, McNair A, Prinsen CAC, Schmitt J, Terwee CB, Young B. The COMET Handbook: version 1.0. Trials 2017; 18:280. [PMID: 28681707 PMCID: PMC5499094 DOI: 10.1186/s13063-017-1978-4] [Citation(s) in RCA: 1057] [Impact Index Per Article: 151.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/08/2023] Open
Abstract
The selection of appropriate outcomes is crucial when designing clinical trials in order to compare the effects of different interventions directly. For the findings to influence policy and practice, the outcomes need to be relevant and important to key stakeholders including patients and the public, health care professionals and others making decisions about health care. It is now widely acknowledged that insufficient attention has been paid to the choice of outcomes measured in clinical trials. Researchers are increasingly addressing this issue through the development and use of a core outcome set, an agreed standardised collection of outcomes which should be measured and reported, as a minimum, in all trials for a specific clinical area.Accumulating work in this area has identified the need for guidance on the development, implementation, evaluation and updating of core outcome sets. This Handbook, developed by the COMET Initiative, brings together current thinking and methodological research regarding those issues. We recommend a four-step process to develop a core outcome set. The aim is to update the contents of the Handbook as further research is identified.
Collapse
Affiliation(s)
- Paula R. Williamson
- MRC North West Hub for Trials Methodology Research, Department of Biostatistics, University of Liverpool, Block F Waterhouse Building, 1-5 Brownlow Street, Liverpool, L69 3GL UK
| | - Douglas G. Altman
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Heather Bagley
- MRC North West Hub for Trials Methodology Research, Department of Biostatistics, University of Liverpool, Block F Waterhouse Building, 1-5 Brownlow Street, Liverpool, L69 3GL UK
| | - Karen L. Barnes
- MRC North West Hub for Trials Methodology Research, Department of Biostatistics, University of Liverpool, Block F Waterhouse Building, 1-5 Brownlow Street, Liverpool, L69 3GL UK
| | - Jane M. Blazeby
- MRC ConDuCT II Hub for Trials Methodology Research, School of Social and Community Medicine, University of Bristol, Bristol, UK
| | - Sara T. Brookes
- MRC ConDuCT II Hub for Trials Methodology Research, School of Social and Community Medicine, University of Bristol, Bristol, UK
| | - Mike Clarke
- Centre for Public Health, Queen’s University Belfast, Belfast, UK
- National University of Ireland Galway and HRB Trials Methodology Research Network, Galway, Ireland
| | - Elizabeth Gargon
- MRC North West Hub for Trials Methodology Research, Department of Biostatistics, University of Liverpool, Block F Waterhouse Building, 1-5 Brownlow Street, Liverpool, L69 3GL UK
| | - Sarah Gorst
- MRC North West Hub for Trials Methodology Research, Department of Biostatistics, University of Liverpool, Block F Waterhouse Building, 1-5 Brownlow Street, Liverpool, L69 3GL UK
| | - Nicola Harman
- MRC North West Hub for Trials Methodology Research, Department of Biostatistics, University of Liverpool, Block F Waterhouse Building, 1-5 Brownlow Street, Liverpool, L69 3GL UK
| | - Jamie J. Kirkham
- MRC North West Hub for Trials Methodology Research, Department of Biostatistics, University of Liverpool, Block F Waterhouse Building, 1-5 Brownlow Street, Liverpool, L69 3GL UK
| | - Angus McNair
- MRC ConDuCT II Hub for Trials Methodology Research, School of Social and Community Medicine, University of Bristol, Bristol, UK
| | - Cecilia A. C. Prinsen
- Department of Epidemiology and Biostatistics, EMGO+ Institute for Health and Care Research, VU University Medical Center, Amsterdam, The Netherlands
| | - Jochen Schmitt
- Center for Evidence-based Healthcare, Medizinische Fakultät, Technische Univesität Dresden, Dresden, Germany
| | - Caroline B. Terwee
- Department of Epidemiology and Biostatistics, EMGO+ Institute for Health and Care Research, VU University Medical Center, Amsterdam, The Netherlands
| | - Bridget Young
- MRC North West Hub for Trials Methodology Research, Department of Biostatistics, University of Liverpool, Block F Waterhouse Building, 1-5 Brownlow Street, Liverpool, L69 3GL UK
| |
Collapse
|
27
|
Clayton GL, Smith IL, Higgins JPT, Mihaylova B, Thorpe B, Cicero R, Lokuge K, Forman JR, Tierney JF, White IR, Sharples LD, Jones HE. The INVEST project: investigating the use of evidence synthesis in the design and analysis of clinical trials. Trials 2017; 18:219. [PMID: 28506284 PMCID: PMC5433067 DOI: 10.1186/s13063-017-1955-y] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2017] [Accepted: 04/26/2017] [Indexed: 11/23/2022] Open
Abstract
BACKGROUND When designing and analysing clinical trials, using previous relevant information, perhaps in the form of evidence syntheses, can reduce research waste. We conducted the INVEST (INVestigating the use of Evidence Synthesis in the design and analysis of clinical Trials) survey to summarise the current use of evidence synthesis in trial design and analysis, to capture opinions of trialists and methodologists on such use, and to understand any barriers. METHODS Our sampling frame was all delegates attending the International Clinical Trials Methodology Conference in November 2015. Respondents were asked to indicate (1) their views on the use of evidence synthesis in trial design and analysis, (2) their own use during the past 10 years and (3) the three greatest barriers to use in practice. RESULTS Of approximately 638 attendees of the conference, 106 (17%) completed the survey, half of whom were statisticians. Support was generally high for using a description of previous evidence, a systematic review or a meta-analysis in trial design. Generally, respondents did not seem to be using evidence syntheses as often as they felt they should. For example, only 50% (42/84 relevant respondents) had used a meta-analysis to inform whether a trial is needed compared with 74% (62/84) indicating that this is desirable. Only 6% (5/81 relevant respondents) had used a value of information analysis to inform sample size calculations versus 22% (18/81) indicating support for this. Surprisingly large numbers of participants indicated support for, and previous use of, evidence syntheses in trial analysis. For example, 79% (79/100) of respondents indicated that external information about the treatment effect should be used to inform aspects of the analysis. The greatest perceived barrier to using evidence synthesis methods in trial design or analysis was time constraints, followed by a belief that the new trial was the first in the area. CONCLUSIONS Evidence syntheses can be resource-intensive, but their use in informing the design, conduct and analysis of clinical trials is widely considered desirable. We advocate additional research, training and investment in resources dedicated to ways in which evidence syntheses can be undertaken more efficiently, offering the potential for cost savings in the long term.
Collapse
Affiliation(s)
- Gemma L. Clayton
- School of Social and Community Medicine, Faculty of Health Sciences, University of Bristol, Canynge Hall, 39 Whatley Road, Bristol, BS8 2PS UK
| | - Isabelle L. Smith
- Leeds Institute of Clinical Trials Research, University of Leeds, Leeds, UK
| | - Julian P. T. Higgins
- School of Social and Community Medicine, Faculty of Health Sciences, University of Bristol, Canynge Hall, 39 Whatley Road, Bristol, BS8 2PS UK
| | - Borislava Mihaylova
- Health Economics Research Centre, Nuffield Department of Population Health, University of Oxford, Oxford, UK
| | - Benjamin Thorpe
- Leeds Institute of Clinical Trials Research, University of Leeds, Leeds, UK
| | - Robert Cicero
- Leeds Institute of Clinical Trials Research, University of Leeds, Leeds, UK
| | - Kusal Lokuge
- Health Economics Research Centre, Nuffield Department of Population Health, University of Oxford, Oxford, UK
| | - Julia R. Forman
- Cambridge Clinical Trials Unit, University of Cambridge, Cambridge, UK
| | | | - Ian R. White
- MRC Biostatistics Unit, Cambridge Institute of Public Health, Cambridge, UK
| | | | - Hayley E. Jones
- School of Social and Community Medicine, Faculty of Health Sciences, University of Bristol, Canynge Hall, 39 Whatley Road, Bristol, BS8 2PS UK
| |
Collapse
|
28
|
Berge E, Salman RAS, van der Worp HB, Stapf C, Sandercock P, Sprigg N, Macleod MR, Kelly PJ, Nederkoorn PJ, Ford GA, Arnold M, Berge E, Diez-Tejedor E, Jatuzis D, Kelly PJ, Krieger DW, Nederkoorn PJ, Sandercock P, Stapf C, Weimar C, Ford GA, Salman RAS. Increasing value and reducing waste in stroke research. Lancet Neurol 2017; 16:399-408. [DOI: 10.1016/s1474-4422(17)30078-9] [Citation(s) in RCA: 33] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2016] [Revised: 02/06/2017] [Accepted: 03/07/2017] [Indexed: 12/21/2022]
|
29
|
Coyle C, Cafferty FH, Vale C, Langley RE. Metformin as an adjuvant treatment for cancer: a systematic review and meta-analysis. Ann Oncol 2016; 27:2184-2195. [PMID: 27681864 PMCID: PMC5178140 DOI: 10.1093/annonc/mdw410] [Citation(s) in RCA: 286] [Impact Index Per Article: 35.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2016] [Revised: 08/17/2016] [Accepted: 08/19/2016] [Indexed: 12/23/2022] Open
Abstract
BACKGROUND Metformin use has been associated with a reduced risk of developing cancer and an improvement in overall cancer survival rates in meta-analyses, but, to date, evidence to support the use of metformin as an adjuvant therapy in individual cancer types has not been presented. PATIENTS AND METHODS We systematically searched research databases, conference abstracts and trial registries for any studies reporting cancer outcomes for individual tumour types in metformin users compared with non-users, and extracted data on patients with early-stage cancer. Studies were assessed for design and quality, and a meta-analysis was conducted to quantify the adjuvant effect of metformin on recurrence-free survival (RFS), overall survival (OS) and cancer-specific survival (CSS), to inform future trial design. RESULTS Of 7670 articles screened, 27 eligible studies were identified comprising 24 178 participants, all enrolled in observational studies. In those with early-stage colorectal cancer, metformin use was associated with a significant benefit in all outcomes [RFS hazard ratio (HR) 0.63, 95% confidence interval (CI) 0.47-0.85; OS HR 0.69, CI 0.58-0.83; CSS HR 0.58, CI 0.39-0.86]. For men with early-stage prostate cancer, metformin was associated with significant, or borderline significant, benefits in all outcomes (RFS HR 0.83, CI 0.69-1.00; OS HR 0.82, CI 0.73-0.93; CSS HR 0.58, CI 0.37-0.93); however, there was significant heterogeneity between studies. The data suggest that prostate cancer patients treated with radical radiotherapy may benefit more from metformin (RFS HR 0.45, CI 0.29-0.70). In breast and urothelial cancer, no significant benefits were identified. Sufficient data were not available to conduct analyses on the impact of metformin dose and duration. CONCLUSIONS Our findings suggest that metformin could be a useful adjuvant agent, with the greatest benefits seen in colorectal and prostate cancer, particularly in those receiving radical radiotherapy, and randomised, controlled trials which investigate dose and duration, alongside efficacy, are advocated.
Collapse
Affiliation(s)
- C Coyle
- MRC Clinical Trials Unit at University College London, London, UK
| | - F H Cafferty
- MRC Clinical Trials Unit at University College London, London, UK
| | - C Vale
- MRC Clinical Trials Unit at University College London, London, UK
| | - R E Langley
- MRC Clinical Trials Unit at University College London, London, UK
| |
Collapse
|
30
|
Dagenais S, Brady O. Reply to the Letter to the Editor: advocating for a moratorium on low quality research in the spinal manipulation field. Spine J 2016; 16:1424. [PMID: 27832881 DOI: 10.1016/j.spinee.2016.05.013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/13/2016] [Accepted: 05/18/2016] [Indexed: 02/03/2023]
Affiliation(s)
- Simon Dagenais
- Spine Research LLC, 540 Main Street #7, Winchester, MA 01890, USA
| | - O'Dane Brady
- School of Physical Therapy and Rehabilitation Sciences, Morsani College of Medicine, University of South Florida, 12901 North Bruce B. Downs Blvd., MDC 077, Tampa, FL, 33612-4766, USA
| |
Collapse
|
31
|
|
32
|
Guthrie S, Bienkowska-Gibbs T, Manville C, Pollitt A, Kirtley A, Wooding S. The impact of the National Institute for Health Research Health Technology Assessment programme, 2003-13: a multimethod evaluation. Health Technol Assess 2016; 19:1-291. [PMID: 26307643 DOI: 10.3310/hta19670] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022] Open
Abstract
BACKGROUND The National Institute for Health Research (NIHR) Health Technology Assessment (HTA) programme supports research tailored to the needs of NHS decision-makers, patients and clinicians. This study reviewed the impact of the programme, from 2003 to 2013, on health, clinical practice, health policy, the economy and academia. It also considered how HTA could maintain and increase its impact. METHODS Interviews (n = 20): senior stakeholders from academia, policy-making organisations and the HTA programme. Bibliometric analysis: citation analysis of publications arising from HTA programme-funded research. Researchfish survey: electronic survey of all HTA grant holders. Payback case studies (n = 12): in-depth case studies of HTA programme-funded research. RESULTS We make the following observations about the impact, and routes to impact, of the HTA programme: it has had an impact on patients, primarily through changes in guidelines, but also directly (e.g. changing clinical practice); it has had an impact on UK health policy, through providing high-quality scientific evidence - its close relationships with the National Institute for Health and Care Excellence (NICE) and the National Screening Committee (NSC) contributed to the observed impact on health policy, although in some instances other organisations may better facilitate impact; HTA research is used outside the UK by other HTA organisations and systematic reviewers - the programme has an impact on HTA practice internationally as a leader in HTA research methods and the funding of HTA research; the work of the programme is of high academic quality - the Health Technology Assessment journal ensures that the vast majority of HTA programme-funded research is published in full, while the HTA programme still encourages publication in other peer-reviewed journals; academics agree that the programme has played an important role in building and retaining HTA research capacity in the UK; the HTA programme has played a role in increasing the focus on effectiveness and cost-effectiveness in medicine - it has also contributed to increasingly positive attitudes towards HTA research both within the research community and the NHS; and the HTA focuses resources on research that is of value to patients and the UK NHS, which would not otherwise be funded (e.g. where there is no commercial incentive to undertake research). The programme should consider the following to maintain and increase its impact: providing targeted support for dissemination, focusing resources when important results are unlikely to be implemented by other stakeholders, particularly when findings challenge vested interests; maintaining close relationships with NICE and the NSC, but also considering other potential users of HTA research; maintaining flexibility and good relationships with researchers, giving particular consideration to the Technology Assessment Report (TAR) programme and the potential for learning between TAR centres; maintaining the academic quality of the work and the focus on NHS need; considering funding research on the short-term costs of the implementation of new health technologies; improving the monitoring and evaluation of whether or not patient and public involvement influences research; improve the transparency of the priority-setting process; and continuing to monitor the impact and value of the programme to inform its future scientific and administrative development.
Collapse
|
33
|
Biased and inadequate citation of prior research in reports of cardiovascular trials is a continuing source of waste in research. J Clin Epidemiol 2016; 69:174-8. [DOI: 10.1016/j.jclinepi.2015.03.026] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2015] [Accepted: 03/19/2015] [Indexed: 01/10/2023]
|
34
|
Bhurke S, Cook A, Tallant A, Young A, Williams E, Raftery J. Using systematic reviews to inform NIHR HTA trial planning and design: a retrospective cohort. BMC Med Res Methodol 2015; 15:108. [PMID: 26715462 PMCID: PMC4696153 DOI: 10.1186/s12874-015-0102-2] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2015] [Accepted: 12/11/2015] [Indexed: 11/12/2022] Open
Abstract
Background Chalmers and Glasziou’s paper published in 2014 recommends research funding bodies should mandate that proposals for additional primary research are built on systematic reviews of existing evidence showing what is already known. Jones et al. identified 11 (23 %) of 48 trials funded during 2006–8 by the National Institute for Health Research Health Technology Assessment (NIHR HTA) Programme did not reference a systematic review. This study did not explore the reasons for trials not referencing a systematic review or consider trials using any other evidence in the absence of a systematic review. Referencing a systematic review may not be possible in certain circumstances, for instance if the systematic review does not address the question being proposed in the trial. The current study extended Jones’ study by exploring the reasons for why trials did not reference a systematic review and included a more recent cohort of trials funded in 2013 to determine if there were any changes in the referencing or use of systematic reviews. Methods Two cohorts of NIHR HTA randomised controlled trials were included. Cohort I included the same trials as Jones et al. (with the exception of one trial which was discontinued). Cohort II included NIHR HTA trials funded in 2013. Data extraction was undertaken independently by two reviewers using full applications and trial protocols. Descriptive statistics was used and no formal statistical analyses were conducted. Results Five (11 %) trials of the 47 funded during 2006–2008 did not reference a systematic review. These 5 trials had warranted reasons for not referencing systematic reviews. All trials from Cohort II referenced a systematic review. A quarter of all those trials with a preceding systematic review used a different primary outcome than those stated in the reviews. Conclusions The NIHR requires that proposals for new primary research are justified by existing evidence and the findings of this study confirm the adherence to this requirement with a high rate of applications using systematic reviews. Electronic supplementary material The online version of this article (doi:10.1186/s12874-015-0102-2) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Sheetal Bhurke
- Wessex Institute, University of Southampton Alpha House, University of Southampton Science Park, Southampton, SO16 7NS, UK. .,National Institute for Health Research (NIHR), Evaluation, Trials and Studies Coordinating Centre (NETSCC), University of Southampton, Southampton, SO16 7NS, UK.
| | - Andrew Cook
- National Institute for Health Research (NIHR), Evaluation, Trials and Studies Coordinating Centre (NETSCC), University of Southampton, Southampton, SO16 7NS, UK. .,University of Southampton and University Hospital Southampton NHS Foundation Trusts, Southampton, UK.
| | - Anna Tallant
- National Institute for Health Research (NIHR), Evaluation, Trials and Studies Coordinating Centre (NETSCC), University of Southampton, Southampton, SO16 7NS, UK.
| | - Amanda Young
- Wessex Institute, University of Southampton Alpha House, University of Southampton Science Park, Southampton, SO16 7NS, UK. .,National Institute for Health Research (NIHR), Evaluation, Trials and Studies Coordinating Centre (NETSCC), University of Southampton, Southampton, SO16 7NS, UK.
| | - Elaine Williams
- National Institute for Health Research (NIHR), Evaluation, Trials and Studies Coordinating Centre (NETSCC), University of Southampton, Southampton, SO16 7NS, UK.
| | - James Raftery
- Wessex Institute, University of Southampton Alpha House, University of Southampton Science Park, Southampton, SO16 7NS, UK. .,National Institute for Health Research (NIHR), Evaluation, Trials and Studies Coordinating Centre (NETSCC), University of Southampton, Southampton, SO16 7NS, UK. .,University of Southampton and University Hospital Southampton NHS Foundation Trusts, Southampton, UK.
| |
Collapse
|
35
|
Catalá-López F, González-Bermejo D, de la Fuente Honrubia C, Macías Saint-Gerons D. [Clinical equipoise and systematic reviews of randomized controlled trials]. Med Clin (Barc) 2015; 145:496-8. [PMID: 26004273 DOI: 10.1016/j.medcli.2015.04.006] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2015] [Revised: 03/30/2015] [Accepted: 04/09/2015] [Indexed: 11/28/2022]
Affiliation(s)
- Ferrán Catalá-López
- División de Farmacoepidemiología y Farmacovigilancia, Agencia Española de Medicamentos y Productos Sanitarios (AEMPS), Madrid, España; Fundación Instituto de Investigación en Servicios de Salud, Valencia, España; Departamento de Medicina, Universidad de Valencia/Centro de Investigación Biomédica en Red de Salud Mental (CIBERSAM), Valencia, España.
| | - Diana González-Bermejo
- División de Farmacoepidemiología y Farmacovigilancia, Agencia Española de Medicamentos y Productos Sanitarios (AEMPS), Madrid, España
| | - César de la Fuente Honrubia
- División de Farmacoepidemiología y Farmacovigilancia, Agencia Española de Medicamentos y Productos Sanitarios (AEMPS), Madrid, España
| | - Diego Macías Saint-Gerons
- División de Farmacoepidemiología y Farmacovigilancia, Agencia Española de Medicamentos y Productos Sanitarios (AEMPS), Madrid, España
| |
Collapse
|
36
|
Raftery J, Young A, Stanton L, Milne R, Cook A, Turner D, Davidson P. Clinical trial metadata: defining and extracting metadata on the design, conduct, results and costs of 125 randomised clinical trials funded by the National Institute for Health Research Health Technology Assessment programme. Health Technol Assess 2015; 19:1-138. [PMID: 25671821 DOI: 10.3310/hta19110] [Citation(s) in RCA: 30] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022] Open
Abstract
BACKGROUND By 2011, the Health Technology Assessment (HTA) programme had published the results of over 100 trials with another 220 in progress. The aim of the project was to develop and pilot 'metadata' on clinical trials funded by the HTA programme. OBJECTIVES The aim of the project was to develop and pilot questions describing clinical trials funded by the HTA programme in terms of it meeting the needs of the NHS with scientifically robust studies. The objectives were to develop relevant classification systems and definitions for use in answering relevant questions and to assess their utility. DATA SOURCES Published monographs and internal HTA documents. REVIEW METHODS A database was developed, 'populated' using retrospective data and used to answer questions under six prespecified themes. Questions were screened for feasibility in terms of data availability and/or ease of extraction. Answers were assessed by the authors in terms of completeness, success of the classification system used and resources required. Each question was scored to be retained, amended or dropped. RESULTS One hundred and twenty-five randomised trials were included in the database from 109 monographs. Neither the International Standard Randomised Controlled Trial Number nor the term 'randomised trial' in the title proved a reliable way of identifying randomised trials. Only limited data were available on how the trials aimed to meet the needs of the NHS. Most trials were shown to follow their protocols but updates were often necessary as hardly any trials recruited as planned. Details were often lacking on planned statistical analyses, but we did not have access to the relevant statistical plans. Almost all the trials reported on cost-effectiveness, often in terms of both the primary outcome and quality-adjusted life-years. The cost of trials was shown to depend on the number of centres and the duration of the trial. Of the 78 questions explored, 61 were well answered, 33 fully with 28 requiring amendment were the analysis updated. The other 17 could not be answered with readily available data. LIMITATIONS The study was limited by being confined to 125 randomised trials by one funder. CONCLUSIONS Metadata on randomised controlled trials can be expanded to include aspects of design, performance, results and costs. The HTA programme should continue and extend the work reported here. FUNDING The National Institute for Health Research HTA programme.
Collapse
Affiliation(s)
- James Raftery
- Wessex Institute, Faculty of Medicine, University of Southampton, Southampton, UK
| | - Amanda Young
- Wessex Institute, Faculty of Medicine, University of Southampton, Southampton, UK
| | - Louise Stanton
- University of Southampton Clinical Trials Unit, Southampton General Hospital, Southampton, UK
| | - Ruairidh Milne
- Wessex Institute, Faculty of Medicine, University of Southampton, Southampton, UK
| | - Andrew Cook
- Wessex Institute, Faculty of Medicine, University of Southampton, Southampton, UK
| | - David Turner
- Wessex Institute, Faculty of Medicine, University of Southampton, Southampton, UK
| | - Peter Davidson
- Wessex Institute, Faculty of Medicine, University of Southampton, Southampton, UK
| |
Collapse
|
37
|
Affiliation(s)
- Ian Roberts
- Clinical Trials Unit, London School of Hygiene & Tropical Medicine, London WC1E 7HT, UK.
| | - Katharine Ker
- Clinical Trials Unit, London School of Hygiene & Tropical Medicine, London WC1E 7HT, UK
| |
Collapse
|
38
|
Dechartres A, Ravaud P. Better prioritization to increase research value and decrease waste. BMC Med 2015; 13:244. [PMID: 26416788 PMCID: PMC4587677 DOI: 10.1186/s12916-015-0492-3] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/10/2015] [Accepted: 09/17/2015] [Indexed: 11/10/2022] Open
Abstract
In a recent study published in BMC Medicine, Singh Ospina and colleagues outlined the important gaps between ongoing research and research needs in the field of endocrinology. Many recommendations from clinical practice guidelines are based on a low level of evidence, thereby resulting in research gaps. Despite the publication of around 25,000 randomized controlled trials each year, ongoing research does not cover most of these gaps. In contrast, trials are planned when sufficient data are already available for decision making, which results in redundant research and exposes patients to unnecessary risks. This lack of prioritization contributes to the enormous problem of waste in research. A systematic approach to accumulate the available body of evidence is necessary to determine when we have sufficient evidence and when we have knowledge gaps, defined as research questions with no or a low level of evidence available. Systematic registration of research gaps and their prioritization may help to organize future research. Some initiatives exist, but they need to be generalized.Please see related research: http://www.biomedcentral.com/1741-7015/13/187.
Collapse
Affiliation(s)
- Agnes Dechartres
- Centre de Recherche Epidémiologie et Statistique, INSERM U1153, Paris, France. .,Centre d'Epidémiologie Clinique, Hôpital Hôtel-Dieu, Assistance Publique-Hôpitaux de Paris, 1 place du Parvis Notre Dame, 75004, Paris, France. .,Faculté de Médecine, Université Paris Descartes, Sorbonne Paris Cité, Paris, France. .,French Cochrane Centre, Paris, France.
| | - Philippe Ravaud
- Centre de Recherche Epidémiologie et Statistique, INSERM U1153, Paris, France.,Centre d'Epidémiologie Clinique, Hôpital Hôtel-Dieu, Assistance Publique-Hôpitaux de Paris, 1 place du Parvis Notre Dame, 75004, Paris, France.,Faculté de Médecine, Université Paris Descartes, Sorbonne Paris Cité, Paris, France.,French Cochrane Centre, Paris, France.,Department of Epidemiology, Mailman School of Public Health, Columbia University, New York, NY, USA
| |
Collapse
|
39
|
Lee S, Kim KH. Acupuncture for knee pain: is there no more room for further research? Acupunct Med 2015; 33:499. [PMID: 26185278 DOI: 10.1136/acupmed-2015-010881] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/27/2015] [Indexed: 11/03/2022]
Affiliation(s)
- Seunghoon Lee
- Department of Acupuncture & Moxibustion Medicine, Kyung Hee University Korean Medicine Hospital, Seoul, South Korea
| | - Kun Hyung Kim
- School of Korean Medicine, Pusan National University, Yangsan, South Korea
| |
Collapse
|
40
|
Tierney JF, Pignon JP, Gueffyier F, Clarke M, Askie L, Vale CL, Burdett S. How individual participant data meta-analyses have influenced trial design, conduct, and analysis. J Clin Epidemiol 2015; 68:1325-35. [PMID: 26186982 PMCID: PMC4635379 DOI: 10.1016/j.jclinepi.2015.05.024] [Citation(s) in RCA: 33] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2014] [Revised: 04/15/2015] [Accepted: 05/27/2015] [Indexed: 12/25/2022]
Abstract
Objectives To demonstrate how individual participant data (IPD) meta-analyses have impacted directly on the design and conduct of trials and highlight other advantages IPD might offer. Study Design and Setting Potential examples of the impact of IPD meta-analyses on trials were identified at an international workshop, attended by individuals with experience in the conduct of IPD meta-analyses and knowledge of trials in their respective clinical areas. Experts in the field who did not attend were asked to provide any further examples. We then examined relevant trial protocols, publications, and Web sites to verify the impacts of the IPD meta-analyses. A subgroup of workshop attendees sought further examples and identified other aspects of trial design and conduct that may inform IPD meta-analyses. Results We identified 52 examples of IPD meta-analyses thought to have had a direct impact on the design or conduct of trials. After screening relevant trial protocols and publications, we identified 28 instances where IPD meta-analyses had clearly impacted on trials. They have influenced the selection of comparators and participants, sample size calculations, analysis and interpretation of subsequent trials, and the conduct and analysis of ongoing trials, sometimes in ways that would not possible with systematic reviews of aggregate data. We identified additional potential ways that IPD meta-analyses could be used to influence trials. Conclusions IPD meta-analysis could be better used to inform the design, conduct, analysis, and interpretation of trials.
Collapse
Affiliation(s)
- Jayne F Tierney
- MRC Clinical Trials Unit at UCL, Aviation House, 125 Kingsway, London WC2B 6NH, UK.
| | - Jean-Pierre Pignon
- LNCC plateforme de méta-analyse en oncologie, Service de Biostatistique et d'Epidemiologie, Gustave-Roussy, Villejuif, France
| | - Francois Gueffyier
- Université Claude Bernard Lyon 1/Université de Lyon, 69365 Lyon Cedex 07, Lyon, France; Service de Pharmacologie Clinique, Hospices Civils de Lyon, Bron cedex, France
| | - Mike Clarke
- All-Ireland Hub for Trials Methodology Research, Queen's University Belfast, University Road, Belfast BT7 1NN, Northern Ireland, UK
| | - Lisa Askie
- NHMRC Clinical Trials Centre, ABN 15 211 513 464, Locked Bag 77, Camperdown, NSW 1450 Australia
| | - Claire L Vale
- MRC Clinical Trials Unit at UCL, Aviation House, 125 Kingsway, London WC2B 6NH, UK
| | - Sarah Burdett
- MRC Clinical Trials Unit at UCL, Aviation House, 125 Kingsway, London WC2B 6NH, UK
| | | |
Collapse
|
41
|
Helfer B, Prosser A, Samara MT, Geddes JR, Cipriani A, Davis JM, Mavridis D, Salanti G, Leucht S. Recent meta-analyses neglect previous systematic reviews and meta-analyses about the same topic: a systematic examination. BMC Med 2015; 13:82. [PMID: 25889502 PMCID: PMC4411715 DOI: 10.1186/s12916-015-0317-4] [Citation(s) in RCA: 37] [Impact Index Per Article: 4.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/24/2014] [Accepted: 03/10/2015] [Indexed: 01/08/2023] Open
Abstract
BACKGROUND As the number of systematic reviews is growing rapidly, we systematically investigate whether meta-analyses published in leading medical journals present an outline of available evidence by referring to previous meta-analyses and systematic reviews. METHODS We searched PubMed for recent meta-analyses of pharmacological treatments published in high impact factor journals. Previous systematic reviews and meta-analyses were identified with electronic searches of keywords and by searching reference sections. We analyzed the number of meta-analyses and systematic reviews that were cited, described and discussed in each recent meta-analysis. Moreover, we investigated publication characteristics that potentially influence the referencing practices. RESULTS We identified 52 recent meta-analyses and 242 previous meta-analyses on the same topics. Of these, 66% of identified previous meta-analyses were cited, 36% described, and only 20% discussed by recent meta-analyses. The probability of citing a previous meta-analysis was positively associated with its publication in a journal with a higher impact factor (odds ratio, 1.49; 95% confidence interval, 1.06 to 2.10) and more recent publication year (odds ratio, 1.19; 95% confidence interval 1.03 to 1.37). Additionally, the probability of a previous study being described by the recent meta-analysis was inversely associated with the concordance of results (odds ratio, 0.38; 95% confidence interval, 0.17 to 0.88), and the probability of being discussed was increased for previous studies that employed meta-analytic methods (odds ratio, 32.36; 95% confidence interval, 2.00 to 522.85). CONCLUSIONS Meta-analyses on pharmacological treatments do not consistently refer to and discuss findings of previous meta-analyses on the same topic. Such neglect can lead to research waste and be confusing for readers. Journals should make the discussion of related meta-analyses mandatory.
Collapse
Affiliation(s)
- Bartosz Helfer
- Department of Psychiatry and Psychotherapy, Technical University Munich, Klinikum rechts der Isar, Ismaningerstr 22, 81675, Munich, Germany.
| | - Aaron Prosser
- Complex Mental Illness Program, Centre for Addiction and Mental Health, Toronto, Canada.
| | - Myrto T Samara
- Department of Psychiatry and Psychotherapy, Technical University Munich, Klinikum rechts der Isar, Ismaningerstr 22, 81675, Munich, Germany.
| | - John R Geddes
- Department of Psychiatry, University of Oxford, Warneford Hospital, Oxford, UK.
| | - Andrea Cipriani
- Department of Psychiatry, University of Oxford, Warneford Hospital, Oxford, UK.
| | - John M Davis
- Department of Psychiatry, University of Illinois at Chicago, Chicago, IL, USA.
| | - Dimitris Mavridis
- Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece. .,Department of Primary Education, University of Ioannina, Ioannina, Greece.
| | - Georgia Salanti
- Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece.
| | - Stefan Leucht
- Department of Psychiatry and Psychotherapy, Technical University Munich, Klinikum rechts der Isar, Ismaningerstr 22, 81675, Munich, Germany.
| |
Collapse
|
42
|
Smyth RMD, Jacoby A, Altman DG, Gamble C, Williamson PR. The natural history of conducting and reporting clinical trials: interviews with trialists. Trials 2015; 16:16. [PMID: 25619208 PMCID: PMC4322554 DOI: 10.1186/s13063-014-0536-6] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2014] [Accepted: 12/19/2014] [Indexed: 11/25/2022] Open
Abstract
BACKGROUND To investigate the nature of the research process as a whole, factors that might influence the way in which research is carried out, and how researchers ultimately report their findings. METHODS Semi-structured qualitative telephone interviews with authors of trials, identified from two sources: trials published since 2002 included in Cochrane systematic reviews selected for the ORBIT project; and trial reports randomly sampled from 14,758 indexed on PubMed over the 12-month period from August 2007 to July 2008. RESULTS A total of 268 trials were identified for inclusion, 183 published since 2002 and included in the Cochrane systematic reviews selected for the ORBIT project and 85 randomly selected published trials indexed on PubMed. The response rate from researchers in the former group was 21% (38/183) and in the latter group was 25% (21/85). Overall, 59 trialists were interviewed from the two different sources. A number of major but related themes emerged regarding the conduct and reporting of trials: establishment of the research question; identification of outcome variables; use of and adherence to the study protocol; conduct of the research; reporting and publishing of findings. Our results reveal that, although a substantial proportion of trialists identify outcome variables based on their clinical experience and knowing experts in the field, there can be insufficient reference to previous research in the planning of a new trial. We have revealed problems with trial recruitment: not reaching the target sample size, over-estimation of recruitment potential and recruiting clinicians not being in equipoise. We found a wide variation in the completeness of protocols, in terms of detailing study rationale, outlining the proposed methods, trial organisation and ethical considerations. CONCLUSION Our results confirm that the conduct and reporting of some trials can be inadequate. Interviews with researchers identified aspects of clinical research that can be especially challenging: establishing appropriate and relevant outcome variables to measure, use of and adherence to the study protocol, recruiting of study participants and reporting and publishing the study findings. Our trialists considered the prestige and impact factors of academic journals to be the most important criteria for selecting those to which they would submit manuscripts.
Collapse
Affiliation(s)
- Rebecca M D Smyth
- School of Nursing, Midwifery and Social Work, University of Manchester, Manchester, M13 9PL, UK.
| | - Ann Jacoby
- Division of Public Health, University of Liverpool, Liverpool, L69 3GB, UK.
| | - Douglas G Altman
- Centre for Statistics in Medicine, University of Oxford, Oxford, OX2 6UD, UK.
| | - Carrol Gamble
- Centre for Medical Statistics and Health Evaluation, University of Liverpool, Liverpool, L69 3GS, UK.
| | - Paula R Williamson
- Centre for Medical Statistics and Health Evaluation, University of Liverpool, Liverpool, L69 3GS, UK.
| |
Collapse
|
43
|
Curley GF, McAuley DF. Clinical trial design in prevention and treatment of acute respiratory distress syndrome. Clin Chest Med 2014; 35:713-27. [PMID: 25453420 DOI: 10.1016/j.ccm.2014.08.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
Abstract
Our ability to define appropriate molecular targets for preclinical development and develop better methods needs to be improved, to determine the clinical value of novel acute respiratory distress syndrome (ARDS) agents. Clinical trials must have realistic sample sizes and meaningful end points and use the available observation and meta-analytical data to inform design. Biomarker-driven studies or defined ARDS subsets should be considered to categorize specific at-risk populations most likely to benefit from a new treatment. Innovations in clinical trial design should be pursued to improve the outlook for future interventional trials in ARDS.
Collapse
Affiliation(s)
- Gerard F Curley
- Department of Anesthesia, Keenan Research Centre for Biomedical Science, Li Ka Shing Knowledge Institute, St Michael's Hospital, 30, Bond Street, Toronto, Ontario M5B 1W8, Canada
| | - Daniel F McAuley
- School of Medicine, Dentistry and Biomedical Science, Centre for Infection and Immunity, Queen's University Belfast, Health Sciences Building, 97 Lisburn Road, Belfast, Northern Ireland BT9 7BL, UK.
| |
Collapse
|
44
|
Affiliation(s)
| | - Magne Nylenna
- The Norwegian Knowledge Centre for the Health Services, Oslo, Norway
| |
Collapse
|
45
|
Nikolakopoulou A, Mavridis D, Salanti G. Using conditional power of network meta-analysis (NMA) to inform the design of future clinical trials. Biom J 2014; 56:973-90. [PMID: 25225031 DOI: 10.1002/bimj.201300216] [Citation(s) in RCA: 26] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2013] [Revised: 05/09/2014] [Accepted: 05/31/2014] [Indexed: 01/19/2023]
Abstract
Clinical trials are typically designed with an aim to reach sufficient power to test a hypothesis about relative effectiveness of two or more interventions. Their role in informing evidence-based decision-making demands, however, that they are considered in the context of the existing evidence. Consequently, their planning can be informed by characteristics of relevant systematic reviews and meta-analyses. In the presence of multiple competing interventions the evidence base has the form of a network of trials, which provides information not only about the required sample size but also about the interventions that should be compared in a future trial. In this paper we present a methodology to evaluate the impact of new studies, their information size, the comparisons involved, and the anticipated heterogeneity on the conditional power (CP) of the updated network meta-analysis. The methods presented are an extension of the idea of CP initially suggested for a pairwise meta-analysis and we show how to estimate the required sample size using various combinations of direct and indirect evidence in future trials. We apply the methods to two previously published networks and we show that CP for a treatment comparison is dependent on the magnitude of heterogeneity and the ratio of direct to indirect information in existing and future trials for that comparison. Our methodology can help investigators calculate the required sample size under different assumptions about heterogeneity and make decisions about the number and design of future studies (set of treatments compared).
Collapse
Affiliation(s)
- Adriani Nikolakopoulou
- Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, University Campus, Ioannina, 45110, Greece
| | | | | |
Collapse
|
46
|
Clarke M, Brice A, Chalmers I. Accumulating research: a systematic account of how cumulative meta-analyses would have provided knowledge, improved health, reduced harm and saved resources. PLoS One 2014; 9:e102670. [PMID: 25068257 PMCID: PMC4113310 DOI: 10.1371/journal.pone.0102670] [Citation(s) in RCA: 67] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2014] [Accepted: 06/20/2014] [Indexed: 12/17/2022] Open
Abstract
Background “Cumulative meta-analysis” describes a statistical procedure to calculate, retrospectively, summary estimates from the results of similar trials every time the results of a further trial in the series had become available. In the early 1990s, comparisons of cumulative meta-analyses of treatments for myocardial infarction with advice promulgated through medical textbooks showed that research had continued long after robust estimates of treatment effects had accumulated, and that medical textbooks had overlooked strong, existing evidence from trials. Cumulative meta-analyses have subsequently been used to assess what could have been known had new studies been informed by systematic reviews of relevant existing evidence and how waste might have been reduced. Methods and Findings We used a systematic approach to identify and summarise the findings of cumulative meta-analyses of studies of the effects of clinical interventions, published from 1992 to 2012. Searches were done of PubMed, MEDLINE, EMBASE, the Cochrane Methodology Register and Science Citation Index. A total of 50 eligible reports were identified, including more than 1,500 cumulative meta-analyses. A variety of themes are illustrated with specific examples. The studies showed that initially positive results became null or negative in meta-analyses as more trials were done; that early null or negative results were over-turned; that stable results (beneficial, harmful and neutral) would have been seen had a meta-analysis been done before the new trial; and that additional trials had been much too small to resolve the remaining uncertainties. Conclusions This large, unique collection of cumulative meta-analyses highlights how a review of the existing evidence might have helped researchers, practitioners, patients and funders make more informed decisions and choices about new trials over decades of research. This would have led to earlier uptake of effective interventions in practice, less exposure of trial participants to less effective treatments, and reduced waste resulting from unjustified research.
Collapse
Affiliation(s)
- Mike Clarke
- All-Ireland Hub for Trials Methodology Research, Queen's University Belfast, Belfast, United Kingdom
- * E-mail:
| | - Anne Brice
- James Lind Initiative, Oxford, United Kingdom
| | | |
Collapse
|
47
|
Lund H. From evidence-based practice to evidence-based research – Reaching research-worthy problems by applying an evidence-based approach. EUROPEAN JOURNAL OF PHYSIOTHERAPY 2014. [DOI: 10.3109/21679169.2014.917838] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022]
|
48
|
Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gülmezoglu AM, Howells DW, Ioannidis JPA, Oliver S. How to increase value and reduce waste when research priorities are set. Lancet 2014; 383:156-65. [PMID: 24411644 DOI: 10.1016/s0140-6736(13)62229-1] [Citation(s) in RCA: 862] [Impact Index Per Article: 86.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
The increase in annual global investment in biomedical research--reaching US$240 billion in 2010--has resulted in important health dividends for patients and the public. However, much research does not lead to worthwhile achievements, partly because some studies are done to improve understanding of basic mechanisms that might not have relevance for human health. Additionally, good research ideas often do not yield the anticipated results. As long as the way in which these ideas are prioritised for research is transparent and warranted, these disappointments should not be deemed wasteful; they are simply an inevitable feature of the way science works. However, some sources of waste cannot be justified. In this report, we discuss how avoidable waste can be considered when research priorities are set. We have four recommendations. First, ways to improve the yield from basic research should be investigated. Second, the transparency of processes by which funders prioritise important uncertainties should be increased, making clear how they take account of the needs of potential users of research. Third, investment in additional research should always be preceded by systematic assessment of existing evidence. Fourth, sources of information about research that is in progress should be strengthened and developed and used by researchers. Research funders have primary responsibility for reductions in waste resulting from decisions about what research to do.
Collapse
Affiliation(s)
| | - Michael B Bracken
- School of Public Health and School of Medicine, Yale University, New Haven, CT, USA
| | - Ben Djulbegovic
- Center for Evidence-Based Medicine and Health Outcomes Research, Division of Internal Medicine, University of South Florida, Tampa, FL, USA; Department of Hematology and Department of Health Outcomes and Behavior, H Lee Moffitt Cancer Center and Research Institute, Tampa, FL, USA
| | - Silvio Garattini
- Istituto di Ricovero e Cura a Carattere Scientifico Istituto di Ricerche Farmacologiche Mario Negri, Milan, Italy
| | | | - A Metin Gülmezoglu
- UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP), WHO, Geneva, Switzerland
| | - David W Howells
- Florey Institute of Neuroscience and Mental Health, Melbourne, VIC, Australia
| | - John P A Ioannidis
- Stanford Prevention Research Center, Department of Medicine, School of Medicine, Stanford University, Stanford, CA, USA; Division of Epidemiology, Department of Health Research and Policy, School of Medicine, Stanford University, Stanford, CA, USA; Department of Statistics, School of Humanities and Sciences, Stanford University, Stanford, CA, USA; Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA
| | | |
Collapse
|
49
|
Jamtvedt G, Lund H, Nortvedt MW. Kunnskapsbasert forskning? TIDSSKRIFT FOR DEN NORSKE LEGEFORENING 2014; 134:10-1. [DOI: 10.4045/tidsskr.13.1324] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022] Open
|