1
|
Cooner F, Liao R, Lin J, Barthel S, Seifu Y, Ruan S. Leveraging Real-World Data in COVID-19 Response. Stat Biopharm Res 2022. [DOI: 10.1080/19466315.2022.2096688] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/17/2022]
Affiliation(s)
- Freda Cooner
- Amgen Inc., One Amgen Center Dr., Thousand Oaks, CA, USA
| | - Ran Liao
- Eli Lilly & Co, Lilly Corporate Center, Indianapolis, IN, USA
| | - Junjing Lin
- Takeda Pharmaceutical Co. Limited, Cambridge, MA, USA
| | | | | | | |
Collapse
|
2
|
Jacklin C, Rodrigues JN, Collins J, Cook J, Harrison CJ. Sample size calculations in high-profile surgical trials that use patient-reported outcome measures: systematic review. Br J Surg 2021; 109:178-181. [PMID: 34915565 DOI: 10.1093/bjs/znab421] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2021] [Accepted: 11/10/2021] [Indexed: 11/14/2022]
Abstract
Sample size calculations from high-profile surgical RCTs that used a patient-reported outcome measure as primary outcome were reviewed systematically against Difference ELicitation in TriAls (DELTA2) standards, with a focus on target differences. In this sample of trials, there was frequent use of suboptimal methods to determine the target difference, and sample size calculations were generally not reported to DELTA2 standards. This risks over-recruitment and/or erroneous trial conclusions, which clinicians should be aware of when interpreting published trials.
Collapse
Affiliation(s)
- Chloe Jacklin
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Jeremy N Rodrigues
- Department of Plastic Surgery, Stoke Mandeville Hospital, Buckinghamshire Healthcare NHS Trust, Aylesbury, UK.,Warwick Clinical Trials Unit, Warwick Medical School, University of Warwick, Coventry, UK
| | - Joanna Collins
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Jonathan Cook
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Conrad J Harrison
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| |
Collapse
|
3
|
Laigaard J, Pedersen C, Rønsbo TN, Mathiesen O, Karlsen APH. Minimal clinically important differences in randomised clinical trials on pain management after total hip and knee arthroplasty: a systematic review. Br J Anaesth 2021; 126:1029-1037. [PMID: 33678402 DOI: 10.1016/j.bja.2021.01.021] [Citation(s) in RCA: 161] [Impact Index Per Article: 53.7] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2020] [Revised: 12/19/2020] [Accepted: 01/10/2021] [Indexed: 01/02/2023] Open
Abstract
BACKGROUND Sample size determination is essential for reliable hypothesis testing in clinical trials and should rely on adequate sample size calculations with alpha, beta, variance, and an effect size being the minimal clinically important difference (MCID). This facilitates interpretation of the clinical relevance of statistically significant results. No gold standard for MCIDs exists in postoperative pain research. METHODS We searched Cochrane Central Register of Controlled Trials, MEDLINE, and Embase for English language articles on randomised trials investigating analgesic interventions after total hip or knee arthroplasty. Primary outcomes were the reported MCIDs for pain score and cumulated rescue opioid consumption. Secondary outcomes included reported sample size calculations and propensity to report statistical significance without reaching MCID. Trend analyses were conducted using statistical process control. RESULTS We included 570 trials. Median MCID for 0-24 h opioid consumption was 10 mg i.v. morphine equivalents for absolute reductions (interquartile range [IQR]: 6.8-14.5) and relative 40% (IQR: 30-50%). Median MCIDs for pain scores were absolute 15 mm at rest (IQR: 10-20) and 18 mm during movement (IQR: 10-20) on a 0-100 mm VAS and relative 30% (IQR: 20-30%). No trends were demonstrated for MCIDs. Adequate sample size calculations were reported in 34% of trials. In 46% of trials with statistically significant primary outcomes, the differences did not reach the predetermined MCID. CONCLUSIONS We provide clinician-perceived MCID estimates for rescue opioid consumption and pain scores that can be used for sample size calculations until reliable evidence-based patient-rated MCIDs emerge. Nearly half of the trials with significant findings did not reach the predetermined MCID.
Collapse
Affiliation(s)
- Jens Laigaard
- Centre for Anaesthesiological Research, Department of Anaesthesia, Zealand University Hospital, Køge, Denmark.
| | - Casper Pedersen
- Centre for Anaesthesiological Research, Department of Anaesthesia, Zealand University Hospital, Køge, Denmark
| | - Thea Nørgaard Rønsbo
- Centre for Anaesthesiological Research, Department of Anaesthesia, Zealand University Hospital, Køge, Denmark
| | - Ole Mathiesen
- Centre for Anaesthesiological Research, Department of Anaesthesia, Zealand University Hospital, Køge, Denmark; Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark
| | - Anders Peder Højer Karlsen
- Centre for Anaesthesiological Research, Department of Anaesthesia, Zealand University Hospital, Køge, Denmark
| |
Collapse
|
4
|
Shen J, Breckons M, Vale L, Pickard R. Using Time Trade-Off Methods to Elicit Short-Term Utilities Associated with Treatments for Bulbar Urethral Stricture. PHARMACOECONOMICS - OPEN 2019; 3:551-558. [PMID: 31240689 PMCID: PMC6861395 DOI: 10.1007/s41669-019-0133-4] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
BACKGROUND Recurrent urethral stricture is usually treated with either open urethroplasty or endoscopic urethrotomy. Both of the procedures cause short-term utility loss, which may not be captured by standard utility questionnaires due to the challenges of completing a standard instrument at the time of an acute episode of short duration, especially within a clinical trial setting. We propose to use time trade-off (TTO) methods to estimate these short-term utility losses. OBJECTIVE The aim was to compare the use of two alternative TTO methods to elicit patients' short-term utilities following surgical treatments for recurrent urethral stricture. METHOD Two variants of TTO (chained and conventional) were used. Six health profiles were developed-three for each procedure. Forty participants took part, with 20 randomly allocated to each TTO method. RESULTS Thirty-eight participants provided usable data for analysis. Estimated utility values decreased as the severity of the health profiles increased. There was no evidence that utility values differed between elicitation methods or procedures for mild {ranging from 0.79 (standard deviation [SD] 0.17) to 0.83 [SD 0.20]} and moderate (ranging from 0.54 [SD 0.24] to 0.67 [SD 0.21]) health states, although they appeared to differ for severe health states (ranging from 0.29 [SD 0.20] to 0.56 [SD 0.24]). CONCLUSION The study demonstrates the feasibility and value of eliciting patients' short-term utilities. Given the small sample size, the study findings are tentative. Further research with a larger sample size is needed to determine the appropriate TTO method to use and how the elicited utilities can be used in combination with standard cost-utility assessments to aid decision making.
Collapse
Affiliation(s)
- Jing Shen
- Health Economics Group, Institute of Health and Society, Newcastle University, Baddiley-Clark Building, Richardson Road, Newcastle upon Tyne, Tyne and Wear, NE2 4AX, UK.
| | - Matthew Breckons
- Health Economics Group, Institute of Health and Society, Newcastle University, Baddiley-Clark Building, Richardson Road, Newcastle upon Tyne, Tyne and Wear, NE2 4AX, UK
| | - Luke Vale
- Health Economics Group, Institute of Health and Society, Newcastle University, Baddiley-Clark Building, Richardson Road, Newcastle upon Tyne, Tyne and Wear, NE2 4AX, UK
| | - Robert Pickard
- Institute of Cellular Medicine, Newcastle University, Newcastle upon Tyne, UK
| |
Collapse
|
5
|
Hemming K, Taljaard M, McKenzie JE, Hooper R, Copas A, Thompson JA, Dixon-Woods M, Aldcroft A, Doussau A, Grayling M, Kristunas C, Goldstein CE, Campbell MK, Girling A, Eldridge S, Campbell MJ, Lilford RJ, Weijer C, Forbes AB, Grimshaw JM. Reporting of stepped wedge cluster randomised trials: extension of the CONSORT 2010 statement with explanation and elaboration. BMJ 2018; 363:k1614. [PMID: 30413417 PMCID: PMC6225589 DOI: 10.1136/bmj.k1614] [Citation(s) in RCA: 220] [Impact Index Per Article: 36.7] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 03/20/2018] [Indexed: 12/14/2022]
Affiliation(s)
- Karla Hemming
- Institute of Applied Health Research, University of Birmingham, Birmingham B15 2TT, UK
| | - Monica Taljaard
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, ON, Canada
- School of Epidemiology and Public Health, University of Ottawa, Ottawa, ON, Canada
| | - Joanne E McKenzie
- School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia
| | - Richard Hooper
- Centre for Primary Care and Public Health, Queen Mary University of London, London, UK
| | - Andrew Copas
- London Hub for Trials Methodology Research, MRC Clinical Trials Unit at University College London, London, UK
| | - Jennifer A Thompson
- London Hub for Trials Methodology Research, MRC Clinical Trials Unit at University College London, London, UK
- Department for Infectious Disease Epidemiology, London School of Hygiene and Tropical Medicine, London, UK
| | - Mary Dixon-Woods
- The Healthcare Improvement Studies Institute, University of Cambridge, Cambridge Biomedical Campus, Cambridge, UK
| | | | - Adelaide Doussau
- Biomedical Ethics Unit, McGill University School of Medicine, Montreal, QC, Canada
| | | | | | - Cory E Goldstein
- Rotman Institute of Philosophy, Western University, London, ON, Canada
| | | | - Alan Girling
- Institute of Applied Health Research, University of Birmingham, Birmingham B15 2TT, UK
| | - Sandra Eldridge
- Centre for Primary Care and Public Health, Queen Mary University of London, London, UK
| | | | | | - Charles Weijer
- Rotman Institute of Philosophy, Western University, London, ON, Canada
| | - Andrew B Forbes
- School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia
| | - Jeremy M Grimshaw
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, ON, Canada
- School of Epidemiology and Public Health, University of Ottawa, Ottawa, ON, Canada
- Department of Medicine, University of Ottawa, Ottawa, ON, Canada
| |
Collapse
|
6
|
Bell ML. New guidance to improve sample size calculations for trials: eliciting the target difference. Trials 2018; 19:605. [PMID: 30396364 PMCID: PMC6219024 DOI: 10.1186/s13063-018-2894-y] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2018] [Accepted: 08/31/2018] [Indexed: 11/26/2022] Open
Abstract
BACKGROUND Sample size calculations are central to the design of health research trials. To ensure that the trial provides good evidence to answer the trial's research question, the target effect size (difference in means or proportions, odds ratio, relative risk or hazard ratio between trial arms) must be specified under the conventional approach to determining the sample size. However, until now, there has not been comprehensive guidance on how to specify this effect. MAIN TEXT This is a commentary on a collection of papers from two important projects, DELTA (Difference ELicitation in TriAls) and DELTA2 that aim to provide evidence-based guidance on systematically determining the target effect size, or difference and the resultant sample sizes for trials. In addition to surveying methods that researchers are using in practice, the research team met with various experts (statisticians, methodologists, clinicians and funders); reviewed guidelines from funding agencies; and reviewed recent methodological literature. The DELTA2 guidance stresses specifying important and realistic differences, and undertaking sensitivity analyses in calculating sample sizes. It gives recommendations on how to find appropriate differences, conduct the sample size calculation(s) and how to report these in grant applications, protocols and manuscripts. It is hoped that this will contribute not only to better powered studies, but better reporting and reproducibility and thinking about what a trial should be designed to achieve. CONCLUSIONS The DELTA researchers have developed a set of comprehensive guidance documents that are welcome and will almost certainly improve the way that trials are designed and reported.
Collapse
Affiliation(s)
- Melanie L Bell
- Department of Epidemiology and Biostatistics, Mel and Enid Zuckerman College of Public Health, University of Arizona, Tucson, AZ, 85724, USA.
| |
Collapse
|
7
|
Sones W, Julious SA, Rothwell JC, Ramsay CR, Hampson LV, Emsley R, Walters SJ, Hewitt C, Bland M, Fergusson DA, Berlin JA, Altman D, Vale LD, Cook JA. Choosing the target difference and undertaking and reporting the sample size calculation for a randomised controlled trial - the development of the DELTA 2 guidance. Trials 2018; 19:542. [PMID: 30305155 PMCID: PMC6180499 DOI: 10.1186/s13063-018-2887-x] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/09/2018] [Accepted: 08/29/2018] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND A key step in the design of a randomised controlled trial is the estimation of the number of participants needed. The most common approach is to specify a target difference in the primary outcome between the randomised groups and then estimate the corresponding sample size. The sample size is chosen to provide reassurance that the trial will have high statistical power to detect the target difference at the planned statistical significance level. Alternative approaches are also available, though most still require specification of a target difference. The sample size has many implications for the conduct of the study, as well as incurring scientific and ethical aspects. Despite the critical role of the target difference for the primary outcome in the design of a randomised controlled trial (RCT), the manner in which it is determined has received little attention. This article reports the development of the DELTA2 guidance on the specification and reporting of the target difference for the primary outcome in a sample size calculation for a RCT. METHODS The DELTA2 (Difference ELicitation in TriAls) project has five components comprising systematic literature reviews of recent methodological developments (stage 1) and existing funder guidance (stage 2), a Delphi study (stage 3), a 2-day consensus meeting bringing together researchers, funders and patient representatives (stage 4), and the preparation and dissemination of a guidance document (stage 5). RESULTS The project started in April 2016. The literature search identified 28 articles of methodological developments relevant to a method for specifying a target difference. A Delphi study involving 69 participants, along with a 2-day consensus meeting were conducted. In addition, further engagement sessions were held at two international conferences. The main guidance text was finalised on April 18, 2018, after revision informed by feedback gathered from stages 2 and 3 and from funder representatives. DISCUSSION The DELTA2 Delphi study identified a number of areas (such as practical recommendations and examples, greater coverage of different trial designs and statistical approaches) of particular interest amongst stakeholders which new guidance was desired to meet. New relevant references were identified by the review. Such findings influenced the scope, drafting and revision of the guidance. While not all suggestions could be accommodated, it is hoped that the process has led to a more useful and practical document.
Collapse
Affiliation(s)
- William Sones
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Botnar Research Centre, Nuffield Orthopaedic Centre, Windmill Rd, Oxford, OX3 7LD, UK
| | - Steven A Julious
- Medical Statistics Group, ScHARR, The University of Sheffield, Regent Court, 30 Regent Street, Sheffield, S1 4DA, UK
| | - Joanne C Rothwell
- Medical Statistics Group, ScHARR, The University of Sheffield, Regent Court, 30 Regent Street, Sheffield, S1 4DA, UK
| | - Craig Robert Ramsay
- Health Services Research Unit, University of Aberdeen, Health Sciences Building, Foresterhill, Aberdeen, AB25 2ZD, UK
| | - Lisa V Hampson
- Department of Mathematics and Statistics, Lancaster University, Lancaster, LA1 4YF, UK.,Statistical Methodology and Consulting, Novartis Pharma AG, Basel, Switzerland
| | - Richard Emsley
- Department of Biostatistics and Health Informatics, Institute of Psychiatry, Psychology and Neuroscience, King's College London, De Crespigny Park, Denmark Hill, London, SE5 8AF, UK
| | - Stephen J Walters
- Medical Statistics Group, ScHARR, The University of Sheffield, Regent Court, 30 Regent Street, Sheffield, S1 4DA, UK
| | - Catherine Hewitt
- Department of Health Sciences, Seebohm Rowntree Building, University of York, Heslington, York, YO10 5DD, UK
| | - Martin Bland
- Department of Health Sciences, Seebohm Rowntree Building, University of York, Heslington, York, YO10 5DD, UK
| | - Dean A Fergusson
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, 501 Smyth Road, Box 201B, Ottawa, ON, K1H 8L6, Canada
| | - Jesse A Berlin
- Johnson & Johnson, One J&J Plaza, New Brunswick, NJ, 08933, USA
| | - Doug Altman
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Botnar Research Centre, Nuffield Orthopaedic Centre, Windmill Rd, Oxford, OX3 7LD, UK
| | - Luke David Vale
- Health Economics Group, Institute of Health and Society, Newcastle University, Newcastle upon Tyne, UK
| | - Jonathan Alistair Cook
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Botnar Research Centre, Nuffield Orthopaedic Centre, Windmill Rd, Oxford, OX3 7LD, UK.
| |
Collapse
|
8
|
Rothwell JC, Julious SA, Cooper CL. A study of target effect sizes in randomised controlled trials published in the Health Technology Assessment journal. Trials 2018; 19:544. [PMID: 30305146 PMCID: PMC6180439 DOI: 10.1186/s13063-018-2886-y] [Citation(s) in RCA: 31] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2018] [Accepted: 08/29/2018] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND When designing a randomised controlled trial (RCT), an important consideration is the sample size required. This is calculated from several components; one of which is the target difference. This study aims to review the currently reported methods of elicitation of the target difference as well as to quantify the target differences used in Health Technology Assessment (HTA)-funded trials. METHODS Trials were identified from the National Institute of Health Research Health Technology Assessment journal. A total of 177 RCTs published between 2006 and 2016 were assessed for eligibility. Eligibility was established by the design of the trial and the quality of data available. The trial designs were parallel-group, superiority RCTs with a continuous primary endpoint. Data were extracted and the standardised anticipated and observed effect size estimates were calculated. Exclusion criteria was based on trials not providing enough detail in the sample size calculation and results, and trials not being of parallel-group, superiority design. RESULTS A total of 107 RCTs were included in the study from 102 reports. The most commonly reported method for effect size derivation was a review of evidence and use of previous research (52.3%). This was common across all clinical areas. The median standardised target effect size was 0.30 (interquartile range: 0.20-0.38), with the median standardised observed effect size 0.11 (IQR 0.05-0.29). The maximum anticipated and observed effect sizes were 0.76 and 1.18, respectively. Only two trials had anticipated target values above 0.60. CONCLUSION The most commonly reported method of elicitation of the target effect size is previous published research. The average target effect size was 0.3. A clear distinction between the target difference and the minimum clinically important difference is recommended when designing a trial. Transparent explanation of target difference elicitation is advised, with multiple methods including a review of evidence and opinion-seeking advised as the more optimal methods for effect size quantification.
Collapse
Affiliation(s)
- Joanne C. Rothwell
- School of Health and Related Research, University of Sheffield, Sheffield, UK
- The Medical Statistics Group, School of Health and Related Research (ScHARR), University of Sheffield, Regent Court, 30 Regent Street, Sheffield, S1 4DA UK
| | - Steven A. Julious
- School of Health and Related Research, University of Sheffield, Sheffield, UK
| | - Cindy L. Cooper
- Sheffield Clinical Trials Unit, University of Sheffield, Sheffield, UK
| |
Collapse
|
9
|
Bigirumurame T, Kasim AS. Can testing clinical significance reduce false positive rates in randomized controlled trials? A snap review. BMC Res Notes 2017; 10:775. [PMID: 29282120 PMCID: PMC5745896 DOI: 10.1186/s13104-017-3117-4] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2017] [Accepted: 12/21/2017] [Indexed: 11/10/2022] Open
Abstract
OBJECTIVE The use of minimum clinically important difference in the hypothesis formulation for superiority trials is similar in principle to the concept of non-inferiority or equivalence trial. However, most clinical trials are analysed testing zero clinical difference. Since the minimum clinically important difference is pre-defined for power calculation, it is important to incorporate it in both the hypothesis testing and the interpretation of findings from clinical trials. RESULTS We reviewed a set of 50 publications (25 with binary outcome, and 25 with survival time outcome). 20% of the 50 published trials that were statistically significant, were also clinically significant based on the minimum clinically important risk differences (or hazard ratio) used for their power calculations. This snap review seems to suggest that most published trials with statistically significant results were less likely to be clinically significant, which may partly explain the high false positive findings associated with findings from superiority trials. Furthermore, none of the reviewed publications explicitly used minimum clinically important difference in the interpretation of their findings. However, a systematic review is needed to critically appraise the impact of the current practice on false positive rate in published trials with significant findings.
Collapse
Affiliation(s)
| | - Adetayo S Kasim
- Durham University, University boulevard, Stockton-on-Tees, TS17 6BH, UK
| |
Collapse
|