1
|
Mutter A, Küchler AM, Idrees AR, Kählke F, Terhorst Y, Baumeister H. StudiCare procrastination - Randomized controlled non-inferiority trial of a persuasive design-optimized internet- and mobile-based intervention with digital coach targeting procrastination in college students. BMC Psychol 2023; 11:273. [PMID: 37700387 PMCID: PMC10496391 DOI: 10.1186/s40359-023-01312-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2023] [Accepted: 09/05/2023] [Indexed: 09/14/2023] Open
Abstract
BACKGROUND Academic procrastination is widespread among college students. Procrastination is strongly negatively correlated with psychological well-being, thus early interventions are needed. Internet- and mobile-based cognitive behavioral therapy (iCBT) could provide a low-threshold treatment option. Human guidance seems to be a decisive mechanism of change in iCBT. Persuasive design optimization of iCBT and guidance by a digital coach might represent a resource-saving alternative. The study evaluated the non-inferiority of a digital coach in comparison to human guidance with regard to the primary outcome procrastination. METHODS The iCBT StudiCare procrastination was optimized by principles of the Persuasive System Design (PSD). A total of 233 college students were randomly assigned to either StudiCare procrastination guided by a digital coach (intervention group, IG) or by a human eCoach (control group, CG). All participants were assessed at baseline, 4-, 8- and 12-weeks post-randomization. Symptom change and between-group differences were assessed with latent growth curve models and supported by effect size levels. The non-inferiority margin was set at Cohen's d = - 0.3. RESULTS The primary outcome procrastination measured by the Irrational Procrastination scale (IPS) significantly decreased across groups (γ = - 0.79, p < .001, Cohen's d = -0.43 to -0.89) from baseline to 12-weeks post-randomization. There were no significant differences between groups (γ = -0.03, p = .84, Cohen's d = -0.03 to 0.08). Regarding symptoms of depression, no significant time x group effect was found (γ = 0.26, p = .09; Cohen's d = -0.15 to 0.21). There was also no significant time x group effect on the improvement of symptoms of anxiety (γ = 0.25, p = .09). However, Cohen's ds were above the non-inferiority margin 8-weeks (Cohen's d = 0.51) and 12-weeks post-randomization (Cohen's d = 0.37), preferring the CG. Of the IG, 34% and of the CG, 36% completed 80% of the modules. CONCLUSIONS The PSD optimized version of StudiCare procrastination is effective in reducing procrastination. The digital coach was not inferior to human guidance. Guidance by a digital coach in iCBT against procrastination for college students could be a resource-saving alternative to human guidance. TRIAL REGISTRATION The trial was registered at the WHO International Clinical Trials Registry Platform via the German Clinical Trial Register (ID: DRKS00025209, 30/04/2021).
Collapse
Affiliation(s)
- Agnes Mutter
- Department of Clinical Psychology and Psychotherapy, Institute of Psychology and Education, Faculty of Engineering, Ulm University, Ulm, Germany.
| | - A-M Küchler
- Department of Clinical Psychology and Psychotherapy, Institute of Psychology and Education, Faculty of Engineering, Ulm University, Ulm, Germany
| | - A R Idrees
- Department of Clinical Psychology and Psychotherapy, Institute of Psychology and Education, Faculty of Engineering, Ulm University, Ulm, Germany
- Institute of Databases and Information Systems (DBIS), Ulm University, Ulm, Germany
| | - F Kählke
- Department of Sport and Health Sciences, Professorship of Psychology and Digital Mental Health Care, Technische Universität München, Munich, Germany
| | - Y Terhorst
- Department of Clinical Psychology and Psychotherapy, Institute of Psychology and Education, Faculty of Engineering, Ulm University, Ulm, Germany
| | - H Baumeister
- Department of Clinical Psychology and Psychotherapy, Institute of Psychology and Education, Faculty of Engineering, Ulm University, Ulm, Germany
| |
Collapse
|
2
|
Engelbrektsson J, Salomonsson S, Högström J, Sorjonen K, Sundell K, Forster M. Parent Training via Internet or in Group for Disruptive Behaviors: A Randomized Clinical Noninferiority Trial. J Am Acad Child Adolesc Psychiatry 2023; 62:987-997. [PMID: 36863414 DOI: 10.1016/j.jaac.2023.01.019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/15/2022] [Revised: 01/01/2023] [Accepted: 02/21/2023] [Indexed: 03/04/2023]
Abstract
OBJECTIVE To evaluate if an internet-delivered parent training program is noninferior to its group-delivered counterpart in reducing child disruptive behavior problems (DBP). METHOD This noninferiority randomized clinical trial enrolled families seeking treatment in primary care in Stockholm, Sweden, for DBP in a child 3-11 years of age. Participants were randomized to internet-delivered (iComet) or group-delivered (gComet) parent training. The primary outcome was parent-rated DBP. Assessments were made at baseline and 3, 6, and 12 months. Secondary outcomes included child and parent behaviors and well-being and treatment satisfaction. The noninferiority analysis was determined by a one-sided 95% CI of the mean difference between gComet and iComet using multilevel modeling. RESULTS This trial included 161 children (mean age 8.0); 102 (63%) were boys. In both intention-to-treat and per-protocol analyses, iComet was noninferior to gComet. There were small differences in between-group effect sizes (d = -0.02 to 0.13) on the primary outcome with the upper limit of the one-sided 95% CI below the noninferiority margin at 3-, 6-, and 12-month follow-up. Parents were more satisfied with gComet (d = 0.49, 95% CI [0.26, 0.71]). At 3-month follow-up, there were also significant differences in treatment effect on attention-deficit/hyperactivity disorder symptoms (d = 0.34, 95% CI [0.07, 0.61]) and parenting behavior (d = 0.41, 95% CI [0.17, 0.65]) favoring gComet. At 12-month follow-up, there were no differences in any outcomes. CONCLUSION Internet-delivered parent training was noninferior to group-delivered parent training in reducing child DBP. The results were maintained at 12-month follow-up. This study supports internet-delivered parent training being used as an alternative to group-delivered parent training in clinical settings. CLINICAL TRIAL REGISTRATION INFORMATION Randomized Controlled Trial of Comet via the Internet or in Group Format; https://www. CLINICALTRIALS gov/; NCT03465384.
Collapse
Affiliation(s)
| | - Sigrid Salomonsson
- Centre for Psychiatry Research, Karolinska Institutet and Region Stockholm, Sweden
| | - Jens Högström
- Centre for Psychiatry Research, Karolinska Institutet and Region Stockholm, Sweden
| | | | - Knut Sundell
- Swedish Agency for Health Technology Assessment and Assessment of Social Services, Stockholm, Sweden
| | | |
Collapse
|
3
|
Hu RT, Royse AG, Royse C, Scott DA, Bowyer A, Boggett S, Summers P, Mazer CD. Health-related quality of life after restrictive versus liberal RBC transfusion for cardiac surgery: Sub-study from a randomized clinical trial. Transfusion 2022; 62:1973-1983. [PMID: 36066319 DOI: 10.1111/trf.17084] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2022] [Revised: 06/20/2022] [Accepted: 06/21/2022] [Indexed: 11/30/2022]
Abstract
BACKGROUND Transfusion Requirements in Cardiac Surgery III (TRICS III), a multi-center randomized controlled trial, demonstrated clinical non-inferiority for restrictive versus liberal RBC transfusion for patients undergoing cardiac surgery. However, it is uncertain if transfusion strategy affects long-term health-related quality of life (HRQOL). STUDY DESIGN AND METHODS In this planned sub-study of Australian patients in TRICS III, we sought to determine the non-inferiority of restrictive versus liberal transfusion strategy on long-term HRQOL and to describe clinical outcomes 24 months postoperatively. The restrictive strategy involved transfusing RBCs when hemoglobin was <7.5 g/dl; the transfusion triggers in the liberal group were: <9.5 g/L intraoperatively, <9.5 g/L in intensive care, or <8.5 g/dl on the ward. HRQOL assessments were performed using the 36-item short form survey version 2 (SF-36v2). Primary outcome was non-inferiority of summary measures of SF-36v2 at 12 months, (non-inferiority margin: -0.25 effect size; restrictive minus liberal scores). Secondary outcomes included non-inferiority of HRQOL at 18 and 24 months. RESULTS Six hundred seventeen Australian patients received allocated randomization; HRQOL data were available for 208/311 in restrictive and 217/306 in liberal group. After multiple imputation, non-inferiority of restrictive transfusion at 12 months was not demonstrated for HRQOL, and the estimates were directionally in favor of liberal transfusion. Non-inferiority also could not be concluded at 18 and 24 months. Sensitivity analyses supported these results. There were no differences in quality-adjusted life years or composite clinical outcomes up to 24 months after surgery. DISCUSSION The non-inferiority of a restrictive compared to a liberal transfusion strategy was not established for long-term HRQOL in this dataset.
Collapse
Affiliation(s)
- Raymond T Hu
- Department of Surgery, University of Melbourne, Parkville, Victoria, Australia.,Department of Anaesthesia, Austin Health, Heidelberg, Victoria, Australia
| | - Alistair G Royse
- Department of Surgery, University of Melbourne, Parkville, Victoria, Australia.,Department of Anaesthesia and Pain Management, Royal Melbourne Hospital, Parkville, Victoria, Australia
| | - Colin Royse
- Department of Surgery, University of Melbourne, Parkville, Victoria, Australia.,Department of Anaesthesia and Pain Management, Royal Melbourne Hospital, Parkville, Victoria, Australia.,Outcomes Research Consortium, The Cleveland Clinic, Cleveland, Ohio, USA
| | - David A Scott
- Department of Surgery, University of Melbourne, Parkville, Victoria, Australia.,Department of Anaesthesia and Acute Pain Medicine, St Vincent's Hospital Melbourne, Fitzroy, Victoria, Australia
| | - Andrea Bowyer
- Department of Surgery, University of Melbourne, Parkville, Victoria, Australia.,Department of Anaesthesia and Pain Management, Royal Melbourne Hospital, Parkville, Victoria, Australia
| | - Stuart Boggett
- Department of Surgery, University of Melbourne, Parkville, Victoria, Australia
| | - Peter Summers
- Statistical Consulting Centre, University of Melbourne, Parkville, Victoria, Australia.,Melbourne Disability Institute, University of Melbourne, Parkville, Victoria, Australia.,Centre for Health Analytics, Murdoch Children's Research Institute, Royal Children's Hospital, Parkville, Victoria, Australia
| | - Cyril David Mazer
- Department of Anaesthesia, St. Michael's Hospital, University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|
4
|
Pong S, Urner M, Fowler RA, Mitsakakis N, Seto W, Hutchison JS, Science M, Daneman N. Testing for non-inferior mortality: a systematic review of non-inferiority margin sizes and trial characteristics. BMJ Open 2021; 11:e044480. [PMID: 33879485 PMCID: PMC8061825 DOI: 10.1136/bmjopen-2020-044480] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/17/2022] Open
Abstract
OBJECTIVE To describe the size and variability of non-inferiority margins used in non-inferiority trials of medications with primary outcomes involving mortality, and to examine the association between trial characteristics and non-inferiority margin size. DESIGN Systematic review. DATA SOURCES Medline, Medline In Process, Medline Epub Ahead of Print and Embase Classic+Embase databases from January 1989 to December 2019. ELIGIBILITY CRITERIA Prospective non-inferiority randomised controlled trials comparing pharmacological therapies, with primary analyses for non-inferiority and primary outcomes involving mortality alone or as part of a composite outcome. Trials had to prespecify non-inferiority margins as absolute risk differences or relative to risks of outcome and provide a baseline risk of primary outcome in the control intervention. RESULTS 3992 records were screened, 195 articles were selected for full text review and 111 articles were included for analyses. 82% of trials were conducted in thrombosis, infectious diseases or oncology. Mortality was the sole primary outcome in 23 (21%) trials, and part of a composite primary outcome in 88 (79%) trials. The overall median non-inferiority margin was an absolute risk difference of 9% (IQR 4.2%-10%). When non-inferiority margins were expressed relative to the baseline risk of primary outcome in control groups, the median relative non-inferiority margin was 1.5 (IQR 1.3-1.7). In multivariable regression analyses examining the association between trial characteristics (medical specialty, inclusion of paediatric patients, mortality as a sole or part of a composite primary outcome, presence of industry funding) and non-inferiority margin size, only medical specialty was significantly associated with non-inferiority margin size. CONCLUSION Absolute and relative non-inferiority margins used in published trials comparing medications are large, allowing conclusions of non-inferiority in the context of large differences in mortality. Accepting the potential for large increases in outcomes involving mortality while declaring non-inferiority is a challenging methodological issue in the conduct of non-inferiority trials.
Collapse
Affiliation(s)
- Sandra Pong
- Department of Pharmacy, The Hospital for Sick Children, Toronto, Ontario, Canada
| | - Martin Urner
- Interdepartmental Division of Critical Care Medicine, University of Toronto, Toronto, Ontario, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
| | - Robert A Fowler
- Interdepartmental Division of Critical Care Medicine, University of Toronto, Toronto, Ontario, Canada
- Tory Trauma Program, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada
| | - Nicholas Mitsakakis
- Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada
| | - Winnie Seto
- Department of Pharmacy, The Hospital for Sick Children, Toronto, Ontario, Canada
- Faculty of Pharmacy, University of Toronto, Toronto, Ontario, Canada
| | - James S Hutchison
- Department of Critical Care Medicine, The Hospital for Sick Children, Toronto, Ontario, Canada
| | - Michelle Science
- Division of Infectious Diseases, The Hospital for Sick Children, Toronto, Ontario, Canada
| | - Nick Daneman
- Division of Infectious Diseases, Department of Medicine, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada
| |
Collapse
|
5
|
Jan SL, Shieh G. A Comparative Study of TOST and UMPT Procedures for Evaluating Dispersion Equivalence. Stat Biopharm Res 2020. [DOI: 10.1080/19466315.2020.1821762] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Affiliation(s)
- Show-Li Jan
- Department of Applied Mathematics, Chung Yuan Christian University, Taoyuan, Taiwan
| | - Gwowen Shieh
- Department of Management Science, National Chiao Tung University, Hsinchu, Taiwan
| |
Collapse
|
6
|
Shieh G. Appraising Minimum Effect of Standardized Contrasts in ANCOVA Designs: Statistical Power, Sample Size, and Covariate Imbalance Considerations. Stat Biopharm Res 2020. [DOI: 10.1080/19466315.2020.1788982] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Affiliation(s)
- Gwowen Shieh
- Department of Management Science, National Chiao Tung University, Hsinchu, Taiwan, ROC
| |
Collapse
|
7
|
Lietman TM, Oldenburg CE, O'Brien KS. Noninferiority: It's All in the Margins. Ophthalmology 2020; 127:711-712. [PMID: 32444016 DOI: 10.1016/j.ophtha.2020.04.010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2019] [Revised: 04/08/2020] [Accepted: 04/08/2020] [Indexed: 11/17/2022] Open
|
8
|
Jan SL, Shieh G. Optimal contrast analysis with heterogeneous variances and budget concerns. PLoS One 2019; 14:e0214391. [PMID: 30913244 PMCID: PMC6435144 DOI: 10.1371/journal.pone.0214391] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2018] [Accepted: 03/12/2019] [Indexed: 11/27/2022] Open
Abstract
The omnibus test is commonly applied to evaluate the overall disparity between group means in ANOVA. Alternatively, linear contrasts are more informative in detecting specific pattern of mean differences that cannot be obtained via the omnibus test. This article concerns power and sample size calculations for contrast analysis with heterogeneous variances and budget concerns. Optimal allocation procedures for the Welch-Satterthwaite tests of standardized and unstandardized contrasts are presented to minimize the total sample size with the designated ratios, to meet a desirable power level for the least cost, and to attain the maximum power performance under a fixed cost. Currently available methods rely exclusively on simple allocation formula and direct rounding rule. The proposed allocation strategies combine the computing techniques of nonlinear optimization search and iterative screening process. Numerical assessments of a randomized control trial for the overcoming depression on the Internet are conducted to demonstrate and confirm that the approximate procedures do not guarantee optimal solution. The suggested approaches extend and outperform the existing findings in methodological soundness and overall performance. The corresponding computer algorithms are developed to implement the recommended power and sample size calculations for optimal contrast analysis.
Collapse
Affiliation(s)
- Show-Li Jan
- Department of Applied Mathematics, Chung Yuan Christian University, Taiwan, Republic of China
| | - Gwowen Shieh
- Department of Management Science, National Chiao Tung University, Taiwan, Republic of China
- * E-mail:
| |
Collapse
|
9
|
Aupiais C, Zohar S, Taverny G, Le Roux E, Boulkedid R, Alberti C. Exploring how non-inferiority and equivalence are assessed in paediatrics: a systematic review. Arch Dis Child 2018; 103:1067-1075. [PMID: 29794107 DOI: 10.1136/archdischild-2018-314874] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/26/2018] [Revised: 04/24/2018] [Accepted: 04/25/2018] [Indexed: 11/04/2022]
Abstract
OBJECTIVE To review characteristics, methodology and reporting of non-inferiority and equivalence trials in the specific context of paediatrics. DESIGN PubMed and Cochrane databases were searched (up to September 2016) for non-inferiority/equivalence randomised controlled trials conducted in children published in high-impact-factor journals (>5.0 for general/specialist medical journals; >2.2 for paediatric journals). RESULTS We found that the statistical hypothesis was inconsistent with the objective in 12 (10%) of the 125 reports included. Non-inferiority (n=98) and equivalence trials (n=27) were mostly used to evaluate interventions with easier administration (45%, n=54/120) and/or better safety profile (34%, n=41/120). All the data needed for targeted sample size recalculation were available for 39 reports (31%). The margin-representing the largest difference between arms that would be clinically acceptable-was reported in 119 (95%), and 44/119 (37%) reported the method used for margin determination. The median sample size was 268 (IQR 125-531). Margins were wider in smaller trials (<125 randomised patients) than in larger trials (p=0.04/p<0.01 for binary/continuous outcomes, respectively). We did not agree with the authors' conclusions in 11% (11/103) of the reports that provided sufficient information. CONCLUSIONS There is still a need to improve the quality of methodology, reporting and interpretation of non-inferiority/equivalence trials in paediatrics. In particular, the margins were often not justified and the conclusion was often not supported by the design and/or the results. As researchers have to cope with small sample size and with lack of evidence, methods for non-inferiority/equivalence trials need to be used and/or developed in this vulnerable population.
Collapse
Affiliation(s)
- Camille Aupiais
- Inserm UMR 1123-ECEVE, UMRS 1138, Team 22, CRC; APHP, Hôpital Robert Debré, University Paris Diderot, Paris, France
| | - Sarah Zohar
- Inserm, UMRS 1138, Team 22, CRC, University Paris Descartes, Sorbonne University, Paris, France
| | - Garry Taverny
- Inserm UMR 1123-ECEVE, University Paris Diderot, Paris, France
| | - Enora Le Roux
- Inserm UMR 1123-ECEVE and CIC-EC 1426, APHP, Hôpital Robert Debré, University Paris Diderot, Paris, France
| | - Rym Boulkedid
- Inserm UMR 1123-ECEVE and CIC-EC 1426, APHP, Hôpital Robert Debré, University Paris Diderot, Paris, France
| | - Corinne Alberti
- Inserm UMR 1123-ECEVE and CIC-EC 1426, APHP, Hôpital Robert Debré, University Paris Diderot, Paris, France
| |
Collapse
|
10
|
Leichsenring F, Abbass A, Driessen E, Hilsenroth M, Luyten P, Rabung S, Steinert C. Equivalence and non-inferiority testing in psychotherapy research. Psychol Med 2018; 48:1917-1919. [PMID: 29747714 DOI: 10.1017/s0033291718001289] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Affiliation(s)
- Falk Leichsenring
- Department of Psychosomatics and Psychotherapy, Justus-Liebig-University Giessen, Ludwigstr 76, D-35392 Giessen, Germany
| | - Allan Abbass
- Department of Psychiatry, Dalhousie University, Centre for Emotions and Health, Halifax 8203-5909 Veterans Memorial Lane, Halifax, NS B3H 2E2, Canada
| | - Ellen Driessen
- Department of Clinical, Neuro and Developmental Psychology, Amsterdam Public Health research institute, Vrije Universiteit Amsterdam, Van der Boechorststraat 1, 1081 BT Amsterdam, The Netherlands
| | - Mark Hilsenroth
- Derner School of Psychology, Adelphi University, Hy Weinberg Center, 1 South Avenue, Garden City, NY 11530-0701, USA
| | - Patrick Luyten
- Faculty of Psychology and Educational Sciences, University of Leuven, Klinische Psychologie (OE), Tiensestraat 102 - bus 3722, 3000 Leuven, Belgium
- Research Department of Clinical, Educational and Health Psychology, University College London, Gower Street, London WC1E 6BT, UK
| | - Sven Rabung
- Department of Psychology, Alpen-Adria-Universität Klagenfurt, Universitätsstr, 65-67, A-9020 Klagenfurt, Austria
| | - Christiane Steinert
- Department of Psychosomatics and Psychotherapy, Justus-Liebig-University Giessen, Ludwigstr 76, D-35392 Giessen, Germany
- Department of Psychology, MSB Medical School Berlin, Calandrellistr. 1-9, 12247 Berlin, Germany
| |
Collapse
|
11
|
Rief W, Hofmann SG. Some problems with non-inferiority tests in psychotherapy research: psychodynamic therapies as an example. Psychol Med 2018; 48:1392-1394. [PMID: 29439745 DOI: 10.1017/s0033291718000247] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
In virtually every field of medicine, non-inferiority trials and meta-analyses with non-inferiority conclusions are increasingly common. This non-inferiority approach has been frequently used by a group of authors favoring psychodynamic therapies (PDTs), concluding that PDTs are just as effective as cognitive-behavioral therapies (CBT). We focus on these examples to exemplify some problems associated with non-inferiority tests of psychological treatments, although the problems also apply to psychopharmacotherapy research, CBT research, and others. We conclude that non-inferiority trials have specific risks of different types of validity problems, usually favoring an (erroneous) non-inferiority conclusion. Non-inferiority trials require the definition of non-inferiority margins, and currently used thresholds have a tendency to be inflationary, not protecting sufficiently against degradation. The use of non-inferiority approaches can lead to the astonishing result that one single analysis can suggest both, superiority of the comparator (here: CBT) and non-inferiority of the other treatment (here PDT) at the same time. We provide recommendations how to improve the quality of non-inferiority trials, and we recommend to consider them among other criteria when evaluating manuscripts examining non-inferiority trials. If psychotherapeutic families (such as PDT and CBT) differ on the number of investigating trials, and in the fields of clinical applications, and in other validity aspects mentioned above, conclusions about their general non-inferiority are no more than a best guess, typically expressing the favored approach of the lead author.
Collapse
|
12
|
Aberegg SK, Hersh AM, Samore MH. Do non-inferiority trials of reduced intensity therapies show reduced effects? A descriptive analysis. BMJ Open 2018; 8:e019494. [PMID: 29500210 PMCID: PMC5855198 DOI: 10.1136/bmjopen-2017-019494] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/30/2022] Open
Abstract
OBJECTIVES To identify non-inferiority trials within a cohort where the experimental therapy is the same as the active control comparator but at a reduced intensity and determine if these non-inferiority trials of reduced intensity therapies have less favourable results than other non-inferiority trials in the cohort. Such a finding would provide suggestive evidence of biocreep in these trials. DESIGN This metaresearch study used a cohort of non-inferiority trials published in the five highest impact general medical journals during a 5-year period. Data relating to the characteristics and results of the trials were abstracted. PRIMARY OUTCOME MEASURES Proportions of trials with a declaration of superiority, non-inferiority and point estimates favouring the experimental therapy and mean absolute risk differences for trials with outcomes expressed as a proportion. RESULTS Our search yielded 163 trials reporting 182 non-inferiority comparisons; 36 comparisons from 31 trials were between the same therapy at reduced and full intensity. Compared with trials not evaluating reduced intensity therapies, fewer comparisons of reduced intensity therapies demonstrated a favourable result (non-inferiority or superiority) (58.3%vs82.2%; P=0.002) and fewer demonstrated superiority (2.8%vs18.5%; P=0.019). Likewise, point estimates for reduced intensity therapies more often favoured active control than those for other trials (77.8%vs39.7%; P<0.001) as did mean absolute risk differences (+2.5% vs -0.7%; P=0.018). CONCLUSIONS Non-inferiority trials comparing a therapy at reduced intensity to the same therapy at full intensity showed reduced effects compared with other non-inferiority trials. This suggests these trials may have a high rate of type 1 errors and biocreep, with significant implications for the design and interpretation of future non-inferiority trials.
Collapse
Affiliation(s)
| | - Andrew M Hersh
- The University of Utah, Salt Lake City, Utah, USA
- Brooke Army Medical Center, San Antonio, Texas, USA
| | - Matthew H Samore
- The University of Utah, Salt Lake City, Utah, USA
- Epidemiology, Veterans Affairs Salt Lake City Health Care System, Salt Lake City, Utah, USA
| |
Collapse
|
13
|
Ganju J, Rom D. Non-inferiority versus superiority drug claims: the (not so) subtle distinction. Trials 2017; 18:278. [PMID: 28619049 PMCID: PMC5472861 DOI: 10.1186/s13063-017-2024-2] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2017] [Accepted: 05/31/2017] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Current regulatory guidance and practice of non-inferiority trials are asymmetric in favor of the test treatment (Test) over the reference treatment (Control). These trials are designed to compare the relative efficacy of Test to Control by reference to a clinically important margin, M. MAIN TEXT Non-inferiority trials allow for the conclusion of: (a) non-inferiority of Test to Control if Test is slightly worse than Control but by no more than M; and (b) superiority if Test is slightly better than Control even if it is by less than M. From Control's perspective, (b) should lead to a conclusion of non-inferiority of Control to Test. The logical interpretation ought to be that, while Test is statistically better, it is not clinically superior to Control (since Control should be able to claim non-inferiority to Test). This article makes a distinction between statistical and clinical significance, providing for symmetry in the interpretation of results. Statistical superiority and clinical superiority are achieved, respectively, when the null and the non-inferiority margins are exceeded. We discuss a similar modification to placebo-controlled trials. CONCLUSION Rules for interpretation should not favor one treatment over another. Claims of statistical or clinical superiority should depend on whether or not the null margin or the clinically relevant margin is exceeded.
Collapse
Affiliation(s)
- Jitendra Ganju
- Global Blood Therapeutics, South San Francisco, CA, 94080, USA.
| | | |
Collapse
|
14
|
The choice of the noninferiority margin in clinical trials was driven by baseline risk, type of primary outcome, and benefits of new treatment. J Clin Epidemiol 2015; 68:1144-51. [PMID: 25716902 DOI: 10.1016/j.jclinepi.2015.01.017] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2014] [Revised: 12/04/2014] [Accepted: 01/21/2015] [Indexed: 11/23/2022]
Abstract
OBJECTIVES To explore characteristics of clinical trials that influence the choice of the noninferiority margin (NIM) when planning the trial. STUDY DESIGN AND SETTING We conducted an experimental survey among corresponding authors of randomized controlled trials indexed in MEDLINE. We described two hypothetical studies and asked the respondents' opinion on the largest loss of effectiveness that is clinically negligible (or the smallest lost of effectiveness that is clinically important in the superiority scenario). We randomly manipulated four study attributes in each vignette, using a factorial design. RESULTS A total of 364 researchers participated. The values for NIMs were significantly lower than the differences to be detected in a superiority trial. The NIM was smaller when the primary outcome was mortality compared with treatment failure, when baseline risk in the control arm was lower, and when the advantage of the new treatment was a lower cost compared with having fewer side effects. In contrast, the population age group under study and the difficulty to recruit patients showed no effect on the choice of the NIM. CONCLUSION In our experimental study, the factors associated with lower NIMs were mortality as a primary outcome, low baseline risk, and a less costly new treatment.
Collapse
|