1
|
Bai AD, Komorowski AS, Lo CKL, Tandon P, Li XX, Mokashi V, Cvetkovic A, Findlater A, Liang L, Tomlinson G, Loeb M, Mertz D. Confidence interval of risk difference by different statistical methods and its impact on the study conclusion in antibiotic non-inferiority trials. Trials 2021; 22:708. [PMID: 34656155 PMCID: PMC8520289 DOI: 10.1186/s13063-021-05686-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2021] [Accepted: 10/05/2021] [Indexed: 11/17/2022] Open
Abstract
Background Numerous statistical methods can be used to calculate the confidence interval (CI) of risk differences. There is consensus in previous literature that the Wald method should be discouraged. We compared five statistical methods for estimating the CI of risk difference in terms of CI width and study conclusion in antibiotic non-inferiority trials. Methods In a secondary analysis of a systematic review, we included non-inferiority trials that compared different antibiotic regimens, reported risk differences for the primary outcome, and described the number of successes and/or failures as well as patients in each arm. For each study, we re-calculated the risk difference CI using the Wald, Agresti-Caffo, Newcombe, Miettinen-Nurminen, and skewness-corrected asymptotic score (SCAS) methods. The CIs by different statistical methods were compared in terms of CI width and conclusion on non-inferiority. A wider CI was considered to be more conservative. Results The analysis included 224 comparisons from 213 studies. The statistical method used to calculate CI was not reported in 134 (59.8%) cases. The median (interquartile range IQR) for CI width by Wald, Agresti-Caffo, Newcombe, Miettinen-Nurminen, and SCAS methods was 13.0% (10.8%, 17.4%), 13.3% (10.9%, 18.5%), 13.6% (11.1%, 18.9%), 13.6% (11.1% and 19.0%), and 13.4% (11.1%, 18.9%), respectively. In 216 comparisons that reported a non-inferiority margin, the conclusion on non-inferiority was the same across the five statistical methods in 211 (97.7%) cases. The differences in CI width were more in trials with a sample size of 100 or less in each group and treatment success rate above 90%. Of the 18 trials in this subgroup with a specified non-inferiority margin, non-inferiority was shown in 17 (94.4%), 16 (88.9%), 14 (77.8%), 14 (77.8%), and 15 (83.3%) cases based on CI by Wald, Agresti-Caffo, Newcombe, Miettinen-Nurminen, and SCAS methods, respectively. Conclusions The statistical method used to calculate CI was not reported in the majority of antibiotic non-inferiority trials. Different statistical methods for CI resulted in different conclusions on non-inferiority in 2.3% cases. The differences in CI widths were highest in trials with a sample size of 100 or less in each group and a treatment success rate above 90%. Trial registration PROSPERO CRD42020165040. April 28, 2020. Supplementary Information The online version contains supplementary material available at 10.1186/s13063-021-05686-8.
Collapse
Affiliation(s)
- Anthony D Bai
- Division of Infectious Diseases, Queen's University, Kingston, ON, Canada. .,Health Research Methodology Program, McMaster University, Hamilton, ON, Canada.
| | - Adam S Komorowski
- Health Research Methodology Program, McMaster University, Hamilton, ON, Canada.,Division of Medical Microbiology, McMaster University, Hamilton, ON, Canada
| | - Carson K L Lo
- Division of Infectious Diseases, McMaster University, Hamilton, ON, Canada
| | - Pranav Tandon
- Global Health Office, McMaster University, Hamilton, ON, Canada
| | - Xena X Li
- Division of Medical Microbiology, McMaster University, Hamilton, ON, Canada.,Division of Infectious Diseases, McMaster University, Hamilton, ON, Canada
| | - Vaibhav Mokashi
- Division of Infectious Diseases, McMaster University, Hamilton, ON, Canada
| | - Anna Cvetkovic
- Division of Infectious Diseases, McMaster University, Hamilton, ON, Canada
| | - Aidan Findlater
- Division of Infectious Diseases, McMaster University, Hamilton, ON, Canada
| | - Laurel Liang
- Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, ON, Canada
| | - George Tomlinson
- Department of Medicine, University Health Network and Mount Sinai Hospital, Toronto, ON, Canada.,Institute of Health Policy Management and Evaluation, University of Toronto, Toronto, ON, Canada
| | - Mark Loeb
- Division of Infectious Diseases, McMaster University, Hamilton, ON, Canada
| | - Dominik Mertz
- Division of Infectious Diseases, McMaster University, Hamilton, ON, Canada
| | | |
Collapse
|