Claims of equivalence in medical research: are they supported by the evidence?

Evaluation of the efficacy of a live Escherichia coli biotherapeutic product (asymptomatic bacteriuria E. coli 212)

BACKGROUND

Recurrent bacterial cystitis, often referred to as recurrent urinary tract infection (UTI), can be difficult to manage and alternative treatments are needed.

HYPOTHESIS/OBJECTIVE

Intravesicular administration of asymptomatic bacteriuria (ASB) E. coli 212 will not be inferior to antimicrobial treatment for the management of recurrent UTI in dogs.

ANIMALS

Thirty-four dogs with >1 UTI in the 12 months before presentation.

METHODS

All dogs were deemed normal otherwise based on absence of abnormalities on physical examination, CBC, serum biochemical panel, and abdominal ultrasonography. Dogs were randomized to 1 of 2 treatment groups: Group 1 antimicrobials for 7 days or group 2 intravesicular administration of ASB E. coli 212. Owners were provided a voiding questionnaire regarding their dogs' clinical signs, which was completed daily for 14 days to assess clinical cure. Dogs were examined on days 7 and 14 to assess clinical cure, and urine specimens were submitted for urinalysis and bacterial culture.

RESULTS

Clinical cure rates for ASB E. coli 212-treated dogs were not inferior to 7 days of antimicrobial treatment with a 12% margin of difference to determine non-inferiority. No significant difference was found between the treatment groups on days 7 and 14 in the proportion of dogs achieving ≥50% or ≥75% reduction in their clinical score compared with baseline.

CONCLUSIONS AND CLINICAL IMPORTANCE

These data suggest that intravesicular administration of ASB E. coli 212 is not inferior to antimicrobials for the treatment of recurrent UTI in dogs. This biotherapeutic agent could help alleviate the need for antimicrobials for some dogs with recurrent UTI, improving antimicrobial stewardship.

Evaluating rural health outcomes: A methodological approach using population-level data

Background

The sustainability of rural surgical and obstetrical facilities depends on their efficacy and quality of care, which are difficult to measure in a rural context. In an evaluation of rural practice, it is often the case that the only comparators are larger referral facilities, for which facility-level comparisons are difficult due to differences in population demographics, acuity of patients, and services offered. This publication outlines these limitations and highlights a best-practice approach to making facility-level comparisons using population-level data, risk stratification, tests of noninferiority, and Firth logistic regression analysis. This includes an investigation of minimum sample-size requirements through Monte Carlo power analysis in the context of low-acuity rural surgical care.

Methods

Monte Carlo power analysis was used to estimate the minimum sample size required to achieve a power of 0.8 for both logistic regression and Firth logistic regression models that compare the proportion of surgical adverse events against facility type, among other confounders. We provide guidelines for the implementation of a recommended methodology that uses risk stratification, Firth penalized logistic regression, and tests of noninferiority.

Results

We illustrate limitations in facility-level comparison of surgical quality among patients undergoing one of four index procedures including hernia repair, colonoscopy, appendectomy, and cesarean delivery. We identified minimum sample sizes for comparison of each index procedure that fluctuate depending on the level of risk stratification used.

Conclusion

The availability of administrative data can provide an adequate sample size to allow for facility-level comparisons in surgical quality, at the rural level and elsewhere. When they are made appropriately, these comparisons can be used to evaluate the efficacy of general practitioners and nurse practitioners in performing low-acuity procedures.

Pharmacoeconomic Aspects of Diabetes Mellitus: Outcomes and Analysis of Health Benefits Approach

Pharmacoeconomics is an important tool for investigating and restructuring healthcare policies. In India, recent statistical studies have shown that the number of diabetic patients is rapidly increasing in the rural, middle and upper-class settings. The aim of this review is to call attention towards the need to carry out pharmacoeconomic studies for diabetes mellitus and highlight the outcome of these studies on healthcare. A well-structured literature search from PubMed, Embase, Springer, ScienceDirect, and Cochrane was done. Studies that evaluated the cost-effectiveness of various anti-diabetic agents for type 2 diabetes were eligible for inclusion in the analysis and review. Two independent reviewers sequentially assessed the titles, abstracts, and full articles to select studies that met the predetermined inclusion and exclusion criteria for data abstraction. Any discrepancies between the reviewers were resolved through consensus. By employing search terms such as pharmacoeconomics, diabetes mellitus, cost-effective analysis, cost minimization analysis, cost-utility analysis, and cost-benefit analysis, a total of 194 papers were gathered. Out of these, 110 papers were selected as they aligned with the defined search criteria and underwent the removal of duplicate entries. This review outlined four basic pharmacoeconomic studies carried out on diabetes mellitus. It gave a direction that early detection, patient counseling, personalized medication, appropriate screening intervals, and early start of pharmacotherapy proved to be a cost-effective as well as health benefits approach.

baymedr: an R package and web application for the calculation of Bayes factors for superiority, equivalence, and non-inferiority designs

BACKGROUND

Clinical trials often seek to determine the superiority, equivalence, or non-inferiority of an experimental condition (e.g., a new drug) compared to a control condition (e.g., a placebo or an already existing drug). The use of frequentist statistical methods to analyze data for these types of designs is ubiquitous even though they have several limitations. Bayesian inference remedies many of these shortcomings and allows for intuitive interpretations, but are currently difficult to implement for the applied researcher.

RESULTS

We outline the frequentist conceptualization of superiority, equivalence, and non-inferiority designs and discuss its disadvantages. Subsequently, we explain how Bayes factors can be used to compare the relative plausibility of competing hypotheses. We present baymedr, an R package and web application, that provides user-friendly tools for the computation of Bayes factors for superiority, equivalence, and non-inferiority designs. Instructions on how to use baymedr are provided and an example illustrates how existing results can be reanalyzed with baymedr.

CONCLUSIONS

Our baymedr R package and web application enable researchers to conduct Bayesian superiority, equivalence, and non-inferiority tests. baymedr is characterized by a user-friendly implementation, making it convenient for researchers who are not statistical experts. Using baymedr, it is possible to calculate Bayes factors based on raw data and summary statistics.

Statistical Power of Randomized Controlled Trials in Trauma Surgery

BACKGROUND

Our purpose was to conduct a bibliometric study investigating the prevalence of underpowered randomized controlled trials (RCTs) in trauma surgery.

STUDY DESIGN

A medical librarian conducted a search of RCTs in trauma published from 2000 to 2021. Data extracted included study type, sample size calculation, and power analyses. Post hoc calculations were performed using a power of 80% and an alpha level of 0.05. A CONSORT checklist was then tabulated from each study as well as a fragility index for studies with statistical significance.

RESULTS

In total 187 RCTs from multiple continents and 60 journals were examined. A total of 133 (71%) were found to have "positive" findings consistent with their hypothesis. When evaluating their methods, 51.3% of articles did not report how they calculated their intended sample size. Of those that did, 25 (27%) did not meet their target enrollment. When examining post hoc power, 46%, 57%, and 65% were adequately powered to detect small, medium, and large effect sizes, respectively. Only 11% of RCTs had complete adherence with CONSORT reporting guidelines and the average CONSORT score was 19 out of 25. For positive superiority trials with binary outcomes, the fragility index median (interquartile range) was 2 (2 to 8).

CONCLUSIONS

A concerningly large proportion of recently published RCTs in trauma surgery do not report a priori sample size calculations, do not meet enrollment targets, and are not adequately powered to detect even large effect sizes. There exists opportunity for improvement of trauma surgery study design, conduct, and reporting.

Acute Anterior Cruciate Ligament Rupture: Repair or Reconstruction? Five-Year Results of a Randomized Controlled Clinical Trial

BACKGROUND

High-level evidence for short-term outcomes of contemporary anterior cruciate ligament (ACL) suture repair (ACLSR) in comparison with those of ACL reconstruction (ACLR) is scarce. High-level evidence for mid- and long-term results is lacking, whereas outcomes of ACLSR in several historical studies were shown to deteriorate at midterm follow-up after initial good short-term outcomes.

HYPOTHESIS

Contemporary ACLSR is noninferior to ACLR in the treatment of acute ACL rupture in terms of patient self-reported outcomes at 5 years postoperatively.

STUDY DESIGN

Randomized controlled trial; Level of evidence, 1.

METHODS

A total of 48 patients were enrolled in the study and, after stratification and randomization, underwent either dynamic augmented (DA) ACLSR or anatomic single-bundle ACLR. The primary outcome measure was the International Knee Documentation Committee 2000 (IKDC) subjective score (IKDCs). Furthermore, the Knee injury and Osteoarthritis Outcome Score (KOOS), Tegner Activity Scale score (TAS), visual analog scale score for satisfaction (VASs), IKDC physical examination score (IKDCpe), limb symmetry index for quadriceps (LSIq) and hamstrings (LSIh) strength and jump test battery (LSIj), Kellgren-Lawrence grade of osteoarthritis (OA), and rate of adverse events were recorded. Analyses were based on an intention-to-treat principle.

RESULTS

The lower limit of the 2-sided 95% CI for the median IKDCs of the DA ACLSR group (n = 23; 75.9) was lower than the prespecified noninferiority margin (n = 21; 86.6). Therefore, the null hypothesis was rejected. However, the upper limit of the 2-sided 95% CI of the DA ACLSR group (100.0) was higher than the median IKDCs of the ACLR group (96.6), rendering the result for noninferiority inconclusive. No statistical difference was found between groups for median IKDCs (repair, 90.2; reconstruction, 96.6). Furthermore, no statistically significant differences were found for any of the secondary outcome measures for the DA ACLSR compared with the ACLR group: KOOS Symptoms, 92.9 versus 96.4; KOOS Pain, 100 versus 97.2; KOOS Activities of Daily Living, 100 versus 100; KOOS Sport and Recreation, 85.0 versus 100; TAS score, 7.0 versus 6.5; VASs, 9.2 versus 8.7; IKDCpe, 81.8% versus 100%; LSIq, ≥91.6 versus ≥88.2; LSIh, ≥95.1 versus ≥90.7; LSIj, ≥94.2 versus ≥97.6; OA grade 0, 90.9% versus 77.8%; clinical ACL failure rate, 20.8% versus 27.2%; and repeat surgery rate, 37.5% versus 20.0%, respectively.

CONCLUSION

It remains inconclusive whether the effectiveness of DA ACLSR is noninferior to that of ACLR in terms of subjective patient-reported outcomes as measured using the IKDCs. Although DA ACLSR may be a viable treatment option for patients with acute ACL rupture, caution must be exercised when considering this treatment for young, active patients, corresponding to the present study population.

Inconsistencies in the Methodological Framework Throughout Published Studies in High-Impact Orthopaedic Journals: A Systematic Review

BACKGROUND

Both the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) and Consolidated Standards of Reporting Trials (CONSORT) guidelines recommend that clinical trials follow a study framework that aligns with their objective to test the relative efficacy or safety (equality) or effectiveness (superiority, noninferiority, or equivalence) between interventions. We conducted a systematic review to assess the proportion of studies that demonstrated inconsistency between the framing of their research question, sample size calculation, and conclusion and those that should have framed their research question differently based on the compared interventions.

METHODS

We included studies from 5 high-impact-factor orthopaedic journals published in 2017 and 2019 that compared at least 2 interventions using patient-reported outcome measures.

RESULTS

We included 228 studies. The sample size calculation was reported in 60.5% (n = 138) of studies. Of these, 52.2% (n = 72) were inconsistent between the framing of their research question, sample size calculation, and conclusion. The majority (n = 137) of sample size calculations were for equality, but 43.8% of these studies concluded superiority, noninferiority, or equivalence. Studies that framed their research question as equality (n = 186) should have been framed as superiority (n = 129), equivalence (n = 52), or noninferiority (n = 3). Only 2 studies correctly framed their research question as equality.

CONCLUSIONS

Studies published in high-impact journals were inconsistent between the framing of their research question, sample size calculation, and conclusion. Authors may be misinterpreting research findings and making clinical recommendations solely based on p values. Researchers are encouraged to state and justify their methodological framework and choice of margin(s) in a publicly published protocol as they have implications for sample size and the applicability of conclusions.

CLINICAL RELEVANCE

The results of clinical research must be interpreted using confidence intervals, with careful consideration as to how the confidence intervals relate to clinically meaningful differences in outcomes between treatments. The more typical practice of relying on p values leaves the clinician at high risk of erroneous interpretation, recommendation, and/or action.

Understanding the Merits and Drawbacks of Noninferiority Trials in Cardiovascular Medicine

A noninferiority trial is designed to show that the new treatment is not unacceptably worse than the standard treatment by more than a predefined noninferiority margin. These trials are typically performed when standard placebo-controlled trials are considered to be unethical or impractical and the new treatment offers advantages over the existing standard treatment in terms of safety, convenience, or cost. Given that noninferiority trials are being performed with increasing frequency in cardiovascular applications, it is important to understand their complex trial design and analysis. This narrative review aims to provide readers with a detailed perspective on the goals, characteristics, design, and analysis of noninferiority trials. Trials designed to show noninferiority require an appropriate reference population, a proven standard treatment and dose, an appropriate margin of noninferiority that is statistically justifiable (based on historical placebo-controlled trials evaluating standard treatment effect) and clinically reasonable (choosing the fraction of the effect of the standard drug that should be "preserved" by the new drug), a high level of adherence to treatment, and adequate statistical power to reliably conclude that a treatment is truly noninferior and therefore effective. The merits and pitfalls of noninferiority trials, with representative contemporary cardiovascular clinical trials in interventional cardiology, cardiac surgery, and atrial fibrillation management as exemplars, are described. The key issues that challenge the design, conduct, analysis, interpretation, and implementation of these trials are discussed, and a variety of ways to identify and mitigate key errors are recommended to allow for optimal evaluation of noninferiority trials conducted in cardiovascular medicine.

Non-inferiority trials in cardiology: what clinicians need to know

Clinical trials traditionally aim to show a new treatment is superior to placebo or standard treatment, that is, superiority trials. There is an increasing number of trials demonstrating a new treatment is non-inferior to standard treatment. The hypotheses, design and interpretation of non-inferiority trials are different to superiority trials. Non-inferiority trials are designed with the notion that the new treatment offers advantages over standard treatment in certain important aspects. The non-inferior margin is a predetermined margin of difference between the new and standard treatment that is considered acceptable or tolerable for the new treatment to be considered ‘similar’ or ‘not worse’. Both relative difference and absolute difference methods can be used to define the non-inferior margin. Sequential testing for non-inferiority and superiority is often performed. Non-inferiority trials may be necessary in situations where it is no longer ethical to test any new treatment against placebo. There are inherent assumptions in non-inferiority trials which may not be correct and which are not being tested. Successive non-inferiority trials may introduce less and less effective treatments even though these treatments may have been shown to be non-inferior. Furthermore, poor quality trials favour non-inferior results. Intention-to-treat analysis, the preferred way to analyse randomised trials, may favour non-inferiority. Both intention-to-treat and per-protocol analyses should be recommended in non-inferiority trials. Clinicians should be aware of the pitfalls of non-inferiority trials and not accept non-inferiority on face value. The focus should not be on the p values but on the effect size and confidence limits.

Are claims of non-inferiority of Internet and computer-based cognitive-behavioural therapy compared with in-person cognitive-behavioural therapy for adults with anxiety disorders supported by the evidence from head-to-head randomised controlled trials? A systematic review

OBJECTIVE

This review examines the evidence from head-to-head randomised controlled trials addressing whether the efficacy of cognitive-behavioural therapy for anxiety disorders, obsessive-compulsive disorder and post-traumatic stress disorders in adults delivered by computer or online (computer- and Internet-delivered cognitive-behavioural therapy) is not inferior to in-person cognitive-behavioural therapy for reducing levels of symptoms and producing clinically significant gains at post-treatment and at follow-up. A supplementary aim is to examine the evidence for severity as a moderator of the relative efficacy of computer- and Internet-delivered cognitive-behavioural therapy and in-person cognitive-behavioural therapy.

METHOD

PubMed, PsycINFO, Embase and Cochrane database of randomised trials were searched for randomised controlled trials of cognitive-behavioural therapy for these disorders with at least an in-person cognitive-behavioural therapy and Internet or computer cognitive-behavioural therapy arm.

RESULTS

A total of 14 randomised controlled trials (9 Internet, 5 computer) of cognitive-behavioural therapy for social anxiety disorder, panic disorder and specific phobia and 3 reports of effect moderators were included. One study showed a low risk of bias when assessed against risk of bias criteria for non-inferiority trials. The remaining studies were assessed as high or unclear risk of bias. One study found that Internet-delivered cognitive-behavioural therapy was superior and non-inferior at post-treatment and follow-up to group in-person cognitive-behavioural therapy for social anxiety disorder. One study of Internet-delivered cognitive-behavioural therapy for panic disorder showed non-inferiority to individual in-person cognitive-behavioural therapy for responder status at post-treatment and one of Internet cognitive-behavioural therapy for panic disorder for symptom severity at follow-up. Other comparisons (22 Internet, 13 computer) and for estimates pooled for Internet cognitive-behavioural therapy for social anxiety disorder, Internet cognitive-behavioural therapy for panic disorder and computer-delivered cognitive-behavioural therapy studies did not support non-inferiority. Evidence of effect moderation by severity and co-morbidity was mixed.

CONCLUSION

There is limited evidence from randomised controlled trials which supports claims that computer- or Internet-delivered cognitive-behavioural therapy for anxiety disorders is not inferior to in-person delivery. Randomised controlled trials properly designed to test non-inferiority are needed before conclusions about the relative benefits of in-person and Internet- and computer-delivered cognitive-behavioural therapy can be made.

PROSPERO

CRD420180961655-6.

Bayes factors for superiority, non-inferiority, and equivalence designs

Background

In clinical trials, study designs may focus on assessment of superiority, equivalence, or non-inferiority, of a new medicine or treatment as compared to a control. Typically, evidence in each of these paradigms is quantified with a variant of the null hypothesis significance test. A null hypothesis is assumed (null effect, inferior by a specific amount, inferior by a specific amount and superior by a specific amount, for superiority, non-inferiority, and equivalence respectively), after which the probabilities of obtaining data more extreme than those observed under these null hypotheses are quantified by p-values. Although ubiquitous in clinical testing, the null hypothesis significance test can lead to a number of difficulties in interpretation of the results of the statistical evidence.

Methods

We advocate quantifying evidence instead by means of Bayes factors and highlight how these can be calculated for different types of research design.

Results

We illustrate Bayes factors in practice with reanalyses of data from existing published studies.

Conclusions

Bayes factors for superiority, non-inferiority, and equivalence designs allow for explicit quantification of evidence in favor of the null hypothesis. They also allow for interim testing without the need to employ explicit corrections for multiple testing.

Acute Anterior Cruciate Ligament Rupture: Repair or Reconstruction? Two-Year Results of a Randomized Controlled Clinical Trial

BACKGROUND

Contemporary anterior cruciate ligament (ACL) suture repair techniques have been subject to renewed interest in recent years. Although several clinical studies have yielded good short-term results, high-quality evidence is lacking in regard to the effectiveness of this treatment compared with ACL reconstruction.

HYPOTHESIS

Dynamic augmented ACL suture repair is at least as effective as anatomic single-bundle ACL reconstruction for the treatment of acute ACL rupture in terms of patient self-reported outcomes at 2 years postoperatively.

STUDY DESIGN

Randomized controlled trial; Level of evidence, 1.

METHODS

After stratification and randomization, 48 patients underwent either dynamic augmented ACL suture repair or ACL reconstruction with a single-bundle, all-inside, semitendinosus technique. The International Knee Documentation Committee (IKDC) subjective score at 2 years postoperatively was the primary outcome measure. Patient-reported outcomes (IKDC subjective score, Knee injury and Osteoarthritis Outcome Score, Tegner score, visual analog scale for satisfaction), clinical outcomes (IKDC physical examination score, leg symmetry index for the quadriceps, hamstrings strength, and jump test battery), and radiological outcomes as well as adverse events including reruptures were recorded. Analyses were based on an intention-to-treat principle.

RESULTS

The lower limit for the median IKDC subjective score of the repair group (86.2) fell within the prespecified noninferiority margin, confirming noninferiority of dynamic augmented ACL suture repair compared with ACL reconstruction. No statistical difference was found between groups for median IKDC subjective score (repair, 95.4; reconstruction, 94.3). Overall, 2 reruptures (8.7%) occurred in the dynamic ACL suture repair group and 4 reruptures (19.0%) in the ACL reconstruction group; further, 5 repeat surgeries-other than for revision ACL surgery-took place in 4 patients from the dynamic ACL suture repair group (20.8%) and in 3 patients from the ACL reconstruction group (14.3%).

CONCLUSION

Dynamic augmented ACL suture repair is not inferior to ACL reconstruction in terms of subjective patient-reported outcomes as measured with the IKDC subjective score 2 years postoperatively. However, for reasons other than revision ACL surgery due to rerupture, a higher number of related adverse events leading to repeat surgery were seen in the dynamic augmented ACL suture repair group within 2 years postoperatively.

CLINICAL RELEVANCE

Dynamic augmented ACL suture repair might be a viable treatment option for patients with an acute ACL rupture.

REGISTRATION

NCT02310854 ( ClinicalTrials.gov identifier).

Unfounded conclusions of equivalence in diagnostic accuracy studies: a pervasive fallacy of inference in pediatric radiology scientific abstracts

BACKGROUND

In studies of diagnostic performance that fail to detect a statistically significant difference between compared techniques, investigators often declare evidence of equivalence or similarity without having actually tested that assertion due to incorrect methodology or insufficient statistical power.

OBJECTIVE

The purpose of our investigation is to measure the prevalence of unfounded assertions of equivalence or similarity in comparison studies presented at the International Pediatric Radiology (IPR) meeting of 2016 and promote awareness of this fallacy of inference to the pediatric radiology community.

MATERIALS AND METHODS

Two pediatric radiologists independently reviewed the methodology and reporting quality of the 194 scientific paper abstracts from the 2016 IPR meeting. All comparison studies were identified and those failing to detect a statistically significance difference and making a claim of equivalence or similarity in the results or conclusion were assessed for a description of the study design type, statistical power and sample size estimator calculation.

RESULTS

Of 194 scientific paper abstracts, 112 (58%) were comparison studies. Of these, 36/112 (32%) made unfounded inferences of equivalence or similarity in diagnostic imaging performance. No study had an equivalence or non-inferiority design. No abstract specified the statistical power of the study, and only one abstract acknowledged a small sample size as a limitation in detecting a statistically significant difference.

CONCLUSION

Inadequate reporting and unfounded inferences of equivalence or similarity were common in diagnostic performance comparison studies presented at IPR 2016. Failure to recognize these limitations could have adverse consequences by leading to the adoption of unvalidated imaging techniques.

Editorial Commentary: "There, It Fits!"-Justifying Nonsignificant P Values

The increasing emphasis placed on value-based medicine has become a powerful motivating factor that has driven the performance and publication of more comparative clinical outcome studies. Although these endeavors are well intentioned and significant progress has been made in our field over the past few decades, I believe that we need both to re-emphasize the importance of results derived from meaningful study designs and to avoid the notion that nonsignificant P values represent study failure.

Equivalency of fecal immunochemical tests and colonoscopy in familial colorectal cancer screening

BACKGROUND & AIMS

Colonoscopy is the recommended screening procedure for first-degree relatives of patients with colorectal cancer (CRC), but few studies have compared its efficacy for CRC detection with that of other screening strategies. We conducted a controlled randomized trial to compare the efficacy of repeated fecal immunochemical tests (FITs) and colonoscopy in detecting advanced neoplasia (advanced adenoma or CRC) in family members of patients with CRC.

METHODS

In a prospective study, 1918 first-degree relatives of patients with CRC were randomly assigned (1:1 ratio) to receive a single colonoscopy examination or 3 FITs (1/year for 3 years; OC-Sensor; cutoff ≥10 μg hemoglobin/g feces, corresponding to 50 ng hemoglobin/mL buffer). The strategies were considered to be equivalent if the 95% confidence interval of the difference for the detection of advanced neoplasia was ±3%. Follow-up analyses were performed to identify false-negative FIT results and interval CRCs.

RESULTS

Of all eligible asymptomatic first-degree relatives, 782 were included in the colonoscopy group and 784 in the FIT group. In the intention-to-screen analysis, advanced neoplasia was detected in 33 (4.2%) and 44 (5.6%) first-degree relatives in the FIT and colonoscopy groups, respectively (odds ratio = 1.41; 95% confidence interval: 0.88-2.26; P = .14). In the per-protocol analysis, 28 first-degree relatives (3.9%) in the FIT group and 43 (5.8%) in the colonoscopy group had advanced neoplasia (odds ratio = 1.56; 95% confidence interval: 0.95-2.56; P = .08). FIT missed 16 of 41 advanced adenomas but no CRCs. The FIT strategy required endoscopic evaluation of 4-fold fewer individuals to detect 1 advanced neoplasia than the colonoscopy strategy.

CONCLUSIONS

Repeated FIT screening (1/year for 3 years) detected all CRCs and proved equivalent to colonoscopy in detecting advanced neoplasia in first-degree relatives of patients with CRC. This strategy should be considered for populations where compliance with FITs is higher than with colonoscopy. ClinicalTrials.gov number: NCT01075633 (COLONFAM Study).

Comparison of noninferiority margins reported in protocols and publications showed incomplete and inconsistent reporting

OBJECTIVES

To compare noninferiority margins defined in study protocols and trial registry records with margins reported in subsequent publications.

STUDY DESIGN AND SETTING

Comparison of protocols of noninferiority trials submitted 2001 to 2005 to ethics committees in Switzerland and The Netherlands with corresponding publications and registry records. We searched MEDLINE via PubMed, the Cochrane Controlled Trials Register (Cochrane Library issue 01/2012), and Google Scholar in September 2013 to identify published reports, and the International Clinical Trials Registry Platform of the World Health Organization in March 2013 to identify registry records. Two readers recorded the noninferiority margin and other data using a standardized data-abstraction form.

RESULTS

The margin was identical in study protocol and publication in 43 (80%) of 54 pairs of study protocols and articles. In the remaining pairs, reporting was inconsistent (five pairs, 9%), or the noninferiority margin was either not reported in the publication (five pairs, 9%) or not defined in the study protocol (one pair). The confidence interval or the exact P-value required to judge whether the result was compatible with noninferior, inferior, or superior efficacy was reported in 43 (80%) publications. Complete and consistent reporting of both noninferiority margin and confidence interval (or exact P-value) was present in 39 (72%) protocol-publication pairs. Twenty-nine trials (54%) were registered in trial registries, but only one registry record included the noninferiority margin.

CONCLUSION

The reporting of noninferiority margins was incomplete and inconsistent with study protocols in a substantial proportion of published trials, and margins were rarely reported in trial registries.

Intraclass differences among antihypertensive drugs

The four major classes of antihypertensive drugs—diuretics, β-blockers, calcium channel blockers, and renin-angiotensin system inhibitors (including angiotensin-converting enzyme inhibitors and angiotensin receptor blockers)—have significant qualitative and quantitative differences in the adverse effects they cause. Structural and chemical differences have been identified within these classes, especially among the calcium channel blockers and, to a lesser extent, among the thiazide/thiazide-like diuretics. However, it has been more difficult to demonstrate that these differences translate into differential effects with respect to either the surrogate endpoint of blood pressure reduction or, more importantly, hypertension-related cardiovascular complications. Based on a hierarchy-of-evidence approach, differences are apparent between hydrochlorothiazide and chlorthalidone based on evidence of moderate quality. Low-quality evidence suggests atenolol is less effective than other β-blockers. However, no significant intraclass differences have been established among the other classes of antihypertensive drugs.

Choice of non-inferiority (NI) margins does not protect against degradation of treatment effects on an average--an observational study of registered and published NI trials

OBJECTIVE

NI margins have to be chosen appropriately to control the risk of degradation of treatment effects in non-inferiority (NI) trials. We aimed to study whether the current choice of NI margins protects sufficiently against a degradation of treatment effect on an average.

STUDY DESIGN AND SETTING

NI trials reflecting current practice were assembled and for each trial, the NI margin was translated into a likelihood of degradation. The likelihood of degradation was calculated as the conditional probability of a treatment being harmful given that it is declared non-inferior in the trial, using simulation. Its distribution among the NI trials was then studied to assess the potential risk of degradation.

RESULTS

The median (lower/upper quartile) NI margin among 112 binary outcome NI trials corresponded to an odds ratio of 0.57(0.45, 0.66), while among 38 NI trials with continuous outcome, to a Cohen's d of -0.42(-0.54, -0.31) and a hazard ratio of 0.82(0.73, 0.86) among 24 survival outcome NI trials. Overall, the median likelihood of degradation was 56% (45%, 62%).

CONCLUSION

Only two fifths of the current NI trials had a likelihood of degradation lower than 50%, suggesting that, in majority of the NI trials, there is no sufficient protection against degradation on an average. We suggest a third hurdle for the choice of NI margins, thus contributing a sufficient degree of protection.

Ophthalmic randomized controlled trials reports: the statement of the hypothesis

PURPOSE

To evaluate whether the ophthalmic randomized controlled trials (RCTs) were designed properly, their hypotheses stated clearly, and their conclusions drawn correctly.

DESIGN

A systematic review of 206 ophthalmic RCTs.

METHODS

The objective statement, methods, and results sections and the conclusions of RCTs published in 4 major general clinical ophthalmology journals from 2009 through 2011 were assessed. The clinical objective and specific hypothesis were the main outcome measures.

RESULTS

The clinical objective of the trial was presented in 199 (96.6%) studies and the hypothesis was specified explicitly in 56 (27.2%) studies. One hundred ninety (92.2%) studies tested superiority. Among them, 17 (8.3%) studies comparing 2 or more active treatments concluded equal or similar effectiveness between the 2 arms after obtaining insignificant results. There were 5 noninferiority studies and 4 equivalence studies. How the treatments were compared was not mentioned in 1 of the noninferiority studies. Two of the equivalence studies did not specify the equivalence margin and used tests for detecting difference rather than confirming equivalence.

CONCLUSIONS

The clinical objective commonly was stated, but the prospectively defined hypothesis tended to be understated in ophthalmic RCTs. Superiority was the most common type of comparison. Conclusions made in some of them with negative results were not consistent with the hypothesis, indicating that noninferiority or equivalence may be a more appropriate design. Flaws were common in the noninferiority and equivalence studies. Future ophthalmic researchers should choose the type of comparison carefully, specify the hypothesis clearly, and draw conclusions that are consistent with the hypothesis.

Non-inferiority cancer clinical trials: scope and purposes underlying their design

BACKGROUND

Non-inferiority clinical trials (NIFCTs) aim to demonstrate that the experimental therapy has advantages over the standard of care, with acceptable loss of efficacy. We evaluated the purposes underlying the selection of a non-inferiority design in oncology and the size of their non-inferiority margins (NIFm's).

PATIENTS AND METHODS

All NIFCTs of cancer-directed therapies and supportive care agents published in a 10-year period were eligible. Two investigators extracted the data and independently classified the trials by their purpose to choose a non-inferiority design.

RESULTS

Seventy-five were included: 43% received funds from industry, overall survival was the most common primary end point and 73% reported positive results. The most frequent purposes underlying the selection of a non-inferiority design were to test more conveniently administered schedules and/or less toxic treatments. In 13 (17%) trials, a clear purpose was not identified. Among the trials that reported a pre-specified NIFm, the median value was 12.5% (range 4%-25%) for trials with binary primary end points and Hazard Ratio of 1.25 (range 1.10-1.50) for trials that used time-to-event primary outcomes.

CONCLUSION

Cancer NIFCT harbor serious methodological and ethical issues. Many use large NIFm and nearly one-fifth did not state a clear purpose for selecting a non-inferiority design.

[Consistency in the analysis and reporting of PEPs in oncology randomized controlled trials from registration to publication: a systematic review]

To improve the quality of reporting of randomized clinical trials (RCTs), international registries for RCTs and guidelines for primary endpoint (PEP) analysis were established. The objectives of this systematic review were to evaluate concordance of PEP between publication and the corresponding registry and to assess the intrapublication consistency in PEP reporting. All adult oncology RCTs in solid tumors published in 10 journals between 2005 and 2009 were reviewed. Registration information was extracted from international trial registries. A total 366 RCTs were identified. Trial registration was found for 215 trials and the rate increased from 43% in 2005 to 82% in 2009 (P < 0.001). There were 134 RCTs with clearly defined PEPs in registry, with the rate increasing from 15 to 67% (P < 0.001). PEP differs between registration and final publication in 14% trials with clearly defined PEPs. Reporting issues in methodology were found in 15% RCTs, mainly due to inadequate reporting of PEP or of sample size calculation. Problems with the interpretation of trial results were found in 22% publications, mostly due to negative superiority studies being interpreted as showing equivalence. The rates of trial registration and of trials with clearly defined PEP have improved over time, however 14% of these trials reported a different PEP in the final publication. Intrapublication inconsistencies in PEP reporting are frequent. Our findings highlight the need for investigators, peer reviewers and readers for increased awareness and scrutiny of reporting outcomes of oncology RCTs.

Cost-analysis of nutrition support in patients with severe acute pancreatitis

PURPOSE

The purpose of this research was to assess the preferred route of nutrition support (enteral versus parenteral) for treatment of severe acute pancreatitis in the acute care setting. Further, in cases when enteral nutrition is the preferred route, is nasal-bridling a lower-morbidity and cost-effective method?

DESIGN/METHODOLOGY/APPROACH

A retrospective review of pre-existing data from an 870-bed hospital system. Medical records were reviewed via an online database system (n = 25 patients) with severe acute pancreatitis. Length of stay and cost were analyzed.

FINDINGS

More patients received TPN versus the nasal-jejunal (post-pyloric) tube feeds group. No significant relationship was found between total cost and number of co-morbidities or between either of the two treatment groups. However, a medium to large effect size was shown which could indicate a significant relationship in a larger sample size.

ORIGINALITY/VALUE

The findings of this research add to the literature already available and will be of interest to those who specialize in this area.

Consistency in the Analysis and Reporting of Primary End Points in Oncology Randomized Controlled Trials From Registration to Publication: A Systematic Review

Purpose

To improve the quality of reporting of randomized clinical trials (RCTs), international registries for RCTs and guidelines for primary end point (PEP) analysis were established. The objectives of this systematic review were to evaluate concordance of PEP between publication and the corresponding registry and to assess intrapublication consistency in PEP reporting.

Methods

All adult oncology RCTs in solid tumors published in 10 journals between 2005 and 2009 were reviewed. Registration information was extracted from international trial registries.

Results

A total 366 RCTs were identified. Trial registration was found for 215 trials, and the rate increased from 43% in 2005 to 82% in 2009 (P < .001). There were 134 RCTs with clearly defined PEPs in registry, with the rate increasing from 15% to 67% (P < .001). PEP differed between registration and final publication in 14% trials with clearly defined PEPs. Reporting issues in methodology were found in 15% of RCTs, mainly because of inadequate reporting of PEP or sample size calculation. Problems with the interpretation of trial results were found in 22% publications, mostly resulting from negative superiority studies being interpreted as showing equivalence.

Conclusion

The rates of trial registration and of trials with clearly defined PEPs have improved over time; however, 14% of these trials reported a different PEP in the final publication. Intrapublication inconsistencies in PEP reporting are frequent. Our findings highlight the need for investigators, peer reviewers, and readers to exercise increased awareness and scrutiny of reporting outcomes of oncology RCTs.

Rationale for and methods of superiority, noninferiority, or equivalence designs in orthopaedic, controlled trials

BACKGROUND

To provide value-based healthcare in orthopaedics, controlled trials are needed to assess the comparative effectiveness of treatments. Typically comparative trials are based on superiority testing using statistical tests that produce a p value. However, as orthopaedic treatments continue to improve, superiority becomes more difficult to show and, perhaps, less important as margins of improvement shrink to clinically irrelevant levels. Alternative methods to compare groups in controlled trials are noninferiority and equivalence. It is important to equip the reader of the orthopaedic literature with the knowledge to understand and critically evaluate the methods and findings of trials attempting to establish superiority, noninferiority, and equivalence.

QUESTIONS/PURPOSES

I will discuss supplemental and alternative methods to superiority for assessment of the outcome of controlled trials in the context of diminishing returns on new therapies over old ones.

METHODS

The three methods-superiority, noninferiority, and equivalence-are presented and compared, with a discussion of implied pitfalls and problems.

RESULTS

Noninferiority and equivalence offer alternatives to superiority testing and allow one to judge whether a new treatment is no worse (within a margin) or substantively the same as an active control. Noninferiority testing also allows for inclusion of superiority testing in the same study without the need for adjustment of the statistical methods.

CONCLUSIONS

Noninferiority and equivalence testing might prove most valuable in orthopaedic, controlled trials as they allow for comparative assessment of treatments with similar primary end points but potentially important differences in secondary outcomes, safety profiles, and cost-effectiveness.

Evaluating equivalence and noninferiority trials

PURPOSE

The conceptual issues related to the design of equivalence and non-inferiority trials and considerations for interpreting the findings of such trials are described.

SUMMARY

Comparative effectiveness research (CER) has recently gained importance in the evaluation of different treatment alternatives. Large, prospective, randomized controlled trials (RCTs) conducted with patient populations under routine practice conditions can yield high-quality CER results. A Phase III RCT, usually conducted for establishing superiority of one treatment over another, is called a superiority trial, and the statistical test associated with it is known as a superiority test. In a pragmatic equivalence trial, a researcher aims to test if two treatments are identical (within a specified range) with respect to some predefined clinical criteria. Pragmatic noninferiority trials aim to show if a test therapy is no worse than a standard therapy with respect to achieving the primary treatment outcome. A nonsignificant result obtained from a superiority test does not indicate that the two treatment options are similar. In other words, the lack of evidence of superiority does not guarantee a lack of difference in the performance shown by the therapies. A researcher can only demonstrate identical effects of two treatments in an equivalence trial. In a noninferiority trial, the test therapy is preferred when there is evidence about its benefits over the standard treatment in terms of secondary outcomes such as cost, adherence, and adverse effects.

CONCLUSION

Equivalence and noninferiority trials are designed differently from superiority trials. The overall quality of equivalence and noninferiority studies depends on study design and the manner in which the results are reported.

Non-inferiority randomized trials, an issue between science and ethics: the case of the SYNTAX study

Non-inferiority trials are questionable when death and serious complications are included among outcomes. The term itself "non-inferiority" is misleading, since such a study would not demonstrate that a new treatment is non-inferior to a control treatment, but simply that the inferiority would not reach a pre-specified level, deemed as acceptable by the designers of the trial. Group cross-over, assay-sensitivity and the need of a placebo arm are major issues for the reliability of non-inferiority trials. The SYNTAX trial for severe coronary artery disease was designed on a non-inferiority margin of 6.6%. In this paper we show that the SYNTAX designers were ready to accept up to 30% higher rate of death and major adverse events to claim the non-inferiority of percutaneous coronary intervention versus coronary artery bypass grafting. Eventually the SYNTAX study failed because percutaneous patients sustained an even higher rate of adverse events. We propose major caution in performing non-inferiority randomized trials.

Challenges of Phase III study design for trials of new drug regimens for the treatment of TB

The standard WHO-recommended 6-month four-drug regimen for the treatment of tuberculosis is highly effective in clinical trial settings, but this level of efficacy is not always achieved in practice. Shorter, simpler regimens that promote better treatment adherence are urgently needed. In addition, it will be necessary to demonstrate noninferiority rather than superiority over the 6-month regimen. There are considerable challenges in designing and conducting noninferiority Phase III trials of new drug regimens. These include the choice of the margin of noninferiority, the primary outcome measure and strategies for handling nonassessable patients and reinfected patients in the analysis. Most of these trials are likely to be large and expensive and public–private partnerships will be required to conduct them.

Problems in the reporting of acne clinical trials: a spot check from the 2009 Annual Evidence Update on Acne Vulgaris

In the course of producing the 2009 NHS Evidence--skin disorders Annual Evidence Update on Acne Vulgaris, 25 randomised controlled trials were examined. From these, at least 12 potentially serious problems of trial reporting were identified. Several trials concluded no effect of a treatment yet they were insufficiently powered to exclude potentially useful benefits. There were examples of duplicate publication and "salami publication", as well as two trials being combined and reported as one. In some cases, an incorrect "within-groups" statistical comparison was made and one trial report omitted original efficacy data and included only P values. Both of the non-inferiority studies examined failed to pre-specify a non-inferiority margin. Trials reported as "double-blind" compared treatments that were dissimilar in appearance or had differing adverse effect profiles. In one case an intention-to-treat analysis was not performed and there was a failure to account for all of the randomized participants. Trial results were made to sound more impressive by selective outcome reporting, emphasizing the statistical significance of treatment effects that were clinically insignificant, and by the use of larger-sounding odds ratios rather than rate ratios for common events. Most of the reporting problems could have been avoided by use of the CONSORT guidelines and prospective trial registration on a public clinical trials database.

Research design considerations for confirmatory chronic pain clinical trials: IMMPACT recommendations

There has been an increase in the number of chronic pain clinical trials in which the treatments being evaluated did not differ significantly from placebo in the primary efficacy analyses despite previous research suggesting that efficacy could be expected. These findings could reflect a true lack of efficacy or methodological and other aspects of these trials that compromise the demonstration of efficacy. There is substantial variability among chronic pain clinical trials with respect to important research design considerations, and identifying and addressing any methodological weaknesses would enhance the likelihood of demonstrating the analgesic effects of new interventions. An IMMPACT consensus meeting was therefore convened to identify the critical research design considerations for confirmatory chronic pain trials and to make recommendations for their conduct. We present recommendations for the major components of confirmatory chronic pain clinical trials, including participant selection, trial phases and duration, treatment groups and dosing regimens, and types of trials. Increased attention to and research on the methodological aspects of confirmatory chronic pain clinical trials has the potential to enhance their assay sensitivity and ultimately provide more meaningful evaluations of treatments for chronic pain.

Do all psychological treatments really work the same in posttraumatic stress disorder?

A recent meta-analysis by Benish, Imel, and Wampold (2008, Clinical Psychology Review, 28, 746-758) concluded that all bona fide treatments are equally effective in posttraumatic stress disorder (PTSD). In contrast, seven other meta-analyses or systematic reviews concluded that there is good evidence that trauma-focused psychological treatments (trauma-focused cognitive behavior therapy and eye movement desensitization and reprocessing) are effective in PTSD; but that treatments that do not focus on the patients' trauma memories or their meanings are either less effective or not yet sufficiently studied. International treatment guidelines therefore recommend trauma-focused psychological treatments as first-line treatments for PTSD. We examine possible reasons for the discrepant conclusions and argue that (1) the selection procedure of the available evidence used in Benish et al.'s (2008)meta-analysis introduces bias, and (2) the analysis and conclusions fail to take into account the need to demonstrate that treatments for PTSD are more effective than natural recovery. Furthermore, significant increases in effect sizes of trauma-focused cognitive behavior therapies over the past two decades contradict the conclusion that content of treatment does not matter. To advance understanding of the optimal treatment for PTSD, we recommend further research into the active mechanisms of therapeutic change, including treatment elements commonly considered to be non-specific. We also recommend transparency in reporting exclusions in meta-analyses and suggest that bona fide treatments should be defined on empirical and theoretical grounds rather than by judgments of the investigators' intent.

Clinical trial structures

Most errors in clinical trials are a result of poor planning. Fancy statistical methods cannot rescue design flaws. Thus careful planning with clear foresight is crucial. The selection of a clinical trial design structure requires logic and creativity. Common structural designs are discussed.

Comparative study of control selection in a national population-based case-control study: Estimating risk of smoking on cancer deaths in Chinese men

PURPOSE

To assess the validation of a novel control selection design by comparing the consistency between the new design and a routine design in a large case-control study that was incorporated into a nationwide mortality survey in China.

METHODS

A nationwide mortality study was conducted during 1989-1991. Surviving spouses or other relatives of all adults who died during 1986-1988 provided detailed information about their own as well as the deceased person's smoking history. In this study, 130,079 males who died of various smoking-related cancers at age 35 or over were taken as cases, while 103,248 male surviving spouses (same age range with cases) of women who died during the same period and 49,331 males who died from causes other than those related to smoking were used as control group 1 and control group 2, respectively. Consistency in the results when comparing cases with each of the control groups was assessed.

RESULTS

Consistency in the results was observed in the analyses using different control groups although cancer deaths varied with region and age. Equivalence could be ascertained using a 15% criterion in most cancer deaths which had high death rates in urban areas, but they were uncertain for most cancers in rural areas irrespective of whether the hypothesis testing showed significant differences or not.

CONCLUSIONS

Sex-matched living spouse control design as an alternative control selection for a case-control study is valid and feasible, and the basic principles of the equivalence study are also supported by epidemiological survey data.

[Interpreting methodological and statistical considerations in studies of rescue therapy]

The methodology used in studies of rescue therapy is sometimes complex. This is because of the heterogeneity of objectives and options. Firstly, the definition of failure has multiple interpretations and subtle distinctions. Secondly, the aim of treatment in these patients has varied according to the available treatment options in each case and at each moment of time. Lastly, the methodology used to develop these studies of rescue therapy has varied over time in line with changes in their aims and options. Currently, a new change can be expected to adjust to the current situation, since the number of therapeutic options for rescue therapy has substantially increased in the last year. The present review discusses changes in the design of these studies, the main methodological issues to be taken into account and the recommendations on this subject.