Improving the information content of categorical clinical trial endpoints

Conflicts of Interest, Selective Inertia, and Research Malpractice in Randomized Clinical Trials: An Unholy Trinity

Recently a great deal of attention has been paid to conflicts of interest in medical research, and the Institute of Medicine has called for more research into this important area. One research question that has not received sufficient attention concerns the mechanisms of action by which conflicts of interest can result in biased and/or flawed research. What discretion do conflicted researchers have to sway the results one way or the other? We address this issue from the perspective of selective inertia, or an unnatural selection of research methods based on which are most likely to establish the preferred conclusions, rather than on which are most valid. In many cases it is abundantly clear that a method that is not being used in practice is superior to the one that is being used in practice, at least from the perspective of validity, and that it is only inertia, as opposed to any serious suggestion that the incumbent method is superior (or even comparable), that keeps the inferior procedure in use, to the exclusion of the superior one. By focusing on these flawed research methods we can go beyond statements of potential harm from real conflicts of interest, and can more directly assess actual (not potential) harm.

Evidence-based diagnostics: adult septic arthritis

BACKGROUND

Acutely swollen or painful joints are common complaints in the emergency department (ED). Septic arthritis in adults is a challenging diagnosis, but prompt differentiation of a bacterial etiology is crucial to minimize morbidity and mortality.

OBJECTIVES

The objective was to perform a systematic review describing the diagnostic characteristics of history, physical examination, and bedside laboratory tests for nongonococcal septic arthritis. A secondary objective was to quantify test and treatment thresholds using derived estimates of sensitivity and specificity, as well as best-evidence diagnostic and treatment risks and anticipated benefits from appropriate therapy.

METHODS

Two electronic search engines (PUBMED and EMBASE) were used in conjunction with a selected bibliography and scientific abstract hand search. Inclusion criteria included adult trials of patients presenting with monoarticular complaints if they reported sufficient detail to reconstruct partial or complete 2 × 2 contingency tables for experimental diagnostic test characteristics using an acceptable criterion standard. Evidence was rated by two investigators using the Quality Assessment Tool for Diagnostic Accuracy Studies (QUADAS). When more than one similarly designed trial existed for a diagnostic test, meta-analysis was conducted using a random effects model. Interval likelihood ratios (LRs) were computed when possible. To illustrate one method to quantify theoretical points in the probability of disease whereby clinicians might cease testing altogether and either withhold treatment (test threshold) or initiate definitive therapy in lieu of further diagnostics (treatment threshold), an interactive spreadsheet was designed and sample calculations were provided based on research estimates of diagnostic accuracy, diagnostic risk, and therapeutic risk/benefits.

RESULTS

The prevalence of nongonococcal septic arthritis in ED patients with a single acutely painful joint is approximately 27% (95% confidence interval [CI] = 17% to 38%). With the exception of joint surgery (positive likelihood ratio [+LR] = 6.9) or skin infection overlying a prosthetic joint (+LR = 15.0), history, physical examination, and serum tests do not significantly alter posttest probability. Serum inflammatory markers such as white blood cell (WBC) counts, erythrocyte sedimentation rate (ESR), and C-reactive protein (CRP) are not useful acutely. The interval LR for synovial white blood cell (sWBC) counts of 0 × 10(9)-25 × 10(9)/L was 0.33; for 25 × 10(9)-50 × 10(9)/L, 1.06; for 50 × 10(9)-100 × 10(9)/L, 3.59; and exceeding 100 × 10(9)/L, infinity. Synovial lactate may be useful to rule in or rule out the diagnosis of septic arthritis with a +LR ranging from 2.4 to infinity, and negative likelihood ratio (-LR) ranging from 0 to 0.46. Rapid polymerase chain reaction (PCR) of synovial fluid may identify the causative organism within 3 hours. Based on 56% sensitivity and 90% specificity for sWBC counts of >50 × 10(9)/L in conjunction with best-evidence estimates for diagnosis-related risk and treatment-related risk/benefit, the arthrocentesis test threshold is 5%, with a treatment threshold of 39%.

CONCLUSIONS

Recent joint surgery or cellulitis overlying a prosthetic hip or knee were the only findings on history or physical examination that significantly alter the probability of nongonococcal septic arthritis. Extreme values of sWBC (>50 × 10(9)/L) can increase, but not decrease, the probability of septic arthritis. Future ED-based diagnostic trials are needed to evaluate the role of clinical gestalt and the efficacy of nontraditional synovial markers such as lactate.

A randomized controlled trial of the effect of service delivery information on patient satisfaction in an emergency department fast track

OBJECTIVES

The objective was to determine the effect on patient satisfaction of providing patients with predicted service completion times.

METHODS

A randomized controlled trial was conducted in an urban, community teaching hospital. Emergency department (ED) patients triaged to fast track on weekdays between October 26, 2009, and December 30, 2009, from 9 am to 5 pm were eligible. Patients were randomized to: 1) usual care (n = 342), 2) provided ED process information (n = 336), or 3) provided ED process information plus predicted service delivery times (n = 333). Patients in group 3 were given an "average" and "upper range" estimate of their waiting room times and treatment times. The average and upper range predictions were calculated from quantile regression models that estimated the 50th and 90th percentiles of the waiting room time and treatment time distributions for fast track patients at the study site based on 2.5 years of historical data. Trained research assistants administered the interventions after triage. Patients completed a brief survey at discharge that measured their satisfaction with overall care, the quality of the information they received, and the timeliness of care. Satisfaction ratings of very good versus good, fair, poor, and very poor were modeled using logistic regression as a function of study group; actual service delivery times; and other patient, clinical, and temporal covariates. The study also modeled satisfaction ratings of fair, poor, and very poor compared to good and very good ratings as a function of the same covariates.

RESULTS

Survey completion rates and patient, clinical, and temporal characteristics were similar by study group. Median waiting room time was 70 minutes (interquartile range [IQR] = 40 to 114 minutes), and median treatment time was 52 minutes (IQR = 31 to 81 minutes). Neither intervention affected any of the satisfaction outcomes. Satisfaction was significantly associated with actual waiting room time, individual providers, and patient age. Every 10-minute increase in waiting room time corresponded with an 8% decrease (odds ratio [OR] = 0.92; 95% confidence interval [CI] = 0.89 to 0.95) in the odds of reporting very good satisfaction with overall care. The odds of reporting very good satisfaction with care were lower for several triage nurses and fast track nurses, compared to the triage nurse and fast track nurse who treated the most study patients. Each 10-minute increase in waiting room time was also associated with a 10% increase in the odds of reporting very poor, poor, or fair satisfaction with overall care (OR = 1.10; 95% CI = 1.06 to 1.14). The odds of reporting very poor, poor, or fair satisfaction with overall care also varied significantly among the triage nurses, fast track doctors, and fast track nurses. The odds of reporting very poor, poor, or fair satisfaction with overall care were significantly lower among patients aged 35 years and older compared to patients aged 18 to 34 years.

CONCLUSIONS

Satisfaction with overall care was influenced by waiting room time and the clinicians who treated them and not by service completion time estimates provided at triage.

Testing for heterogeneity among the components of a binary composite outcome in a clinical trial

Background

Investigators designing clinical trials often use composite outcomes to overcome many statistical issues. Trialists want to maximize power to show a statistically significant treatment effect and avoid inflation of Type I error rate due to evaluation of multiple individual clinical outcomes. However, if the treatment effect is not similar among the components of this composite outcome, we are left not knowing how to interpret the treatment effect on the composite itself. Given significant heterogeneity among these components, a composite outcome may be judged as being invalid or un-interpretable for estimation of the treatment effect. This paper compares the power of different tests to detect heterogeneity of treatment effect across components of a composite binary outcome.

Methods

Simulations were done comparing four different models commonly used to analyze correlated binary data. These models included: logistic regression for ignoring correlation, logistic regression weighted by the intra cluster correlation coefficient, population average logistic regression using generalized estimating equations (GEE), and random effects logistic regression.

Results

We found that the population average model based on generalized estimating equations (GEE) had the greatest power across most scenarios. Adequate power to detect possible composite heterogeneity or variation between treatment effects of individual components of a composite outcome was seen when the power for detecting the main study treatment effect for the composite outcome was also reasonably high.

Conclusions

It is recommended that authors report tests of composite heterogeneity for composite outcomes and that this accompany the publication of the statistically significant results of the main effect on the composite along with individual components of composite outcomes.

A general framework for the evaluation of clinical trial quality

Flawed evaluation of clinical trial quality allows flawed trials to thrive (get funded, obtain IRB approval, get published, serve as the basis of regulatory approval, and set policy). A reasonable evaluation of clinical trial quality must recognize that any one of a large number of potential biases could by itself completely invalidate the trial results. In addition, clever new ways to distort trial results toward a favored outcome may be devised at any time. Finally, the vested financial and other interests of those conducting the experiments and publishing the reports must cast suspicion on any inadequately reported aspect of clinical trial quality. Putting these ideas together, we see that an adequate evaluation of clinical quality would need to enumerate all known biases, update this list periodically, score the trial with regard to each potential bias on a scale of 0% to 100%, offer partial credit for only that which can be substantiated, and then multiply (not add) the component scores to obtain an overall score between 0% and 100%. We will demonstrate that current evaluations fall well short of these ideals.

Re-formulating non-inferiority trials as superiority trials: The case of binary outcomes

Non-inferiority trials are conducted for a variety of reasons including to show that a new treatment has a negligible reduction in efficacy or safety when compared to the current standard treatment, or a more complex setting of showing that a new treatment has a negligible reduction in efficacy when compared to the current standard yet is superior in terms of other treatment characteristics. The latter reason for conducting a non-inferiority trial presents the challenge of deciding on a balance between a suitable reduction in efficacy, known as the non-inferiority margin, in return for a gain in other important treatment characteristics/findings. It would be ideal to alleviate the dilemma on the choice of margin in this setting by reverting to a traditional superiority trial design where a single p -value for superiority of both the most important endpoint (efficacy) and the most important finding (treatment characteristic) is provided. We discuss how this can be done using the information-preserving composite endpoint (IPCE) approach and consider binary outcome cases in which the combination of efficacy and treatment characteristics, but not one itself, paints a clear picture that the novel treatment is superior to the active control.

Biomedical research and corporate interests: a question of academic freedom

The current situation in medicine has been described as a crisis of credibility, as the profit motive of industry has taken control of clinical trials and the dissemination of data. Pharmaceutical companies maintain a stranglehold over the content of medical journals in three ways: (1) by ghostwriting articles that bias the results of clinical trials, (2) by the sheer economic power they exert on journals due to the purchase of drug advertisements and journal reprints, and (3) by the threat of legal action against those researchers who seek to correct the misrepresentation of study results. This paper argues that Karl Popper's critical rationalism provides a corrective to the failure of academic freedom in biomedical research.

Disciplined analyses in clinical trials: the dark heart of the matter

Clinical research analyses must balance the desire to `learn all that is learnable' from the database with the observation that sample-based data commonly lead to conclusions that are perfectly correct for the sample, but wholly incorrect for the population from which the data were based.

Investigators who defend exploratory analyses as reliable, misuse important tools that have taken over three hundred years to develop. Statistical estimators in clinical trials function appropriately when they incorporate random data that is gathered in response to a “xed research question. Their prediction ability degrades rapidly when the selection of the research question is itself random, that is, left to the data. Operating like blind guides, these estimators mislead the medical community about what it would see in the population, based on sample observations. The result is a wavering research focus, leaping from one provocative but misleading “nding to the next on the powerful waves of sampling error. Therefore, a primary purpose of the prospective design is to “x the research questions prospectively, thereby anchoring the analysis plan.

Prospective statements of the research questions and rejection of tempting databased changes to the protocol preserve the best estimates of effect sizes, standard errors, con“dence intervals and p-values. Embracing these principles promotes the prosecution of a successful research program, that is, the construction and protection of a research environment that permits an objective assessment of the therapy or exposure being studied. If there is any “xed star in the research constellation, it is that sample-based research must be hypothesis-driven and concordantly executed to have real meaning for both the scienti“c community and the patient populations that we serve.

A placebo-controlled randomized clinical trial of perioperative administration of gabapentin, rofecoxib and their combination for spontaneous and movement-evoked pain after abdominal hysterectomy

Current treatments for post-injury movement-evoked pain are inadequate. Non-opioids may complement opioids, which preferentially reduce spontaneous pain, but most have incomplete efficacy as single agents. This trial evaluates efficacy of a gabapentin-rofecoxib combination following hysterectomy. In addition to IV-PCA morphine, 110 patients received either placebo, gabapentin (1800 mg/day), rofecoxib (50 mg/day) or a gabapentin-rofecoxib combination (1800/50 mg/day) starting 1 h pre-operatively for 72 h. Outcomes included pain at rest, evoked by sitting, peak expiration and cough, morphine consumption and peak expiratory flow (PEF). For placebo, gabapentin, rofecoxib and combination, 24 h pain (100 mm VAS) was: at rest-23.6 (P<0.05 vs. all treatments), 13.8, 14.4 and 12.1; during cough-50.7 (P<0.05 vs. all treatments), 41.5, 44.8 and 30.8; 48 h morphine consumption (mg) was: 130.4 (P<0.05 vs. all treatments), 81.7, 75.6 and 57.2 (P<0.05 vs. gabapentin and rofecoxib) and 48 h PEF (% baseline) was: 63.9 (P<0.05 vs. all treatments), 77.2, 76.7 and 87.5 (P<0.05 vs. gabapentin and rofecoxib). Adverse effects were similar in all groups except sedation which was more frequent with gabapentin. Combination and rofecoxib reduced pain interference with movement, mood and sleep (P<0.05) and combination was superior to gabapentin for all these three (P<0.05). These data suggest that a gabapentin-rofecoxib combination is superior to either single agent for postoperative pain. Other benefits include opioid sparing, reduced interference with movement, mood and sleep and increased PEF suggesting accelerated pulmonary recovery. Future research should identify optimal dose-ratios for this and other analgesic combinations.