Totality of outcomes: A different paradigm in assessing interventions for treatment of tuberculosis

A desirability of outcome ranking (DOOR) for periprosthetic joint infection - a Delphi analysis

Background: Treatment outcomes in studies on prosthetic joint infection are generally assessed using a dichotomous outcome relating to treatment success or failure. These outcome measures neither include patient-centred outcome measures including joint function and quality of life, nor do they account for adverse effects of treatment. A desirability of outcome ranking (DOOR) measure can include these factors and has previously been proposed and validated for other serious infections. We aimed to develop a novel DOOR for prosthetic joint infections (PJIs). Methods: The Delphi method was used to develop a DOOR for PJI research. An international working group of 18 clinicians (orthopaedic surgeons and infectious disease specialists) completed the Delphi process. The final DOOR comprised the dimensions established to be most important by consensus with > 75  % of participant agreement. Results: The consensus DOOR comprised four main dimensions. The primary dimension was patient-reported joint function. The secondary dimensions were infection cure and mortality. The final dimension of quality of life was selected as a tie-breaker. Discussion: A desirability of outcome ranking for periprosthetic joint infection has been proposed. It focuses on patient-centric outcome measures of joint function, cure and quality of life. This DOOR provides a multidimensional assessment to comprehensively rank outcomes when comparing treatments for prosthetic joint infection.

A Composite Endpoint for Treatment Benefit According to Patient Preference

Complex disorders usually affect multiple symptom domains measured by several outcomes. The importance of these outcomes is often different among patients. Current approaches integrate multiple outcomes without considering patient preferences at the individual level. In this paper, we propose a new composite Desirability of Outcome Ranking (DOOR) that integrates individual level ranking of outcome importance and define a winning probability measuring the overall treatment effect. Stratified randomization can be performed based on the participants' baseline outcome rankings. A Wilcoxon-Mann-Whitney U-statistic is used to average the pairwise DOOR between one treated and one control patient, considering the difference in these patients' ranking of outcome importance. We use both theoretical and empirical methods to examine the statistical properties of our method and to compare with conventional approaches. We conclude that the proposed composite DOOR properly reflects patient-level preferences and can be used in pivotal trials or comparative effectiveness trials for a patient-centered evaluation of overall treatment benefits.

A framework for considering the risk-benefit trade-off in designing noninferiority trials using composite outcome approaches

When a new treatment regimen is expected to have comparable or slightly worse efficacy to that of the control regimen but has benefits in other domains such as safety and tolerability, a noninferiority (NI) trial may be appropriate but is fraught with difficulty in justifying an acceptable NI margin that is based on both clinical and statistical input. To overcome this, we propose to utilize composite risk-benefit outcomes that combine elements from domains of importance (eg, efficacy, safety, and tolerability). The composite outcome itself may be analyzed using a superiority framework, or it can be used as a tool at the design stage of a NI trial for selecting an NI margin for efficacy that balances changes in risks and benefits. In the latter case, the choice of NI margin may be based on a novel quantity called the maximum allowable decrease in efficacy (MADE), defined as the marginal difference in efficacy between arms that would yield a null treatment effect for the composite outcome given an assumed distribution for the composite outcome. We observe that MADE: (1) is larger when the safety improvement for the experimental arm is larger, (2) depends on the association between the efficacy and safety outcomes, and (3) depends on the control arm efficacy rate. We use a numerical example for power comparisons between a superiority test for the composite outcome vs a noninferiority test for efficacy using the MADE as the NI margin, and apply the methods to a TB treatment trial.

Methodologies for pragmatic and efficient assessment of benefits and harms: Application to the SOCRATES trial

BACKGROUND/AIMS

Standard approaches to trial design and analyses can be inefficient and non-pragmatic. Failure to consider a range of outcomes impedes evidence-based interpretation and reduces power. Traditional approaches synthesizing information obtained from separate analysis of each outcome fail to incorporate associations between outcomes and recognize the cumulative nature of outcomes in individual patients, suffer from competing risk complexities during interpretation, and since efficacy and safety analyses are often conducted on different populations, generalizability is unclear. Pragmatic and efficient approaches to trial design and analyses are needed.

METHODS

Approaches providing a pragmatic assessment of benefits and harms of interventions, summarizing outcomes experienced by patients, and providing sample size efficiencies are described. Ordinal outcomes recognize finer gradations of patient responses. Desirability of outcome ranking is an ordinal outcome combining benefits and harms within patients. Analysis of desirability of outcome ranking can be based on rank-based methodologies including the desirability of outcome ranking probability, the win ratio, and the proportion in favor of treatment. Partial credit analyses, involving grading the levels of the desirability of outcome ranking outcome similar to an academic test, provides an alternative approach. The methodologies are demonstrated using the acute stroke or transient ischemic attack treated with aspirin or ticagrelor and patient outcomes study (SOCRATES; NCT01994720), a randomized clinical trial.

RESULTS

Two 5-level ordinal outcomes were developed for SOCRATES. The first was based on a modified Rankin scale. The odds ratio is 0.86 (95% confidence interval = 0.75, 0.99; p = 0.04) indicating that the odds of worse stroke categorization for a trial participant assigned to ticagrelor is 0.86 times that of a trial participant assigned to aspirin. The 5-level desirability of outcome ranking outcome incorporated and prioritized survival; the number of strokes, myocardial infarction, and major bleeding events; and whether a stroke event was disabling. The desirability of outcome ranking probability and win ratio are 0.504 (95% confidence interval = 0.499, 0.508; p = 0.10) and 1.11 (95% confidence interval = 0.98, 1.26; p = 0.10), respectively, implying that the probability of a more desirable result with ticagrelor is 50.4% and that a more desirable result occurs 1.11 times more frequently on ticagrelor versus aspirin.

CONCLUSION

Ordinal outcomes can improve efficiency through required pre-specification, careful construction, and analyses. Greater pragmatism can be obtained by composing outcomes within patients. Desirability of outcome ranking provides a global assessment of the benefits and harms that more closely reflect the experience of patients. The desirability of outcome ranking probability, the proportion in favor of treatment, the win ratio, and partial credit can more optimally inform patient treatment, enhance the understanding of the totality of intervention effects on patients, and potentially provide efficiencies over standard analyses. The methods provide the infrastructure for incorporating patient values and estimating personalized effects.

Applying a Risk-benefit Analysis to Outcomes in Tuberculosis Clinical Trials

Although it is common to analyze efficacy and safety separately in clinical trials, this could yield a misleading study conclusion if an increase in efficacy is accompanied by a decrease in safety. A risk-benefit analysis is a systematic approach to examine safety and efficacy jointly. Both the "rank-based" and "partial-credit" methods described in this paper allow researchers to create a single, composite outcome incorporating efficacy, safety, and other factors. The first approach compares the distribution of rankings between arms. In the second approach, a score can be assigned to each outcome category, considering its severity and comparing the mean or median scores of arms. The methods were applied to the A5279/Brief Rifapentine-Isoniazid Efficacy for TB Prevention study, and design considerations for future clinical trials are discussed, including the challenge of arriving at a consensus on rankings/scorings. If well designed, a risk-benefit analysis may allow for a superiority comparison and, therefore, avoid setting a noninferiority margin. Clinical Trials Registration. NCT01404312 (A5279).

Statistical considerations for pediatric multidrug-resistant tuberculosis efficacy trials

The inclusion of newly licensed or repurposed drugs in regimens to treat children for multidrug-resistant tuberculosis (TB) may lead to treatment that is shorter than traditional regimens and composed only of oral medications. As an all-oral regimen may be more acceptable and have a better safety profile than current regimens, demonstrating non-inferiority may be satisfactory. Demonstrating non-inferior efficacy requires setting a non-inferiority margin and safeguarding study assay sensitivity. Multi-arm, multistage designs may currently not be appropriate in pediatric trials because of the lack of sensitive and specific intermediate outcomes. However, including an arm with an agent to ameliorate toxicity would be efficient. Covariates can be used to stratify randomization, define subgroups, and improve efficiency of analysis. Enriching the sample for the confirmed-TB subgroup to ensure that they are well represented may be important. Primary outcomes using a fixed timepoint from randomization for all study arms will result in variations in post-treatment duration, but may be the best choice. While blinding of site personnel and patients may not be possible when regimens differ substantially in drugs and modes of administration, blinding should be maintained for independent endpoint review groups and other personnel. Type I error and family-wise error rates should be tightly controlled.

Use of composite outcomes to assess risk-benefit in clinical trials

Before a novel treatment can be deemed a clinical success, an assessment of its risk-benefit profile must be made. One of the inherent challenges for this assessment comes from the multiplicity that arises from comparing treatment groups across multiple outcomes. Composite outcomes that summarize a patient's clinical status, or severity, across a prioritized list of safety and efficacy outcomes have become increasing popular. In this article, we review these approaches and illustrate through examples some of the challenges and complexities of a composite derived from prioritized outcomes, such as the win ratio. These challenges include the difficult tension between the analytical validity that comes from choosing a pre-specified outcome and an evaluation that is responsive to unexpected safety events that arise during the course of a trial. Other challenges include a sensitivity of the resulting test statistic to the underlying censoring distribution and other nuisance parameters. Approaches that resolve some of the difficulties of the analytical challenges associated with prioritized outcomes are then discussed. Ultimately, a composite outcome of net clinical benefit is another decision tool, but one to be used alongside more traditional analyses of efficacy and safety, and with the broader perspective that investigators, the data safety monitoring board, and regulators bring to an evaluation of risk-benefit.