The Net Benefit of a treatment should take the correlation between benefits and harms into account

Assessing the benefit-risk balance of drugs. Some lessons from the COVID pandemic

INTRODUCTION

Drug efficacy and effectiveness are assessed respectively through clinical trials and pharmaco-epidemiological studies. However, relative and absolute benefits of drugs are distinct measures that must be considered in relation to the baseline risk of disease incidence, complication or progression. On the other hand, adverse drug reactions are independent of the basic risk but depend on the characteristics of the population treated. Given these prerequisites, how can we balance the benefits and risks of drugs?

AREAS COVERED

We use the example of therapeutics evaluated during Covid to describe how assessing the benefit-risk balance of drugs is a complex process.

EXPERT OPINION

Clinical trials are not designed to identify rare adverse events, underscoring the necessity for a pharmacovigilance system. Evaluating the balance between the benefits and risks of drugs is an ongoing process, demanding the simultaneous analysis of data from clinical trials, potential drug-drug interactions, pharmacovigilance monitoring and pharmaco-epidemiological studies, to identify potential safety concerns. In addition, pharmacologists must play a major role in educating the general public about drugs, aiding in the accurate interpretation of the benefit-risk balance and preventing misinformation.

Evaluating the balance of benefits and harms in chronic pain clinical trials: prioritizing individual participants over individual outcomes

BACKGROUND

Randomized clinical trials (RCTs) generally assess efficacy and safety separately, with the conclusion of whether a treatment is beneficial based solely on the efficacy endpoint. However, assessing and combining efficacy and safety domains, using a single composite outcome measure, can provide a more comprehensive assessment of the overall effect of a treatment. Furthermore, composite outcomes can incorporate information regarding the relationship between the individual outcomes. In fact, such outcomes have been suggested in the clinical trials literature for at least 15 years.

OBJECTIVES

To (1) identify whether recent primary publications of chronic pain RCTs from major pain journals included a composite outcome measure of benefits and harms and (2) discuss the potential benefits of such outcomes in various stages of treatment development, including as outcome measures in RCTs, and to support decisions of Data and Safety Monitoring Boards and ordering of treatments in the context of treatment guidelines.

EVIDENCE REVIEW

RCTs published in 6 major pain journals published between 2016 and 2021 that investigated interventions for chronic pain were reviewed.

FINDINGS

Of 73 RCTs identified, only 2 included a composite outcome measure of benefits and harms. Both of these articles compared 2 active treatments.

CONCLUSIONS

Composite outcomes of benefits and harms are underutilized in chronic pain RCTs. The advantages and challenges of using such outcomes are discussed.

Design of a clinical trial using generalized pairwise comparisons to test a less intensive treatment regimen

BACKGROUND/AIMS

Showing "similar efficacy" of a less intensive treatment typically requires a non-inferiority trial. Yet such trials may be challenging to design and conduct. In acute promyelocytic leukemia, great progress has been achieved with the introduction of targeted therapies, but toxicity remains a major clinical issue. There is a pressing need to show the favorable benefit/risk of less intensive treatment regimens.

METHODS

We designed a clinical trial that uses generalized pairwise comparisons of five prioritized outcomes (alive and event-free at 2 years, grade 3/4 documented infections, differentiation syndrome, hepatotoxicity, and neuropathy) to confirm a favorable benefit/risk of a less intensive treatment regimen. We conducted simulations based on historical data and assumptions about the differences expected between the standard of care and the less intensive treatment regimen to calculate the sample size required to have high power to show a positive Net Treatment Benefit in favor of the less intensive treatment regimen.

RESULTS

Across 10,000 simulations, average sample sizes of 260 to 300 patients are required for a trial using generalized pairwise comparisons to detect typical Net Treatment Benefits of 0.19 (interquartile range 0.14-0.23 for a sample size of 280). The Net Treatment Benefit is interpreted as a difference between the probability of doing better on the less intensive treatment regimen than on the standard of care, minus the probability of the opposite situation. A Net Treatment Benefit of 0.19 translates to a number needed to treat of about 5.3 patients (1/0.19 ≃ 5.3).

CONCLUSION

Generalized pairwise comparisons allow for simultaneous assessment of efficacy and safety, with priority given to the former. The sample size required would be of the order of 300 patients, as compared with more than 700 patients for a non-inferiority trial using a margin of 4% against the less intensive treatment regimen for the absolute difference in event-free survival at 2 years, as considered here.

Impact of correlations between prioritized outcomes on the net benefit and its estimate by generalized pairwise comparisons

Benefit-risk balance is gaining interest in clinical trials. For the comprehensive assessment of benefits and risks, generalized pairwise comparisons are increasingly used to estimate the net benefit based on multiple prioritized outcomes. Although previous research has demonstrated that the correlations between the outcomes impact the net benefit and its estimate, the direction and magnitude of this impact remain unclear. In this study, we investigated the impact of correlations between two binary or Gaussian variables on the true net benefit values via theoretical and numerical analyses. We also explored the impact of correlations between survival and categorical variables on the net benefit estimates based on four existing methods (Gehan, Péron, Gehan with correction, and Péron with correction) in the presence of right censoring via simulation and application to actual oncology clinical trial data. Our theoretical and numerical analyses revealed that the true net benefit values were impacted by the correlations in various directions depending on the outcome distributions. With binary endpoints, this direction was governed by a simple rule with a threshold of 50% for a favorable outcome. Our simulation showed that the net benefit estimates based on Gehan's or Péron's scoring rule could be substantially biased in the presence of right censoring, and that the direction and magnitude of this bias were associated with the outcome correlations. The recently proposed correction method greatly reduced this bias, even in the presence of strong outcome correlations. The impact of correlations should be carefully considered when interpreting the net benefit and its estimate.

Individualized Net Benefit estimation and meta-analysis using generalized pairwise comparisons in N-of-1 trials

BACKGROUND

The Net Benefit (Δ) is a measure of the benefit-risk balance in clinical trials, based on generalized pairwise comparisons (GPC) using several prioritized outcomes and thresholds of clinical relevance. We extended Δ to N-of-1 trials, with a focus on patient-level and population-level Δ.

METHODS

We developed a Δ estimator at the individual level as an extension of the stratum-specific Δ, and at the population-level as an extension of the stratified Δ. We performed a simulation study mimicking PROFIL, a series of 38 N-of-1 trials testing sildenafil in Raynaud's phenomenon, to assess the power for such an analysis with realistic data. We then reanalyzed PROFIL using GPC. This reanalysis was finally interpreted in the context of the main analysis of PROFIL which used Bayesian individual probabilities of efficacy.

RESULTS

Simulations under the null showed good size of the test for both individual and population levels. The test lacked power when being simulated from the true PROFIL data, even when increasing the number of repetitions up to 140 days per patient. PROFIL individual-level estimated Δ were well correlated with the probabilities of efficacy from the Bayesian analysis while showing similarly wide confidence intervals. Population-level estimated Δ was not significantly different from zero, consistently with the previous Bayesian analysis.

CONCLUSION

GPC can be used to estimate individual Δ which can then be aggregated in a meta-analytic way in N-of-1 trials. GPC ability to easily incorporate patient preferences allow for more personalized treatment evaluation, while needing much less computing time than Bayesian modeling.

The patient-reported outcomes measurement information systems (PROMIS®) physical function and its derivative measures in adults: a systematic review of content validity

PURPOSE

This study aims to systematically review and critically appraise the content validity of the adult versions of the Patient-Reported Outcomes Measurement Information System Physical Function (PROMIS-PF) item bank and its derivative measures in any adult population.

METHODS

MEDLINE and EMBASE were searched in October 2021 for studies on measurement properties of PROMIS-PF measures in an adult population. Studies were included if the study described the development of a PROMIS-PF measure or investigated its relevance, comprehensiveness, or comprehensibility. Assessment of the methodological quality of eligible studies, rating of results, and summarizing evidence was performed following the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) methodology for assessing content validity. A modified GRADE approach was used to determine the level of evidence.

RESULTS

Three development studies and eight studies on the content validity of one or more of the PROMIS-PF measures were identified. The methodological quality of most studies was rated doubtful. There was low to high level evidence for sufficient relevance, comprehensiveness, and comprehensibility of most PROMIS-PF measures for healthy seniors and various disease populations. We found low to moderate level evidence for insufficient relevance of PROMIS-PF measures for patients with conditions that affected only one body part, and insufficient comprehensibility of the PROMIS-PF measures for minority elderly.

CONCLUSION

Most PROMIS-PF measures demonstrate sufficient content validity in healthy seniors and various disease populations. However, the quality of this evidence is generally low to moderate, due to limitations in the methodological quality of the studies.

The Impasse on Overall Survival in Oncology Reimbursement Decision-Making: How Can We Resolve This?

Mature overall survival (OS) data are often unavailable at the time of regulatory and reimbursement decisions for a new cancer treatment. For patients with early-stage cancers treated with potentially curative treatments, demonstrating an OS benefit may take years and may be confounded by subsequent lines of therapy or crossover to the investigational treatment. For patients with advanced-stage cancers, mature OS data may be available but difficult to interpret for similar reasons. There are strong opinions about approval and reimbursement in the absence of mature OS data, with concerns over delay in patient access set against concerns about uncertainty in long-term benefit. This position paper reflects our individual views as patient advocate, clinician or health economist on one aspect of this debate. We look at payer decisions in the absence of mature OS data, considering when and how non-OS trial outcomes could inform decision-making and how uncertainty can be addressed beyond the trial, supporting these views with evidence from the literature. We consider when it is reasonable for payers to expect or not expect mature OS data at the initial reimbursement decision (based on criteria such as cancer stage and treatment efficacy) acknowledging that there are settings in which mature OS data are expected. We propose flexible strategies for generating and appraising patient-relevant evidence, including context-relevant endpoints and quality of life measures, when survival rates are good and mature OS data are not expected. We note that fair reimbursement is important; this means valuing patient benefit as shown through prespecified endpoints and reappraising if there is ongoing uncertainty or failure to show a sustained benefit. We suggest that reimbursement systems continue to evolve to align with scientific advances, because innovation is only meaningful if readily accessible to patients. The proposed strategies have the potential to promote thorough assessment of potential benefit to patients and lead to timely access to effective medicines.