Misleading Statistics: The Problems Surrounding Number Needed to Treat and Number Needed to Harm

Interpreting Randomized Controlled Trials

This article describes rationales and limitations for making inferences based on data from randomized controlled trials (RCTs). We argue that obtaining a representative random sample from a patient population is impossible for a clinical trial because patients are accrued sequentially over time and thus comprise a convenience sample, subject only to protocol entry criteria. Consequently, the trial's sample is unlikely to represent a definable patient population. We use causal diagrams to illustrate the difference between random allocation of interventions within a clinical trial sample and true simple or stratified random sampling, as executed in surveys. We argue that group-specific statistics, such as a median survival time estimate for a treatment arm in an RCT, have limited meaning as estimates of larger patient population parameters. In contrast, random allocation between interventions facilitates comparative causal inferences about between-treatment effects, such as hazard ratios or differences between probabilities of response. Comparative inferences also require the assumption of transportability from a clinical trial's convenience sample to a targeted patient population. We focus on the consequences and limitations of randomization procedures in order to clarify the distinctions between pairs of complementary concepts of fundamental importance to data science and RCT interpretation. These include internal and external validity, generalizability and transportability, uncertainty and variability, representativeness and inclusiveness, blocking and stratification, relevance and robustness, forward and reverse causal inference, intention to treat and per protocol analyses, and potential outcomes and counterfactuals.

A framework for assessing the impact of accelerated approval

The FDA's Accelerated Approval program (AA) is a regulatory program to expedite availability of products to treat serious or life-threatening illnesses that lack effective treatment alternatives. Ideally, all of the many stakeholders such as patients, physicians, regulators, and health technology assessment [HTA] agencies that are affected by AA should benefit from it. In practice, however, there is intense debate over whether evidence supporting AA is sufficient to meet the needs of the stakeholders who collectively bring an approved product into routine clinical care. As AAs have become more common, it becomes essential to be able to determine their impact objectively and reproducibly in a way that provides for consistent evaluation of therapeutic decision alternatives. We describe the basic features of an approach for evaluating AA impact that accommodates stakeholder-specific views about potential benefits, risks, and costs. The approach is based on a formal decision-analytic framework combining predictive distributions for therapeutic outcomes (efficacy and safety) based on statistical models that incorporate findings from AA trials with stakeholder assessments of various actions that might be taken. The framework described here provides a starting point for communicating the value of a treatment granted AA in the context of what is important to various stakeholders.

NVX‐Cov2373 Novavax Covid‐19 vaccine: A further analysis of its efficacy using multiple modes of expression

A fifth vaccine against Covid‐19, NVX‐CoV2373 Nuvavoxid® (Novavax), a protein‐based adjuvanted vaccine, was recently marketed in Europe. The main clinical trial before marketing concluded to a ‘vaccine efficacy’ of 89.7% without talking about other validated efficacy parameters. We further analysed the data of this clinical trial using the different validated methods of risk expression: absolute risks (AR), AR reduction (ARR) and number needed to treat (NNT). ARR and NNT values were 1.22% and 82, respectively, for an RR value of 0.10. Description of these parameters allowed defining some interesting characteristics of NVX‐CoV2373 efficacy according to age, race, variant and coexisting illness. Finally, we ask that the results of clinical trials be systematically presented, using not only RR but also including AR, ARR and NNT.

Efficacy of COVID-19 vaccines: Several modes of expression should be presented in scientific publications

Several vaccines are being developed as part of the COVID‐19 pandemic. The results of clinical trials for these vaccines were published with efficacy values of more than 90%, using mainly relative risk (RR). In this paper, we decided to reanalyse the data using the different validated methods of risk expression. Using main publications, absolute risks (AR), AR reduction (ARR), number needed to treat (NNT) were calculated for five COVID‐19 vaccines (tozinameran Comirnaty®, Moderna, Vaxzevria®, Janssen, and Sputnik V vaccines). AR, ARR, NNT, and RR values varied according to COVID‐19 vaccines. The order of the different vaccines was not the same according to the chosen efficacy parameters. This is a further example of the need to express results of clinical trials, using not only RR, but also AR, ARR, and NNT in order to clearly present the clinical interest of drugs.

Making Patient-Specific Treatment Decisions Using Prognostic Variables and Utilities of Clinical Outcomes

We argue that well-informed patient-specific decision-making may be carried out as three consecutive tasks: (1) estimating key parameters of a statistical model, (2) using prognostic information to convert these parameters into clinically interpretable values, and (3) specifying joint utility functions to quantify risk-benefit trade-offs between clinical outcomes. Using the management of metastatic clear cell renal cell carcinoma as our motivating example, we explain the role of prognostic covariates that characterize between-patient heterogeneity in clinical outcomes. We show that explicitly specifying the joint utility of clinical outcomes provides a coherent basis for patient-specific decision-making.

Optimum duration of regimens for Helicobacter pylori eradication

BACKGROUND

The optimal duration for Helicobacter pylori (H. pylori) eradication therapy is controversial, with recommendations ranging from 7 to 14 days. Several systematic reviews have attempted to address this issue but have given conflicting results and limited their analysis to proton pump inhibitor (PPI), two antibiotics (PPI triple) therapy. We performed a systematic review and meta-analysis to investigate the optimal duration of multiple H. pylori eradication regimens.

OBJECTIVES

The primary objective was to assess the relative effectiveness of different durations (7, 10 or 14 days) of a variety of regimens for eradicating H. pylori. The primary outcome was H. pylori persistence. The secondary outcome was adverse events.

SEARCH METHODS

The Cochrane Library, MEDLINE, EMBASE, and CINAHL were searched up to December 2011 to identify eligible randomised controlled trials (RCTs). We also searched the proceedings of six conferences from 1995 to 2011, dissertations and theses, and grey literature. There were no language restrictions applied to any search.

SELECTION CRITERIA

Only parallel group RCTs assessing the efficacy of one to two weeks duration of first line H. pylori eradication regimens in adults were eligible. Within each regimen, the same combinations of drugs at the same dose were compared over different durations. Studies with at least two arms comparing 7, 10, or 14 days were eligible. Enrolled participants needed to be diagnosed with at least one positive test for H. pylori on the basis of a rapid urease test (RUT), histology, culture, urea breath test (UBT), or a stool antigen test (HpSA) before treatment. Eligible trials needed to confirm eradication of H. pylori as their primary outcome at least 28 days after completion of eradication treatment. Trials using only serology or a polymerase chain reaction (PCR) to determine H. pylori infection or eradication were excluded.

DATA COLLECTION AND ANALYSIS

Study eligibility and data extraction were performed by two independent review authors. Data analyses were performed within each type of intervention, for both primary and secondary outcomes. The relative risk (RR) and number needed to treat (NNT)/number needed to harm (NNTH) according to duration of therapy were calculated using the outcomes of H. pylori persistence and adverse events. A random-effects model was used. Subgroup analyses and sensitivity analyses were planned a priori.

MAIN RESULTS

In total, 75 studies met the inclusion criteria. Eight types of regimens were reported with at least two comparative eligible durations. They included: PPI + two antibiotics triple therapy (n = 59), PPI bismuth-based quadruple therapy (n = 6), PPI + three antibiotics quadruple therapy (n = 1), PPI dual therapy (n = 2), histamine H2-receptor antagonist (H₂RA) bismuth quadruple therapy (n = 3), H₂RA bismuth-based triple therapy (n = 2), H₂RA + two antibiotics triple therapy (n = 3), and bismuth + two antibiotics triple therapy (n = 2). Some studies provided data for more than one regimen or more than two durations.For the PPI triple therapy, 59 studies with five regimens were reported: PPI + clarithromycin + amoxicillin (PCA); PPI + clarithromycin + a nitroimidazole (PCN); PPI + amoxicillin + nitroimidazole (PAN); PPI + amoxicillin + a quinolone (PAQ); and PPI + amoxicillin + a nitrofuran (PANi). Regardless of type and dose of antibiotics, increased duration of PPI triple therapy from 7 to 14 days significantly increased the H. pylori eradication rate (45 studies, 72.9% versus 81.9%), the RR for H. pylori persistence was 0.66 (95% CI 0.60 to 0.74), NNT was 11 (95% CI 9 to 14). Significant effects were seen in the subgroup of PCA (34 studies, RR 0.65, 95% CI 0.57 to 0.75; NNT 12, 95% CI 9 to 16); PAN (10 studies, RR 0.67, 95% CI 0.52 to 0.86; NNT = 11, 95% CI 8 to 25); and in PAQ (2 studies, RR 0.37, 95% CI 0.16 to 0.83; NNT 3, 95% CI 2 to 10); but not in PCN triple therapy (4 studies, RR 0.87, 95% CI 0.71 to 1.07). Significantly increased eradication rates were also seen for PPI triple therapy with 10 versus 7 days (24 studies, 79.9% versus 75.7%; RR 0.80, 95% CI 0.72 to 0.89; NNT 21, 95% CI 15 to 38) and 14 versus 10 days (12 studies, 84.4% versus 78.5%; RR 0.72, 95% CI 0.58 to 0.90; NNT 17, 95% CI 11 to 46); especially in the subgroup of PAC for 10 versus 7 days (17 studies, RR 0.80, 95% CI 0.70 to 0.91) and for 14 versus 10 days (10 studies, RR 0.69, 95% CI 0.52 to 0.91). A trend towards increased H. pylori eradication rates was seen with increased duration of PCN for 10 versus 7 days, and of PAN for 10 versus 7 days and 14 versus 10 days, though this was not statistical significant. The proportion of patients with adverse events, defined by authors, was marginally significantly increased only between 7 days and 14 days (15.5% versus 19.4%; RR 1.21, 95% CI 1.06 to 1.37; NNTH 31, 95% CI 18 to 104) but not for other duration comparisons. The proportion of patients discontinuing treatment due to adverse events was not significantly different between treatment durations.Only limited data were reported for different durations of regimens other than PPI triple therapy. No significant difference of the eradication rate was seen for all regimens according to different durations except for H₂RA bismuth quadruple therapy, where a significantly higher eradication rate was seen for 14 days versus 7 days, however only one study reported outcome data.

AUTHORS' CONCLUSIONS

Increasing the duration of PPI-based triple therapy increases H. pylori eradication rates. For PCA, prolonging treatment duration from 7 to 10 or from 10 to 14 days is associated with a significantly higher eradication rate. The optimal duration of therapy for PCA and PAN is at least 14 days. More data are needed to confirm if there is any benefit of increasing the duration of therapy for PCN therapy. Information is limited for regimens other than PPI triple therapy; more studies are needed to draw meaningful conclusions for optimal duration of other H. pylori eradication regimens.

Is structured group psychoeducation for bipolar patients effective in ordinary mental health services? A controlled trial in Italy

BACKGROUND

Recent reviews of evidence-based guidelines for the clinical management of Bipolar Disorders (BD) have recommended that "all patients with BD be offered group or individual psychoeducation" to prevent relapse, improve treatment adherence, quality of life, and functioning. The present study evaluated the effectiveness of psychoeducation in routine mental health services in reducing number of hospitalisations and number of days spent in hospital, at a 1-year follow-up.

METHODS

A total of 102 outpatients were recruited from two Italian Departments of Mental Health. Inclusion criteria were a lifetime BD type I or II diagnosis, assessed with SCID, and ≥ 3 months of euthymia. Exclusion criteria were DSM-IV Axis I comorbidity, mental retardation (IQ<70), organic brain damage, or deafness. All participants received standard psychiatric care, including standard pharmacological treatment; the experimental group also received 21 group psychoeducation sessions, weekly held and conducted according to Colom and Vieta's model.

RESULTS

The number of patients hospitalised during the 1-year follow-up, the mean number of hospitalisations per patient, and the mean number of hospitalisation days were significantly lower for psychoeducated patients.

CONCLUSION

Our findings support the view that group psychoeducation is an effective way to prevent hospitalisation and decrease hospital days in pharmacologically treated patients with bipolar disorder also in routine clinical settings. The results confirm that psychoeducation promotes improvement in illness course by preventing acute phases and enhancing mood stability, and consequently, improvement in the quality of life for people with BD.

Number needed to harm: its limitations in psychotropic drug safety research

Commonly used statistical measures to quantify the likelihood of an adverse drug event (ADE) from clinical trials include risk ratio; odds ratio; and number needed to harm (NNH), the reciprocal of absolute risk. This critical review focused on NNH, specifically on its limitations in controlled trials with psychotropic medication. Data for this evaluation were obtained primarily from articles in MEDLINE from 1988 to 2012. Limitations of NNH were found to include the following: a) arbitrary binary cutoffs for continuous measures, b) limited use of confidence intervals, c) limited adjustments for potential baseline confounders, d) limited adjustments for differences in dose and treatment duration, e) rare consideration of high attrition rates, f) variable use of the term harm, g) oversimplified single harm comparisons, h) frequent biased design and reporting, i) undue emphasis on less severe ADEs, j) application primarily to short-term clinical trials, and k) little or no generalizability in community practice. In sum, the NNH metric supplies very limited information on the risks of psychotropic medication. Postmarketing surveillance of community treatment populations using case-control methodology provides far more useful data on serious ADEs.