Measuring and illustrating statistical evidence in a cost-effectiveness analysis

A Tutorial on Net Benefit Regression for Real-World Cost-Effectiveness Analysis Using Censored Data from Randomized or Observational Studies

HIGHLIGHTS

We illustrate the steps involved in carrying out cost-effectiveness analysis using net benefit regressions with possibly censored demo data by providing step-by-step guidance and code applied to a data set.We demonstrate the importance of these new methods by illustrating how naïve methods for handling censoring can lead to biased cost-effectiveness results.

Cost-effectiveness of universal iron supplementation and iron-containing micronutrient powders for anemia among young children in rural Bangladesh: analysis of a randomized, placebo-controlled trial

BACKGROUND

Universal provision of iron supplements or iron-containing multiple micronutrient powders (MNPs) is widely used to prevent anemia in young children in low- and middle-income countries. The BRISC (Benefits and Risks of Iron Interventions in Children) trial compared iron supplements and MNPs with placebo in children <2 y old in rural Bangladesh.

OBJECTIVES

We aimed to assess the cost-effectiveness of iron supplements or iron-containing MNPs among young children in rural Bangladesh.

METHODS

We did a cost-effectiveness analysis of MNPs and iron supplements using the BRISC trial outcomes and resource use data, and programmatic data from the literature. Health care costs were assessed from a health system perspective. We calculated incremental cost-effectiveness ratios (ICERs) in terms of US$ per disability-adjusted life-year (DALY) averted. To explore uncertainty, we constructed cost-effectiveness acceptability curves using bootstrapped data over a range of cost-effectiveness thresholds. One- and 2-way sensitivity analyses tested the impact of varying key parameter values on our results.

RESULTS

Provision of MNPs was estimated to avert 0.0031 (95% CI: 0.0022, 0.0041) DALYs/child, whereas iron supplements averted 0.0039 (95% CI: 0.0030, 0.0048) DALYs/child, over 1 y compared with no intervention. Incremental mean costs were $0.75 (95% CI: 0.73, 0.77) for MNPs compared with no intervention and $0.64 ($0.62, $0.67) for iron supplements compared with no intervention. Iron supplementation dominated MNPs because it was cheaper and averted more DALYs. Iron supplementation had an ICER of $1645 ($1333, $2153) per DALY averted compared with no intervention, and had a 0% probability of being the optimal strategy at cost-effectiveness thresholds of $200 (reflecting health opportunity costs in Bangladesh) and $985 [half of gross domestic product (GDP) per capita] per DALY averted. Scenario and sensitivity analyses supported the base case findings.

CONCLUSIONS

These findings do not support universal iron supplementation or micronutrient powders as a cost-effective intervention for young children in rural Bangladesh.This trial was registered at anzctr.org.au as ACTRN1261700066038 and trialsearch.who.int as U1111-1196-1125.

Net benefit separation and the determination curve: A probabilistic framework for cost-effectiveness estimation

Considerations regarding clinical effectiveness and cost are essential in comparing the overall value of two treatments. There has been growing interest in methodology to integrate cost and effectiveness measures in order to inform policy and promote adequate resource allocation. The net monetary benefit aggregates information on differences in mean cost and clinical outcomes; the cost-effectiveness acceptability curve was developed to characterize the extent to which the strength of evidence regarding net monetary benefit changes with fluctuations in the willingness-to-pay threshold. Methods to derive insights from characteristics of the cost/clinical outcomes besides mean differences remain undeveloped but may also be informative. We propose a novel probabilistic measure of cost-effectiveness based on the stochastic ordering of the individual net benefit distribution under each treatment. Our approach is able to accommodate features frequently encountered in observational data including confounding and censoring, and complements the net monetary benefit in the insights it provides. We conduct a range of simulations to evaluate finite-sample performance and illustrate our proposed approach using simulated data based on a study of endometrial cancer patients.

Impact of small study bias on cost-effectiveness acceptability curves and value of information analyses

It is well known that small, randomized, controlled trials (RCTs) have limited validity. When comparing the results of meta-analyses with those of later large trials or with those of large trials removed from the meta-analyses, discrepancies were reported. This paper addresses two issues: (1) how measures of the uncertainty in cost-effectiveness, i.e., cost-effectiveness acceptability curves (CEACs), and the expected value of perfect information (EVPI) are affected by the limited validity of small trials and (2) how to deal with this bias. To this end, the paper adopts a Bayesian approach. Using empirical estimates for the validity of small RCTs compared to larger RCTs, the probability of cost-effectiveness drops by almost 10 %, while the EVPI is three times higher. In conclusion, traditional CEACs and EVPI analyses based on (small) RCTs may need careful appraisal. Ignoring prior evidence on the validity of small-size trials leads to an underestimation of uncertainty in cost-effectiveness. For future economic analyses, it is important to incorporate aspects of uncertainty which are caused by flawed data on effectiveness .

A randomized trial of the effect of centralized reminder/recall on immunizations and preventive care visits for adolescents

OBJECTIVE

To assess the impact of a managed care-based patient reminder/recall system on immunization rates and preventive care visits among low-income adolescents.

METHODS

We conducted a randomized controlled trial between December 2009 and December 2010 that assigned adolescents aged 11-17 years to one of three groups: mailed letter, telephone reminders, or control. Publicly insured youths (n = 4115) were identified in 37 participating primary care practices. The main outcome measures were immunization rates for routine vaccines (meningococcus, pertussis, HPV) and preventive visit rates at study end.

RESULTS

Intervention and control groups were similar at baseline for demographics, immunization rates, and preventive visits. Among adolescents who were behind at the start, immunization rates at study end increased by 21% for mailed (P < .01 vs control), 17% for telephone (P < .05), and 13% for control groups. The proportion of adolescents with a preventive visit (within 12 months) was: mailed (65%; P < .01), telephone (63%; P < .05), and controls (59%). The number needed to treat for an additional fully vaccinated adolescent was 14 for mailed and 25 for telephone reminders; for an additional preventive visit, it was 17 and 29. The intervention cost $18.78 (mailed) or $16.68 (phone) per adolescent per year to deliver. The cost per additional adolescent fully vaccinated was $463.99 for mailed and $714.98 for telephone; the cost per additional adolescent receiving a preventive visit was $324.75 and $487.03.

CONCLUSIONS

Managed care-based mail or telephone reminder/recall improved adolescent immunizations and preventive visits, with modest costs and modest impact.

Applying the net-benefit framework for assessing cost-effectiveness of interventions towards universal health coverage

In assessing the cost-effectiveness of an intervention, the interpretation and handling of uncertainties of the traditional summary measure, the Incremental Cost Effectiveness Ratio (ICER), can be problematic. This is particularly the case with strategies towards universal health coverage in which the decision makers are typically concerned with coverage and equity issues. We explored the feasibility and relative advantages of the net-benefit framework (NBF) (compared to the more traditional Incremental Cost-Effectiveness Ratio, ICER) in presenting results of cost-effectiveness analysis of a community based health insurance (CBHI) scheme in Nouna, a rural district of Burkina Faso. Data were collected from April to December 2007 from Nouna's longitudinal Demographic Surveillance System on utilization of health services, membership of the CBHI, covariates, and CBHI costs. The incremental cost of a 1 increase in utilization of health services by household members of the CBHI was 433,000 XOF ($1000 approximately). The incremental cost varies significantly by covariates. The probability of the CBHI achieving a 1% increase in utilization of health services, when the ceiling ratio is $1,000, is barely 30% for households in Nouna villages compared to 90% for households in Nouna town. Compared to the ICER, the NBF provides more useful information for policy making.

Using parametric multipoint lods and mods for linkage analysis requires a shift in statistical thinking

Multipoint (MP) linkage analysis represents a valuable tool for whole-genome studies but suffers from the disadvantage that its probability distribution is unknown and varies as a function of marker information and density, genetic model, number and structure of pedigrees, and the affection status distribution [Xing and Elston: Genet Epidemiol 2006;30:447-458; Hodge et al.: Genet Epidemiol 2008;32:800-815]. This implies that the MP significance criterion can differ for each marker and each dataset, and this fact makes planning and evaluation of MP linkage studies difficult. One way to circumvent this difficulty is to use simulations or permutation testing. Another approach is to use an alternative statistical paradigm to assess the statistical evidence for linkage, one that does not require computation of a p value. Here we show how to use the evidential statistical paradigm for planning, conducting, and interpreting MP linkage studies when the disease model is known (lod analysis) or unknown (mod analysis). As a key feature, the evidential paradigm decouples uncertainty (i.e. error probabilities) from statistical evidence. In the planning stage, the user calculates error probabilities, as functions of one's design choices (sample size, choice of alternative hypothesis, choice of likelihood ratio (LR) criterion k) in order to ensure a reliable study design. In the data analysis stage one no longer pays attention to those error probabilities. In this stage, one calculates the LR for two simple hypotheses (i.e. trait locus is unlinked vs. trait locus is located at a particular position) as a function of the parameter of interest (position). The LR directly measures the strength of evidence for linkage in a given data set and remains completely divorced from the error probabilities calculated in the planning stage. An important consequence of this procedure is that one can use the same criterion k for all analyses. This contrasts with the situation described above, in which the value one uses to conclude significance may differ for each marker and each dataset in order to accommodate a fixed test size, α. In this study we accomplish two goals that lead to a general algorithm for conducting evidential MP linkage studies. (1) We provide two theoretical results that translate into guidelines for investigators conducting evidential MP linkage: (a) Comparing mods to lods, error rates (including probabilities of weak evidence) are generally higher for mods when the null hypothesis is true, but lower for mods in the presence of true linkage. Royall [J Am Stat Assoc 2000;95:760-780] has shown that errors based on lods are bounded and generally small. Therefore when the true disease model is unknown and one chooses to use mods, one needs to control misleading evidence rates only under the null hypothesis; (b) for any given pair of contiguous marker loci, error rates under the null are greatest at the midpoint between the markers spaced furthest apart, which provides an obvious simple alternative hypothesis to specify for planning MP linkage studies. (2) We demonstrate through extensive simulation that this evidential approach can yield low error rates under the null and alternative hypotheses for both lods and mods, despite the fact that mod scores are not true LRs. Using these results we provide a coherent approach to implement a MP linkage study using the evidential paradigm.

Methodological concerns with economic evaluations of meningococcal vaccines

The evolution of meningococcal vaccines illustrates nicely the incremental technological process that is common to much medical innovation and particularly appropriate to economic analysis. However, the economic evaluation of vaccines is complicated by several features unique to vaccines, including the possibility of indirect (herd immunity) benefits and the shear breadth of vaccination strategies available to decision makers. As with the vaccines themselves, the modelling approaches applied to the economic evaluation of meningococcal vaccines have evolved to become increasingly complex. Despite such innovation, concerns remain about the quality of economic studies of meningococcal vaccines. This article reviews evaluations of meningococcal vaccine strategies in developed countries and provides recommendations for future work in this area. Four potentially problematic areas in the existing literature are identified - indirect costs, herd immunity, quality of life and comparison programmes - and approaches to these issues are discussed. Recommendations for reporting a 'minimum analysis profile' case are also made. Although comparisons across studies are difficult at best, recent work shows that incorporating the indirect benefits and costs of vaccination substantially improves cost effectiveness.