Optimal global value of information trials: better aligning manufacturer and decision maker interests and enabling feasible risk sharing

Risk sharing arrangements relate to adjusting payments for new health technologies given evidence of their performance over time. Such arrangements rely on prospective information regarding the incremental net benefit of the new technology, and its use in practice. However, once the new technology has been adopted in a particular jurisdiction, randomized clinical trials within that jurisdiction are likely to be infeasible and unethical in the cases where they would be most helpful, i.e. with current evidence of positive while uncertain incremental health and net monetary benefit. Informed patients in these cases would likely be reluctant to participate in a trial, preferring instead to receive the new technology with certainty. Consequently, informing risk sharing arrangements within a jurisdiction is problematic given the infeasibility of collecting prospective trial data. To overcome such problems, we demonstrate that global trials facilitate trialling post adoption, leading to more complete and robust risk sharing arrangements that mitigate the impact of costs of reversal on expected value of information in jurisdictions who adopt while a global trial is undertaken. More generally, optimally designed global trials offer distinct advantages over locally optimal solutions for decision makers and manufacturers alike: avoiding opportunity costs of delay in jurisdictions that adopt; overcoming barriers to evidence collection; and improving levels of expected implementation. Further, the greater strength and translatability of evidence across jurisdictions inherent in optimal global trial design reduces barriers to translation across jurisdictions characteristic of local trials. Consequently, efficiently designed global trials better align the interests of decision makers and manufacturers, increasing the feasibility of risk sharing and the expected strength of evidence over local trials, up until the point that current evidence is globally sufficient.

A flexible nonparametric approach to find candidate genes associated with disease in microarray experiments

Very often biologists are interested to know the biological function of a particular gene. Its true biological function may depend on other genes. Finding other genes in the same biological pathway of that gene may enhance further understanding of its biological function. Therefore, we are interested in finding other candidate genes whose expression values are highly correlated with that of a "seed" gene. The "seed" gene, which is known and associated with a disease, is used as a reference to extract candidate genes from microarray experiments and enriched pathways. We propose a nonparametric procedure for selecting the candidate genes. The expression levels for these candidate genes are correlated with that of a "seed" gene in microarray experiments. The proposed test statistic compares two Area Under Receiver Operating Characteristic Curves (AUC) for gene pairs, taking implicit correlation between two AUCs into account. The performance of our method is compared to the other well-known methods through the use of simulation and real data analysis.

Determining optimal sample sizes for multistage adaptive randomized clinical trials from an industry perspective using value of information methods

Background

 Most often, sample size determinations for randomized clinical trials are based on frequentist approaches that depend on somewhat arbitrarily chosen factors, such as type I and II error probabilities and the smallest clinically important difference. As an alternative, many authors have proposed decision-theoretic (full Bayesian) approaches, often referred to as value of information methods that attempt to determine the sample size that maximizes the difference between the trial’s expected utility and its expected cost, referred to as the expected net gain. Taking an industry perspective, Willan proposes a solution in which the trial’s utility is the increase in expected profit. Furthermore, Willan and Kowgier, taking a societal perspective, show that multistage designs can increase expected net gain.

Purpose

 The purpose of this article is to determine the optimal sample size using value of information methods for industry-based, multistage adaptive randomized clinical trials, and to demonstrate the increase in expected net gain realized. At the end of each stage, the trial’s sponsor must decide between three actions: continue to the next stage, stop the trial and seek regulatory approval, or stop the trial and abandon the drug.

Methods

 A model for expected total profit is proposed that includes consideration of per-patient profit, disease incidence, time horizon, trial duration, market share, and the relationship between trial results and probability of regulatory approval. The proposed method is extended to include multistage designs with a solution provided for a two-stage design. An example is given.

Results

 Significant increases in the expected net gain are realized by using multistage designs.

Limitations

 The complexity of the solutions increases with the number of stages, although far simpler near-optimal solutions exist. The method relies on the central limit theorem, assuming that the sample size is sufficiently large so that the relevant statistics are normally distributed.

Conclusion

 From a value of information perspective, the use of multistage designs in industry trials leads to significant gains in the expected net gain.

Accounting for between-study variation in incremental net benefit in value of information methodology

Previous applications of value of information methods for determining optimal sample size in randomized clinical trials have assumed no between-study variation in mean incremental net benefit. By adopting a hierarchical model, we provide a solution for determining optimal sample size with this assumption relaxed. The solution is illustrated with two examples from the literature. Expected net gain increases with increasing between-study variation, reflecting the increased uncertainty in incremental net benefit and reduced extent to which data are borrowed from previous evidence. Hence, a trial can become optimal where current evidence is sufficient assuming no between-study variation. However, despite the expected net gain increasing, the optimal sample size in the illustrated examples is relatively insensitive to the amount of between-study variation. Further percentage losses in expected net gain were small even when choosing sample sizes that reflected widely different between-study variation.

Bias-corrected estimator for intraclass correlation coefficient in the balanced one-way random effects model

Background

Intraclass correlation coefficients (ICCs) are used in a wide range of applications. However, most commonly used estimators for the ICC are known to be subject to bias.

Methods

Using second order Taylor series expansion, we propose a new bias-corrected estimator for one type of intraclass correlation coefficient, for the ICC that arises in the context of the balanced one-way random effects model. A simulation study is performed to assess the performance of the proposed estimator. Data have been generated under normal as well as non-normal scenarios.

Results

Our simulation results show that the new estimator has reduced bias compared to the least square estimator which is often referred to as the conventional or analytical estimator. The results also show marked bias reduction both in normal and non-normal data scenarios. In particular, our estimator outperforms the analytical estimator in a non-normal setting producing estimates that are very close to the true ICC values.

Conclusions

The proposed bias-corrected estimator for the ICC from a one-way random effects analysis of variance model appears to perform well in the scenarios we considered in this paper and can be used as a motivation to construct bias-corrected estimators for other types of ICCs that arise in more complex scenarios. It would also be interesting to investigate the bias-variance trade-off.

Value of information and pricing new healthcare interventions

Previous application of value-of-information methods to optimal clinical trial design have predominantly taken a societal decision-making perspective, implicitly assuming that healthcare costs are covered through public expenditure and trial research is funded by government or donation-based philanthropic agencies. In this paper, we consider the interaction between interrelated perspectives of a societal decision maker (e.g. the National Institute for Health and Clinical Excellence [NICE] in the UK) charged with the responsibility for approving new health interventions for reimbursement and the company that holds the patent for a new intervention. We establish optimal decision making from societal and company perspectives, allowing for trade-offs between the value and cost of research and the price of the new intervention. Given the current level of evidence, there exists a maximum (threshold) price acceptable to the decision maker. Submission for approval with prices above this threshold will be refused. Given the current level of evidence and the decision maker's threshold price, there exists a minimum (threshold) price acceptable to the company. If the decision maker's threshold price exceeds the company's, then current evidence is sufficient since any price between the thresholds is acceptable to both. On the other hand, if the decision maker's threshold price is lower than the company's, then no price is acceptable to both and the company's optimal strategy is to commission additional research. The methods are illustrated using a recent example from the literature.

Value of information methods for planning and analyzing clinical studies optimize decision making and research planning

OBJECTIVE

The results of two randomized clinical trials (RCTs) demonstrate the clinical effectiveness of alternatives to casting for certain ankle and wrist fractures. We illustrate the use of value of information (VOI) methods for evaluating the evidence provided by these studies with respect to decision making.

STUDY DESIGN AND SETTING

Using cost-effectiveness data from these studies, the expected value of sample information (EVSI) of a future RCT can be determined. If the EVSI exceeds the cost of the future trial for any sample size, then the current evidence is considered insufficient for decision making and a future trial is considered worthwhile. If, on the other hand, there is no sample size for which the EVSI exceeds the cost, then the evidence is considered sufficient, and no future trial is required.

RESULTS

We found that the evidence from the ankle study was insufficient to support the adoption of the removable device and determined the optimal sample size for a future trial. Conversely, the evidence from the wrist study was sufficient to support the adoption of the removable device.

CONCLUSIONS

VOI methods provide a decision-analytic alternative to the standard hypothesis testing approach for assessing the evidence provided by cost-effectiveness studies and for determining sample sizes for RCTs.

Haemophilia prophylaxis: how can we justify the costs?

Severe haemophilia results in increased mortality and poorer quality of life. Factor prophylaxis leads to a more normal life, but is very costly; most of the cost is due to the high cost of replacement factor. Despite its high cost, factor prophylaxis has been adopted throughout the developed world--even in different health care systems. We argue that there are at least five possible reasons why societies may value factor prophylaxis despite its cost: (i) it is directed towards an inherited disease, (ii) the treatment is largely directed towards children, (iii) the disease is rare and the overall cost to society is small, (iv) the treatment is preventative, and v) the high cost is largely the result of providing safe products. In an era of rising health care costs, there is a strong research agenda to establish the factors that determine the value of expensive therapies for rare diseases like haemophilia.

Rapid versus standard intravenous rehydration in paediatric gastroenteritis: pragmatic blinded randomised clinical trial

OBJECTIVE

To determine if rapid rather than standard intravenous rehydration results in improved hydration and clinical outcomes when administered to children with gastroenteritis.

DESIGN

Single centre, two arm, parallel randomised pragmatic controlled trial. Blocked randomisation stratified by site. Participants, caregivers, outcome assessors, investigators, and statisticians were blinded to the treatment assignment.

SETTING

Paediatric emergency department in a tertiary care centre in Toronto, Canada.

PARTICIPANTS

226 children aged 3 months to 11 years; complete follow-up was obtained on 223 (99%). Eligible children were aged over 90 days, had a diagnosis of dehydration secondary to gastroenteritis, had not responded to oral rehydration, and had been prescribed intravenous rehydration. Children were excluded if they weighed less than 5 kg or more than 33 kg, required fluid restriction, had a suspected surgical condition, or had an insurmountable language barrier. Children were also excluded if they had a history of a chronic systemic disease, abdominal surgery, bilious or bloody vomit, hypotension, or hypoglycaemia or hyperglycaemia.

INTERVENTIONS

Rapid (60 mL/kg) or standard (20 mL/kg) rehydration with 0.9% saline over an hour; subsequent fluids administered according to protocol.

MAIN OUTCOME MEASURES

PRIMARY OUTCOME

clinical rehydration, assessed with a validated scale, two hours after the start of treatment.

SECONDARY OUTCOMES

prolonged treatment, mean clinical dehydration scores over the four hour study period, time to discharge, repeat visits to emergency department, adequate oral intake, and physician's comfort with discharge. Data from all randomised patients were included in an intention to treat analysis.

RESULTS

114 patients were randomised to rapid rehydration and 112 to standard. One child was withdrawn because of severe hyponatraemia at baseline. There was no evidence of a difference between the rapid and standard rehydration groups in the proportions of participants who were rehydrated at two hours (41/114 (36%) v 33/112 (30%); difference 6.5% (95% confidence interval -5.7% to 18.7%; P=0.32). The results did not change after adjustment for weight, baseline dehydration score, and baseline pH (odds ratio 1.8, 0.90 to 3.5; P=0.10). The rates of prolonged treatment were similar (52% rapid v 43% standard; difference 8.9%, 21% to -5%; P=0.19). Although dehydration scores were similar throughout the study period (P=0.96), the median time to discharge was longer in the rapid group (6.3 v 5.0 hours; P=0.03).

CONCLUSIONS

There are no relevant clinical benefits from the administration of rapid rather than standard intravenous rehydration to haemodynamically stable children deemed to require intravenous rehydration. Trail registration Clinical Trials NCT00392145.

Sample size determination for cost-effectiveness trials

Methods for determining sample size requirements for cost-effectiveness studies are reviewed and illustrated. Traditional methods based on tests of hypothesis and power arguments are given for the incremental cost-effectiveness ratio and incremental net benefit (INB). In addition, a full Bayesian approach using decision theory to determine optimal sample size is given for INB. The full Bayesian approach, based on the value of information, is proposed in reaction to concerns that traditional methods rely on arbitrarily chosen error probabilities and an ill-defined notion of the smallest clinically important difference. Furthermore, the results of cost-effectiveness studies are used for decision making (e.g. should a new intervention be adopted or the old one retained), and employing decision theory, which permits optimal use of current information and the optimal design of new studies, provides a more consistent approach.

Maternal side-effects after multiple courses of antenatal corticosteroids (MACS): the three-month follow-up of women in the randomized controlled trial of MACS for preterm birth study

OBJECTIVE

A single course of antenatal corticosteroids (ACS) is associated with a reduction in respiratory distress syndrome and neonatal death. Multiple Courses of Antenatal Corticosteroids Study (MACS), a study involving 1858 women, was a multicentre randomized placebo-controlled trial of multiple courses of ACS, given every 14 days until 33+6 weeks or birth, whichever came first. The primary outcome of the study, a composite of neonatal mortality and morbidity, was similar for the multiple ACS and placebo groups (12.9% vs. 12.5%), but infants exposed to multiple courses of ACS weighed less, were shorter, and had smaller head circumferences. Thus for women who remain at increased risk of preterm birth, multiple courses of ACS (every 14 days) are not recommended. Chronic use of corticosteroids is associated with numerous side effects including weight gain and depression. The aim of this postpartum assessment was to ascertain if multiple courses of ACS were associated with maternal side effects.

METHODS

Three months postpartum, women who participated in MACS were asked to complete a structured questionnaire that asked about maternal side effects of corticosteroid use during MACS and included the Edinburgh Postnatal Depression Scale. Women were also asked to evaluate their study participation.

RESULTS

Of the 1858 women randomized, 1712 (92.1%) completed the postpartum questionnaire. There were no significant differences in the risk of maternal side effects between the two groups. Large numbers of women met the criteria for postpartum depression (14.1% in the ACS vs. 16.0% in the placebo group). Most women (94.1%) responded that they would participate in the trial again.

CONCLUSION

In pregnancy, corticosteroids are given to women for fetal lung maturation and for the treatment of various maternal diseases. In this international multicentre randomized controlled trial, multiple courses of ACS (every 14 days) were not associated with maternal side effects, and the majority of women responded that they would participate in such a study again.

Presenting evidence and summary measures to best inform societal decisions when comparing multiple strategies

Multiple strategy comparisons in health technology assessment (HTA) are becoming increasingly important, with multiple alternative therapeutic actions, combinations of therapies and diagnostic and genetic testing alternatives. Comparison under uncertainty of incremental cost, effects and cost effectiveness across more than two strategies is conceptually and practically very different from that for two strategies, where all evidence can be summarized in a single bivariate distribution on the incremental cost-effectiveness plane. Alternative methods for comparing multiple strategies in HTA have been developed in (i) presenting cost and effects on the cost-disutility plane and (ii) summarizing evidence with multiple strategy cost-effectiveness acceptability (CEA) and expected net loss (ENL) curves and frontiers. However, critical questions remain for the analyst and decision maker of how these techniques can be best employed across multiple strategies to (i) inform clinical and cost inference in presenting evidence, and (ii) summarize evidence of cost effectiveness to inform societal reimbursement decisions where preferences may be risk neutral or somewhat risk averse under the Arrow-Lind theorem. We critically consider how evidence across multiple strategies can be best presented and summarized to inform inference and societal reimbursement decisions, given currently available methods. In the process, we make a number of important original findings. First, in presenting evidence for multiple strategies, the joint distribution of costs and effects on the cost-disutility plane with associated flexible comparators varying across replicates for cost and effect axes ensure full cost and effect inference. Such inference is usually confounded on the cost-effectiveness plane with comparison relative to a fixed origin and axes. Second, in summarizing evidence for risk-neutral societal decision making, ENL curves and frontiers are shown to have advantages over the CEA frontier in directly presenting differences in expected net benefit (ENB). The CEA frontier, while identifying strategies that maximize ENB, only presents their probability of maximizing net benefit (NB) and, hence, fails to explain why strategies maximize ENB at any given threshold value. Third, in summarizing evidence for somewhat risk-averse societal decision making, trade-offs between the strategy maximizing ENB and other potentially optimal strategies with higher probability of maximizing NB should be presented over discrete threshold values where they arise. However, the probabilities informing these trade-offs and associated discrete threshold value regions should be derived from bilateral CEA curves to prevent confounding by other strategies inherent in multiple strategy CEA curves. Based on these findings, a series of recommendations are made for best presenting and summarizing cost-effectiveness evidence for reimbursement decisions when comparing multiple strategies, which are contrasted with advice for comparing two strategies. Implications for joint research and reimbursement decisions are also discussed.

The Early External Cephalic Version (ECV) 2 Trial: an international multicentre randomised controlled trial of timing of ECV for breech pregnancies

Objective

To investigate whether initiating external cephalic version (ECV) earlier in pregnancy might increase the rate of successful ECV procedures, and be more effective in decreasing the rate of non-cephalic presentation at birth and of caesarean section.

Design

An unblinded multicentred randomised controlled trial.

Setting

A total of 1543 women were randomised from 68 centres in 21 countries.

Population

Women with a singleton breech fetus at a gestational age of 33^0/7 weeks (231 days) to 35^6/7 weeks (251 days) of gestation were included.

Methods

Participants were randomly assigned to having a first ECV procedure between the gestational ages of 34^0/7 (238 days) and 35^6/7 weeks of gestation (early ECV group) or at or after 37^0/7 (259 days) weeks of gestation (delayed ECV group).

Main outcome measures

The primary outcome was the rate of caesarean section; the secondary outcome was the rate of preterm birth.

Results

Fewer fetuses were in a non-cephalic presentation at birth in the early ECV group (314/765 [41.1%] versus 377/768 [49.1%] in the delayed ECV group; relative risk [RR] 0.84, 95% CI 0.75, 0.94, P = 0.002). There were no differences in rates of caesarean section (398/765 [52.0%] versus 430/768 [56.0%]; RR 0.93, 95% CI 0.85, 1.02, P = 0.12) or in risk of preterm birth (50/765 [6.5%] versus 34/768 [4.4%]; RR 1.48, 95% CI 0.97, 2.26, P = 0.07) between groups.

Conclusion

External cephalic version at 34–35 weeks versus 37 or more weeks of gestation increases the likelihood of cephalic presentation at birth but does not reduce the rate of caesarean section and may increase the rate of preterm birth.

Consistently estimating absolute risk difference when translating evidence to jurisdictions of interest

Economic analysis and assessment of net clinical benefit often requires estimation of absolute risk difference (ARD) for binary outcomes (e.g. survival, response, disease progression) given baseline epidemiological risk in a jurisdiction of interest and trial evidence of treatment effects. Typically, the assumption is made that relative treatment effects are constant across baseline risk, in which case relative risk (RR) or odds ratios (OR) could be applied to estimate ARD. The objective of this article is to establish whether such use of RR or OR allows consistent estimates of ARD. ARD is calculated from alternative framing of effects (e.g. mortality vs survival) applying standard methods for translating evidence with RR and OR. For RR, the RR is applied to baseline risk in the jurisdiction to estimate treatment risk; for OR, the baseline risk is converted to odds, the OR applied and the resulting treatment odds converted back to risk. ARD is shown to be consistently estimated with OR but changes with framing of effects using RR wherever there is a treatment effect and epidemiological risk differs from trial risk. Additionally, in indirect comparisons, ARD is shown to be consistently estimated with OR, while calculation with RR allows inconsistency, with alternative framing of effects in the direction, let alone the extent, of ARD. OR ensures consistent calculation of ARD in translating evidence from trial settings and across trials in direct and indirect comparisons, avoiding inconsistencies from RR with alternative outcome framing and associated biases. These findings are critical for consistently translating evidence to inform economic analysis and assessment of net clinical benefit, as translation of evidence is proposed precisely where the advantages of OR over RR arise.

Clinical inertia in patients with T2DM requiring insulin in family practice

OBJECTIVE

To describe the clinical status of patients with type 2 diabetes mellitus (T2DM) in the primary care setting at insulin initiation and during follow-up, and to assess the efficacy of insulin initiation and intensification.

DESIGN

Ontario FPs from the IMS Health database who had prescribed insulin at least once in the 12 months before November 2006 were randomly selected to receive an invitation to participate. Eligible and consenting FPs completed a questionnaire for each of up to 10 consecutive eligible patients. Patient data were recorded from 3 time points.

SETTING

Family practices in Ontario, Canada.

PARTICIPANTS

One hundred and nine FPs and 379 of their T2DM patients taking insulin (with or without oral agents).

MAIN OUTCOME MEASURES

Glycated hemoglobin (HbA₁(c)) levels, daily insulin dose, and use of concomitant oral agents at insulin initiation and 2 subsequent visits.

RESULTS

Data from each patient were obtained on insulin initiation and intensification, glycemic control, further pharmacologic therapy, and related complications. Mean time from diagnosis of T2DM to insulin initiation was 9.2 years. Mean HbA₁(c) values were 9.5% before insulin initiation, 8.1% at visit 2 (median 1.2 years later), and 7.9% at visit 3 (median 3.9 years after initiation). Mean insulin dose was 24 units at initiation, 48 units at visit 2, and 65 units at visit 3. At visit 3, 20% of patients continued to have very poor glycemic control (HbA₁(c) > 9.0%). With the exception of a decrease in sulfonylurea use, concomitant use of oral antihyperglycemic agents remained static over time.

CONCLUSION

Even in patients identified as being sufficiently high risk to warrant insulin therapy, a clinical care gap exists in physician efforts to achieve and sustain recommended HbA₁(c) target levels. Family physicians need strategies to facilitate earlier initiation and ongoing intensification of insulin therapy.

Multiple courses of antenatal corticosteroids for preterm birth study: 2-year outcomes

OBJECTIVE

The aim of this study was to determine the effects of repeated courses of prenatal corticosteroid therapy versus placebo on death or neurologic impairment among the children enrolled in the Multiple Courses of Antenatal Corticosteroids for Preterm Birth Study, at 18 to 24 months of age.

METHODS

A total of 2305 infants were eligible for follow-up evaluation; 2104 infants (1069 in the prenatal corticosteroid therapy group and 1035 in the placebo group) were monitored. The primary outcome was death or neurologic impairment, defined as either cerebral palsy or cognitive delay, at 18 to 24 months of age. The secondary outcomes were measurements of growth (height, weight, and head circumference).

RESULTS

Children exposed to multiple courses of prenatal corticosteroid therapy had similar rates of death or neurologic impairment, compared with children exposed to placebo (148 children [13.8%] vs 142 children [13.7%]; odds ratio: 1.001[95% confidence interval: 0.75-1.30]; P = .95). They had a mean weight of 11.94 kg, compared with 12.14 kg in the placebo group (P = .04), a mean height of 85.51 cm, compared with 85.46 cm (P = .87), and a mean head circumference of 48.18 cm, compared with 48.25 cm (P = .45).

CONCLUSIONS

Multiple courses of prenatal corticosteroid therapy, given every 14 days, did not increase or decrease the risk of death or neurologic impairment at 18 to 24 months of age, compared with a single course of prenatal corticosteroid therapy. Continued follow-up monitoring of these children is necessary to assess neurobehavioral function, school performance, and possible susceptibility to disease.

Survival distributions impact the power of randomized placebo-phase design and parallel groups randomized clinical trials

OBJECTIVES

The study evaluated the power of the randomized placebo-phase design (RPPD)-a new design of randomized clinical trials (RCTs), compared with the traditional parallel groups design, assuming various response time distributions. In the RPPD, at some point, all subjects receive the experimental therapy, and the exposure to placebo is for only a short fixed period of time.

STUDY DESIGN AND SETTING

For the study, an object-oriented simulation program was written in R. The power of the simulated trials was evaluated using six scenarios, where the treatment response times followed the exponential, Weibull, or lognormal distributions. The median response time was assumed to be 355 days for the placebo and 42 days for the experimental drug.

RESULTS

Based on the simulation results, the sample size requirements to achieve the same level of power were different under different response time to treatment distributions. The scenario where the response times followed the exponential distribution had the highest sample size requirement. In most scenarios, the parallel groups RCT had higher power compared with the RPPD.

CONCLUSION

The sample size requirement varies depending on the underlying hazard distribution. The RPPD requires more subjects to achieve a similar power to the parallel groups design.

Using the incremental net benefit framework for quantitative benefit-risk analysis in regulatory decision-making--a case study of alosetron in irritable bowel syndrome

OBJECTIVE

There is consensus that a more transparent, explicit, and rigorous approach to benefit-risk evaluation is required. The objective of this study is to evaluate the incremental net benefit (INB) framework for undertaking quantitative benefit-risk assessment by performing a quantitative benefit-risk analysis of alosetron for the treatment of irritable bowel syndrome from the patients' perspective.

METHODS

A discrete event simulation model was developed to determine the INB of alosetron relative to placebo, calculated as "relative value-adjusted life-years (RVALYs)."

RESULTS

In the base case analysis, alosetron resulted in a mean INB of 34.1 RVALYs per 1000 patients treated relative to placebo over 52 weeks of treatment. Incorporating parameter uncertainty into the model, probabilistic sensitivity analysis revealed a mean INB of 30.4 (95% confidence interval 15.9-45.4) RVALYs per 1000 patients treated relative to placebo over 52 weeks of treatment. Overall, there was >99% chance that both the incremental benefit and incremental risk associated with alosetron are greater than placebo. As hypothesized, the INB of alosetron was greatest in patients with the worst quality of life experienced at baseline. The mean INB associated with alosetron in patients with mild, moderate, and severe symptoms at baseline was 17.97 (-0.55 to 36.23), 29.98 (17.05-43.37), and 35.98 (23.49-48.77) RVALYs per 1000 patients treated, respectively.

CONCLUSIONS

This study demonstrates the potential utility of applying the INB framework to real-life decision-making, and the ability to use simulation modeling incorporating outcomes data from different sources as a benefit-risk decision aid.

Optimal clinical trial design using value of information methods with imperfect implementation

Traditional sample size calculations for randomized clinical trials are based on the tests of hypotheses and depend on somewhat arbitrarily chosen factors, such as type I and II errors rates and the smallest clinically important difference. In response to this, many authors have proposed the use of methods based on the value of information as an alternative. Previous attempts have assumed perfect implementation, i.e. if current evidence favors the new intervention and no new information is sought or expected, all future patients will receive it. A framework is proposed to allow for this assumption to be relaxed. The profound effect that this can have on the optimal sample size and expected net gain is illustrated on two recent examples. In addition, a model for assessing the value of implementation strategies is proposed and illustrated.

The value of value of information: best informing research design and prioritization using current methods

Value of information (VOI) methods have been proposed as a systematic approach to inform optimal research design and prioritization. Four related questions arise that VOI methods could address. (i) Is further research for a health technology assessment (HTA) potentially worthwhile? (ii) Is the cost of a given research design less than its expected value? (iii) What is the optimal research design for an HTA? (iv) How can research funding be best prioritized across alternative HTAs? Following Occam's razor, we consider the usefulness of VOI methods in informing questions 1-4 relative to their simplicity of use. Expected value of perfect information (EVPI) with current information, while simple to calculate, is shown to provide neither a necessary nor a sufficient condition to address question 1, given that what EVPI needs to exceed varies with the cost of research design, which can vary from very large down to negligible. Hence, for any given HTA, EVPI does not discriminate, as it can be large and further research not worthwhile or small and further research worthwhile. In contrast, each of questions 1-4 are shown to be fully addressed (necessary and sufficient) where VOI methods are applied to maximize expected value of sample information (EVSI) minus expected costs across designs. In comparing complexity in use of VOI methods, applying the central limit theorem (CLT) simplifies analysis to enable easy estimation of EVSI and optimal overall research design, and has been shown to outperform bootstrapping, particularly with small samples. Consequently, VOI methods applying the CLT to inform optimal overall research design satisfy Occam's razor in both improving decision making and reducing complexity. Furthermore, they enable consideration of relevant decision contexts, including option value and opportunity cost of delay, time, imperfect implementation and optimal design across jurisdictions. More complex VOI methods such as bootstrapping of the expected value of partial EVPI may have potential value in refining overall research design. However, Occam's razor must be seriously considered in application of these VOI methods, given their increased complexity and current limitations in informing decision making, with restriction to EVPI rather than EVSI and not allowing for important decision-making contexts. Initial use of CLT methods to focus these more complex partial VOI methods towards where they may be useful in refining optimal overall trial design is suggested. Integrating CLT methods with such partial VOI methods to allow estimation of partial EVSI is suggested in future research to add value to the current VOI toolkit.

Can montelukast shorten prednisolone therapy in children with mild to moderate acute asthma? A randomized controlled trial

OBJECTIVE

To examine whether outpatient post-stabilization therapy with montelukast produces more treatment failures than prednisolone.

STUDY DESIGN

In this randomized, double-blind, double-dummy non-inferiority trial, 130 children 2 to 17 years of age with mild to moderate acute asthma stabilized with prednisolone in the emergency department received 5 daily treatments with either prednisolone or montelukast after discharge. The primary outcome was treatment failure within 8 days (ie, an asthma-related unscheduled visit, hospitalization, or additional systemic corticosteroids).

RESULTS

The rates of treatment failure were 7.9% in the prednisolone group and 22.4% in the montelukast group (95% CI, 26.5%-2.4%). Treatment was more likely to fail in younger patients (odds ratio, 4.9). In the montelukast group, more patients received additional pharmacotherapy than in patients receiving prednisolone (23.9% versus 9.5%, P = .03). The differences in the daily salbutamol treatments, asymptomatic days, and changes in the Pediatric Respiratory Assessment Measure score were not significant (P = .85, .75, and .26, respectively).

CONCLUSION

Montelukast does not represent an adequate alternative to corticosteroids after outpatient stabilization in mild to moderate acute asthma. This population should receive oral corticosteroids after discharge.

Optimal sample size determinations from an industry perspective based on the expected value of information

BACKGROUND

Traditional sample size calculations for randomized clinical trials depend on somewhat arbitrarily chosen factors, such as type I and II errors. As an alternative, taking a societal perspective, and using the expected value of information based on Bayesian decision theory, a number of authors have recently shown how to determine the sample size that maximizes the expected net gain, i.e., the difference between the cost of the trial and the value of the information gained from the results. Other authors have proposed Bayesian methods to determine sample sizes from an industry perspective.

PURPOSE

The purpose of this article is to propose a Bayesian approach to sample size calculations from an industry perspective that attempts to determine the sample size that maximizes expected profit.

METHODS

A model is proposed for expected total profit that includes consideration of per-patient profit, disease incidence, time horizon, trial duration, market share, discount rate, and the relationship between the results and the probability of regulatory approval. The expected value of information provided by trial data is related to the increase in expected profit from increasing the probability of regulatory approval. The methods are applied to an example, including an examination of robustness. The model is extended to consider market share as a function of observed treatment effect.

RESULTS

The use of methods based on the expected value of information can provide, from an industry perspective, robust sample size solutions that maximize the difference between the expected cost of the trial and the expected value of information gained from the results.

LIMITATIONS

The method is only as good as the model for expected total profit. Although the model probably has all the right elements, it assumes that market share, per-patient profit, and incidence are insensitive to trial results. The method relies on the central limit theorem which assumes that the sample sizes involved ensure that the relevant test statistics are normally distributed.

CONCLUSIONS

Industry officials should consider the use of expected value of information methods for determining sample sizes for proposed trials. The method also can be used to select, from a number of proposed trials, those which maximize return from a fixed research budget.

Globally optimal trial design for local decision making

Value of information methods allows decision makers to identify efficient trial design following a principle of maximizing the expected value to decision makers of information from potential trial designs relative to their expected cost. However, in health technology assessment (HTA) the restrictive assumption has been made that, prospectively, there is only expected value of sample information from research commissioned within jurisdiction. This paper extends the framework for optimal trial design and decision making within jurisdiction to allow for optimal trial design across jurisdictions. This is illustrated in identifying an optimal trial design for decision making across the US, the UK and Australia for early versus late external cephalic version for pregnant women presenting in the breech position. The expected net gain from locally optimal trial designs of US$0.72M is shown to increase to US$1.14M with a globally optimal trial design. In general, the proposed method of globally optimal trial design improves on optimal trial design within jurisdictions by: (i) reflecting the global value of non-rival information; (ii) allowing optimal allocation of trial sample across jurisdictions; (iii) avoiding market failure associated with free-rider effects, sub-optimal spreading of fixed costs and heterogeneity of trial information with multiple trials.

Multiple courses of antenatal corticosteroids for preterm birth (MACS): a randomised controlled trial

BACKGROUND

One course of antenatal corticosteroids reduces the risk of respiratory distress syndrome and neonatal death. Weekly doses given to women who remain undelivered after a single course may have benefits (less respiratory morbidity) or cause harm (reduced growth in utero). We aimed to find out whether multiple courses of antenatal corticosteroids would reduce neonatal morbidity and mortality without adversely affecting fetal growth.

METHODS

1858 women at 25-32 weeks' gestation who remained undelivered 14-21 days after an initial course of antenatal corticosteroids and continued to be at high risk of preterm birth were randomly assigned to multiple courses of antenatal corticosteroids (n=937) or placebo (n=921), every 14 days until week 33 or delivery, whichever came first. The primary outcome was a composite of perinatal or neonatal mortality, severe respiratory distress syndrome, intraventricular haemorrhage (grade III or IV), periventricular leucomalacia, bronchopulmonary dysplasia, or necrotising enterocolitis. Analysis was by intention to treat. All patients and caregivers were unaware of the treatment given. This trial is registered as number ISRCTN2654148.

FINDINGS

Infants exposed to multiple courses of antenatal corticosteroids had similar morbidity and mortality to those exposed to placebo (150 [12.9%] vs 143 [12.5%]). Those receiving multiple doses of corticosteroids also weighed less at birth than those exposed to placebo (2216 g vs 2330 g, p=0.0026), were shorter (44.5 cm vs 45.4 cm, p<0.001), and had a smaller head circumference (31.1 cm vs 31.7 cm, p<0.001).

INTERPRETATION

Multiple courses of antenatal corticosteroids, every 14 days, do not improve preterm-birth outcomes, and are associated with a decreased weight, length, and head circumference at birth. Therefore, this treatment schedule is not recommended.

FUNDING

Canadian Institutes of Health Research.