Value of Information: A Tool to Improve Research Prioritization and Reduce Waste

The Social Value Misconception in Clinical Research

Clinical researchers should help respect the autonomy and promote the well-being of prospective study participants by helping them make voluntary, informed decisions about enrollment. However, participants often exhibit poor understanding of important information about clinical research. Bioethicists have given special attention to "misconceptions" about clinical research that can compromise participants' decision-making, most notably the "therapeutic misconception." These misconceptions typically involve false beliefs about a study's purpose, or risks or potential benefits for participants. In this article, we describe a misconception involving false beliefs about a study's potential benefits for non-participants, or its expected social value. This social value misconception can compromise altruistically motivated participants' decision-making, potentially threatening their autonomy and well-being. We show how the social value misconception raises ethical concerns for inherently low-value research, hyped research, and even ordinary research, and advocate for empirical and normative work to help understand and counteract this misconception's potential negative impacts on participants.

Value of Information for Clinical Trial Design: The Importance of Considering All Relevant Comparators

Value of Information (VOI) analyses calculate the economic value that could be generated by obtaining further information to reduce uncertainty in a health economic decision model. VOI has been suggested as a tool for research prioritisation and trial design as it can highlight economically valuable avenues for future research. Recent methodological advances have made it increasingly feasible to use VOI in practice for research; however, there are critical differences between the VOI approach and the standard methods used to design research studies such as clinical trials. We aimed to highlight key differences between the research design approach based on VOI and standard clinical trial design methods, in particular the importance of considering the full decision context. We present two hypothetical examples to demonstrate that VOI methods are only accurate when (1) all feasible comparators are included in the decision model when designing research, and (2) all comparators are retained in the decision model once the data have been collected and a final treatment recommendation is made. Omitting comparators from either the design or analysis phase of research when using VOI methods can lead to incorrect trial designs and/or treatment recommendations. Overall, we conclude that incorrectly specifying the health economic model by ignoring potential comparators can lead to misleading VOI results and potentially waste scarce research resources.

What, Where, and How to Collect Real-World Data and Generate Real-World Evidence to Support Drug Reimbursement Decision-Making in Asia: A reflection Into the Past and A Way Forward

BACKGROUND

Globally, there is increasing interest in the use of real-world data (RWD) and real-world evidence (RWE) to inform health technology assessment (HTA) and reimbursement decision-making. Using current practices and case studies shared by eleven health systems in Asia, a non-binding guidance that seeks to align practices for generating and using RWD/RWE for decision-making in Asia was developed by the REAL World Data In ASia for HEalth Technology Assessment in Reimbursement (REALISE) Working Group, addressing a current gap and needs among HTA users and generators.

METHODS

The guidance document was developed over two face-to-face workshops, in addition to an online survey, a face-to-face interview and pragmatic search of literature. The specific focus was on what, where and how to collect RWD/ RWE.

RESULTS

All 11 REALISE member jurisdictions participated in the online survey and the first in-person workshop, 10 participated in the second in-person workshop, and 8 participated in the in-depth face-to-face interviews. The guidance document was iteratively reviewed by all working group members and the International Advisory Panel. There was substantial variation in: (a) sources and types of RWD being used in HTA, and (b) the relative importance and prioritization of RWE being used for policy-making. A list of national-level databases and other sources of RWD available in each country was compiled. A list of useful guidance on data collection, quality assurance and study design were also compiled.

CONCLUSION

The REALISE guidance document serves to align the collection of better quality RWD and generation of reliable RWE to ultimately inform HTA in Asia.

Cost-effectiveness of direct surgery versus preoperative octreotide therapy for growth-hormone secreting pituitary adenomas

PURPOSE

The objective of this study was to compare the cost-effectiveness of preoperative octreotide therapy followed by surgery versus the standard treatment modality for growth-hormone secreting pituitary adenomas, direct surgery (that is, surgery without preoperative treatment) from a public third-party payer perspective.

METHODS

We developed an individual-level state-transition microsimulation model to simulate costs and outcomes associated with preoperative octreotide therapy followed by surgery and direct surgery for patients with growth-hormone secreting pituitary adenomas. Transition probabilities, utilities, and costs were estimated from recent published data and discounted by 3% annually over a lifetime time horizon. Model outcomes included lifetime costs [2020 United States (US) Dollars], quality-adjusted life-years (QALYs) and incremental cost-effectiveness ratios (ICERs).

RESULTS

Under base case assumptions, direct surgery was found to be the dominant strategy as it yielded lower costs and greater health effects (QALYs) compared to preoperative octreotide strategy in the second-order Monte Carlo microsimulation. The ICER was most sensitive to probability of remission following primary therapy and duration of preoperative octreotide therapy. Accounting for joint parameter uncertainty, direct surgery had a higher probability of demonstrating a cost-effective profile compared to preoperative octreotide treatment at 77% compared to 23%, respectively.

CONCLUSIONS

Using standard benchmarks for cost-effectiveness in the US ($100,000/QALY), preoperative octreotide therapy followed by surgery may not be cost-effective compared to direct surgery for patients with growth-hormone secreting pituitary adenomas but the result is highly sensitive to initial treatment failure and duration of preoperative treatment.

Communication of personalised disease risk by general practitioners to motivate smoking cessation in England: a cost-effectiveness and research prioritisation study

BACKGROUND AND AIMS

Communication of personalised disease risk can motivate smoking cessation. We assessed whether routine implementation of this intervention by general practitioners (GPs) in England is cost-effective or whether we need further research to better establish its effectiveness.

DESIGN

Cost-effectiveness analysis (CEA) with value of information (VoI) analysis from the UK National Health Service perspective, using GP communication of personalised disease risk on smoking cessation versus usual care.

SETTING

GP practices in England.

STUDY POPULATION

Healthy smokers aged 35-60 years attending the GP practice.

MEASUREMENTS

Effectiveness of GP communication of personalised disease risk on smoking cessation was estimated through systematic review and meta-analysis. A Bayesian CEA was then performed using a lifetime Markov model on smokers aged 35-60 years that measured lifetime costs and quality-adjusted life-years (QALYs) assigned to the four diseases contributing the most to smoking-related morbidity, mortality and costs: chronic obstructive pulmonary disease, lung cancer, myocardial infarction and stroke. Costs and QALYs for each disease state were obtained from the literature. VoI analysis identified sources of uncertainty in the CEA and assessed how much would be worth investing in further research to reduce this uncertainty.

FINDINGS

The meta-analysis odds ratio for the effectiveness estimate of GP communication of personalised disease risk was 1.48 (95% credibility interval, 0.91-2.26), an absolute increase in smoking cessation rates of 3.84%. The probability of cost-effectiveness ranged 89-94% depending on sex and age. VoI analysis indicated that: (i) uncertainty in the effectiveness of the intervention was the driver of the overall uncertainty in the CEA; and (ii) a research investment to reduce this uncertainty is justified if lower than £27.6 million (£7 per smoker).

CONCLUSIONS

Evidence to date shows that, in England, incorporating disease risk communication into general practitioners' practices to motivate smoking cessation is likely to be cost-effective compared with usual care.

Prioritizing nurturing care at the municipal and district level with the Brazilian Early Childhood Friendly Municipal Index (IMAPI)

The Brazilian Early Childhood Friendly Municipal Index (IMAPI) is a population‐based approach to monitor the nurturing care environment for early childhood development (ECD) using routine information system data. It is unknown whether IMAPI can be applied to document metropolitan urban territorial differences in nurturing care environments. We used Brasilia, Brazil's capital with a large metropolitan population of 2,881,854 inhabitants divided into 31 districts, as a case study to examine whether disaggregation of nurturing care data can inform a more equitable prioritization for ECD in metropolitan areas. IMAPI scores were estimated at the municipal level (IMAPI‐M, 31 indicators) and at the district level (IMAPI‐D, 29 indicators). We developed a quantitative prioritization process for indicators in each IMAPI analysis, and those selected were jointly mapped in the socioecological model for the role of indicators in relation to the enabling environment for nurturing care. Out of 28 common nurturing care indicators across IMAPI analysis, only four were prioritized in both analyses: one from the Adequate nutrition, two from the Opportunities for early learning, and one from the Responsive caregiving domains. These four indicators were mapped as enabling policies, supportive services, and caregivers’ capabilities (socioecological model) and Effort, Coverage, and Quality (indicator's role). In conclusion, the different levels of nurturing care data disaggregation in the IMAPI can better inform decision‐making than each one individually, especially in metropolitan areas where municipalities and districts within metropolitan areas have relative decision‐making autonomy.

The Brazilian Early Childhood Friendly Municipal Index (IMAPI) is an innovative population‐based approach created to systematically measure nurturing care for early childhood development (ECD) using routine information systems data at the municipal level.

Brasilia was used as a case study to evaluate whether different levels of nurturing care data disaggregation (municipal vs. district level) can inform more equitable prioritization for ECD investments in metropolitan areas.

We developed a quantitative prioritization process of indicators to analyse IMAPI at the municipal and district levels, and the selected indicators were jointly mapped into the socioecological model, specifying the roles of the indicator in the enabling environment for nurturing care.

The two different levels of data disaggregation of the IMAPI can better inform nurturing care‐related decisions than each individual level in metropolitan areas where municipalities and districts within metropolitan areas have relative decision‐making autonomy.

Prioritizing Research in an Era of Personalized Medicine: The Potential Value of Unexplained Heterogeneity

Background

Clinical care is moving from a “one size fits all” approach to a setting in which treatment decisions are based on individual treatment response, needs, preferences, and risk. Research into personalized treatment strategies aims to discover currently unknown markers that identify individuals who would benefit from treatments that are nonoptimal at the population level. Before investing in research to identify these markers, it is important to assess whether such research has the potential to generate value. Thus, this article aims to develop a framework to prioritize research into the development of new personalized treatment strategies by creating a set of measures that assess the value of personalizing care based on a set of unknown patient characteristics.

Methods

Generalizing ideas from the value of heterogeneity framework, we demonstrate 3 measures that assess the value of developing personalized treatment strategies. The first measure identifies the potential value of personalizing medicine within a given disease area. The next 2 measures highlight specific research priorities and subgroup structures that would lead to improved patient outcomes from the personalization of treatment decisions.

Results

We graphically present the 3 measures to perform sensitivity analyses around the key drivers of value, in particular, the correlation between the individual treatment benefits across the available treatment options. We illustrate these 3 measures using a previously published decision model and discuss how they can direct research in personalized medicine.

Conclusion

We discuss 3 measures that form the basis of a novel framework to prioritize research into novel personalized treatment strategies. Our novel framework ensures that research targets personalized treatment strategies that have high potential to improve patient outcomes and health system efficiency.

Highlights

Methods for Identifying Health Research Gaps, Needs, and Priorities: a Scoping Review

BACKGROUND

Well-defined, systematic, and transparent processes to identify health research gaps, needs, and priorities are vital to ensuring that available funds target areas with the greatest potential for impact.

OBJECTIVE

The purpose of this review is to characterize methods conducted or supported by research funding organizations to identify health research gaps, needs, or priorities.

METHOD

We searched MEDLINE, PsycINFO, and the Web of Science up to September 2019. Eligible studies reported on methods to identify health research gaps, needs, and priorities that had been conducted or supported by research funding organizations. Using a published protocol, we extracted data on the method, criteria, involvement of stakeholders, evaluations, and whether the method had been replicated (i.e., used in other studies).

RESULTS

Among 10,832 citations, 167 studies were eligible for full data extraction. More than half of the studies employed methods to identify both needs and priorities, whereas about a quarter of studies focused singularly on identifying gaps (7%), needs (6%), or priorities (14%) only. The most frequently used methods were the convening of workshops or meetings (37%), quantitative methods (32%), and the James Lind Alliance approach, a multi-stakeholder research needs and priority setting process (28%). The most widely applied criteria were importance to stakeholders (72%), potential value (29%), and feasibility (18%). Stakeholder involvement was most prominent among clinicians (69%), researchers (66%), and patients and the public (59%). Stakeholders were identified through stakeholder organizations (51%) and purposive (26%) and convenience sampling (11%). Only 4% of studies evaluated the effectiveness of the methods and 37% employed methods that were reproducible and used in other studies.

DISCUSSION

To ensure optimal targeting of funds to meet the greatest areas of need and maximize outcomes, a much more robust evidence base is needed to ascertain the effectiveness of methods used to identify research gaps, needs, and priorities.

Early health technology assessment using the MAFEIP tool. A case study on a wearable device for fall prediction in elderly patients

By using a case-study on a fall-prediction device for elderly patients with orthostatic hypotension we aim to demonstrate how the MAFEIP tool, developed as part of the European Innovation Programme on Active and Healthy Ageing (EIP on AHA), can be used to inform manufacturers on their product development based on a cost-effectiveness criterion. Secondly, we critically appraise the tool and suggest further improvements that may be needed for a larger-scale adoption of MAFEIP within and beside the EIP on AHA initiative. The model was implemented using the MAFEIP tool. Within the tool one way sensitivity analyses were performed to assess the robustness of the model against the relative effectiveness of the fall-prevention device at different price levels. The MAFEIP tool was applied to a novel fall-prediction device and used to estimate the expected cost-effectiveness and perform threshold analysis. In our case study, the device produced estimated gains of 0.035 QALYs per patient and incremental costs of £ 518 (incremental cost-effectiveness ratio £14,719). Based on the one-way sensitivity analysis, the maximum achievable price at a willingness to pay threshold of £20,000 per QALY is estimated close to £900. The MAFEIP allows to quickly create early economic models, and to explore model uncertainty by performing deterministic sensitivity analysis for single parameters. However, the integration within the MAFEIP of common analytical tools such as probabilistic sensitivity analysis and Value of information would greatly contribute to its relevance for evaluating innovative technologies within and beside the EIP on AHA initiative.

Implementation Barriers to Value of Information Analysis in Health Technology Decision Making: Results From a Process Evaluation

OBJECTIVES

Value of information (VOI) analysis can support health technology assessment decision making, but it is a long way from being standard use. The objective of this study was to understand barriers to the implementation of VOI analysis and propose actions to overcome these.

METHODS

We performed a process evaluation of VOI analysis use within decision making on tomosynthesis versus digital mammography for use in the Dutch breast cancer population screening. Based on steering committee meeting attendance and regular meetings with analysts, we developed a list of barriers to VOI use, which were analyzed using an established diffusion model. We proposed actions to address these barriers. Barriers and actions were discussed and validated in a workshop with stakeholders representing patients, clinicians, regulators, policy advisors, researchers, and the industry.

RESULTS

Consensus was reached on groups of barriers, which included characteristics of VOI analysis itself, stakeholder's attitudes, analysts' and policy makers' skills and knowledge, system readiness, and implementation in the organization. Observed barriers did not only pertain to VOI analysis itself but also to formulating the objective of the assessment, economic modeling, and broader aspects of uncertainty assessment. Actions to overcome these barriers related to organizational changes, knowledge transfer, cultural change, and tools.

CONCLUSIONS

This in-depth analysis of barriers to implementation of VOI analysis and resulting actions and tools may be useful to health technology assessment organizations that wish to implement VOI analysis in technology assessment and research prioritization. Further research should focus on application and evaluation of the proposed actions in real-world assessment processes.

Prioritisation and design of clinical trials

Clinical trials require participation of numerous patients, enormous research resources and substantial public funding. Time-consuming trials lead to delayed implementation of beneficial interventions and to reduced benefit to patients. This manuscript discusses two methods for the allocation of research resources and reviews a framework for prioritisation and design of clinical trials. The traditional error-driven approach of clinical trial design controls for type I and II errors. However, controlling for those statistical errors has limited relevance to policy makers. Therefore, this error-driven approach can be inefficient, waste research resources and lead to research with limited impact on daily practice. The novel value-driven approach assesses the currently available evidence and focuses on designing clinical trials that directly inform policy and treatment decisions. Estimating the net value of collecting further information, prior to undertaking a trial, informs a decision maker whether a clinical or health policy decision can be made with current information or if collection of extra evidence is justified. Additionally, estimating the net value of new information guides study design, data collection choices, and sample size estimation. The value-driven approach ensures the efficient use of research resources, reduces unnecessary burden to trial participants, and accelerates implementation of beneficial healthcare interventions.

Building a trusted framework for uncertainty assessment in rare diseases: suggestions for improvement (Response to "TRUST4RD: tool for reducing uncertainties in the evidence generation for specialised treatments for rare diseases")

The aim of this letter to the editor is to provide a comprehensive summary of uncertainty assessment in Health Technology Assessment, with a focus on transferability to the setting of rare diseases. The authors of "TRUST4RD: tool for reducing uncertainties in the evidence generation for specialised treatments for rare diseases" presented recommendations for reducing uncertainty in rare diseases. Their article is of great importance but unfortunately suffers from a lack of references to the wider uncertainty in Health Technology Assessment and research prioritisation literature and consequently fails to provide a trusted framework for decision-making in rare diseases. In this letter to the editor we critique the authors' tool and provide pointers as to how their proposal can be strengthened. We present references to the literature, including our own tool for uncertainty assessment (TRUST; unrelated to the authors' research), apply TRUST to two assessments of orphan drugs in rare diseases and provide a broader perspective on uncertainty and risk management in rare diseases, including a detailed research agenda.

Managing Uncertainties Due to Limited Evidence in Economic Evaluations of Novel Anti-Tuberculosis Regimens: A Systematic Review

BACKGROUND

Limited evidence for the implementation of new health technologies in low- and middle-income countries (LMICs) may lead to uncertainties in economic evaluations and cause the evaluations to produce inaccurate information for decision making. We performed a systematic review of economic evaluations on implementing new short-course regimens (SCR) for drug-sensitive and drug-resistant tuberculosis (TB), to explore how uncertainties due to the limited evidence in the studies were dealt with and to identify useful information for decision making from these studies.

METHODS

We searched in electronic databases PubMed, EMBASE, NHSEED, and CEA registry for economic evaluations addressing the implementation of new anti-TB SCRs in LMICs published until September 2018. We included studies addressing both the cost and outcomes of implementing a new regimen for drug-sensitive and drug-resistant TB with a shorter treatment duration than the currently used regimens. The quality of the included studies was assessed using The Consensus Health Economic Criteria checklist. We extracted information from the included studies on uncertainties and how they were managed. The management of uncertainties was compared with approaches used in early health technology assessments (HTAs), including sensitivity analyses and pragmatic scenario analyses. We extracted information that could be useful for decision making such as cost-effectiveness conclusions, and barriers to implementing the intervention.

RESULTS

Four of the 322 studies found in the search met the eligibility criteria. Three studies were model-based studies that investigated the cost effectiveness of a new first-line SCR. One study was an empirical study investigating the cost effectiveness of new regimens for drug-resistant TB. The model-based studies addressed uncertainties due to limited evidence through various sensitivity analyses as in early HTAs. They performed a deterministic sensitivity analysis and found the main drivers of the cost-effectiveness outcomes, that is, the rate of treatment default and treatment delivery costs. Additionally, two of the model-based studies performed a pragmatic scenario analysis and found a potential barrier to implementing the new first-line SCR, that is, a weak health system with a low TB care utilization rate. The empirical study only performed a few scenario analyses with different regimen prices and volumes of TB care utilization. Therefore, the study could only provide information on the main cost drivers.

CONCLUSION

Using an approach similar to that used in early HTAs, where uncertainties due to the limited evidence are rigorously explored upfront, the economic evaluations could inform not only the decision to implement the intervention but also how to manage risks and implementation barriers.

The relationship between government research funding and the cancer burden in South Korea: implications for prioritising health research

Background

In this study, we aimed to assess health research funding allocation in South Korea by analysing the relationship between government funding and disease burden in South Korea, specifically focusing on cancers.

Methods

The relationship between research funding and the cancer burden, measured in disability-adjusted life-years (DALYs), was analysed using a linear regression method over a 10-year interval. Funding information on 25 types of cancer was obtained from the National Science and Technology Information Service portal in South Korea. Measures of cancer burden were obtained from Global Burden of Disease studies. The funding predictions were derived from regression analysis and compared with actual funding allocations. In addition, we evaluated how the funding distribution reflected long-term changes in the burden and the burden specific to South Korea compared with global values.

Results

Korean funding in four periods, 2005–2007, 2008–2010, 2011–2013 and 2015–2017, were associated with the cancer burden in 2003, 2006, 2009 and 2013, respectively. For DALYs, the correlation coefficients were 0.79 and 0.82 in 2003 and 2013, respectively, which were higher than the values from other countries. However, the changes in DALYs (1990–2006) were not associated with the funding changes (from 2005 to 2007 to 2015–2017). In addition, the value differences between Korean and global DALYs were not associated with Korean government research funding.

Conclusions

Although research funding was associated with the cancer burden in South Korea during the last decade, the distribution of research funds did not appropriately reflect the changes in burden nor the differences between the South Korean and global burden levels. The policy-makers involved in health research budgeting should consider not only the absolute burden values for singular years but also the long-term changes in burden and the country-specific burden when they prioritise public research projects.

Error associated with estimates of Minimum Infection Rate for Endemic West Nile Virus in areas of low mosquito trap density

West Nile Virus (WNV) is a mosquito-borne infection that can cause serious illness in humans. Surveillance for WNV primarily focuses on a measure of infection prevalence in the Culex spp. mosquitos, its primary vectors, known as the Minimum Infection Rate (MIR). The calculation of MIR for a given area considers the number of mosquitos tested, but not the relative effort to collect mosquitos, leading to a potential underestimation of the uncertainty around the estimate. We performed Value of Information analysis on simulated data sets including a range of mosquito trap densities in two well-studied counties in Illinois between 2005 and 2016 to determine the relative error introduced into MIR associated with changing the density of mosquito traps. We found that low trap density increases the potential for error in MIR estimation, and that it does so synergistically with low true MIR values. We propose that these results could be used to better estimate uncertainty in WNV risk.

A Framework to Prioritise Health Research Proposals for Funding: Integrating Value for Money

When making funding decisions, research organisations largely consider the merits (e.g. scientific rigour and feasibility) of submitted research proposals; yet, there is often little or no reference to their value for money. This may be attributed to the challenges of assessing and integrating value of research into existing research prioritisation processes. We propose a framework that considers both the merits of research and its value for money to guide health research funding decisions. A practical framework is developed based on current processes followed by funding organizations for assessing investigator-initiated research proposals, and analytical methods for evaluating the expected value of research. We apply the analytical methods to estimate the expected return on investment of two real-world grant applications. The framework comprises four sequential steps: (1) initial screening of applications for eligibility and completeness; (2) merit assessment of eligible proposals; (3) estimating the expected value of research for the shortlisted proposals that pass the first two steps and ranking of proposals based on return on investment; and (4) selecting research proposals for funding. We demonstrate how the expected value for money can be efficiently estimated using certain information provided in funding applications. The proposed framework integrates value-for-money assessment into the existing research prioritisation processes. Considering value for money to inform research funding decisions is vital to achieve efficient utilisation of research budgets and maximise returns on research investments.

A Value of Information Analysis of Research on the 21-Gene Assay for Breast Cancer Management

OBJECTIVES

The 21-gene assay Oncotype DX (21-GA) shows promise as a guide in deciding when to initiate adjuvant chemotherapy in women with hormone receptor-positive early-stage breast cancer. Nevertheless, its routine use remains controversial, owing to insufficient evidence of its clinical utility and cost-effectiveness. Accordingly, we aim to quantify the value of conducting further research to reduce decision uncertainty in the use of the 21-GA.

METHODS

Using value of information methods, we first generated probability distributions of survival and costs for decision making with and without the 21-GA alongside traditional risk prediction. These served as the input to a comparison of 3 alternative study designs: a retrospective observational study to update risk classification from the 21-GA, a prospective observational study to estimate prevalence of chemotherapy use, and a randomized controlled trial (RCT) of the 21-GA predictive value.

RESULTS

We found that current evidence strongly supports the use of the 21-GA in intermediate- and high-risk women. Further research should focus on low-risk women, among whom the cost-effectiveness findings remained equivocal. For this population, we identified a high value of reducing uncertainty in the 21-GA use for all proposed research studies. The RCT had the greatest potential to efficiently reduce the likelihood of choosing a suboptimal strategy, providing a value between $162 million and $1.1 billion at willingness-to-pay thresholds of $150 000 to $200 000/quality-adjusted life years.

CONCLUSION

Future research to inform 21-GA decision making is of high value. The RCT of the 21-GA predictive value has the greatest potential to efficiently reduce decision uncertainty around 21-GA use in women with low-risk early-stage breast cancer.

Cost-effectiveness model for a hypothetical monotherapy vs standard of care in adult patients with treatment-resistant depression

Background

Patients with treatment-resistant major depressive disorder (TRD) have limited treatment options. We developed an early stage cost-effectiveness model of TRD to explore the potential value of a hypothetical monotherapy relative to the standard of care (SOC). The relative impacts of the monotherapy’s three differentiating features over SOC are explored: efficacy advantage, tolerability advantage, and price premium.

Methods

We adapted an existing economic model of TRD to evaluate the cost-effectiveness of a hypothetical monotherapy for TRD with a 25% efficacy advantage, a 10% tolerability advantage, and a 50% price premium over SOC (selective serotonin reuptake inhibitor plus atypical antipsychotics [SSRI + AAP]). The model is a hybrid of a decision tree that captures patients’ outcomes after an 8-week acute treatment phase and a Markov model that simulates patients’ depression course through a 10-month maintenance phase. Sensitivity (deterministic and probabilistic) and scenario analyses were conducted to characterize the relative impacts of the monotherapy’s three differentiating features over SOC.

Results

Over the 12-month time horizon, the hypothetical monotherapy is shown to dominate SOC; it generates lower costs and higher quality-adjusted life years in comparison to SSRI + AAP. Sensitivity and scenario analyses showed that this dominance depends largely on the monotherapy’s efficacy and tolerability advantages over SOC. Specifically, a monotherapy with ≥ 12% efficacy or ≥70% tolerability advantage (and a 50% price premium) will always be superior to SSRI + AAP. Between these two extremes, most profiles, nonetheless, generate incremental cost-utility ratios for the monotherapy, which fall below common payer willingness-to-pay thresholds.

Conclusion

Our adaptation of an existing economic model of TRD provides a flexible platform for researchers to evaluate the efficacy/tolerability improvements required for a successful new TRD product and for decision-makers to assess the cost-effectiveness impact of uncertainties inherent in early stage product development in TRD.

Prospective Assessment of the Feasibility of a Trial of Low-Tidal Volume Ventilation for Patients with Acute Respiratory Failure

RATIONALE

Low-tidal volume ventilation (LTVV; 6 ml/kg) benefits patients with acute respiratory distress syndrome and may aid those with other causes of respiratory failure. Current early ventilation practices are poorly defined.

OBJECTIVES

We observed patients with acute respiratory failure to assess the feasibility of a pragmatic trial of LTVV and to guide experimental design.

METHODS

We prospectively enrolled consecutive patients with acute respiratory failure admitted to intensive care units expected to participate in the proposed trial. We collected clinical data as well as information on initial and daily ventilator settings and inpatient mortality. We estimated the benefit of LTVV using predictive linear and nonlinear models. We simulated models to estimate power and feasibility of a cluster-randomized trial of LTVV versus usual care in acute respiratory failure.

RESULTS

We included 2,484 newly mechanically ventilated patients (31% with acute respiratory distress syndrome) from 49 hospitals. Hospital mortality was 28%. Mean initial tidal volume was 7.1 ml/kg predicted body weight (95% confidence interval, 7.1-7.2), with 78% of patients receiving tidal volumes less than or equal to 8 ml/kg. Our models estimated a mortality benefit of 0-2% from LTVV compared with usual care. Simulation of a stepped-wedged cluster-randomized trial suggested that enrollment of 106,361 patients would be necessary to achieve greater than 90% power.

CONCLUSIONS

Use of initial tidal volumes less than 8 ml/kg predicted body weight was common at hospitals participating in the National Heart, Lung, and Blood Institute Prevention and Early Treatment of Acute Lung Injury (PETAL) Network. After considering the size and budgetary requirement for a cluster-randomized trial of LTVV versus usual care in acute respiratory failure, the PETAL Network deemed the proposed trial infeasible. A rapid observational study and simulations to model anticipated power may help better design trials.

When Is Enough, Enough? When Are More Observational Epidemiologic Studies Needed to Resolve a Research Question: Illustrations Using Biomarker-Cancer Associations

BACKGROUND

Research reproducibility is vital for translation of epidemiologic findings. However, repeated studies of the same question may be undertaken without enhancing existing knowledge. To identify settings in which additional research is or is not warranted, we adapted research synthesis metrics to determine number of additional observational studies needed to change the inference from an existing meta-analysis.

METHODS

The fail-safe number (FSN) estimates number of additional studies of average weight and null effect needed to drive a statistically significant meta-analysis to null (P ≥ 0.05). We used conditional power to determine number of additional studies of average weight and equivalent heterogeneity to achieve 80% power in an updated meta-analysis to detect the observed summary estimate as statistically significant. We applied these metrics to a curated set of 98 meta-analyses on biomarkers and cancer risk.

RESULTS

Both metrics were influenced by number of studies, heterogeneity, and summary estimate size in the existing meta-analysis. For the meta-analysis on Helicobacter pylori and gastric cancer with 15 studies [OR = 2.29; 95% confidence interval (CI), 1.71-3.05], FSN was 805 studies, supporting futility of further study. For the meta-analysis on dehydroepiandrosterone sulfate and prostate cancer with 7 studies (OR = 1.29; 95% CI, 0.99-1.69), 5 more studies would be needed for 80% power, suggesting further study could change inferences.

CONCLUSIONS

Along with traditional assessments, these metrics could be used by stakeholders to decide whether additional studies addressing the same question are needed.

IMPACT

Systematic application of these metrics could lead to more judicious use of resources and acceleration from discovery to population-health impact.

Value of Information Choices that Influence Estimates: A Systematic Review of Prevailing Considerations

Background. Although value of information (VOI) analyses are increasingly advocated and used for research prioritization and reimbursement decisions, the interpretation and usefulness of VOI outcomes depend critically on the underlying choices and assumptions used in the analysis. In this article, we present a structured overview of all items reported in literature to potentially influence VOI outcomes. Use of this overview increases awareness and transparency of choices and assumptions underpinning VOI outcomes. Methods. A systematic literature review was performed to identify aspects of VOI analyses that were found to potentially influence VOI outcomes. Identified aspects were grouped to develop a structured overview. Explanations were defined for all items included in the overview. Results. We retrieved 687 unique papers, of which 71 original papers and 8 reviews were included. In the full text of these 79 papers, 16 aspects were found that may influence VOI outcomes. These aspects related to the underlying evidence (bias, synthesis, heterogeneity, correlation), uncertainty (structural, future pricing), model (relevance, approach, population), choices in VOI calculation (estimation technique, implementation level, population size, perspective), and aspects specifically for assessing the value of future study designs (reversal costs, efficient estimator). These aspects were aggregated into 7 items to provide a structured overview. Conclusion. The developed overview should increase awareness of key choices underlying VOI analysis and facilitate structured reporting of such choices and interpretation of the ensuing VOI outcomes by researchers and policy makers. Use of this overview should improve prioritization and reimbursement decisions.

Using HTA and guideline development as a tool for research priority setting the NICE way: reducing research waste by identifying the right research to fund

BACKGROUND

The National Institute for Health and Care Excellence (NICE) was established in 1999 and provides national guidance and advice to improve health and social care. Several steps in the research cycle have been identified that can support the reduction of waste that occurs in biomedical research. The first step in the process is ensuring appropriate research priority setting occurs so only the questions that are needed to fill existing gaps in the evidence are funded. This paper summarises the research priority setting processes at NICE.

METHODS

NICE uses its guidance production processes to identify and prioritise research questions through systematic reviews, economic analyses and stakeholder consultations and then highlights those priorities by engagement with the research community. NICE also highlights its methodological areas for research to ensure the appropriate development and growth of the evidence landscape.

RESULTS

NICE has prioritised research questions through its guidance production and methodological work and has successfully had several research products funded through the National Institute for Health Research and Medical Research Council. This paper summarises those activities and results.

CONCLUSIONS

This activity of NICE therefore reduces research waste by ensuring that the research it recommends has been systematically prioritised through evidence reviews and stakeholder input.

Predictive Assessment of Cancer Center Catchment Area from Electronic Health Records

Healthcare facilities (HF) may identify catchment areas (CA) by selecting criteria that depend on various factors. These refer to hospital activities, geographical definition, patient covariates, and more. The analyses that were traditionally pursued have a limiting factor in the consideration of only static conditions. Instead, some of the CA determinants involve influences occurring at both temporal and spatial scales. The study of CA in the cancer context means choosing between HF, usually divided into general hospitals versus oncological centers (OCs). In the CA context, electronic health records (EHRs) promise to be a valuable source of information, one driving the next-generation patient-driven clinical decision support systems. Among the challenges, digital health requires the re-definition of a role of stochastic modeling to deal with emerging complexities from data heterogeneity. To model CA with cancer EHR, we have chosen a computational framework centered on a logistic model, as a reference, and on a multivariate statistical approach. We also provided a battery of tests for CA assessment. Our results indicate that a more refined CA model’s structure yields superior discrimination power between health facilities. The increased significance was also visualized by comparative evaluations with ad hoc geo-localized maps. Notably, a cancer-specific spatial effect can be noticed, especially for breast cancer and through OCs. To mitigate the data distributional influences, bootstrap analysis was performed, and gains in some cancer-specific and spatially concentrated regions were obtained. Finally, when the temporal dynamics are assessed along a 3-year timeframe, negligible differential effects appear between predicted probabilities observed between standard critical values and bootstrapped values. In conclusion, for interpreting CA in terms of both spatial and temporal dynamics, sophisticated models are required. The one here proposed suggests that bootstrap can improve test accuracy. We recommend that evidences from stochastic modeling are merged with visual analytics, as this combination may be exploited by policy-makers in support to quantitative CA assessment.

Uncertainty on the effectiveness and safety of rivaroxaban in premenopausal women with atrial fibrillation: empirical evidence needed

Background

Novel anticoagulations (NOACs) are increasingly prescribed for the prevention of stroke in premenopausal women with atrial fibrillation. Small studies suggest NOACs are associated with a higher risk of abnormal uterine bleeds than vitamin K antagonists (VKAs). Because there is no direct empirical evidence on the benefit/risk profile of rivaroxaban compared to VKAs in this subgroup, we synthesize available indirect evidence, estimate decision uncertainty on the treatments, and assess whether further research in premenopausal women is warranted.

Methods

A Markov model with annual cycles and a lifetime horizon was developed comparing rivaroxaban (the most frequently prescribed NOAC in this population) and VKAs. Clinical event rates, associated quality adjusted life years, and health care costs were obtained from different sources and adjusted for gender, age, and history of stroke. A Monte Carlo simulation with 10,000 iterations was then performed for a hypothetical cohort of premenopausal women, estimated to be reflective of the population of premenopausal women with AF in The Netherlands.

Results

In the simulation, rivaroxaban is the better treatment option for the prevention of ischemic strokes in premenopausal women in 61% of the iterations. Similarly, this is 98% for intracranial hemorrhages, 24% for major abnormal uterine bleeds, 1% for minor abnormal uterine bleeds, 9% for other major extracranial hemorrhages, and 23% for other minor extracranial hemorrhages. There is a 78% chance that rivaroxaban offers the most quality-adjusted life years. The expected value of perfect information in The Netherlands equals 122 quality-adjusted life years and 22 million Euros.

Conclusions

There is a 22% risk that rivaroxaban offers a worse rather than a better benefit/risk profile than vitamin K antagonists in premenopausal women. Although rivaroxaban is preferred over VKAs in this population, further research is warranted, and should preferably take the shape of an internationally coordinated registry study including other NOACs.

Electronic supplementary material

The online version of this article (10.1186/s12872-017-0692-1) contains supplementary material, which is available to authorized users.

Why Most Clinical Research Is Not Useful

John Ioannidis argues that problem base, context placement, information gain, pragmatism, patient centeredness, value for money, feasibility, and transparency define useful clinical research. He suggests most clinical research is not useful and reform is overdue.