Value of information analysis in oncology: the value of evidence and evidence of value

'First ensure no regret': a decision-theoretic approach to informed consent in clinical practice

Decision theorists recognise that information is valuable only insofar as it has the potential to change a decision. This means that since acquiring more information is time-consuming and sometimes expensive, judgements need to be made about what information is most valuable to acquire, and whether it is worth acquiring at all. In this article I apply this idea to informed consent and argue that the most valuable information relates not to what the best treatment option may be but to possible futures a patient may regret. I conclude by proposing a regret-minimisation framework for informed consent that I contend better captures the true nature of shared decision making than existing formulations.

Emerging Therapies for COVID-19: The Value of Information From More Clinical Trials

OBJECTIVES

The COVID-19 pandemic necessitates time-sensitive policy and implementation decisions regarding new therapies in the face of uncertainty. This study aimed to quantify consequences of approving therapies or pursuing further research: immediate approval, use only in research, approval with research (eg, emergency use authorization), or reject.

METHODS

Using a cohort state-transition model for hospitalized patients with COVID-19, we estimated quality-adjusted life-years (QALYs) and costs associated with the following interventions: hydroxychloroquine, remdesivir, casirivimab-imdevimab, dexamethasone, baricitinib-remdesivir, tocilizumab, lopinavir-ritonavir, interferon beta-1a, and usual care. We used the model outcomes to conduct cost-effectiveness and value of information analyses from a US healthcare perspective and a lifetime horizon.

RESULTS

Assuming a $100 000-per-QALY willingness-to-pay threshold, only remdesivir, casirivimab-imdevimab, dexamethasone, baricitinib-remdesivir, and tocilizumab were (cost-) effective (incremental net health benefit 0.252, 0.164, 0.545, 0.668, and 0.524 QALYs and incremental net monetary benefit $25 249, $16 375, $54 526, $66 826, and $52 378). Our value of information analyses suggest that most value can be obtained if these 5 therapies are approved for immediate use rather than requiring additional randomized controlled trials (RCTs) (net value $20.6 billion, $13.4 billion, $7.4 billion, $54.6 billion, and $7.1 billion), hydroxychloroquine (net value $198 million) is only used in further RCTs if seeking to demonstrate decremental cost-effectiveness and otherwise rejected, and interferon beta-1a and lopinavir-ritonavir are rejected (ie, neither approved nor additional RCTs).

CONCLUSIONS

Estimating the real-time value of collecting additional evidence during the pandemic can inform policy makers and clinicians about the optimal moment to implement therapies and whether to perform further research.

Toward an Ecologically Valid Conceptual Framework for the Use of Artificial Intelligence in Clinical Settings: Need for Systems Thinking, Accountability, Decision-making, Trust, and Patient Safety Considerations in Safeguarding the Technology and Clinicians

The health care management and the medical practitioner literature lack a descriptive conceptual framework for understanding the dynamic and complex interactions between clinicians and artificial intelligence (AI) systems. As most of the existing literature has been investigating AI's performance and effectiveness from a statistical (analytical) standpoint, there is a lack of studies ensuring AI's ecological validity. In this study, we derived a framework that focuses explicitly on the interaction between AI and clinicians. The proposed framework builds upon well-established human factors models such as the technology acceptance model and expectancy theory. The framework can be used to perform quantitative and qualitative analyses (mixed methods) to capture how clinician-AI interactions may vary based on human factors such as expectancy, workload, trust, cognitive variables related to absorptive capacity and bounded rationality, and concerns for patient safety. If leveraged, the proposed framework can help to identify factors influencing clinicians' intention to use AI and, consequently, improve AI acceptance and address the lack of AI accountability while safeguarding the patients, clinicians, and AI technology. Overall, this paper discusses the concepts, propositions, and assumptions of the multidisciplinary decision-making literature, constituting a sociocognitive approach that extends the theories of distributed cognition and, thus, will account for the ecological validity of AI.

High-dimensional role of AI and machine learning in cancer research

The role of Artificial Intelligence and Machine Learning in cancer research offers several advantages, primarily scaling up the information processing and increasing the accuracy of the clinical decision-making. The key enabling tools currently in use in Precision, Digital and Translational Medicine, here named as 'Intelligent Systems' (IS), leverage unprecedented data volumes and aim to model their underlying heterogeneous influences and variables correlated with patients' outcomes. As functionality and performance of IS are associated with complex diagnosis and therapy decisions, a rich spectrum of patterns and features detected in high-dimensional data may be critical for inference purposes. Many challenges are also present in such discovery task. First, the generation of interpretable model results from a mix of structured and unstructured input information. Second, the design, and implementation of automated clinical decision processes for drawing disease trajectories and patient profiles. Ultimately, the clinical impacts depend on the data effectively subjected to steps such as harmonisation, integration, validation, etc. The aim of this work is to discuss the transformative value of IS applied to multimodal data acquired through various interrelated cancer domains (high-throughput genomics, experimental biology, medical image processing, radiomics, patient electronic records, etc.).

The Curve of Optimal Sample Size (COSS): A Graphical Representation of the Optimal Sample Size from a Value of Information Analysis

Value of information (VOI) analysis quantifies the opportunity cost associated with decision uncertainty, and thus informs the value of collecting further information to avoid this cost. VOI can inform study design, optimal sample size selection, and research prioritization. Recent methodological advances have reduced the computational burden of conducting VOI analysis and have made it easier to evaluate the expected value of sample information, the expected net benefit of sampling, and the optimal sample size of a study design ([Formula: see text]). The volume of VOI analyses being published is increasing, and there is now a need for VOI studies to conduct sensitivity analyses on VOI-specific parameters. In this practical application, we introduce the curve of optimal sample size (COSS), which is a graphical representation of [Formula: see text] over a range of willingness-to-pay thresholds and VOI parameters (example data and R code are provided). In a single figure, the COSS presents summary data for decision makers to determine the sample size that optimizes research funding given their operating characteristics. The COSS also presents variation in the optimal sample size given variability or uncertainty in VOI parameters. The COSS represents an efficient and additional approach for summarizing results from a VOI analysis.

Cost-Effectiveness of Treatments for the Management of Bone Metastases: A Systematic Literature Review

BACKGROUND

Metastatic cancers occur when cancer cells break away from the primary tumour. One of the most common sites of metastasis is the bone, with several therapeutic options currently available for managing bone metastases. In a resource-constrained environment, policy makers and practitioners need to know which options are cost effective.

OBJECTIVE

The aim of this systematic review was to review and appraise published economic evaluations on treatments for the management of bone metastases.

METHODS

We searched eight bibliographic databases (MEDLINE, MEDLINE in Process, EMBASE, CSDR, DARE, HTA, EED and CPCI) for relevant economic evaluations published from each database's inception date until March 2017. Study selection, quality assessment and data extraction were carried out according to published guidelines.

RESULTS

Twenty-four relevant economic analyses were identified. Seventeen of these studies focused on bone metastases resulting from a particular type of cancer, i.e. prostate (n = 8), breast (n = 7), lung (n = 1) or renal (n = 1), while seven report results for various primary tumours. Across types of cancer, evidence suggests that bisphosphonates result in lower morbidity and improved quality of life, for an additional cost, which is typically below conventional cost-effectiveness thresholds. While denosumab leads to health gains compared with zoledronic acid, it also results in substantial additional costs and is unlikely to represent value for money. The limited literature on the radiopharmaceutical strontium-89 (Sr89) and external beam radiotherapy (EBR) suggest that these treatments are cost effective compared with no treatment.

CONCLUSIONS

The reviewed evidence suggests that bisphosphonate treatments are cost-effective options for bone metastases, while denosumab is unlikely to represent value for money. Evidence on EBR and Sr89 is limited and less conclusive.

Research Costs Investigated: A Study Into the Budgets of Dutch Publicly Funded Drug-Related Research

BACKGROUND

The costs of performing research are an important input in value of information (VOI) analyses but are difficult to assess.

OBJECTIVE

The aim of this study was to investigate the costs of research, serving two purposes: (1) estimating research costs for use in VOI analyses; and (2) developing a costing tool to support reviewers of grant proposals in assessing whether the proposed budget is realistic.

METHODS

For granted study proposals from the Netherlands Organization for Health Research and Development (ZonMw), type of study, potential cost drivers, proposed budget, and general characteristics were extracted. Regression analysis was conducted in an attempt to generate a 'predicted budget' for certain combinations of cost drivers, for implementation in the costing tool.

RESULTS

Of 133 drug-related research grant proposals, 74 were included for complete data extraction. Because an association between cost drivers and budgets was not confirmed, we could not generate a predicted budget based on regression analysis, but only historic reference budgets given certain study characteristics. The costing tool was designed accordingly, i.e. with given selection criteria the tool returns the range of budgets in comparable studies. This range can be used in VOI analysis to estimate whether the expected net benefit of sampling will be positive to decide upon the net value of future research.

CONCLUSION

The absence of association between study characteristics and budgets may indicate inconsistencies in the budgeting or granting process. Nonetheless, the tool generates useful information on historical budgets, and the option to formally relate VOI to budgets. To our knowledge, this is the first attempt at creating such a tool, which can be complemented with new studies being granted, enlarging the underlying database and keeping estimates up to date.

Health Departments' Engagement in Emergency Preparedness Activities: The Influence of Health Informatics Capacity

BACKGROUND

Local health departments (LHDs) operate in a complex and dynamic public health landscape, with changing demands on their emergency response capacities. Informatics capacities might play an instrumental role in aiding LHDs emergency preparedness. This study aimed to explore the extent to which LHDs' informatics capacities are associated with their activity level in emergency preparedness and to identify which health informatics capacities are associated with improved emergency preparedness.

METHODS

We used the 2013 National Profile of LHDs study to perform Poisson regression of emergency preparedness activities.

RESULTS

Only 38.3% of LHDs participated in full-scale exercises or drills for an emergency in the 12 months period prior to the survey, but a much larger proportion provided emergency preparedness training to staff (84.3%), and/or participated in tabletop exercises (76.4%). Our multivariable analysis showed that after adjusting for several resource-related LHD characteristics, LHDs with more of the 6 information systems still tend to have slightly more preparedness activities. In addition, having a designated emergency preparedness coordinator, and having one or more emergency preparedness staff were among the most significant factors associated with LHDs performing more emergency preparedness activities.

CONCLUSION

LHDs might want to utilize better health information systems and information technology tools to improve their activity level in emergency preparedness, through improved information dissemination, and evidence collection.

Value of Information Analysis Informing Adoption and Research Decisions in a Portfolio of Health Care Interventions

Background: Value of information (VOI) analysis quantifies the value of additional research in reducing decision uncertainty. It addresses adoption and research decisions simultaneously by comparing the expected benefits and costs of research studies. Nevertheless, the application of this approach in practice remains limited. Objectives: To apply VOI analysis in health care interventions to guide adoption decisions, optimize trial design, and prioritize research. Methods: The analysis was from the perspective of Queensland Health, Australia. It included four interventions: clinically indicated catheter replacement, tissue adhesive for securing catheters, negative pressure wound therapy (NPWT) in caesarean sections, and nutritional support for preventing pressure ulcers. For each intervention, cost-effectiveness analysis was performed, decision uncertainty characterized, and VOI calculated using Monte Carlo simulations. The benefits and costs of additional research were considered together with the costs and consequences of acting now versus waiting for more information. All values are reported in 2014 Australian dollars (AU$). Results: All interventions were cost-effective, but with various levels of decision uncertainty. The current evidence is sufficient to support the adoption of clinically indicated catheter replacement. For the tissue adhesive, an additional study before adoption is worthwhile with a four-arm trial of 220 patients per arm. Additional research on NPWT before adoption is worthwhile with a two-arm trial of 200 patients per arm. Nutritional support should be adopted with a two-arm trial of 1200 patients per arm. Based on the expected net monetary benefits, the studies were ranked as follows: 1) NPWT (AU$1.2 million), 2) tissue adhesive (AU$0.3 milliion), and 3) nutritional support (AU$0.1 million). Conclusions: VOI analysis is a useful and practical approach to inform adoption and research decisions. Efforts should be focused on facilitating its integration into decision making frameworks.

Efficient Value of Information Calculation Using a Nonparametric Regression Approach: An Applied Perspective

BACKGROUND

Value-of-information (VOI) analysis provides an analytical framework to assess whether obtaining additional evidence is worthwhile to reduce decision uncertainty. The reporting of VOI measures, particularly the expected value of perfect parameter information (EVPPI) and the expected value of sample information (EVSI), is limited because of the computational burden associated with typical two-level Monte-Carlo-based solution. Recently, a nonparametric regression approach was proposed that allows the estimation of multiparameter EVPPI and EVSI directly from a probabilistic sensitivity analysis sample.

OBJECTIVES

To demonstrate the value of the nonparametric regression approach in calculating VOI measures in real-world cases and to compare its performance with the standard approach of the Monte-Carlo simulation.

METHODS

We used the regression approach to calculate EVPPI and EVSI in two models, and compared the results with the estimates obtained via the standard Monte-Carlo simulation.

RESULTS

The VOI values from the two approaches were very close; computation using the regression method, however, was faster.

CONCLUSION

The nonparametric regression approach provides an efficient and easy-to-implement alternative for EVPPI and EVSI calculation in economic models.

Use of Value of Information in Healthcare Decision Making: Exploring Multiple Perspectives

BACKGROUND

Value of information (VOI) is a tool that can be used to inform decisions concerning additional research in healthcare. VOI estimates the value of obtaining additional information and indicates the optimal design for additional research. Although it is recognized as good practice in handling uncertainty, it is still hardly used in decision making in the Netherlands.

OBJECTIVE

This paper aims to examine the potential value of VOI, barriers and facilitators and the way forward with the use of VOI in the decision-making process for reimbursement of pharmaceuticals in the Netherlands.

METHODS

Three focus group interviews were conducted with researchers, policy makers, and representatives of pharmaceutical companies.

RESULTS

The results revealed that although all stakeholders recognize the relevance of VOI, it is hardly used and many barriers to the performance and use of VOI were identified. One of these barriers is that not all uncertainties are easily incorporated in VOI, and the results may be biased if structural uncertainties are ignored. Furthermore, not all research designs indicated by VOI may be feasible in practice.

CONCLUSIONS

To fully embed VOI into current decision-making processes, a threshold incremental cost-effectiveness ratio and guidelines that clarify when and how VOI should be performed are needed. In addition, it should be clear to all stakeholders how the results of VOI are used in decision making.

Value of Information: A Tool to Improve Research Prioritization and Reduce Waste

In a Guest Editorial, Cosetta Minelli and Gianluca Baio explain how VOI analysis can prioritize research projects by identifying uncertainty in existing knowledge and then estimating expected benefits from reducing that uncertainty.

Value of information analysis in healthcare: a review of principles and applications

BACKGROUND

Economic evaluations are increasingly utilized to inform decisions in healthcare; however, decisions remain uncertain when they are not based on adequate evidence. Value of information (VOI) analysis has been proposed as a systematic approach to measure decision uncertainty and assess whether there is sufficient evidence to support new technologies.

SCOPE

The objective of this paper is to review the principles and applications of VOI analysis in healthcare. Relevant databases were systematically searched to identify VOI articles. The findings from the selected articles were summarized and narratively presented.

FINDINGS

Various VOI methods have been developed and applied to inform decision-making, optimally designing research studies and setting research priorities. However, the application of this approach in healthcare remains limited due to technical and policy challenges.

CONCLUSION

There is a need to create more awareness about VOI analysis, simplify its current methods, and align them with the needs of decision-making organizations.