Accounting for parameter uncertainty in the definition of parametric distributions used to describe individual patient variation in health economic models

Health Economic Evidence and Modeling Challenges for Liquid Biopsy Assays in Cancer Management: A Systematic Literature Review

BACKGROUND

Cancer-derived material circulating in the bloodstream and other bodily fluids, referred to as liquid biopsies (LBs), has become an appealing adjunct or alternative to tissue biopsies, showing vital promise in several clinical applications.

PURPOSE

A systematic literature review was conducted to (1) summarize the current health economic evidence for LB assays and (2) identify and analyze the studies addressed or reported on the challenges of health economic modeling in precision medicine.

METHODS

Relevant studies were identified in the EMBASE, MEDLINE, Cochrane Library, EconLit, and the University of Melbourne Full Text Journal databases from 1 January 2013 to 16 September 2022. Included papers were selected if they were economic evaluations and/or budget impact analyses.

RESULTS

A total of 24 studies were included and analyzed, with the majority being full economic evaluations (n = 19, 79.2%). Four studies (16.7%) were health and budget impact analyses, and one study (4.1%) incorporated both an economic evaluation and a budget impact analysis. Cohort-level modeling techniques were the most common approach (n = 16; 80%). LB technologies were cost-effective in 15 studies (75%) considering different biomarkers, cancer types and stages, and economic analyses. These studies evaluated LBs for screening and early detection (66.7%), treatment selection (26.7%), and monitoring treatment response (6.6%). Budget impact analysis results were varied among included studies, with the majority of studies (n = 4; 80%) reporting either cost savings, minimal, or modest budget impact, while one study (20%) reported LBs as an efficient strategy. The reviewed studies often inadequately reported or addressed modeling challenges, such as patient-level processes, the combination of tests and treatments, preferences, and uncertainty.

CONCLUSION

LBs could provide a cost-effective approach for treatment selection in lung cancer and aid in the screening and early detection of other cancers, including colorectal, gastric, breast, and brain cancers. This is in comparison with various alternatives, such as the standard of care (SOC) and no screening scenario. However, it is important to mention that in some comparisons, LBs were used in combination with SOC instead of replacing it. Importantly, few studies have pointed toward LBs' cost-effectiveness for monitoring treatment response. Most health and budget impact analyses, especially those focused on lung cancer, suggest potential cost savings or a minimal-to-moderate budget impact. Nevertheless, additional research is needed to ascertain their effectiveness across various stages of lung and colorectal cancer, as well as to address potential modeling challenges.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42022307939.

Economic evaluation of using polygenic risk score to guide risk screening and interventions for the prevention of type 2 diabetes in individuals with high overall baseline risk

Type 2 diabetes (T2D) with increasing prevalence is a significant global public health challenge. Obesity, unhealthy diet, and low physical activity are one of the major determinants of the rise in T2D prevalence. In addition, family history and genetic risk of diabetes also play a role in the process of developing T2D. Therefore, solutions for the early identification of individuals at high risk for T2D for early targeted detection of T2D, prevention, and intervention are highly preferred. Recently, novel genomic-based polygenic risk scores (PRSs) have been suggested to improve the accuracy of risk prediction supporting the targeting of preventive interventions to those at highest risk for T2D. Therefore, the aim of the present study was to assess the cost-utility of an additional PRS testing information (as a part of overall risk assessment) followed by a lifestyle intervention and an additional medical therapy when estimated 10-year overall risk for T2D exceeded 20% among Finnish individuals screened as at the high-risk category (i.e., 10%–20% 10-year overall risk of T2D) based on traditional risk factors only. For a cost-utility analysis, an individual-level state-transition model with probabilistic sensitivity analysis was constructed. A 1-year cycle length and a lifetime time horizon were applied in the base-case. A 3% discount rate was used for costs and QALYs. Cost-effectiveness acceptability curve (CEAC) and estimates for the expected value of perfect information (EVPI) were calculated to assist decision makers. The use of the targeted PRS strategy reclassified 12.4 percentage points of individuals to be very high-risk individuals who would have been originally classified as high risk using the usual strategy only. Over a lifetime horizon, the targeted PRS was a dominant strategy (i.e., less costly, more effective). One-way and scenario sensitivity analyses showed that results remained dominant in almost all simulations. However, there is uncertainty, since the probability (EVPI) of cost-effectiveness at a WTP of 0€/QALY was 63.0% (243€) indicating the probability that the PRS strategy is a dominant option. In conclusion, the results demonstrated that the PRS provides moderate additional value in Finnish population in risk screening leading to potential cost savings and better quality of life when compared with the current screening methods for T2D risk.

A scoping review on patient heterogeneity in economic evaluations of precision medicine based on basket trials

INTRODUCTION

Considerable challenges in the economic evaluation of precision medicines have been mentioned in previous studies. However, they have not addressed how an economic assessment would be conducted based on basket trials (novel studies for evaluation of precision medicine effects) in which the included populations have specific biomarkers and various cancers. Since basket trial populations have remarkable heterogeneity, this study aims to investigate the concept of heterogeneity and specific method(s) for considering it in economic evaluations through guidelines and studies that could be applicable in economic evaluation based on basket trials.

AREA COVERED

We searched PubMed, Web of Science, Scopus, Google Scholar, and Google to find studies and pharmacoeconomics guidelines. The inclusion criteria included subjects of patient heterogeneity and suggested explicit method(s). Thirty-nine guidelines and 43 studies were included and evaluated. None of these materials mentioned disease types in a target population as a factor causing heterogeneity. Moreover, in economic evaluations, patient heterogeneity has been considered with four general approaches subgroup analysis, individual-based models, sensitivity analysis, and regression models.

EXPERT OPINION

Type of disease is not considered a contributing factor in population heterogeneity, and the probable appropriate method for this issue could be individual-based models.

Calibrating Natural History of Cancer Models in the Presence of Data Incompatibility: Problems and Solutions

The calibration of cancer natural history models is often challenged by a lack of representative calibration targets, forcing modellers to rely on potentially incompatible datasets. Using a microsimulation colorectal cancer model as an example, the purposes of this paper are to (1) highlight the reasons for uncertainty in calibration targets, (2) illustrate practical and generalisable approaches for dealing with incompatibility in calibration targets, and (3) discuss the importance of future research in the area of incorporating uncertainty in calibration. The low quality of data and differences in populations, outcome definitions, and healthcare systems may result in incompatibility between the model and the data. Acknowledging reasons for data incompatibility allows assessment of the risk of incompatibility before calibrating the model. Only a few approaches are available to address data incompatibility, for instance addressing biases in calibration targets and their adjustment, relaxing the goodness-of-fit metric, and validation of the calibration targets to the data not used in the calibration. However, these approaches lack explicit comparison and validation, and so more research is needed to describe the nature and causes of indirect uncertainty (i.e. uncertainty that cannot be expressed in absolute quantitative forms) and identify methods for managing this uncertainty in healthcare modelling.

Simulating Study Data to Support Expected Value of Sample Information Calculations: A Tutorial

The expected value of sample information (EVSI) can be used to prioritize avenues for future research and design studies that support medical decision making and offer value for money spent. EVSI is calculated based on 3 key elements. Two of these, a probabilistic model-based economic evaluation and updating model uncertainty based on simulated data, have been frequently discussed in the literature. By contrast, the third element, simulating data from the proposed studies, has received little attention. This tutorial contributes to bridging this gap by providing a step-by-step guide to simulating study data for EVSI calculations. We discuss a general-purpose algorithm for simulating data and demonstrate its use to simulate 3 different outcome types. We then discuss how to induce correlations in the generated data, how to adjust for common issues in study implementation such as missingness and censoring, and how individual patient data from previous studies can be leveraged to undertake EVSI calculations. For all examples, we provide comprehensive code written in the R language and, where possible, Excel spreadsheets in the supplementary materials. This tutorial facilitates practical EVSI calculations and allows EVSI to be used to prioritize research and design studies.

Comparing Modeling Approaches for Discrete Event Simulations With Competing Risks Based on Censored Individual Patient Data: A Simulation Study and Illustration in Colorectal Cancer

OBJECTIVES

This study aimed to provide detailed guidance on modeling approaches for implementing competing events in discrete event simulations based on censored individual patient data (IPD).

METHODS

The event-specific distributions (ESDs) approach sampled times from event-specific time-to-event distributions and simulated the first event to occur. The unimodal distribution and regression approach sampled a time from a combined unimodal time-to-event distribution, representing all events, and used a (multinomial) logistic regression model to select the event to be simulated. A simulation study assessed performance in terms of relative absolute event incidence difference and relative entropy of time-to-event distributions for different types and levels of right censoring, numbers of events, distribution overlap, and sample sizes. Differences in cost-effectiveness estimates were illustrated in a colorectal cancer case study.

RESULTS

Increased levels of censoring negatively affected the modeling approaches' performance. A lower number of competing events and higher overlap of distributions improved performance. When IPD were censored at random times, ESD performed best. When censoring occurred owing to a maximum follow-up time for 2 events, ESD performed better for a low level of censoring (ie, 10%). For 3 or 4 competing events, ESD better represented the probabilities of events, whereas unimodal distribution and regression better represented the time to events. Differences in cost-effectiveness estimates, both compared with no censoring and between approaches, increased with increasing censoring levels.

CONCLUSIONS

Modelers should be aware of the different modeling approaches available and that selection between approaches may be informed by data characteristics. Performing and reporting extensive validation efforts remains essential to ensure IPD are appropriately represented.

Lifetime Health and Economic Outcomes of Active Surveillance, Radical Prostatectomy, and Radiotherapy for Favorable-Risk Localized Prostate Cancer

OBJECTIVES

To estimate the lifetime health and economic outcomes of selecting active surveillance (AS), radical prostatectomy (RP), or radiation therapy (RT) as initial management for low- or favorable-risk localized prostate cancer.

METHODS

A discrete-event simulation model was developed using evidence from published randomized trials. Health outcomes were measured in life-years and quality-adjusted life-years (QALYs). Costs were included from a public payer perspective in Australian dollars. Outcomes were discounted at 5% over a lifetime horizon. Probabilistic and scenario analyses quantified parameter and structural uncertainty.

RESULTS

A total of 60% of patients in the AS arm eventually received radical treatment (surgery or radiotherapy) compared with 90% for RP and 91% for RT. Although AS resulted in fewer treatment-related complications, it led to increased clinical progression (AS 40.7%, RP 17.6%, RT 19.9%) and metastatic disease (AS 13.4%, RP 6.1%, RT 7.0%). QALYs were 10.88 for AS, 11.10 for RP, and 11.13 for RT. Total costs were A$17 912 for AS, A$15 609 for RP, and A$15 118 for RT. At a willingness to pay of A$20 000/QALY, RT had a 61.4% chance of being cost-effective compared to 38.5% for RP and 0.1% for AS.

CONCLUSIONS

Although AS resulted in fewer and delayed treatment-related complications, it was not found to be a cost-effective strategy for favorable-risk localized prostate cancer over a lifetime horizon because of an increase in the number of patients developing metastatic disease. RT was the dominant strategy yielding higher QALYs at lower cost although differences compared with RP were small.

Circulating Tumour DNA as a Potential Cost-Effective Biomarker to Reduce Adjuvant Chemotherapy Overtreatment in Stage II Colorectal Cancer

BACKGROUND AND OBJECTIVE

Substantial adjuvant chemotherapy (AC) overtreatment for stage II colorectal cancer results in a health and financial burden. Circulating tumour DNA (ctDNA) can improve patient selection for AC by detecting micro-metastatic disease. We estimated the health economic potential of ctDNA-guided AC for stage II colorectal cancer.

METHODS

A cost-utility analysis was performed to compare ctDNA-guided AC to standard of care, where 22.6% of standard of care patients and all ctDNA-positive patients (8.7% of tested patients) received AC and all ctDNA-negative patients (91.3%) did not. A third preference-sensitive ctDNA strategy was included where 6.8% of ctDNA-negative patients would receive AC. A state-transition model was populated using data from a prospective cohort study and clinical registries. Health and economic outcomes were discounted at 5% over a lifetime horizon from a 2019 Australian payer perspective. Extensive scenario and probabilistic analyses quantified model uncertainty.

RESULTS

Compared to standard of care, the ctDNA and preference-sensitive ctDNA strategies increased quality-adjusted life-years by 0.20 (95% confidence interval - 0.40 to 0.81) and 0.19 (- 0.40 to 0.78), and resulted in incremental costs of AUD - 4055 (- 16,853 to 8472) and AUD - 2284 (- 14,685 to 10,116), respectively. Circulating tumour DNA remained cost effective at a willingness to pay of AUD 20,000 per quality-adjusted life-year gained throughout most scenario analyses in which the proportion of ctDNA-positive patients cured by AC and compliance to a ctDNA-negative test results were decreased.

CONCLUSIONS

Circulating tumour-guided AC is a potentially cost-effective strategy towards reducing overtreatment in stage II colorectal cancer. Results from ongoing randomised clinical studies will be important to reduce uncertainty in the estimates.

Evaluating the resource implications of different service delivery models for offering additional genomic findings

PURPOSE

To evaluate the resource implications of different delivery models for the provision of additional findings (AF) in genomics from a health-care purchaser perspective.

METHODS

Data from the Additional Findings study were used to develop and validate a discrete event simulation model that represented the pathway of delivering AF. Resource implications were estimated by microcosting the consultations, sample verifications, bioinformatics, curation, and multidisciplinary case review meetings. A proof-of-concept model was used to generate costing, and then the simulation model was varied to assess the impact of an automated analysis pipeline, use of telehealth consultation, full automation with electronic decision support, and prioritizing case review for cases with pathogenic variants.

RESULTS

For the proof-of-concept delivery model, the average total cost to report AF was US$430 per patient irrespective of result pathogenicity (95% confidence interval [CI] US$375-US$489). However, the cost of per AF diagnosis was US$4349 (95% CI US$3794-US$4953). Alternative approaches to genetic counseling (telehealth, decision support materials) and to multidisciplinary case review (pathogenic AF cases only) lowered the total per patient cost of AF analysis and reporting by 41-51%.

CONCLUSION

Resources required to provide AF can be reduced substantially by implementing alternative approaches to counseling and multidisciplinary case review.

Simulating Progression-Free and Overall Survival for First-Line Doublet Chemotherapy With or Without Bevacizumab in Metastatic Colorectal Cancer Patients Based on Real-World Registry Data

BACKGROUND

Simulation models utilizing real-world data have potential to optimize treatment sequencing strategies for specific patient subpopulations, including when conducting clinical trials is not feasible. We aimed to develop a simulation model to estimate progression-free survival (PFS) and overall survival for first-line doublet chemotherapy with or without bevacizumab for specific subgroups of metastatic colorectal cancer (mCRC) patients based on registry data.

METHODS

Data from 867 patients were used to develop two survival models and one logistic regression model that populated a discrete event simulation (DES). Discrimination and calibration were used for internal validation of these models separately and predicted and observed medians and Kaplan-Meier plots were compared for the integrated DES. Bootstrapping was performed to correct for optimism in the internal validation and to generate correlated sets of model parameters for use in a probabilistic analysis to reflect parameter uncertainty.

RESULTS

The survival models showed good calibration based on the regression slopes and modified Hosmer-Lemeshow statistics at 1 and 2 years, but not for short-term predictions at 0.5 years. Modified C-statistics indicated acceptable discrimination. The simulation estimated that median first-line PFS (95% confidence interval) of 219 (25%) patients could be improved from 175 days (156-199) to 269 days (246-294) if treatment would be targeted based on the highest expected PFS.

CONCLUSIONS

Extensive internal validation showed that DES accurately estimated the outcomes of treatment combination strategies for specific subpopulations, with outcomes suggesting treatment could be optimized. Although results based on real-world data are informative, they cannot replace randomized trials.

Health Economic Models for Metastatic Colorectal Cancer: A Methodological Review

OBJECTIVE

The aim of this systematic review was to provide a comprehensive and detailed review of structural and methodological assumptions in model-based cost-effectiveness analyses of systemic metastatic colorectal cancer (mCRC) treatments, and discuss their potential impact on health economic outcome estimates.

METHODS

Five databases (EMBASE, MEDLINE, Cochrane Library, Health Technology Assessment and National Health Service Health Economic Evaluation Database) were searched on 26 August 2019 for model-based full health economic evaluations of systemic mCRC treatment using a combination of free-text terms and subject headings. Full-text publications in English were eligible for inclusion if they were published in or after the year 2000. The Consolidated Health Economic Evaluation Reporting Standards checklist was used to assess the reporting quality of included publications. Study selection, appraisal and data extraction were performed by two reviewers independently.

RESULTS

The search yielded 1418 publications, of which 54 were included, representing 51 unique studies. Most studies focused on first-line treatment (n = 29, 57%), followed by third-line treatment (n = 13, 25%). Model structures were health-state driven (n = 27, 53%), treatment driven (n = 19, 37%), or a combination (n = 5, 10%). Cohort-level state-transition modelling (STM) was the most common technique (n = 33, 65%), followed by patient-level STM and partitioned survival analysis (both n = 6, 12%). Only 15 studies (29%) reported some sort of model validation. Health economic outcomes for specific strategies differed substantially between studies. For example, survival following first-line treatment with fluorouracil, leucovorin and oxaliplatin ranged from 1.21 to 7.33 years, with treatment costs ranging from US$8125 to US$126,606.

CONCLUSIONS

Model-based cost-effectiveness analyses of systemic mCRC treatments have adopted varied modelling methods and structures, resulting in substantially different outcomes. As models generally focus on first-line treatment without consideration of downstream treatments, there is a profound source of structural uncertainty implying that the cost-effectiveness of treatments across the mCRC pathway remains uncertain.

Addressing Challenges of Economic Evaluation in Precision Medicine Using Dynamic Simulation Modeling

OBJECTIVES

The objective of this article is to describe the unique challenges and present potential solutions and approaches for economic evaluations of precision medicine (PM) interventions using simulation modeling methods.

METHODS

Given the large and growing number of PM interventions and applications, methods are needed for economic evaluation of PM that can handle the complexity of cascading decisions and patient-specific heterogeneity reflected in the myriad testing and treatment pathways. Traditional approaches (eg, Markov models) have limitations, and other modeling techniques may be required to overcome these challenges. Dynamic simulation models, such as discrete event simulation and agent-based models, are used to design and develop mathematical representations of complex systems and intervention scenarios to evaluate the consequence of interventions over time from a systems perspective.

RESULTS

Some of the methodological challenges of modeling PM can be addressed using dynamic simulation models. For example, issues regarding companion diagnostics, combining and sequencing of tests, and diagnostic performance of tests can be addressed by capturing patient-specific pathways in the context of care delivery. Issues regarding patient heterogeneity can be addressed by using patient-level simulation models.

CONCLUSION

The economic evaluation of PM interventions poses unique methodological challenges that might require new solutions. Simulation models are well suited for economic evaluation in PM because they enable patient-level analyses and can capture the dynamics of interventions in complex systems specific to the context of healthcare service delivery.

Comparing Strategies for Modeling Competing Risks in Discrete-Event Simulations: A Simulation Study and Illustration in Colorectal Cancer

BACKGROUND

Different strategies toward implementing competing risks in discrete-event simulation (DES) models are available. This study aims to provide recommendations regarding modeling approaches that can be defined based on these strategies by performing a quantitative comparison of alternative modeling approaches.

METHODS

Four modeling approaches were defined: 1) event-specific distribution (ESD), 2) event-specific probability and distribution (ESPD), 3) unimodal joint distribution and regression model (UDR), and 4) multimodal joint distribution and regression model (MDR). Each modeling approach was applied to uncensored individual patient data in a simulation study and a case study in colorectal cancer. Their performance was assessed in terms of relative event incidence difference, relative absolute event incidence difference, and relative entropy of time-to-event distributions. Differences in health economic outcomes were also illustrated for the case study.

RESULTS

In the simulation study, the ESPD and MDR approaches outperformed the ESD and UDR approaches, in terms of both event incidence differences and relative entropy. Disease pathway and data characteristics, such as the number of competing risks and overlap between competing time-to-event distributions, substantially affected the approaches' performance. Although no considerable differences in health economic outcomes were observed, the case study showed that the ESPD approach was most sensitive to low event rates, which negatively affected performance.

CONCLUSIONS

Based on overall performance, the recommended modeling approach for implementing competing risks in DES models is the MDR approach, which is defined according to the general strategy of selecting the time-to-event first and the corresponding event second. The ESPD approach is a less complex and equally performing alternative if sufficient observations are available for each competing event (i.e., the internal validity shows appropriate data representation).

Indirect Economic Impact of Chronic Pain on Education Workers: A Company Perspective

OBJECTIVE

The aim of this study was to estimate indirect cost (IC) related to chronic pain (CP) from an employer's perspective.

METHODS

A cost-of-illness study was performed on active workers and retirees due to CP, between October 2017 and March 2018, in one of the Brazilian public universities. IC was measured as a sum of absenteeism, presenteeism, and disability pensions. The analysis of factors associated with IC was based on Tweedie model.

RESULTS

CP had an average IC of R$9258.20 [95% confidence interval (95% CI) = 6907.37 to 11,950.17], which generates an impact of 6.42 million (95% CI = 4.37 to 10.99) per year, corresponding to 3.42% (95% CI = 2.33 to 5.85) of the payroll. The position (Measure of 2.00, 95% CI = 1.19 to 3.38) and pain intensity (Measure of 1.15; 95% CI = 1.02 to 1.30) presented independent association.

CONCLUSION

CP generates the high levels of IC for the education's employer. There is an urgent need to implement prevention programs aimed at improving CP control in the workplace.

Estimating Maximum Daily Precipitation in the Upper Vistula Basin, Poland

The aim of this study was to determine the best probability distributions for calculating the maximum annual daily precipitation with the specific probability of exceedance (Pmaxp%). The novelty of this study lies in using the peak-weighted root mean square error (PWRMSE), the root mean square error (RMSE), and the coefficient of determination (R2) for assessing the fit of empirical and theoretical distributions. The input data included maximum daily precipitation records collected in the years 1971–2014 at 51 rainfall stations from the Upper Vistula Basin, Southern Poland. The value of Pmaxp% was determined based on the following probability distributions of random variables: Pearson’s type III (PIII), Weibull’s (W), log-normal, generalized extreme value (GEV), and Gumbel’s (G). Our outcomes showed a lack of significant trends in the observation series of the investigated random variables for a majority of the rainfall stations in the Upper Vistula Basin. We found that the peak-weighted root mean square error (PWRMSE) method, a commonly used metric for quality assessment of rainfall-runoff models, is useful for identifying the statistical distributions of the best fit. In fact, our findings demonstrated the consistency of this approach with the RMSE goodness-of-fit metrics. We also identified the GEV distribution as recommended for calculating the maximum daily precipitation with the specific probability of exceedance in the catchments of the Upper Vistula Basin.

Matching the model with the evidence: comparing discrete event simulation and state-transition modeling for time-to-event predictions in a cost-effectiveness analysis of treatment in metastatic colorectal cancer patients

BACKGROUND

Individual patient data, e.g. from clinical trials, often need to be extrapolated or combined with additional evidence when assessing long-term impact in cost-effectiveness modeling studies. Different modeling methods can be used to represent the complex dynamics of clinical practice; the choice of which may impact cost-effectiveness outcomes. We compare the use of a previously designed cohort discrete-time state-transition model (DT-STM) with a discrete event simulation (DES) model.

METHODS

The original DT-STM was replicated and a DES model developed using AnyLogic software. Models were populated using individual patient data of a phase III study in metastatic colorectal cancer patients, and compared based on their evidence structure, internal validity, and cost-effectiveness outcomes. The DT-STM used time-dependent transition probabilities, whereas the DES model was populated using parametric distributions.

RESULTS

The estimated time-dependent transition probabilities for the DT-STM were irregular and more sensitive to single events due to the required small cycle length and limited number of event observations, whereas parametric distributions resulted in smooth time-to-event curves for the DES model. Although the DT-STM and DES model both yielded similar time-to-event curves, the DES model represented the trial data more accurately in terms of mean health-state durations. The incremental cost-effectiveness ratio (ICER) was €172,443 and €168,383 per Quality Adjusted Life Year gained for the DT-STM and DES model, respectively.

CONCLUSION

DES represents time-to-event data from clinical trials more naturally and accurately than DT-STM when few events are observed per time cycle. As a consequence, DES is expected to yield a more accurate ICER.