A Flexible Open-Source Decision Model for Value Assessment of Biologic Treatment for Rheumatoid Arthritis

Value-based evidence across health care sectors: a push for transparent real-world studies, data, and evidence dissemination

There is currently a heightened need for transparency in pharmaceutical sectors. The inclusion of real-world (RW) evidence, in addition to clinical trial evidence, in decision-making processes, was an important step forward toward a more inclusive established value proposition. This advance has introduced new transparency challenges. Increasing transparency is a critical step toward accelerating improvement in type, quality, and access to data, regardless of whether these originate from clinical trials or from RW studies. However, so far, advances in transparency have been relatively restricted to clinical trials, and there remains a lack of similar expectations or standards of transparency concerning the generation and reporting of RW data. This perspective paper aims to highlight the need for transparency concerning RW studies, data, and evidence across health care sectors, to identify areas for improvement, and provide concrete recommendations and practices for the future. Specific issues are discussed from different stakeholder perspectives, culminating in recommended actions, from individual stakeholder perspectives, for improved RW study, data, and evidence transparency. Furthermore, a list of potential guidelines for consideration by stakeholders is proposed. While recommendations from different stakeholder perspectives are made, true transparency in the processes involved in the generation, reporting, and use of RW evidence will require a concerted effort from all stakeholders across health care sectors.

Sequences of biological treatments for patients with moderate-to-severe rheumatoid arthritis in the era of treat-to-target in China: a cost-effectiveness analysis

BACKGROUND

Biologic disease-modifying antirheumatic drugs (bDMARDs) are recommended to be added in sequentially in the treatment of moderate-to-severe rheumatoid arthritis (RA). All these drugs, however, are substantially more expensive than conventional synthetic DMARDs throughout the world, including in China. The objective of this study is to evaluate the cost-effectiveness of treatment sequences of bDMARDs for patients with moderate-to-severe rheumatoid arthritis from the Chinese healthcare system perspective.

METHODS

An individual patient simulation model was used to track the course of patients from first treatment through switches to further lines in a sequence. The comparator treatment sequence commenced with methotrexate, followed by non-biologic therapy. The intervention sequences were assumed to be the combinations of bDMARDs available, followed by non-biologic therapy. Life-years, quality-adjusted life-years (QALYs), and lifetime costs were estimated. Univariable and probabilistic sensitivity analyses and scenario analyses were performed to evaluate the model uncertainty.

RESULTS

Compared with the comparator treatment sequence, bDMARDs sequences were associated with more life years, QALYs, and cost. These produced ICERs ranged from $27,441.36/QALY to $40,149.2/QALY, above the willingness-to-pay threshold of $10,378 per QALY. The uncertainty analyses and the scenario analyses confirmed the result of the base case analysis.

CONCLUSIONS

From the perspective of the Chinese healthcare system, bDMARDs sequences are estimated not to be cost-effective compared with conventional synthetic disease-modifying antirheumatic drug strategy for patients with moderate-to-severe RA at a WTP threshold of $10,378 per QALY. Price reductions are warranted to make bDMARDs cost-effective and affordable.

Tofacitinib in the Treatment of Moderate-to-Severe Rheumatoid Arthritis in China: A Cost-Effectiveness Analysis Based on a Mapping Algorithm Derived from a Chinese Population

INTRODUCTION

To estimate the cost-effectiveness of tofacitinib for patients with moderate-to-severe rheumatoid arthritis (RA) who failed conventional synthetic disease-modifying antirheumatic drugs from the Chinese healthcare system perspective.

METHODS

An individual patient simulation model was used to estimate the lifetime cost and effectiveness. The comparator sequence commenced with etanercept, followed by rituximab-tocilizumab- non-biologic therapy. The intervention sequences were assumed to add tofacitinib to different positions in the comparator sequence. Quality-of-life estimates were generated by mapping Health Assessment Questionnaire scores to utility with the algorithm derived from a Chinese population. Scenario analyses, univariable and probabilistic sensitivity analyses were performed to evaluate the model uncertainty.

RESULTS

Compared with the comparator sequence, patients receiving tofacitinib as the first-, second-, third- and fourth-line treatment gained additional 0.49, 0.59, 0.44 and 0.53 QALYs, respectively, and the use of tofacitinib as the first- and second-line treatment was less costly, whereas the use of tofacitinib as the third- and fourth-line treatment cost an additional $234,998 and $381,116, respectively. This produced an incremental cost-effectiveness ratio of $333.73 and $9669.34/QALY, respectively.

CONCLUSION

Tofacitinib is estimated to be dominant in both the first- and second-line settings and to be highly cost-effective in both the third- and fourth-line settings.

Cost-Effectiveness of Baricitinib for Patients with Moderate-to-Severe Rheumatoid Arthritis After Methotrexate Failed in China

INTRODUCTION

A phase 3 (RA-BEAM study) clinical trial reported that baricitinib (BCT) + methotrexate (MTX) had clinical improvement compared with adalimumab (ADA) + MTX as a first-line strategy in patients with rheumatoid arthritis (RA) who had inadequate responses to MTX monotherapy. However, from the perspective of the Chinese healthcare system, the cost-effectiveness of introducing BCT into current treatment for patients with RA unresponsive to MTX remains unclear.

METHODS

A patient-level microsimulation model was used to extrapolate the lifetime incremental cost per quality-adjusted life-year (QALY) and other outcomes. This study compared treatment sequences with or without first-line BCT with current treatment sequences, including adalimumab, etanercept, tocilizumab, and palliative care. Effectiveness and physical function were assessed using the American College of Rheumatology (ACR) 20/50/70 response and Health Assessment Questionnaire (HAQ). The input parameters of the model, comprising patient characteristics (sex and age) and treatment efficacy (ACR responses and HAQ score), were derived from a phase III clinical trial and network meta-analysis. The total cost estimation included direct costs and indirect costs. Probabilistic and univariate sensitivity analyses were performed, as were a series of scenario analyses.

RESULTS

The lifetime analysis revealed that adding BCT as a first-line treatment resulted in a QALY gain of 2.66 years; this gain would cost an incremental $26,662, leading to an incremental cost-effectiveness ratio of $10,036/QALY per patient compared with the current treatment sequence. Sensitivity and scenario analyses showed the results to be robust.

CONCLUSIONS

From a Chinese payer perspective, the introduction of BCT into the current treatment sequence is projected to be a cost-effective option as first-, second-, third-, and fourth-line treatment for patients with moderate-to-severe RA.

Cost-effectiveness of Triple Therapy vs. Biologic Treatment Sequence as First-line Therapy for Rheumatoid Arthritis Patients after Methotrexate Failure

Introduction

A clinical trial (RACAT) reported the noninferiority of triple therapy compared to biologic agents (etanercept + methotrexate), and previous studies confirmed that biologic disease-modifying antirheumatic drugs (bDMARDs) are more expensive but less beneficial than triple therapy for patients with rheumatoid arthritis (RA) in whom methotrexate (MTX) fails. However, from the perspective of the Chinese healthcare system, the cost-effectiveness of triple therapy versus bDMARD treatment sequences as a first-line therapy for patients with RA is still unclear.

Methods

An individual patient simulation model was used to extrapolate the lifetime cost and health outcomes by tracing patients from initial treatment through switches to further treatment lines in a sequence. Therapeutic efficacy and physical function were evaluated using the American College of Rheumatology (ACR) response, 28-Joint Disease Activity Score (DAS28), and Health Assessment Questionnaire score. All input parameters in the model were derived from published studies, national databases, local hospitals, and experts’ opinions. Both direct costs and indirect costs were taken into consideration. Probabilistic and one-way sensitivity analyses were performed to test the uncertainty of the model, as were multiple scenario analyses.

Results

The lifetime analysis demonstrated that triple therapy was associated with lower costs and quality-adjusted life years (QALYs) than bDMARD sequences. These resulted in incremental cost-effectiveness ratios (ICERs) ranging from $87,090/QALY to $104,032/QALY, higher than the willingness-to-pay (WTP) threshold in China ($30,950/QALY). The baseline DAS28 impacted the model outcomes the most. Scenario analyses indicated that adding triple therapy to bDMARD sequences as a first-, second-, third-, or fourth-line therapy is very cost-effective, at a WTP of $10,316/QALY.

Conclusions

From a Chinese payer perspective, triple therapy as first-line treatment in treatment sequence could be regarded as cost-effectiveness option for patients who failed MTX, compared to bDMARDs as first-line treatment, and instead of prescribing triple therapy as a substitute for bDMARDs as a first-line treatment, adding triple therapy to the bDMARD treatment sequence is likely to be very cost-effective for patients with active RA compared to bDMARD sequences.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40744-021-00300-4.

The Effect of Dose Escalation on the Cost-Effectiveness of Etanercept and Adalimumab with Methotrexate Among Patients with Moderate to Severe Rheumatoid Arthritis

BACKGROUND

Patients with moderate to severe rheumatoid arthritis (RA) occasionally increase their doses of tumor necrosis factor (TNF) inhibitors, especially the monoclonal antibody origin drugs such as adalimumab and infliximab, after inadequate response to the initial dose. Previous studies have evaluated the cost-effectiveness of various sequences of treatment for RA in the United States but have not considered the effect of dose escalation.

OBJECTIVE

To assess the cost-effectiveness of etanercept and adalimumab by incorporating the effect of dose escalation in moderate to severe RA patients.

METHODS

We adapted the open-source Innovation and Value Initiative - Rheumatoid Arthritis model, version 1.0 to separately simulate the magnitude and time to dose escalation among RA patients taking adalimumab plus methotrexate or etanercept plus methotrexate from a societal perspective and lifetime horizon. An important assumption in the model was that dose escalation would increase treatment costs through its effect on the number of doses but would have no effect on effectiveness. We estimated the dose escalation parameters using the IBM MarketScan Commercial and Medicare Supplemental Databases. We fit competing parametric survival models to model time to dose escalation and used model diagnostics to compare the fit of the competing models. We measured the magnitude of dose escalation as the percentage increase in the number of doses conditional on dose escalation. Finally, we used the parameterized model to simulate treatment sequences beginning with a TNF inhibitor (adalimumab, etanercept) followed by nonbiologic treatment.

RESULTS

In baseline models without dose escalation, the incremental cost per quality-adjusted life-year of the etanercept treatment sequence relative to the adalimumab treatment sequence was $85,593. Incorporating dose escalation increased treatment costs for each sequence, but costs increased more with adalimumab, lowering the incremental cost-effectiveness ratio to $9,001. At willingness-to-pay levels of $100,000, the etanercept sequence was more cost-effective compared with the adalimumab sequence, with probability 0.55 and 0.85 in models with and without dose escalation, respectively.

CONCLUSIONS

Dose escalation has important effects on cost-effectiveness and should be considered when comparing biologic medications for the treatment of RA.

DISCLOSURES

Funding for this study was contributed by Amgen. When this work was conducted, Incerti and Jansen were employees of Precision Health Economics, which received financial support from Amgen. Maksabedian Hernandez, Collier, Gharaibeh, and Stolshek were employees and stockholders of Amgen, and Tkacz and Moore-Schiltz were employees of IBM Watson Health, which received financial support from Amgen. Some of the results of this work were previously presented as a poster at the 2019 AMCP Managed Care & Specialty Pharmacy Annual Meeting, March 25-28, 2019, in San Diego, CA.

Making health economic models Shiny: A tutorial

Health economic evaluation models have traditionally been built in Microsoft Excel, but more sophisticated tools are increasingly being used as model complexity and computational requirements increase. Of all the programming languages, R is most popular amongst health economists because it has a plethora of user created packages and is highly flexible. However, even with an integrated development environment such as R Studio, R lacks a simple point and click user interface and therefore requires some programming ability. This might make the switch from Microsoft Excel to R seem daunting, and it might make it difficult to directly communicate results with decisions makers and other stakeholders. The R package Shiny has the potential to resolve this limitation. It allows programmers to embed health economic models developed in R into interactive web browser based user interfaces. Users can specify their own assumptions about model parameters and run different scenario analyses, which, in the case of regular a Markov model, can be computed within seconds. This paper provides a tutorial on how to wrap a health economic model built in R into a Shiny application. We use a four-state Markov model developed by the Decision Analysis in R for Technologies in Health (DARTH) group as a case-study to demonstrate main principles and basic functionality. A more extensive tutorial, all code, and data are provided in a  GitHub repository.

R and Shiny for Cost-Effectiveness Analyses: Why and When? A Hypothetical Case Study

INTRODUCTION

Health economics models are typically built in Microsoft Excel^® owing to its wide familiarity, accessibility and perceived transparency. However, given the increasingly rapid and analytically complex decision-making needs of both the pharmaceutical industry and the field of health economics and outcomes research (HEOR), the demands of cost-effectiveness analyses may be better met by the programming language R.

OBJECTIVE

This case study provides an explicit comparison between Excel and R for contemporary cost-effectiveness analysis.

METHODS

We constructed duplicate cost-effectiveness models using Excel and R (with a user interface built using the Shiny package) to address a hypothetical case study typical of contemporary health technology assessment.

RESULTS

We compared R and Excel versions of the same model design to determine the advantages and limitations of the modelling platforms in terms of (i) analytical capability, (ii) data safety, (iii) building considerations, (iv) usability for technical and non-technical users and (v) model adaptability.

CONCLUSIONS

The findings of this explicit comparison are used to produce recommendations for when R might be more suitable than Excel in contemporary cost-effectiveness analyses. We conclude that selection of appropriate modelling software needs to consider case-by-case modelling requirements, particularly (i) intended audience, (ii) complexity of analysis, (iii) nature and frequency of updates and (iv) anticipated model run time.

More Unnecessary Imaginary Worlds - Part 1: The Institute for Clinical and Economic Review's Evidence Report on Janus Kinase (JAK) Inhibitors in Rheumatoid Arthritis

Previous commentaries in the Formulary Evaluation section of INNOVATIONS in Pharmacy have pointed to the lack of credibility in modeled claims for cost-effectiveness and associated recommendations for pricing by the Institute for Clinical and Economic Review (ICER). The principal objection to ICER reports has been that their modeled claims fail the standards of normal science: they are best seen as pseudoscience. The purpose of this latest commentary is to consider the recently released ICER evidence report for Janus Kinase (JAK) Inhibitors. As ICER continues, in the case of JAK Inhibitors, to apply its modeled cost utility framework with consequent recommendations for pricing adjustments, these recommendations also lack credibility. In contrast with previous ICER evidence reports, the present report adopts only a 12-month timeframe, one due, in large part, to ICER being unable to justify assumptions to drive its construction of imaginary worlds beyond 12 months. This commentary emphasizesagain, why the ICER methodology fails to meet the standards of normal science. Claims made by ICER for the competing JAK Inhibitor therapies lack credibility, are impossible to evaluate, let alone replicate across treatment settings. Even so, it is important to examine a number of key elements in the ICER invention of the 12-month JAK Inhibitor imaginary world. While this does not imply any degree of acceptance of the ICER methodology, one element that merits particular attention is thefailure of the ICER modeling to meet logically defensible measurement standards in its application of generic health related quality of life (HRQoL) ordinal metrics to create its QALY claims. The failure to meet the required standards of fundamental measurement means that the cost-per-QALY claims are invalid. This raises the issue of the application of Rasch Measurement Theory (RMT) in instrument development and the potential role of patient centric outcome (PCO) instruments that represent the patient voice in value claims. The case made here is that the ICER approach should be abandoned as an unnecessary distraction. If we are to meet standards for the discovery of new facts in therapy response then our focus must be on proposing credible, evaluable and replicable claims within disease states. Instruments, such as the Rheumatoid Arthritis Quality of Life (RAQoL)questionnaire that build on the common construct that QoL is the extent to which human needs are fulfilled should be the basis for value claims. HRQoL Instruments that are clinically focused and reflect the value calculus of providers and not patients in measuring response by symptoms and activity limitations are irrelevant. This puts to one side the belief that incremental cost-per-QALY models, the construction of imaginary worlds are, in any sense, a 'gold standard'; a meme embraced by the health technology assessment profession. Claims for incremental cost per QALY outcomes and recommendations for pricing and access driven by willingness to pay thresholds are irrelevant to formulary decisions.

Developing Open-Source Models for the US Health System: Practical Experiences and Challenges to Date with the Open-Source Value Project

The Innovation and Value Initiative started the Open-Source Value Project with the aim to improve the credibility and relevance of model-based value assessment in the context of the US healthcare environment. As a core activity of the Open-Source Value Project, the Innovation and Value Initiative develops and provides access to flexible open-source economic models that are developed iteratively based on public feedback and input. In this article, we describe our experience to date with the development of two currently released, Open-Source Value Project models, one in rheumatoid arthritis and one in epidermal growth factor receptor-positive non-small-cell lung cancer. We developed both Open-Source Value Project models using the statistical programming language R instead of spreadsheet software (i.e., Excel), which allows the models to capture multiple model structures, model sequential treatment with individual patient simulations, and improve integration with formal evidence synthesis. By developing the models in R, we were also able to use version control systems to manage changes to the source code, which is needed for iterative and collaborative model development. Similarly, Open-Source Value Project models are freely available to the public to provide maximum transparency and facilitate collaboration. Development of the rheumatoid arthritis and non-small-cell lung cancer model platforms has presented multiple challenges. The development of multiple components of the model platform tailored to different audiences, including web interfaces, required more resources than a cost-effectiveness analysis for a publication would. Furthermore, we faced methodological hurdles, in particular related to the incorporation of multiple competing model structures and novel elements of value. The iterative development based on public feedback also posed some challenges during the review phase, where methodological experts did not always understand feedback from clinicians and vice versa. Response to the Open-Source Value Project by the modeling community and patient organizations has been positive, but feedback from US decision makers has been limited to date. As we progress with this project, we hope to learn more about the feasibility, benefits, and challenges of an open-source and collaborative approach to model development for value assessment.

Transparency in Decision Modelling: What, Why, Who and How?

Transparency in decision modelling is an evolving concept. Recently, discussion has moved from reporting standards to open-source implementation of decision analytic models. However, in the debate about the supposed advantages and disadvantages of greater transparency, there is a lack of definition. The purpose of this article is not to present a case for or against transparency, but rather to provide a more nuanced understanding of what transparency means in the context of decision modelling and how it could be addressed. To this end, we review and summarise the discourse to date, drawing on our collective experience. We outline a taxonomy of the different manifestations of transparency, including reporting standards, reference models, collaboration, model registration, peer review and open-source modelling. Further, we map out the role and incentives for the various stakeholders, including industry, research organisations, publishers and decision makers. We outline the anticipated advantages and disadvantages of greater transparency with respect to each manifestation, as well as the perceived barriers and facilitators to greater transparency. These are considered with respect to the different stakeholders and with reference to issues including intellectual property, legality, standards, quality assurance, code integrity, health technology assessment processes, incentives, funding, software, access and deployment options, data protection and stakeholder engagement. For each manifestation of transparency, we discuss the 'what', 'why', 'who' and 'how'. Specifically, their meaning, why the community might (or might not) wish to embrace them, whose engagement as stakeholders is required and how relevant objectives might be realised. We identify current initiatives aimed to improve transparency to exemplify efforts in current practice and for the future.

A Need for Change! A Coding Framework for Improving Transparency in Decision Modeling

The use of open-source programming languages, such as R, in health decision sciences is growing and has the potential to facilitate model transparency, reproducibility, and shareability. However, realizing this potential can be challenging. Models are complex and primarily built to answer a research question, with model sharing and transparency relegated to being secondary goals. Consequently, code is often neither well documented nor systematically organized in a comprehensible and shareable approach. Moreover, many decision modelers are not formally trained in computer programming and may lack good coding practices, further compounding the problem of model transparency. To address these challenges, we propose a high-level framework for model-based decision and cost-effectiveness analyses (CEA) in R. The proposed framework consists of a conceptual, modular structure and coding recommendations for the implementation of model-based decision analyses in R. This framework defines a set of common decision model elements divided into five components: (1) model inputs, (2) decision model implementation, (3) model calibration, (4) model validation, and (5) analysis. The first four components form the model development phase. The analysis component is the application of the fully developed decision model to answer the policy or the research question of interest, assess decision uncertainty, and/or to determine the value of future research through value of information (VOI) analysis. In this framework, we also make recommendations for good coding practices specific to decision modeling, such as file organization and variable naming conventions. We showcase the framework through a fully functional, testbed decision model, which is hosted on GitHub for free download and easy adaptation to other applications. The use of this framework in decision modeling will improve code readability and model sharing, paving the way to an ideal, open-source world.

Transparency in Health Economic Modeling: Options, Issues and Potential Solutions

Economic models are increasingly being used by health economists to assess the value of health technologies and inform healthcare decision making. However, most published economic models represent a kind of black box, with known inputs and outputs but undisclosed internal calculations and assumptions. This lack of transparency makes the evaluation of the model results challenging, complicates comparisons between models, and limits the reproducibility of the models. Here, we aim to provide an overview of the possible steps that could be undertaken to make economic models more transparent and encourage model developers to share more detailed calculations and assumptions with their peers. Scenarios with different levels of transparency (i.e., how much information is disclosed) and reach of transparency (i.e., who has access to the disclosed information) are discussed, and five key concerns (copyrights, model misuse, confidential data, software, and time/resources) pertaining to model transparency are presented, along with possible solutions. While a shift toward open-source models is underway in health economics, as has happened before in other research fields, the challenges ahead should not be underestimated. Importantly, there is a pressing need to find an acceptable trade-off between the added value of model transparency and the time and resources needed to achieve such transparency. To this end, it will be crucial to set incentives at different stakeholder levels. Despite the many challenges, the many benefits of publicly sharing economic models make increased transparency a goal worth pursuing.