Estimation of the transition matrix of a discrete-time Markov chain

Comparative effectiveness and cost-effectiveness of natalizumab and fingolimod in rapidly evolving severe relapsing-remitting multiple sclerosis in the United Kingdom

AIM

To evaluate the real-world comparative effectiveness and the cost-effectiveness, from a UK National Health Service perspective, of natalizumab versus fingolimod in patients with rapidly evolving severe relapsing-remitting multiple sclerosis (RES-RRMS).

METHODS

Real-world data from the MSBase Registry were obtained for patients with RES-RRMS who were previously either naive to disease-modifying therapies or had been treated with interferon-based therapies, glatiramer acetate, dimethyl fumarate, or teriflunomide (collectively known as BRACETD). Matched cohorts were selected by 3-way multinomial propensity score matching, and the annualized relapse rate (ARR) and 6-month-confirmed disability worsening (CDW6M) and improvement (CDI6M) were compared between treatment groups. Comparative effectiveness results were used in a cost-effectiveness model comparing natalizumab and fingolimod, using an established Markov structure over a lifetime horizon with health states based on the Expanded Disability Status Scale. Additional model data sources included the UK MS Survey 2015, published literature, and publicly available sources.

RESULTS

In the comparative effectiveness analysis, we found a significantly lower ARR for patients starting natalizumab compared with fingolimod (rate ratio [RR] = 0.65; 95% confidence interval [CI], 0.57-0.73) or BRACETD (RR = 0.46; 95% CI, 0.42-0.53). Similarly, CDI6M was higher for patients starting natalizumab compared with fingolimod (hazard ratio [HR] = 1.25; 95% CI, 1.01-1.55) and BRACETD (HR = 1.46; 95% CI, 1.16-1.85). In patients starting fingolimod, we found a lower ARR (RR = 0.72; 95% CI, 0.65-0.80) compared with starting BRACETD, but no difference in CDI6M (HR = 1.17; 95% CI, 0.91-1.50). Differences in CDW6M were not found between the treatment groups. In the base-case cost-effectiveness analysis, natalizumab dominated fingolimod (0.302 higher quality-adjusted life-years [QALYs] and £17,141 lower predicted lifetime costs). Similar cost-effectiveness results were observed across sensitivity analyses.

CONCLUSIONS

This MSBase Registry analysis suggests that natalizumab improves clinical outcomes when compared with fingolimod, which translates to higher QALYs and lower costs in UK patients with RES-RRMS.

Utilizing Clinical Trial Data to Assess Timing of Surgical Treatment for Emphysema Patients

BACKGROUND

Lung volume reduction surgery (LVRS) and medical therapy are 2 available treatment options in dealing with severe emphysema, which is a chronic lung disease. However, or there are currently limited guidelines on the timing of LVRS for patients with different characteristics.

OBJECTIVE

The objective of this study is to assess the timing of receiving LVRS in terms of patient outcomes, taking into consideration a patient's characteristics.

METHODS

A finite-horizon Markov decision process model for patients with severe emphysema was developed to determine the short-term (5 y) and long-term timing of emphysema treatment. Maximizing the expected life expectancy, expected quality-adjusted life-years, and total expected cost of each treatment option were applied as the objective functions of the model. To estimate parameters in the model, the data provided by the National Emphysema Treatment Trial were used.

RESULTS

The results indicate that the treatment timing strategy for patients with upper-lobe predominant emphysema is to receive LVRS regardless of their specific characteristics. However, for patients with non-upper-lobe-predominant emphysema, the optimal strategy depends on the age, maximum workload level, and forced expiratory volume in 1 second level.

CONCLUSION

This study demonstrates the utilization of clinical trial data to gain insights into the timing of surgical treatment for patients with emphysema, considering patient age, observable health condition, and location of emphysema.

HIGHLIGHTS

Both short-term and long-term Markov decision process models were developed to assess the timing of receiving lung volume reduction surgery in patients with severe emphysema.How clinical trial data can be used to estimate the parameters and obtain short-term results from the Markov decision process model is demonstrated.The results provide insights into the timing of receiving lung volume reduction surgery as a function of a patient's characteristics, including age, emphysema location, maximum workload, and forced expiratory volume in 1 second level.

Networked control strategy of dual linear switched reluctance motors based time delay tracking system

To reduce the influence of time delay on the tracking performance of a direct-drive motion control system, this paper concentrates on stability analysis and cooperative position tracking control issues for the dual linear switched reluctance motors (LSRMs) system with network-induced time delays. The closed-loop network control system (NCS) is constructed by modeling random and bounded network-induced time delays existing in forward and feedback channels as a discrete-time Markov chain. Incremental time delay information is introduced in Lyapunov functional analysis to satisfy the high-precision movement of master and secondary motors and improve the system control performance. The time delay compensation method is proposed to compensate for the damage to the networked control system caused by random delay. With Lyapunov stability theory and LMI are applied, stability and stabilization conditions with less computational complexity and low conservatism are obtained based on incremental time delay information insertion. Finally, the numerical simulation and the experimental platform of the motor control system are built. Simulation and experiment results demonstrate that the networked control strategy can compensate the negative impact of delay on the tracking performance of LSRMs based motion control system.

Comparative Effectiveness and Cost-Effectiveness of Natalizumab and Fingolimod in Patients with Inadequate Response to Disease-Modifying Therapies in Relapsing-Remitting Multiple Sclerosis in the United Kingdom

BACKGROUND

Patients with highly active relapsing-remitting multiple sclerosis inadequately responding to first-line therapies (interferon-based therapies, glatiramer acetate, dimethyl fumarate, and teriflunomide, known collectively as "BRACETD") often switch to natalizumab or fingolimod.

OBJECTIVE

The aim was to estimate the comparative effectiveness of switching to natalizumab or fingolimod or within BRACETD using real-world data and to evaluate the cost-effectiveness of switching to natalizumab versus fingolimod using a United Kingdom (UK) third-party payer perspective.

METHODS

Real-world data were obtained from MSBase for patients relapsing on BRACETD in the year before switching to natalizumab or fingolimod or within BRACETD. Three-way-multinomial-propensity-score-matched cohorts were identified, and comparisons between treatment groups were conducted for annualised relapse rate (ARR) and 6-month-confirmed disability worsening (CDW6M) and improvement (CDI6M). Results were applied in a cost-effectiveness model over a lifetime horizon using a published Markov structure with health states based on the Expanded Disability Status Scale. Other model parameters were obtained from the UK MS Survey 2015, published literature, and publicly available UK sources.

RESULTS

The MSBase analysis found a significant reduction in ARR (rate ratio [RR] = 0.64; 95% confidence interval [CI] 0.57-0.72; p < 0.001) and an increase in CDI6M (hazard ratio [HR] = 1.67; 95% CI 1.30-2.15; p < 0.001) for switching to natalizumab compared with BRACETD. For switching to fingolimod, the reduction in ARR (RR = 0.91; 95% CI 0.81-1.03; p = 0.133) and increase in CDI6M (HR = 1.30; 95% CI 0.99-1.72; p = 0.058) compared with BRACETD were not significant. Switching to natalizumab was associated with a significant reduction in ARR (RR = 0.70; 95% CI 0.62-0.79; p < 0.001) and an increase in CDI6M (HR = 1.28; 95% CI 1.01-1.62; p = 0.040) compared to switching to fingolimod. No evidence of difference in CDW6M was found between treatment groups. Natalizumab dominated (higher quality-adjusted life-years [QALYs] and lower costs) fingolimod in the base-case cost-effectiveness analysis (0.453 higher QALYs and £20,843 lower costs per patient). Results were consistent across sensitivity analyses.

CONCLUSIONS

This novel real-world analysis suggests a clinical benefit for therapy escalation to natalizumab versus fingolimod based on comparative effectiveness results, translating to higher QALYs and lower costs for UK patients inadequately responding to BRACETD.

Estimation of Transition Probabilities for State-Transition Models: A Review of NICE Appraisals

State transition models are used to inform health technology reimbursement decisions. Within state transition models, the movement of patients between the model health states over discrete time intervals is determined by transition probabilities (TPs). Estimating TPs presents numerous issues, including missing data for specific transitions, data incongruence and uncertainty around extrapolation. Inappropriately estimated TPs could result in biased models. There is limited guidance on how to address common issues associated with TP estimation. To assess current methods for estimating TPs and to identify issues that may introduce bias, we reviewed National Institute for Health and Care Excellence Technology Appraisals published from 1 January, 2019 to 27 May, 2020. Twenty-eight models (from 26 Technology Appraisals) were included in the review. Several methods for estimating TPs were identified: survival analysis (n = 11); count method (n = 9); multi-state modelling (n = 7); logistic regression (n = 2); negative binomial regression (n = 2); Poisson regression (n = 1); and calibration (n = 1). Evidence Review Groups identified several issues relating to TP estimation within these models, including important transitions being excluded (n = 5); potential selection bias when estimating TPs for post-randomisation health states (n = 2); issues concerning the use of multiple data sources (n = 4); potential biases resulting from the use of data from different populations (n = 2), and inappropriate assumptions around extrapolation (n = 3). These issues remained unresolved in almost every instance. Failing to address these issues may bias model results and lead to sub-optimal decision making. Further research is recommended to address these methodological problems.

Estimating Transition Probabilities from Published Evidence: A Tutorial for Decision Modelers

This tutorial presents practical guidance on transforming various types of information published in journals, or available online from government and other sources, into transition probabilities for use in state-transition models, including cost-effectiveness models. Much, but not all, of the guidance has been previously published in peer-reviewed journals. Our purpose is to collect it in one location to serve as a stand-alone resource for decision modelers who draw most or all of their information from the published literature. Our focus is on the technical aspects of manipulating data to derive transition probabilities. We explain how to derive model transition probabilities from the following types of statistics: relative risks, odds, odds ratios, and rates. We then review the well-known approach for converting probabilities to match the model's cycle length when there are two health-state transitions and how to handle the case of three or more health-state transitions, for which the two-state approach is not appropriate. Other topics discussed include transition probabilities for population subgroups, issues to keep in mind when using data from different sources in the derivation process, and sensitivity analyses, including the use of sensitivity analysis to allocate analyst effort in refining transition probabilities and ways to handle sources of uncertainty that are not routinely formalized in models. The paper concludes with recommendations to help modelers make the best use of the published literature.

Modeling Internal Movement of Children Born in Hong Kong to Nonlocal Mothers

In Hong Kong, approximately 300,000 children were born to Mainland China couples in the period 1991–2012. According to Basic Law, the mini constitution of Hong Kong Special Administrative Region (SAR) government, these parents do not have residence rights, but their children do. As a result, most of these children have returned to Mainland China with their parents. An important consideration for policymakers is how many of these children (who are now adults in some cases) will return to Hong Kong for good, and when, as this will have a significant impact on social service provision, especially in the education sector, where it will be necessary to ensure there is capacity to meet the additional demand. Prior survey results conducted by the government suggested that more than 50% of these children would return to Hong Kong before age six. It is important to be able to provide a timely projection of the demand into the future. Here, we make use of the immigration records on the actual movement of these children and propose a Markov chain model to estimate their return rates in the future. Our results show that only about 25% of these children would return rather than 50% estimated by the survey. We also find that parents with better educational attainment levels are associated with lower return rates of their children. Timely and relevant social and public policies are needed to prepare for their return to minimize disruption to the local population and promote social harmony for the whole community.

Estimating the number and length of episodes in disability using a Markov chain approach

Background

Markov models are a key tool for calculating expected time spent in a state, such as active life expectancy and disabled life expectancy. In reality, individuals often enter and exit states recurrently, but standard analytical approaches are not able to describe this dynamic. We develop an analytical matrix approach to calculating the expected number and length of episodes spent in a state.

Methods

The approach we propose is based on Markov chains with rewards. It allows us to identify the number of entries into a state and to calculate the average length of episodes as total time in a state divided by the number of entries. For sampling variance estimation, we employ the block bootstrap. Two case studies that are based on published literature illustrate how our methods can provide new insights into disability dynamics.

Results

The first application uses a classic textbook example on prednisone treatment and liver functioning among liver cirrhosis patients. We replicate well-known results of no association between treatment and survival or recovery. Our analysis of the episodes of normal liver functioning delivers the new insight that the treatment reduced the likelihood of relapse and extended episodes of normal liver functioning. The second application assesses frailty and disability among elderly people. We replicate the prior finding that frail individuals have longer life expectancy in disability. As a novel finding, we document that frail individuals experience three times as many episodes of disability that were on average twice as long as the episodes of nonfrail individuals.

Conclusions

We provide a simple analytical approach for calculating the number and length of episodes in Markov chain models. The results allow a description of the transition dynamics that goes beyond the results that can be obtained using standard tools for Markov chains. Empirical applications using published data illustrate how the new method is helpful in unraveling the dynamics of the modeled process.

Optimisation of pavement maintenance scheduling using transition matrices

Highway networks play a vital role in the nation’s economy. With the continuing increase in traffic and road users’ expectations of comfort and safety, road authorities are increasingly interested in the upgrading and strengthening of pavements to enhance their service life. However, maintenance of road infrastructures is restricted by a limited budget. In such a situation, effective pavement maintenance scheduling is essential to maintain pavements at the desired level of quality. The performance of pavements is expressed in terms of matrices called transition probability matrices (TPMs). Likewise, pavement condition is expressed as a condition vector, which represents a particular distress (roughness and deflection). In the present study, TPMs for pavement distress and treatment effectiveness are derived from regression models. Pavement condition vectors are obtained from the condition probability distribution as a proportion of each state to which the distress level belongs. After each duty cycle (1 year), the mean distress level is calculated by the expectation of the condition probability distribution. This methodology is implemented in a Microsoft Excel spreadsheet, and optimisation is done using Excel Solver. By adopting the aforementioned procedure, optimisation is performed for scheduling maintenance treatment to a road network.

Learning Discrete-Time Markov Chains Under Concept Drift

Learning under concept drift is a novel and promising research area aiming at designing learning algorithms able to deal with nonstationary data-generating processes. In this research field, most of the literature focuses on learning nonstationary probabilistic frameworks, while some extensions about learning graphs and signals under concept drift exist. For the first time in the literature, this paper addresses the problem of learning discrete-time Markov chains (DTMCs) under concept drift. More specifically, following a hybrid active/passive approach, this paper introduces both a family of change-detection mechanisms (CDMs), differing in the required assumptions and performance, for detecting changes in DTMCs and an adaptive learning algorithm able to deal with DTMCs under concept drift. The effectiveness of both the proposed CDMs and the adaptive learning algorithm has been extensively tested on synthetically generated experiments and real data sets.

Alternative Conversion Methods for Transition Probabilities in State-Transition Models: Validity and Impact on Comparative Effectiveness and Cost-Effectiveness

Background. In state-transition models (STMs), decision problems are conceptualized using health states and transitions among those health states after predefined time cycles. The naive, commonly applied method (C) for cycle length conversion transforms all transition probabilities separately. In STMs with more than 2 health states, this method is not accurate. Therefore, we aim to describe and compare the performance of method C with that of alternative matrix transformation methods. Design. We compare 2 alternative matrix transformation methods (Eigenvalue method [E], Schure-Padé method [SP]) to method C applied in an STM of 3 different treatment strategies for women with breast cancer. We convert the given annual transition matrix into a monthly-cycle matrix and evaluate induced transformation errors for the transition matrices and the long-term outcomes: life years, quality-adjusted life-years, costs and incremental cost-effectiveness ratios, and the performance related to the decisions. In addition, we applied these transformation methods to randomly generated annual transition matrices with 4, 7, 10, and 20 health states. Results. In theory, there is no generally applicable correct transformation method. Based on our simulations, SP resulted in the smallest transformation-induced discrepancies for generated annual transition matrices for 2 treatment strategies. E showed slightly smaller discrepancies than SP in the strategy, where one of the direct transitions between health states was excluded. For long-term outcomes, the largest discrepancy occurred for estimated costs applying method C. For higher dimensional models, E performs best. Conclusions. In our modeling examples, matrix transformations (E, SP) perform better than transforming all transition probabilities separately (C). Transition probabilities based on alternative conversion methods should therefore be applied in sensitivity analyses.

Preference-Sensitive Management of Post-Mammography Decisions in Breast Cancer Diagnosis

Decision models representing the clinical situations where treatment options entail a significant risk of morbidity or mortality should consider the variations in risk preferences of individuals. In this study, we develop a stochastic modeling framework that optimizes risk-sensitive diagnostic decisions after a mammography exam. For a given patient, our objective is to find the utility maximizing diagnostic decisions where we define the utility over quality-adjusted survival duration. We use real data from a private mammography database to numerically solve our model for various utility functions. Our choice of utility functions for the numerical analysis is driven by actual patient behavior encountered in clinical practice. We find that invasive diagnostic procedures such as biopsies are more aggressively used than what the optimal risk-neutral policy would suggest, implying a far-sighted (or equivalently risk-seeking) behavior. When risk preferences are incorporated into the clinical practice, policy makers should bear in mind that a welfare loss in terms of survival duration is inevitable as evidenced by our structural and empirical results.

Estimation of annual probabilities of changing disability levels in Australians with relapsing-remitting multiple sclerosis

BACKGROUND

Transition probabilities are the engine within many health economics decision models. However, the probabilities of progression of disability due to multiple sclerosis (MS) have not previously been estimated in Australia.

OBJECTIVES

To estimate annual probabilities of changing disability levels in Australians with relapsing-remitting MS (RRMS).

METHODS

Combining data from Ausimmune/Ausimmune Longitudinal (2003-2011) and Tasmanian MS Longitudinal (2002-2005) studies (n = 330), annual transition probabilities were obtained between no/mild (Expanded Disability Status Scale (EDSS) levels 0-3.5), moderate (EDSS 4-6.0) and severe (EDSS 6.5-9.5) disability.

RESULTS

From no/mild disability, 6.4% (95% confidence interval (CI): 4.7-8.4) and 0.1% (0.0-0.2) progressed to moderate and severe disability annually, respectively. From moderate disability, 6.9% (1.0-11.4) improved (to no/mild state) and 2.6% (1.1-4.5) worsened. From severe disability, 0.0% improved to moderate and no/mild disability. Male sex, age at onset, longer disease duration, not using immunotherapies greater than 3 months and a history of relapse were related to higher probabilities of worsening.

CONCLUSION

We have estimated probabilities of changing disability levels in Australians with RRMS. Probabilities differed between various subgroups, but due to small sample sizes, results should be interpreted with caution. Our findings will be helpful in predicting long-term disease outcomes and in health economic evaluations of MS.

Natural Course of Clinically Isolated Syndrome: A Longitudinal Analysis Using a Markov Model

Clinically isolated syndrome (CIS) refers to the initial clinical episode with symptoms suggestive of multiple sclerosis (MS). Due to limited number of long-term follow-up studies, progression pattern from CIS to more advanced stages remains unclear. In the current study, we constructed a Markov model to simulate the natural course of CIS. The model estimated the probabilities of transition from CIS to more advanced disease stages and the duration needed for the progression. The analysis showed: (1) CIS is a solid disease identity: more than 85% of the subjects with a diagnosis of CIS progress to RRMS or more advanced stages within 20 years; (2) the reduction of life expectancy in subjects with CIS is marginal.

Infliximab, adalimumab and golimumab for treating moderately to severely active ulcerative colitis after the failure of conventional therapy (including a review of TA140 and TA262): clinical effectiveness systematic review and economic model

BACKGROUND

Ulcerative colitis (UC) is the most common form of inflammatory bowel disease in the UK. UC can have a considerable impact on patients' quality of life. The burden for the NHS is substantial.

OBJECTIVES

To evaluate the clinical effectiveness and safety of interventions, to evaluate the incremental cost-effectiveness of all interventions and comparators (including medical and surgical options), to estimate the expected net budget impact of each intervention, and to identify key research priorities.

DATA SOURCES

Peer-reviewed publications, European Public Assessment Reports and manufacturers' submissions. The following databases were searched from inception to December 2013 for clinical effectiveness searches and from inception to January 2014 for cost-effectiveness searches for published and unpublished research evidence: MEDLINE, EMBASE, Cumulative Index to Nursing and Allied Health Literature, The Cochrane Library including the Cochrane Systematic Reviews Database, Cochrane Controlled Trials Register, Database of Abstracts of Reviews of Effects, the Health Technology Assessment database and NHS Economic Evaluation Database; ISI Web of Science, including Science Citation Index, and the Conference Proceedings Citation Index-Science and Bioscience Information Service Previews. The US Food and Drug Administration website and the European Medicines Agency website were also searched, as were research registers, conference proceedings and key journals.

REVIEW METHODS

A systematic review [including network meta-analysis (NMA)] was conducted to evaluate the clinical effectiveness and safety of named interventions. The health economic analysis included a review of published economic evaluations and the development of a de novo model.

RESULTS

Ten randomised controlled trials were included in the systematic review. The trials suggest that adult patients receiving infliximab (IFX) [Remicade(®), Merck Sharp & Dohme Ltd (MSD)], adalimumab (ADA) (Humira(®), AbbVie) or golimumab (GOL) (Simponi(®), MSD) were more likely to achieve clinical response and remission than those receiving placebo (PBO). Hospitalisation data were limited, but suggested more favourable outcomes for ADA- and IFX-treated patients. Data on the use of surgical intervention were sparse, with a potential benefit for intervention-treated patients. Data were available from one trial to support the use of IFX in paediatric patients. Safety issues identified included serious infections, malignancies and administration site reactions. Based on the NMA, in the induction phase, all biological treatments were associated with statistically significant beneficial effects relative to PBO, with the greatest effect associated with IFX. For patients in response following induction, all treatments except ADA and GOL 100 mg at 32-52 weeks were associated with beneficial effects when compared with PBO, although these were not significant. The greatest effects at 8-32 and 32-52 weeks were associated with 100 mg of GOL and 5 mg/kg of IFX, respectively. For patients in remission following induction, all treatments except ADA at 8-32 weeks and GOL 50 mg at 32-52 weeks were associated with beneficial effects when compared with PBO, although only the effect of ADA at 32-52 weeks was significant. The greatest effects were associated with GOL (at 8-32 weeks) and ADA (at 32-52 weeks). The economic analysis suggests that colectomy is expected to dominate drug therapies, but for some patients, colectomy may not be considered acceptable. In circumstances in which only drug options are considered, IFX and GOL are expected to be ruled out because of dominance, while the incremental cost-effectiveness ratio for ADA versus conventional treatment is approximately £50,300 per QALY gained.

LIMITATIONS

The health economic model is subject to several limitations: uncertainty associated with extrapolating trial data over a lifetime horizon, the model does not consider explicit sequential pathways of non-biological treatments, and evidence relating to complications of colectomy was identified through consideration of approaches used within previous models rather than a full systematic review.

CONCLUSIONS

Adult patients receiving IFX, ADA or GOL were more likely to achieve clinical response and remission than those receiving PBO. Further data are required to conclusively demonstrate the effect of interventions on hospitalisation and surgical outcomes. The economic analysis indicates that colectomy is expected to dominate medical treatments for moderate to severe UC.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42013006883.

FUNDING

The National Institute for Health Research Health Technology Assessment programme.

The development of a stochastic mathematical model of Alzheimer's disease to help improve the design of clinical trials of potential treatments

Alzheimer's disease (AD) is a neurodegenerative disorder characterised by a slow progressive deterioration of cognitive capacity. Drugs are urgently needed for the treatment of AD and unfortunately almost all clinical trials of AD drug candidates have failed or been discontinued to date. Mathematical, computational and statistical tools can be employed in the construction of clinical trial simulators to assist in the improvement of trial design and enhance the chances of success of potential new therapies. Based on the analysis of a set of clinical data provided by the Alzheimer's Disease Neuroimaging Initiative (ADNI) we developed a simple stochastic mathematical model to simulate the development and progression of Alzheimer's in a longitudinal cohort study. We show how this modelling framework could be used to assess the effect and the chances of success of hypothetical treatments that are administered at different stages and delay disease development. We demonstrate that the detection of the true efficacy of an AD treatment can be very challenging, even if the treatment is highly effective. An important reason behind the inability to detect signals of efficacy in a clinical trial in this therapy area could be the high between- and within-individual variability in the measurement of diagnostic markers and endpoints, which consequently results in the misdiagnosis of an individual's disease state.

Current recommendations on the estimation of transition probabilities in Markov cohort models for use in health care decision-making: a targeted literature review

OBJECTIVE

Although Markov cohort models represent one of the most common forms of decision-analytic models used in health care decision-making, correct implementation of such models requires reliable estimation of transition probabilities. This study sought to identify consensus statements or guidelines that detail how such transition probability matrices should be estimated.

METHODS

A literature review was performed to identify relevant publications in the following databases: Medline, Embase, the Cochrane Library, and PubMed. Electronic searches were supplemented by manual-searches of health technology assessment (HTA) websites in Australia, Belgium, Canada, France, Germany, Ireland, Norway, Portugal, Sweden, and the UK. One reviewer assessed studies for eligibility.

RESULTS

Of the 1,931 citations identified in the electronic searches, no studies met the inclusion criteria for full-text review, and no guidelines on transition probabilities in Markov models were identified. Manual-searching of the websites of HTA agencies identified ten guidelines on economic evaluations (Australia, Belgium, Canada, France, Germany, Ireland, Norway, Portugal, Sweden, and UK). All identified guidelines provided general guidance on how to develop economic models, but none provided guidance on the calculation of transition probabilities. One relevant publication was identified following review of the reference lists of HTA agency guidelines: the International Society for Pharmacoeconomics and Outcomes Research taskforce guidance. This provided limited guidance on the use of rates and probabilities.

CONCLUSIONS

There is limited formal guidance available on the estimation of transition probabilities for use in decision-analytic models. Given the increasing importance of cost-effectiveness analysis in the decision-making processes of HTA bodies and other medical decision-makers, there is a need for additional guidance to inform a more consistent approach to decision-analytic modeling. Further research should be done to develop more detailed guidelines on the estimation of transition probabilities.

Markov chain evaluation of acute postoperative pain transition states

Previous investigations on acute postoperative pain dynamicity have focused on daily pain assessments, and so were unable to examine intraday variations in acute pain intensity. We analyzed 476,108 postoperative acute pain intensity ratings, which were clinically documented on postoperative days 1 to 7 from 8346 surgical patients using Markov chain modeling to describe how patients are likely to transition from one pain state to another in a probabilistic fashion. The Markov chain was found to be irreducible and positive recurrent, with no absorbing states. Transition probabilities ranged from 0.0031, for the transition from state 10 to state 1, to 0.69 for the transition from state 0 to state 0. The greatest density of transitions was noted in the diagonal region of the transition matrix, suggesting that patients were generally most likely to transition to the same pain state as their current state. There were also slightly increased probability densities in transitioning to a state of asleep or 0 from the current state. An examination of the number of steps required to traverse from a particular first pain score to a target state suggested that overall, fewer steps were required to reach a state of 0 (range 6.1-8.8 steps) or asleep (range 9.1-11) than were required to reach a mild pain intensity state. Our results suggest that using Markov chains is a feasible method for describing probabilistic postoperative pain trajectories, pointing toward the possibility of using Markov decision processes to model sequential interactions between pain intensity ratings, and postoperative analgesic interventions.

Estimation of Markov Chain Transition Probabilities and Rates from Fully and Partially Observed Data: Uncertainty Propagation, Evidence Synthesis, and Model Calibration

Markov transition models are frequently used to model disease progression. The authors show how the solution to Kolmogorov’s forward equations can be exploited to map between transition rates and probabilities from probability data in multistate models. They provide a uniform, Bayesian treatment of estimation and propagation of uncertainty of transition rates and probabilities when 1) observations are available on all transitions and exact time at risk in each state (fully observed data) and 2) observations are on initial state and final state after a fixed interval of time but not on the sequence of transitions (partially observed data). The authors show how underlying transition rates can be recovered from partially observed data using Markov chain Monte Carlo methods in WinBUGS, and they suggest diagnostics to investigate inconsistencies between evidence from different starting states. An illustrative example for a 3-state model is given, which shows how the methods extend to more complex Markov models using the software WBDiff to compute solutions. Finally, the authors illustrate how to statistically combine data from multiple sources, including partially observed data at several follow-up times and also how to calibrate a Markov model to be consistent with data from one specific study.

Changing Cycle Lengths in State-Transition Models: Challenges and Solutions

The choice of a cycle length in state-transition models should be determined by the frequency of clinical events and interventions. Sometimes there is need to decrease the cycle length of an existing state-transition model to reduce error in outcomes resulting from discretization of the underlying continuous-time phenomena or to increase the cycle length to gain computational efficiency. Cycle length conversion is also frequently required if a new state-transition model is built using observational data that have a different measurement interval than the model's cycle length. We show that a commonly used method of converting transition probabilities to different cycle lengths is incorrect and can provide imprecise estimates of model outcomes. We present an accurate approach that is based on finding the root of a transition probability matrix using eigendecomposition. We present underlying mathematical challenges of converting cycle length in state-transition models and provide numerical approximation methods when the eigendecomposition method fails. Several examples and analytical proofs show that our approach is more general and leads to more accurate estimates of model outcomes than the commonly used approach. MATLAB codes and a user-friendly online toolkit are made available for the implementation of the proposed methods.

Transition probabilities of HER2-positive and HER2-negative breast cancer patients treated with Trastuzumab obtained from a clinical cancer registry dataset

Records of female breast cancer patients were selected from a clinical cancer registry and separated into three cohorts according to HER2-status (human epidermal growth factor receptor 2) and treatment with or without Trastuzumab (a humanized monoclonal antibody). Propensity score matching was used to balance the cohorts. Afterwards, documented information about disease events (recurrence of cancer, metastases, remission of local/regional recurrences, remission of metastases and death) found in the dataset was leveraged to calculate the annual transition probabilities for every cohort.

A method for using real world data in breast cancer modeling

OBJECTIVES

Today, hospitals and other health care-related institutions are accumulating a growing bulk of real world clinical data. Such data offer new possibilities for the generation of disease models for the health economic evaluation. In this article, we propose a new approach to leverage cancer registry data for the development of Markov models. Records of breast cancer patients from a clinical cancer registry were used to construct a real world data driven disease model.

METHODS

We describe a model generation process which maps database structures to disease state definitions based on medical expert knowledge. Software was programmed in Java to automatically derive a model structure and transition probabilities. We illustrate our method with the reconstruction of a published breast cancer reference model derived primarily from clinical study data. In doing so, we exported longitudinal patient data from a clinical cancer registry covering eight years. The patient cohort (n=892) comprised HER2-positive and HER2-negative women treated with or without Trastuzumab.

RESULTS

The models generated with this method for the respective patient cohorts were comparable to the reference model in their structure and treatment effects. However, our computed disease models reflect a more detailed picture of the transition probabilities, especially for disease free survival and recurrence.

CONCLUSIONS

Our work presents an approach to extract Markov models semi-automatically using real world data from a clinical cancer registry. Health care decision makers may benefit from more realistic disease models to improve health care-related planning and actions based on their own data.

Transition probabilities for four states of alcohol use in adolescence and young adulthood: what factors matter when?

BACKGROUND AND AIMS

Risky single-occasion drinking (RSOD) is a health threat, particularly at younger ages. The study aimed to quantify transition probabilities (TPs) between abstinence, use of alcohol, RSOD and frequent RSOD, and to understand how TPs are associated with key demographic factors.

DESIGN

Cohort study (baseline, two follow-ups). A Markov model was fitted to estimate annual TPs and hazard ratios (HRs) for age, sex and socio-economic status (SES).

SETTING

Adolescent and young adult general population of Munich (Germany) and surrounding areas.

PARTICIPANTS

A total of 3021 people aged 14-25 years at baseline in 1995 followed-up in 1998/1999 (n = 2548) and 2003-2005 (n = 2210).

MEASUREMENTS

Alcohol use, RSOD status, age, sex and SES (subjective financial situation) were assessed in a standardized interview.

FINDINGS

The highest TPs (> 65%) were found for staying in the same drinking state. Higher age [hazard ratio (HR) for 1-year increase = 0.87, 95% confidence interval (CI) = 0.84-0.91], being female (HR = 0.30, 95% CI = 0.21-0.42), and a high SES (HR = 0.64, 95% CI = 0.43-0.97) were associated with a lower hazard to progress from use to RSOD. While age was associated predominantly with transitions between abstinence and alcohol use, sex was more relevant for transitions associated with RSOD and frequent RSOD.

CONCLUSIONS

German adolescents and young adults tend to be stable in the drinking states of abstinence, use of alcohol, risky single-occasion drinking and frequent risky single-occasion drinking. Females are less likely to transition to riskier states and more likely to transition back from frequent risky single-occasion drinking, higher age is associated with lower hazard of transitioning and participants of higher socio-economic status are less likely to transition from 'use of alcohol' to 'risky single-occasion drinking'.

Cost-effectiveness of Sativex in multiple sclerosis spasticity: new data and application to Italy

Multiple sclerosis (MS) is a chronic progressive disease that carries a high socioeconomic burden. Spasticity (rigidity and spasms) is common in MS and contributes to MS-related disability. This study aims to evaluate the cost-effectiveness of Sativex(®) (9-delta-tetrahydrocannabinol plus cannabidiol oromucosal spray) when used as add-on therapy for management of resistant MS-related spasticity in the context of the Italian healthcare system. A previously published Markov model-based analysis for the German and Spanish context was replicated, adapting it to the Italian setting. Model parameters were updated to reflect recent findings about MS-related spasticity and use of Sativex in daily clinical practice. The base case incremental cost-effectiveness ratio for Sativex use in Italy over a 5-year period was estimated to be €4968 per quality-adjusted life-year gained (year of costing: 2013). Sativex remained an efficient option in the Italian healthcare context - below the commonly accepted incremental threshold of €30,000 per quality-adjusted life-year gained - when deterministic and probabilistic sensitivity analyses were conducted. Sativex can be regarded as a cost-effective treatment option for patients with MS-related spasticity in Italy.

An early warning indicator for atmospheric blocking events using transfer operators

The existence of persistent midlatitude atmospheric flow regimes with time-scales larger than 5-10 days and indications of preferred transitions between them motivates to develop early warning indicators for such regime transitions. In this paper, we use a hemispheric barotropic model together with estimates of transfer operators on a reduced phase space to develop an early warning indicator of the zonal to blocked flow transition in this model. It is shown that the spectrum of the transfer operators can be used to study the slow dynamics of the flow as well as the non-Markovian character of the reduction. The slowest motions are thereby found to have time scales of three to six weeks and to be associated with meta-stable regimes (and their transitions) which can be detected as almost-invariant sets of the transfer operator. From the energy budget of the model, we are able to explain the meta-stability of the regimes and the existence of preferred transition paths. Even though the model is highly simplified, the skill of the early warning indicator is promising, suggesting that the transfer operator approach can be used in parallel to an operational deterministic model for stochastic prediction or to assess forecast uncertainty.

Using Markov Chains to predict the natural progression of diabetic retinopathy

AIM

To study the natural progression of diabetic retinopathy in patients with type 2 diabetes.

METHODS

This was an observational study of 153 cases with type 2 diabetes from 2010 to 2013. The state of patient was noted at end of each year and transition matrices were developed to model movement between years. Patients who progressed to severe non-proliferative diabetic retinopathy (NPDR) were treated. Markov Chains and Chi-square test were used for statistical analysis.

RESULTS

We modelled the transition of 153 patients from NPDR to blindness on an annual basis. At the end of year 3, we compared results from the Markov model versus actual data. The results from Chi-square test confirmed that there was statistically no significant difference (P=0.70) which provided assurance that the model was robust to estimate mean sojourn times. The key finding was that a patient entering the system in mild NPDR state is expected to stay in that state for 5y followed by 1.07y in moderate NPDR, be in the severe NPDR state for 1.33y before moving into PDR for roughly 8y. It is therefore expected that such a patient entering the model in a state of mild NPDR will enter blindness after 15.29y.

CONCLUSION

Patients stay for long time periods in mild NPDR before transitioning into moderate NPDR. However, they move rapidly from moderate NPDR to proliferative diabetic retinopathy (PDR) and stay in that state for long periods before transitioning into blindness.

Modeling the transmission of community-associated methicillin-resistant Staphylococcus aureus: a dynamic agent-based simulation

BACKGROUND

Methicillin-resistant Staphylococcus aureus (MRSA) has been a deadly pathogen in healthcare settings since the 1960s, but MRSA epidemiology changed since 1990 with new genetically distinct strain types circulating among previously healthy people outside healthcare settings. Community-associated (CA) MRSA strains primarily cause skin and soft tissue infections, but may also cause life-threatening invasive infections. First seen in Australia and the U.S., it is a growing problem around the world. The U.S. has had the most widespread CA-MRSA epidemic, with strain type USA300 causing the great majority of infections. Individuals with either asymptomatic colonization or infection may transmit CA-MRSA to others, largely by skin-to-skin contact. Control measures have focused on hospital transmission. Limited public health education has focused on care for skin infections.

METHODS

We developed a fine-grained agent-based model for Chicago to identify where to target interventions to reduce CA-MRSA transmission. An agent-based model allows us to represent heterogeneity in population behavior, locations and contact patterns that are highly relevant for CA-MRSA transmission and control. Drawing on nationally representative survey data, the model represents variation in sociodemographics, locations, behaviors, and physical contact patterns. Transmission probabilities are based on a comprehensive literature review.

RESULTS

Over multiple 10-year runs with one-hour ticks, our model generates temporal and geographic trends in CA-MRSA incidence similar to Chicago from 2001 to 2010. On average, a majority of transmission events occurred in households, and colonized rather than infected agents were the source of the great majority (over 95%) of transmission events. The key findings are that infected people are not the primary source of spread. Rather, the far greater number of colonized individuals must be targeted to reduce transmission.

CONCLUSIONS

Our findings suggest that current paradigms in MRSA control in the United States cannot be very effective in reducing the incidence of CA-MRSA infections. Furthermore, the control measures that have focused on hospitals are unlikely to have much population-wide impact on CA-MRSA rates. New strategies need to be developed, as the incidence of CA-MRSA is likely to continue to grow around the world.

UK multiple sclerosis risk-sharing scheme: a new natural history dataset and an improved Markov model

OBJECTIVES

In 2002, the UK's National Institute for Health and Care Excellence concluded that the multiple sclerosis (MS) disease modifying therapies; interferon-β and glatiramer acetate, were not cost effective over the short term but recognised that reducing disability over the longer term might dramatically improve the cost effectiveness. The UK Risk-sharing Scheme (RSS) was established to ensure cost-effective provision by prospectively collecting disability-related data from UK-treated patients with MS and comparing findings to a natural history (untreated) cohort. However, deficiencies were found in the originally selected untreated cohort and the resulting analytical approach. This study aims to identify a more suitable natural history cohort and to develop a robust analytical approach using the new cohort.

DESIGN

The Scientific Advisory Group, recommended the British Columbia Multiple Sclerosis (BCMS) database, Canada, as providing a more suitable natural history comparator cohort. Transition probabilities were derived and different Markov models (discrete and continuous) with and without baseline covariates were applied.

SETTING

MS clinics in Canada and the UK.

PARTICIPANTS

From the BCMS database, 898 'untreated' patients with MS considered eligible for drug treatment based on the UK's Association of British Neurologists criteria.

OUTCOME MEASURE

The predicted Expanded Disability Status Scale (EDSS) score was collected and assessed for goodness of fit when compared with actual outcome.

RESULTS

The BCMS untreated cohort contributed 7335 EDSS scores over a median 6.4 years (6357 EDSS 'transitions' recorded at consecutive visits) during the period 1980-1995. A continuous Markov model with 'onset age' as a binary covariate was deemed the most suitable model for future RSS analysis.

CONCLUSIONS

A new untreated MS cohort from British Columbia has been selected and will be modelled using a continuous Markov model with onset age as a baseline covariate. This approach will now be applied to the treated UK RSS MS cohort for future price adjustment calculations.

HIV-1 disease progression during highly active antiretroviral therapy: an application using population-level data in British Columbia: 1996-2011

BACKGROUND

Accurately estimating rates of disease progression is of central importance in developing mathematical models used to project outcomes and guide resource allocation decisions. Our objective was to specify a multivariate regression model to estimate changes in disease progression among individuals on highly active antiretroviral treatment in British Columbia, Canada, 1996-2011.

METHODS

We used population-level data on disease progression and antiretroviral treatment utilization from the BC HIV Drug Treatment Program. Disease progression was captured using longitudinal CD4 and plasma viral load testing data, linked with data on antiretroviral treatment. The study outcome was categorized into (CD4 count ≥ 500, 500-350, 350-200, <200 cells/mm, and mortality). A 5-state continuous-time Markov model was used to estimate covariate-specific probabilities of CD4 progression, focusing on temporal changes during the study period.

RESULTS

A total of 210,083 CD4 measurements among 7421 individuals with HIV/AIDS were included in the study. Results of the multivariate model suggested that current highly active antiretroviral treatment at baseline, lower baseline CD4 (<200 cells/mm), and extended durations of elevated plasma viral load were each associated with accelerated progression. Immunological improvement was accelerated significantly from 2004 onward, with 23% and 46% increases in the probability of CD4 improvement from the fourth CD4 stratum (CD4 < 200) in 2004-2008 and 2008-2011, respectively.

CONCLUSION

Our results demonstrate the impact of innovations in antiretroviral treatment and treatment delivery at the population level. These results can be used to estimate a transition probability matrix flexible to changes in the observed mix of clients in different clinical stages and treatment regimens over time.

Estimating lifetime healthcare costs with morbidity data

BACKGROUND

In many developed countries, the economic crisis started in 2008 producing a serious contraction of the financial resources spent on healthcare. Identifying which individuals will require more resources and the moment in their lives these resources have to be allocated becomes essential. It is well known that a small number of individuals with complex healthcare needs consume a high percentage of health expenditures. Conversely, little is known on how morbidity evolves throughout life. The aim of this study is to introduce a longitudinal perspective to chronic disease management.

METHODS

Data used relate to the population of the county of Baix Empordà in Catalonia for the period 2004-2007 (average population was N = 88,858). The database included individual information on morbidity, resource consumption, costs and activity records. The population was classified using the Clinical Risk Groups (CRG) model. Future morbidity evolution was simulated under different assumptions using a stationary Markov chain. We obtained morbidity patterns for the lifetime and the distribution function of the random variable lifetime costs. Individual information on acute episodes, chronic conditions and multimorbidity patterns were included in the model.

RESULTS

The probability of having a specific health status in the future (healthy, acute process or different combinations of chronic illness) and the distribution function of healthcare costs for the individual lifetime were obtained for the sample population. The mean lifetime cost for women was €111,936, a third higher than for men, at €81,566 (all amounts calculated in 2007 Euros). Healthy life expectancy at birth for females was 46.99, lower than for males (50.22). Females also spent 28.41 years of life suffering from some type of chronic disease, a longer period than men (21.9).

CONCLUSIONS

Future morbidity and whole population costs can be reasonably predicted, combining stochastic microsimulation with a morbidity classification system. Potential ways of efficiency arose by introducing a time perspective to chronic disease management.

A mathematical approach for evaluating Markov models in continuous time without discrete-event simulation

Markov models are a simple and powerful tool for analyzing the health and economic effects of health care interventions. These models are usually evaluated in discrete time using cohort analysis. The use of discrete time assumes that changes in health states occur only at the end of a cycle period. Discrete-time Markov models only approximate the process of disease progression, as clinical events typically occur in continuous time. The approximation can yield biased cost-effectiveness estimates for Markov models with long cycle periods and if no half-cycle correction is made. The purpose of this article is to present an overview of methods for evaluating Markov models in continuous time. These methods use mathematical results from stochastic process theory and control theory. The methods are illustrated using an applied example on the cost-effectiveness of antiviral therapy for chronic hepatitis B. The main result is a mathematical solution for the expected time spent in each state in a continuous-time Markov model. It is shown how this solution can account for age-dependent transition rates and discounting of costs and health effects, and how the concept of tunnel states can be used to account for transition rates that depend on the time spent in a state. The applied example shows that the continuous-time model yields more accurate results than the discrete-time model but does not require much computation time and is easily implemented. In conclusion, continuous-time Markov models are a feasible alternative to cohort analysis and can offer several theoretical and practical advantages.

State-space size considerations for disease-progression models

Markov models of disease progression are widely used to model transitions in patients' health state over time. Usually, patients' health status may be classified according to a set of ordered health states. Modelers lump together similar health states into a finite and usually small, number of health states that form the basis of a Markov chain disease-progression model. This increases the number of observations used to estimate each parameter in the transition probability matrix. However, lumping together observably distinct health states also obscures distinctions among them and may reduce the predictive power of the model. Moreover, as we demonstrate, precision in estimating the model parameters does not necessarily improve as the number of states in the model declines. This paper explores the tradeoff between lumping error introduced by grouping distinct health states and sampling error that arises when there are insufficient patient data to precisely estimate the transition probability matrix.

Modeling the relationship between health and health care expenditures using a latent Markov model

We investigate the dynamic relationship between several dimensions of health and health care expenditures for older individuals. Health data from the Longitudinal Aging Survey Amsterdam is combined with data on hospital and long term care use. We estimate a latent variable based jointly on observed health indicators and expenditures. Annual transition probabilities between states of the latent variable are estimated using a Markov model. States associated with good current health and low annual health care expenditures are not associated with lower cumulative health care expenditures over remaining lifetime. We conclude that, although the direct health care cost saving effect is limited, the considerable gain in healthy lifeyears can make investing in the improvement of health of the older population worthwhile.

Parameterization of treatment effects for meta-analysis in multi-state Markov models

Standard approaches to analysis of randomized controlled trials (RCTs) using Markov models make it difficult to generalize treatment effects to new patient groups and synthesize evidence across trials. This paper demonstrates how pair-wise and mixed treatment comparison meta-analysis can be applied to event history data for disease progression reported by RCTs. The data, in the form of aggregated discrete time transitions, have a multi-nomial likelihood. In order for evidence synthesis to be performed a structured approach to modelling the differences in the effects of the different treatments must be taken. A multi-state continuous-time Markov model similar to others used in published economic evaluations of asthma treatments is developed, with transition rates related to the likelihood via Kolmogorov's forward equations. The formulation in terms of rates allows a flexible characterization of summary treatment effects. These ideas are applied to an illustrative data set consisting of a set of five trials comparing eight different treatments for asthma. A range of models is developed in which the relative treatment effects act on forward, backward transitions, or both, and models are compared using the DIC. Bayesian inferential techniques are used and the parameters are estimated using MCMC simulation in WinBUGS. An intuitively appealing mechanism of action involving a single parameter acting on all backward transitions was identified for the relative effects of the treatments, which allowed the estimation of a pooled treatment effect, allowing us to rank the different treatment options within each connected evidence network to ascertain which were the most clinically effective.

Clinical and economic outcomes in a population-based European cohort of 948 ulcerative colitis and Crohn's disease patients by Markov analysis

BACKGROUND

Forecasting clinical and economic outcomes in ulcerative colitis (UC) and Crohn's disease (CD) patients is complex, but necessary.

AIMS

To determine: the frequency of treatment-classified clinical states; the probability of transition between states; and the economic outcomes.

METHODS

Newly diagnosed UC and CD patients, allocated into seven clinical states by medical and surgical treatments recorded in serial 3-month cycles, underwent Markov analysis.

RESULTS

Over 10 years, 630 UC and 318 CD patients had 22,823 and 11,871 cycles. The most frequent clinical outcomes were medical/surgical remission (medication-free) and mild disease (on 5-aminosalicylates, antibiotics, topical corticosteroids), comprising 28% and 62% of UC cycles and 24% and 51% of CD cycles respectively. The probability of drug-response in patients receiving systemic corticosteroids/immunomodulators was 0.74 in UC, 0.66 in CD. Both diseases had similar likelihood of persistent drug-dependency or drug-refractoriness. Surgery was more probable in CD, 0.20, than UC, 0.08. In terms of economic outcomes, surgery was costlier in UC per cycle, but the outlay over 10 years was greater in CD. Drug-refractory UC and CD cases engendered high costs in the cohort.

CONCLUSIONS

Most patients on 5-aminosalicylates, corticosteroids and immunomodulators had favourable clinical and economic outcomes over 10 years. Drug-refractory and surgical patients exhibited greater long-term expenses.