Multi-state Markov models for analysing incomplete disease history data with illustrations for HIV disease

Maximum likelihood estimation for semiparametric regression models with interval-censored multistate data

Interval-censored multistate data arise in many studies of chronic diseases, where the health status of a subject can be characterized by a finite number of disease states and the transition between any two states is only known to occur over a broad time interval. We relate potentially time-dependent covariates to multistate processes through semiparametric proportional intensity models with random effects. We study nonparametric maximum likelihood estimation under general interval censoring and develop a stable expectation-maximization algorithm. We show that the resulting parameter estimators are consistent and that the finite-dimensional components are asymptotically normal with a covariance matrix that attains the semiparametric efficiency bound and can be consistently estimated through profile likelihood. In addition, we demonstrate through extensive simulation studies that the proposed numerical and inferential procedures perform well in realistic settings. Finally, we provide an application to a major epidemiologic cohort study.

Belatacept-based immunosuppression does not confer increased risk of BK polyomavirus-DNAemia relative to tacrolimus-based immunosuppression

BACKGROUND

The effect of belatacept on BK polyomavirus (BKPyV) control remains largely unknown.

METHODS

This is a propensity matched retrospective cohort study in adult kidney transplant recipients (KTR) transplanted between 2016-2020 who received a belatacept- versus tacrolimus-based immunosuppression regimen. A continuous time multi-state Markov model was used to evaluate BKPyV replication dynamics (BKPyV-dyn). Three BKPyV-dyn states were defined: BKPyV-dyn1 (viral load <3 log10), BKPyV-dyn2 (viral load ≥ 3 log10 and ≤4 log10), and BKPyV-dyn3 (viral load >4 log10).

RESULTS

Two hundred eighty KTR on belatacept- and 280 KTR on tacrolimus-based regimens were compared. The probability of transitioning between BKPyV-dyn states and time spent in each state in both groups was comparable. Total duration in BKPyV-dyn-1 was 632.1 days (95% CI 612.1, 648.5) for belatacept versus 615.2 days (95% CI 592.5, 635.8) for tacrolimus, BKPyV-dyn-2 was 49.2 days (95% CI 41.3, 58.4) for belatacept versus 55.6 days (95% CI 46.5, 66.8) for tacrolimus, and BKPyV-dyn-3 was 48.7 days (95% CI 37.1, 363.1) for belatacept versus 59.2 days (95% CI 45.8, 73.5) for tacrolimus. BKPyV associated nephropathy (PyVAN) occurred in 3.9% in belatacept- and 3.9% tacrolimus-treated KRT (P > .9).

CONCLUSIONS

Compared with tacrolimus-based immunosuppression, belatacept based immunosuppression was not associated with increased risk of BKPyV-DNAemia or nephropathy.

The analysis of Asthma Control using Markov Models: MOSAR study (Multicenter Observational Study of Asthma in Rabat-Morocco)

This study aimed to analyze the probabilities of transitioning between controlled, uncontrolled, and partially controlled states of asthma patients and investigate the influence of age, smoking, dust allergy, and obesity on these probabilities. Four hundred twenty-four asthma patients from three hospitals in Morocco were included in the study, spanning 42 months.A discrete-time homogeneous Markov model with three states and a single aperiodic recurrent class was used to model the asthma evolution, assuming the regularity of consultations. Results showed that controlled patients were more likely to remain in that state, with approximately 79 out of 100 patients expected to stay in optimal control in the long term.A discrete non-homogeneous time Markov Model with the stationarity criterion was used to examine the factors affecting patient states and transitions. Patients seen during the spring and summer seasons were more likely to move into a controlled state compared with those seen in the fall and winter seasons. Patients with dust allergies and obesity significantly impacted asthma exacerbation, with overweight patients more likely to transition into a controlled state.The study estimated the transition intensities matrix under certain conditions, assuming the regularity of patients. In the long term, the probability of an asthmatic patient being in a controlled state was approximately 0.8.Overall, this study provided insights into the probabilities and factors influencing asthma progression in Morocco. Dust allergy and obesity were identified as significant contributors to asthma exacerbation, emphasizing the need for effective management strategies.

Risk factors for stroke-related functional disability and mortality at Felege Hiwot Referral Hospital, Ethiopia

BACKGROUND

Stroke is one of the top causes of functional disability around the world. The main objective was to identify stroke-related functional outcomes and risk factors. A good functional outcome is defined as the absence of problems secondary to the stroke event, a poor functional outcome as the presence of complications, and mortality as the existence of complications.

METHOD

A retrospective cohort analysis was used to observe factors in 298 eligible adult (18 or older) stroke patients who attend outpatient clinics every three months at Felege Hiwot Referral Hospital between September 2019 and August 2021 to predict outcomes.

RESULT

The likelihood of dying from a poor outcome was 9%, and the likelihood of recovering was 24%. The average time spent on good and poor outcomes for different levels of independent variables varies according to their risk. During the first three years of follow-up, the instantaneous risk with a 95% confidence interval of transitioning from good to poor outcome in the women, aged 60 or older, with hypertension, atrial fibrillation, and hemorrhage stroke versus men stroke patients, aged 18 to 59, without hypertension, atrial fibrillation, and ischemic stroke were 1.54 (1.10, 2.15), 1.73 (1.19, 2.52), 2.34 (1.55, 3.53), 2.74 (1.64, 4.56), and 1.52 (1.10, 2.19) respectively. The hazard ratio of transitioning from poor outcome to death for patients with diabetes mellitus and atrial fibrillation versus those without diabetes mellitus and atrial fibrillation was estimated to be 1.95 (1.10, 3.46) and 3.39 (1.67, 6.89), respectively.

CONCLUSION

Women over 60 with hypertension, atrial fibrillation, and hemorrhagic stroke were more likely to progress from a good to a poor outcome. Diabetes and atrial fibrillation were also risk factors for progressing from a poor outcome to death. The states and transitions, as well as a clinical control of the hazards for the transition through states, should improve the physician's decision-making process. Since gender and age are difficult to control, early intervention by patients and the hospital may be critical in influencing functional outcomes.

A new framework for semi-Markovian parametric multi-state models with interval censoring

There are few computational and methodological tools available for the analysis of general multi-state models with interval censoring. Here, we propose a general framework for parametric inference with interval censored multi-state data. Our framework can accommodate any parametric model for the transition times, and covariates may be included in various ways. We present a general method for constructing the likelihood, which we have implemented in a ready-to-use R package, smms, available on GitHub. The R package also computes the required high-dimensional integrals in an efficient manner. Further, we explore connections between our modelling framework and existing approaches: our models fall under the class of semi-Markovian multi-state models, but with a different, and sparser parameterisation than what is often seen. We illustrate our framework through a dataset monitoring heart transplant patients. Finally, we investigate the effect of some forms of misspecification of the model assumptions through simulations.

Impact of belatacept and tacrolimus on cytomegalovirus viral load control and relapse in moderate and high-risk cytomegalovirus serostatus kidney transplant recipients

BACKGROUND

Belatacept improves long-term graft survival, but control of some primary viral infections may be impaired. We evaluated the impact of belatacept and tacrolimus on cytomegalovirus (CMV) viral control, remission and relapse in CMV high-risk and moderate-risk recipients.

METHODS

Using a multistate Markov model, we evaluated viral load state transitions of 173 kidney transplant recipients with at least one episode of viremia within 1 year after transplant: state 1, undetectable/low viral load; state 2, moderate viremia; and state 3, severe viremia.

RESULTS

Among high-risk recipients, belatacept-treated recipients exhibited a significantly higher probability of entering moderate viremia (.36; 95% CI = .31, .41) than tacrolimus-treated recipients (.20; 95% CI = .13, .29). The expected number of days in viremic states differed. High-risk belatacept-treated recipients persisted in moderate viremia for significantly longer (128 days, 95% CI = 110, 146) than did tacrolimus-treated recipients (70.0 days, 95% CI = 45.2, 100) and showed a trend of shorter duration in low/undetectable viral load state (172 days, 95% CI = 148, 195) than did tacrolimus-treated recipients (239 days, 95% CI = 195, 277). Moderate-risk recipients showed better viral load control and with no differences by immunosuppression.

CONCLUSION

High-risk belatacept-treated recipients showed defects in sustaining viral control relative to tacrolimus-treated recipients. Avoidance of initial use belatacept in high-risk recipients or development of modified management protocols should be considered.

Bayesian model of disease progression in mucopolysaccaridosis IIIA

Mucopolysaccaridosis IIIA (MPS IIIA) is a rare genetic disease that afflicts children and leads to neurocognitive degeneration. We develop a Bayesian disease progression model (DPM) of MPS IIIA that characterizes the pattern of cognitive growth and decline in this disease. The DPM is a repeated measures model that incorporates a nonlinear developmental trajectory and shape-invariant random effects. This approach quantifies the pattern of cognitive development in MPS IIIA and addresses differences in biological age, length of follow-up, and clinical outcomes across natural history subjects. The DPM can be used in clinical trials to estimate the percent slowing in disease progression for treatment relative to natural history. Simulations demonstrate that the DPM provides substantial improvements in power relative to alternative analyses.

A review of multistate modelling approaches in monitoring disease progression: Bayesian estimation using the Kolmogorov-Chapman forward equations

There are numerous fields of science in which multistate models are used, including biomedical research and health economics. In biomedical studies, these stochastic continuous-time models are used to describe the time-to-event life history of an individual through a flexible framework for longitudinal data. The multistate framework can describe more than one possible time-to-event outcome for a single individual. The standard estimation quantities in multistate models are transition probabilities and transition rates which can be mapped through the Kolmogorov-Chapman forward equations from the Bayesian estimation perspective. Most multistate models assume the Markov property and time homogeneity; however, if these assumptions are violated, an extension to non-Markovian and time-varying transition rates is possible. This manuscript extends reviews in various types of multistate models, assumptions, methods of estimation and data features compatible with fitting multistate models. We highlight the contrast between the frequentist (maximum likelihood estimation) and the Bayesian estimation approaches in the multistate modeling framework and point out where the latter is advantageous. A partially observed and aggregated dataset from the Zimbabwe national ART program was used to illustrate the use of Kolmogorov-Chapman forward equations. The transition rates from a three-stage reversible multistate model based on viral load measurements in WinBUGS were reported.

Risk Factor Identification in Heterogeneous Disease Progression with L1-Regularized Multi-state Models

Multi-state model (MSM) is a useful tool to analyze longitudinal data for modeling disease progression at multiple time points. While the regularization approaches to variable selection have been widely used, extending them to MSM remains largely unexplored. In this paper, we have developed the L1-regularized multi-state model (L1MSTATE) framework that enables parameter estimation and variable selection simultaneously. The regularized optimization problem was solved by deriving a one-step coordinate descent algorithm with great computational efficiency. The L1MSTATE approach was evaluated using extensive simulation studies, and it showed that L1MSTATE outperformed existing regularized multi-state models in terms of the accurate identification of risk factors. It also outperformed the un-regularized multi-state models (MSTATE) in terms of identifying the important risk factors in situations with small sample sizes. The power of L1MSTATE in predicting the transition probabilities comparing with MSTATE was demonstrated using the Europe Blood and Marrow Transplantation (EBMT) dataset. The L1MSTATE was implemented in the open-access R package 'L1mstate'.

A semiparametric method for the analysis of outcomes during a gap in HIV care under incomplete outcome ascertainment

Objectives

Estimation of the cascade of HIV care is essential for evaluating care and treatment programs, informing policy makers and assessing targets such as 90-90-90. A challenge to estimating the cascade based on electronic health record concerns patients "churning" in and out of care. Correctly estimating this dynamic phenomenon in resource-limited settings, such as those found in sub-Saharan Africa, is challenging because of the significant death under-reporting. An approach to partially recover information on the unobserved deaths is a double-sampling design, where a small subset of individuals with a missed clinic visit is intensively outreached in the community to actively ascertain their vital status. This approach has been adopted in several programs within the East Africa regional IeDEA consortium, the context of our motivating study. The objective of this paper is to propose a semiparametric method for the analysis of competing risks data with incomplete outcome ascertainment.

Methods

Based on data from double-sampling designs, we propose a semiparametric inverse probability weighted estimator of key outcomes during a gap in care, which are crucial pieces of the care cascade puzzle.

Results

Simulation studies suggest that the proposed estimators provide valid estimates in settings with incomplete outcome ascertainment under a set of realistic assumptions. These studies also illustrate that a naïve complete-case analysis can provide seriously biased estimates. The methodology is applied to electronic health record data from the East Africa IeDEA Consortium to estimate death and return to care during a gap in care.

Conclusions

The proposed methodology provides a robust approach for valid inferences about return to care and death during a gap in care, in settings with death under-reporting. Ultimately, the resulting estimates will have significant consequences on program construction, resource allocation, policy and decision making at the highest levels.

Risk factors for assaultive reinjury and death following a nonfatal firearm assault injury: A population-based retrospective cohort study

Individuals with a firearm injury are at high risk of subsequent firearm victimization, but characteristics associated with sustaining recurrent firearm injuries are not well understood. In this retrospective cohort study, we sought to quantify the hazards of sustaining subsequent assaultive firearm injuries among people with an initial firearm assault injury and to identify characteristics associated with recurrent victimization. Using hospital discharge, emergency department, and mortality records, we identified and followed all individuals aged ≥15 years with a nonfatal firearm assault injury resulting in an emergency department visit or hospital admission in California, 2005-2013. We model transitions from one injury to the next and from injury to death, accounting for event history, covariates, and competing risks using multistate models. 29,156 people had an index nonfatal firearm assault injury. Among individuals with 1 such injury, 3.1% had additional nonfatal firearm assault injuries and 1.0% subsequently died from firearm homicide. Among individuals with 2+ nonfatal firearm assaults, 2.0% died from firearm homicide. The estimated transition probability for 1 to 2+ nonfatal injuries reached 10% by 8.5 years post-index injury. The rate of subsequent nonfatal firearm assault injury was highest among men (hazard ratio [HR]: 3.87; 95% confidence interval [CI]: 2.63-5.69) and Blacks (vs. whites) (HR: 2.69; 95% CI: 1.99-3.64). Identification of additional risk markers will require more detailed individual-level data; nonetheless, this study supports the generalizability of findings from smaller studies, provides broad guidance for allocating scarce resources, and suggests that interventions on root causes of violence disparities may have downstream effects on recurrence.

Score tests based on a finite mixture model of Markov processes under intermittent observation

A mixture model is described, which accommodates different Markov processes governing disease progression in a finite set of latent classes. We give special attention to the setting in which individuals are examined intermittently and transition times are consequently interval censored. A score test is developed to identify genetic markers associated with class membership. Simulation studies are conducted to validate the algorithm, assess the finite sample properties of the estimators, and assess the frequency properties of the score tests. A permutation test is recommended for settings when there is concern that the asymptotic approximation to the score test is poor. An application involving progression in joint damage in psoriatic arthritis (PsA) provides illustration and identifies human leukocyte antigen markers associated with unilateral and bilateral sacroiliac damage in individuals with PsA.

Hidden three-state survival model for bivariate longitudinal count data

A model is presented that describes bivariate longitudinal count data by conditioning on a progressive illness-death process where the two living states are latent. The illness-death process is modelled in continuous time, and the count data are described by a bivariate extension of the binomial distribution. The bivariate distributions for the count data approach include the correlation between two responses even after conditioning on the state. An illustrative data analysis is discussed, where the bivariate data consist of scores on two cognitive tests, and the latent states represent two stages of underlying cognitive function. By including a death state, possible association between cognitive function and the risk of death is accounted for.

Approximate Bayesian inference for discretely observed continuous-time multi-state models

Inference for continuous time multi-state models presents considerable computational difficulties when the process is only observed at discrete time points with no additional information about the state transitions. In fact, for general multi-state Markov model, evaluation of the likelihood function is possible only via intensive numerical approximations. Moreover, in real applications, transitions between states may depend on the time since entry into the current state, and semi-Markov models, where the likelihood function is not available in closed form, should be fitted to the data. Approximate Bayesian Computation (ABC) methods, which make use only of comparisons between simulated and observed summary statistics, represent a solution to intractable likelihood problems and provide alternative algorithms when the likelihood calculation is computationally too costly. In this article we investigate the potentiality of ABC techniques for multi-state models both to obtain the posterior distributions of the model parameters and to compare Markov and semi-Markov models. In addition, we will also exploit ABC methods to estimate and compare hidden Markov and semi-Markov models when observed states are subject to classification errors. We illustrate the performance of the ABC methodology both with simulated data and with a real data example.

5-year versus risk-category-specific screening intervals for cardiovascular disease prevention: a cohort study

BACKGROUND

Clinical guidelines suggest preventive interventions such as statin therapy for individuals with a high estimated 10-year risk of major cardiovascular events. For those with a low or intermediate estimated risk, risk-factor screenings are recommended at 5-year intervals; this interval is based on expert opinion rather than on direct research evidence. Using longitudinal data on the progression of cardiovascular disease risk over time, we compared different screening intervals in terms of timely detection of high-risk individuals, cardiovascular events prevented, and health-care costs.

METHODS

We used data from participants in the British Whitehall II study (aged 40-64 years at baseline) who had repeated biomedical screenings at 5-year intervals and linked these data to electronic health records between baseline (Aug 7, 1991, to May 10, 1993) and June 30, 2015. We estimated participants' 10-year risk of a major cardiovascular event (myocardial infarction, cardiac death, and fatal or non-fatal stroke) using the revised Atherosclerotic Cardiovascular Disease (ASCVD) calculator. We used multistate Markov modelling to estimate optimum screening intervals on the basis of progression rates from low-risk and intermediate-risk categories to the high-risk category (ie, ≥7·5% 10-year risk of a major cardiovascular event). Our assessment criteria included person-years spent in a high-risk category before detection, the number of major cardiovascular events prevented and quality-adjusted life-years (QALYs) gained, and screening costs.

FINDINGS

Of 6964 participants (mean age 50·0 years [SD 6·0] at baseline) with 152 700 person-years of follow-up (mean follow-up 22·0 years [SD 5·0]), 1686 participants progressed to the high-risk category and 617 had a major cardiovascular event. With the 5-year screening intervals, participants spent 7866 (95% CI 7130-8658) person-years unrecognised in the high-risk group. For individuals in the low, intermediate-low, and intermediate-high risk categories, 21 alternative risk category-based screening intervals outperformed the 5-yearly screening protocol. Screening intervals at 7 years, 4 years, and 1 year for those in the low, intermediate-low, and intermediate-high-risk category would reduce the number of person-years spent unrecognised in the high-risk group by 62% (95% CI 57-66; 4894 person-years), reduce the number of major cardiovascular events by 8% (7-9; 49 events), and raise 44 QALYs (40-49) for the study population.

INTERPRETATION

In terms of timely preventive interventions, the 5-year screening intervals were unnecessarily frequent for low-risk individuals and insufficiently frequent for intermediate-risk individuals. Screening intervals based on risk-category-specific progression rates would perform better in terms of preventing major cardiovascular disease events and improving cost-effectiveness.

FUNDING

Medical Research Council, British Heart Association, National Institutes on Aging, NordForsk, Academy of Finland.

Estimating transition probability of different states of type 2 diabetes and its associated factors using Markov model

AIMS

Type 2 diabetes is a chronic metabolic disorder and one of the most common non-contagious diseases which is on the rise all over the world. The present study aims to assess the trend of change in fasting blood sugar (FBS) and factors associated with the progression and regression of type 2 diabetes. Moreover, this study estimates transition intensities and transition probabilities among various states using the multi-state Markov model.

METHODS

In this study Multi-Ethnic Study of Atherosclerosis (MESA) dataset, from a longitudinal study, was used. The study, at the beginning, included 6814 individuals who were followed during the five phases of the study. FBS, serving as the criterion to assess the progression of diabetes, was classified into four states including (a) normal (FBS<100mg/dl), (b) impaired fasting glucose I (IFG I) (100mg/dl<FBS<110mg/dl), (c) impaired fasting glucose II (IFG II) (110mg/dl<FBS<126mg/dl), and (d) diabetes status (FBS>126mg/dl). A continuous-time Markov process was used to describe the evaluation of disease changes over the four states. The model estimated the mean sojourn time for each state.

RESULTS

Based on the results obtained from fitting the Markov model, the transition probability for a normal individual to remain in the same status over a 10-year period was 0.63, while the probability for a person in the diabetes state was 0.40. The mean sojourn time for the normal and diabetic individuals aged 45-84 years was 6.26 and 5.20 respectively. The covariates of age, race, body mass index (BMI), physical activity, waist-to-hip ratio (WHR) and blood pressure, significantly affected the progression and regression of diabetes.

CONCLUSION

An increase in physical activity could be the most important factor in the regression of diabetes, while an increase in WHR and BMI could be the most significant factors in progression of the disease.

Flexible multistate models for interval-censored data: Specification, estimation, and an application to ageing research

Continuous-time multistate survival models can be used to describe health-related processes over time. In the presence of interval-censored times for transitions between the living states, the likelihood is constructed using transition probabilities. Models can be specified using parametric or semiparametric shapes for the hazards. Semiparametric hazards can be fitted using P-splines and penalised maximum likelihood estimation. This paper presents a method to estimate flexible multistate models that allow for parametric and semiparametric hazard specifications. The estimation is based on a scoring algorithm. The method is illustrated with data from the English Longitudinal Study of Ageing.

Population sensitivity of acute flaccid paralysis and environmental surveillance for serotype 1 poliovirus in Pakistan: an observational study

Background

To support poliomyelitis eradication in Pakistan, environmental surveillance (ES) of wastewater has been expanded alongside surveillance for acute flaccid paralysis (AFP). ES is a relatively new method of surveillance, and the population sensitivity of detecting poliovirus within endemic settings requires estimation.

Methods

Data for wild serotype 1 poliovirus from AFP and ES from January 2011 to September 2015 from 14 districts in Pakistan were analysed using a multi-state model framework. This framework was used to estimate the sensitivity of poliovirus detection from each surveillance source and parameters such as the duration of infection within a community.

Results

The location and timing of poliomyelitis cases showed spatial and temporal variability. The sensitivity of AFP surveillance to detect serotype 1 poliovirus infection in a district and its neighbours per month was on average 30.0% (95% CI 24.8–35.8) and increased with the incidence of poliomyelitis cases. The average population sensitivity of a single environmental sample was 59.4% (95% CI 55.4–63.0), with significant variation in site-specific estimates (median varied from 33.3–79.2%). The combined population sensitivity of environmental and AFP surveillance in a given month was on average 98.1% (95% CI 97.2–98.7), assuming four samples per month for each site.

Conclusions

ES can be a highly sensitive supplement to AFP surveillance in areas with converging sewage systems. As ES for poliovirus is expanded, it will be important to identify factors associated with variation in site sensitivity, leading to improved site selection and surveillance system performance.

Electronic supplementary material

The online version of this article (10.1186/s12879-018-3070-4) contains supplementary material, which is available to authorized users.

Regularized continuous-time Markov Model via elastic net

Continuous-time Markov models are commonly used to analyze longitudinal transitions between multiple disease states in panel data, where participants' disease states are only observed at multiple time points, and the exact state paths between observations are unknown. However, when covariate effects are incorporated and allowed to vary for different transitions, the number of potential parameters to estimate can become large even when the number of covariates is moderate, and traditional maximum likelihood estimation and subset model selection procedures can easily become unstable due to overfitting. We propose a novel regularized continuous-time Markov model with the elastic net penalty, which is capable of simultaneous variable selection and estimation for large number of parameters. We derive an efficient coordinate descent algorithm to solve the penalized optimization problem, which is fully automatic and data driven. We further consider an extension where one of the states is death, and time of death is exactly known but the state path leading to death is unknown. The proposed method is extensively evaluated in a simulation study, and demonstrated in an application to real-world data on airflow limitation state transitions.

A state transition framework for patient-level modeling of engagement and retention in HIV care using longitudinal cohort data

The human immunodeficiency virus (HIV) care cascade is a conceptual model used to outline the benchmarks that reflects effectiveness of HIV care in the whole HIV care continuum. The models can be used to identify barriers contributing to poor outcomes along each benchmark in the cascade such as disengagement from care or death. Recently, the HIV care cascade has been widely applied to monitor progress towards HIV prevention and care goals in an attempt to develop strategies to improve health outcomes along the care continuum. Yet, there are challenges in quantifying successes and gaps in HIV care using the cascade models that are partly due to the lack of analytic approaches. The availability of large cohort data presents an opportunity to develop a coherent statistical framework for analysis of the HIV care cascade. Motivated by data from the Academic Model Providing Access to Healthcare, which has provided HIV care to nearly 200,000 individuals in Western Kenya since 2001, we developed a state transition framework that can characterize patient-level movements through the multiple stages of the HIV care cascade. We describe how to transform large observational data into an analyzable format. We then illustrate the state transition framework via multistate modeling to quantify dynamics in retention aspects of care. The proposed modeling approach identifies the transition probabilities of moving through each stage in the care cascade. In addition, this approach allows regression-based estimation to characterize effects of (time-varying) predictors of within and between state transitions such as retention, disengagement, re-entry into care, transfer-out, and mortality. Copyright © 2017 John Wiley & Sons, Ltd.

A stochastic estimation procedure for intermittently-observed semi-Markov multistate models with back transitions

Multistate models provide an important method for analyzing a wide range of life history processes including disease progression and patient recovery following medical intervention. Panel data consisting of the states occupied by an individual at a series of discrete time points are often used to estimate transition intensities of the underlying continuous-time process. When transition intensities depend on the time elapsed in the current state and back transitions between states are possible, this intermittent observation process presents difficulties in estimation due to intractability of the likelihood function. In this manuscript, we present an iterative stochastic expectation-maximization algorithm that relies on a simulation-based approximation to the likelihood function and implement this algorithm using rejection sampling. In a simulation study, we demonstrate the feasibility and performance of the proposed procedure. We then demonstrate application of the algorithm to a study of dementia, the Nun Study, consisting of intermittently-observed elderly subjects in one of four possible states corresponding to intact cognition, impaired cognition, dementia, and death. We show that the proposed stochastic expectation-maximization algorithm substantially reduces bias in model parameter estimates compared to an alternative approach used in the literature, minimal path estimation. We conclude that in estimating intermittently observed semi-Markov models, the proposed approach is a computationally feasible and accurate estimation procedure that leads to substantial improvements in back transition estimates.

Markov Chain-Based Acute Effect Estimation of Air Pollution on Elder Asthma Hospitalization

Background

Asthma caused substantial economic and health care burden and is susceptible to air pollution. Particularly, when it comes to elder asthma patient (older than 65), the phenomenon is more significant. The aim of this study is to investigate the Markov-based acute effects of air pollution on elder asthma hospitalizations, in forms of transition probabilities.

Methods

A retrospective, population-based study design was used to assess temporal patterns in hospitalizations for asthma in a region of Sichuan province, China. Approximately 12 million residents were covered during this period. Relative risk analysis and Markov chain model were employed on daily hospitalization state estimation.

Results

Among PM2.5, PM10, NO₂, and SO₂, only SO₂ was significant. When air pollution is severe, the transition probability from a low-admission state (previous day) to high-admission state (next day) is 35.46%, while it is 20.08% when air pollution is mild. In particular, for female-cold subgroup, the counterparts are 30.06% and 0.01%, respectively.

Conclusions

SO₂ was a significant risk factor for elder asthma hospitalization. When air pollution worsened, the transition probabilities from each state to high admission states increase dramatically. This phenomenon appeared more evidently, especially in female-cold subgroup (which is in cold season for female admissions). Based on our work, admission amount forecast, asthma intervention, and corresponding healthcare allocation can be done.

Modelling environmental drivers of black band disease outbreaks in populations of foliose corals in the genus Montipora

Seawater temperature anomalies associated with warming climate have been linked to increases in coral disease outbreaks that have contributed to coral reef declines globally. However, little is known about how seasonal scale variations in environmental factors influence disease dynamics at the level of individual coral colonies. In this study, we applied a multi-state Markov model (MSM) to investigate the dynamics of black band disease (BBD) developing from apparently healthy corals and/or a precursor-stage, termed ‘cyanobacterial patches’ (CP), in relation to seasonal variation in light and seawater temperature at two reef sites around Pelorus Island in the central sector of the Great Barrier Reef. The model predicted that the proportion of colonies transitioning from BBD to Healthy states within three months was approximately 57%, but 5.6% of BBD cases resulted in whole colony mortality. According to our modelling, healthy coral colonies were more susceptible to BBD during summer months when light levels were at their maxima and seawater temperatures were either rising or at their maxima. In contrast, CP mostly occurred during spring, when both light and seawater temperatures were rising. This suggests that environmental drivers for healthy coral colonies transitioning into a CP state are different from those driving transitions into BBD. Our model predicts that (1) the transition from healthy to CP state is best explained by increasing light, (2) the transition between Healthy to BBD occurs more frequently from early to late summer, (3) 20% of CP infected corals developed BBD, although light and temperature appeared to have limited impact on this state transition, and (4) the number of transitions from Healthy to BBD differed significantly between the two study sites, potentially reflecting differences in localised wave action regimes.

A Multi-State Model Examining Patterns of Transitioning Among States of Engagement in Care in HIV-Positive Individuals Initiating Combination Antiretroviral Therapy

BACKGROUND

Common measures of engagement in care fail to acknowledge that infrequent follow-up may occur either intentionally among patients with sustained virologic suppression or unintentionally among patients with poor clinical outcomes.

METHODS

Five states of HIV care were defined within the Canadian Observational Cohort Collaboration following combination antiretroviral therapy (cART) initiation: (1) guidelines HIV care [suppressed viral load (VL) and CD4 >200 cells per cubic millimeter, no gaps in cART >3 months, no gaps in CD4 or VL measurement >6 months], (2) successful care with decreased frequency of follow-up (as above except no gaps in CD4 or VL measurement >12 months), (3) suboptimal care (unsuppressed VL, CD4 <200 cells per cubic millimeter on 2 consecutive visits, ≥1 gap in cART >3 months, or ≥1 gap in CD4 or VL measurement >12 months), (4) loss to follow-up (no contact for 18 months), and (5) death. Multi-state models were used to determine factors associated with transitioning among states.

RESULTS

In total, 7810 participants were included. Younger age, female gender, Indigenous ethnicity, and people who have injected drugs were associated with increased likelihoods of transitioning from guidelines to suboptimal care and decreased likelihoods of transitioning from suboptimal to guidelines care. One-fifth of individuals in successful, decreased follow-up after cART initiation (mean sojourn time 0.72 years) were in suboptimal care in subsequent years.

CONCLUSIONS

Using routinely collected data, we have developed a flexible framework that characterizes patient transitions among states of HIV clinical care. We have demonstrated that multi-state models provide a useful approach to supplement "cascade of care" work.

Estimation of state occupancy probabilities in multistate models with dependent intermittent observation, with application to HIV viral rebounds

In follow-up studies on chronic disease cohorts, individuals are often observed at irregular visit times that may be related to their previous disease history and other factors. This can produce bias in standard methods of estimation. Working in the context of multistate models, we consider a method of nonparametric estimation for state occupancy probabilities that adjusts for dependent follow-up through the use of inverse-intensity-of-visit weighted estimating functions and smoothing. The methodology is applied to the estimation of viral rebound probabilities in the Canadian Observational Cohort on HIV-positive persons. Copyright © 2016 John Wiley & Sons, Ltd.

Estimating hazard ratios in cohort data with missing disease information due to death

In clinical and epidemiological studies information on the primary outcome of interest, that is, the disease status, is usually collected at a limited number of follow-up visits. The disease status can often only be retrieved retrospectively in individuals who are alive at follow-up, but will be missing for those who died before. Right-censoring the death cases at the last visit (ad-hoc analysis) yields biased hazard ratio estimates of a potential risk factor, and the bias can be substantial and occur in either direction. In this work, we investigate three different approaches that use the same likelihood contributions derived from an illness-death multistate model in order to more adequately estimate the hazard ratio by including the death cases into the analysis: a parametric approach, a penalized likelihood approach, and an imputation-based approach. We investigate to which extent these approaches allow for an unbiased regression analysis by evaluating their performance in simulation studies and on a real data example. In doing so, we use the full cohort with complete illness-death data as reference and artificially induce missing information due to death by setting discrete follow-up visits. Compared to an ad-hoc analysis, all considered approaches provide less biased or even unbiased results, depending on the situation studied. In the real data example, the parametric approach is seen to be too restrictive, whereas the imputation-based approach could almost reconstruct the original event history information.

Continuous time Markov chain approaches for analyzing transtheoretical models of health behavioral change: A case study and comparison of model estimations

Continuous time Markov chain models are frequently employed in medical research to study the disease progression but are rarely applied to the transtheoretical model, a psychosocial model widely used in the studies of health-related outcomes. The transtheoretical model often includes more than three states and conceptually allows for all possible instantaneous transitions (referred to as general continuous time Markov chain). This complicates the likelihood function because it involves calculating a matrix exponential that may not be simplified for general continuous time Markov chain models. We undertook a Bayesian approach wherein we numerically evaluated the likelihood using ordinary differential equation solvers available from the gnu scientific library. We compared our Bayesian approach with the maximum likelihood method implemented with the R package MSM. Our simulation study showed that the Bayesian approach provided more accurate point and interval estimates than the maximum likelihood method, especially in complex continuous time Markov chain models with five states. When applied to data from a four-state transtheoretical model collected from a nutrition intervention study in the next step trial, we observed results consistent with the results of the simulation study. Specifically, the two approaches provided comparable point estimates and standard errors for most parameters, but the maximum likelihood offered substantially smaller standard errors for some parameters. Comparable estimates of the standard errors are obtainable from package MSM, which works only when the model estimation algorithm converges.

Analysis of transtheoretical model of health behavioral changes in a nutrition intervention study--a continuous time Markov chain model with Bayesian approach

Continuous time Markov chain (CTMC) models are often used to study the progression of chronic diseases in medical research but rarely applied to studies of the process of behavioral change. In studies of interventions to modify behaviors, a widely used psychosocial model is based on the transtheoretical model that often has more than three states (representing stages of change) and conceptually permits all possible instantaneous transitions. Very little attention is given to the study of the relationships between a CTMC model and associated covariates under the framework of transtheoretical model. We developed a Bayesian approach to evaluate the covariate effects on a CTMC model through a log-linear regression link. A simulation study of this approach showed that model parameters were accurately and precisely estimated. We analyzed an existing data set on stages of change in dietary intake from the Next Step Trial using the proposed method and the generalized multinomial logit model. We found that the generalized multinomial logit model was not suitable for these data because it ignores the unbalanced data structure and temporal correlation between successive measurements. Our analysis not only confirms that the nutrition intervention was effective but also provides information on how the intervention affected the transitions among the stages of change. We found that, compared with the control group, subjects in the intervention group, on average, spent substantively less time in the precontemplation stage and were more/less likely to move from an unhealthy/healthy state to a healthy/unhealthy state.

Effect of common neuropathologies on progression of late life cognitive impairment

Brain pathologies of Alzheimer's (AD), cerebrovascular, and Lewy body diseases are common in old age, but the relationship of these pathologies with progression from normal cognitive function to the various stages of cognitive impairment is unknown. In this study, we fit latent Markov models from longitudinal cognitive data to empirically derive 3 latent stages corresponding to no impairment, mild impairment, and moderate impairment; then, we examined the associations of common neuropathologies with the rates of transition among these stages. Cognitive and neuropathological data were available from 653 autopsied participants in 2 ongoing cohort studies of aging who were cognitively healthy at baseline (mean baseline age 79.1 years) and had longitudinal cognitive data. On average, participants in these analyses developed mild impairment 5 years after enrollment, progressed to moderate impairment after an additional 3.4 years, and stayed impaired for 2.8 years until death. AD and chronic macroscopic infarcts were associated with a higher risk of progression to mild impairment and subsequently to moderate impairment. By contrast, Lewy bodies were associated only with progression from mild to moderate impairment. The 5-year probability of progression to mild or moderate impairment was 20% for persons without any of these 3 pathologies, 38% for AD only, 51% for AD and macroscopic infarcts, and 56% for AD, infarcts, and Lewy bodies. Thus, the presence of AD pathology alone nearly doubles the risk of developing cognitive impairment in late life, and the presence of multiple pathologies further increases this risk over multiple years before death.

Characterization of dementia and Alzheimer's disease in an older population: updated incidence and life expectancy with and without dementia

OBJECTIVES

We estimated dementia incidence rates, life expectancies with and without dementia, and percentage of total life expectancy without dementia.

METHODS

We studied 3605 members of Group Health (Seattle, WA) aged 65 years or older who did not have dementia at enrollment to the Adult Changes in Thought study between 1994 and 2008. We estimated incidence rates of Alzheimer's disease and dementia, as well as life expectancies with and without dementia, defined as the average number of years one is expected to live with and without dementia, and percentage of total life expectancy without dementia.

RESULTS

Dementia incidence increased through ages 85 to 89 years (74.2 cases per 1000 person-years) and 90 years or older (105 cases per 1000 person-years). Life expectancy without dementia and percentage of total life expectancy without dementia decreased with age. Life expectancy with dementia was longer in women and people with at least a college degree. Percentage of total life expectancy without dementia was greater in younger age groups, men, and those with more education.

CONCLUSIONS

Efforts to delay onset of dementia, if successful, would likely benefit older adults of all ages.

Estimation and assessment of markov multistate models with intermittent observations on individuals

Multistate models provide important methods of analysis for many life history processes, and this is an area where John Klein made numerous contributions. When individuals in a study group are observed continuously so that all transitions between states, and their times, are known, estimation and model checking is fairly straightforward. However, individuals in many studies are observed intermittently, and only the states occupied at the observation times are known. We review methods of estimation and assessment for Markov models in this situation. Numerical studies that show the effects of inter-observation times are provided, and new methods for assessing fit are given. An illustration involving viral load dynamics for HIV-positive persons is presented.

A model-informed rank test for right-censored data with intermediate states

The generalized Wilcoxon and log-rank tests are commonly used for testing differences between two survival distributions. We modify the Wilcoxon test to account for auxiliary information on intermediate disease states that subjects may pass through before failure. For a disease with multiple states where patients are monitored periodically but exact transition times are unknown (e.g. staging in cancer), we first fit a multi-state Markov model to the full data set; when censoring precludes the comparison of survival times between two subjects, we use the model to estimate the probability that one subject will have survived longer than the other given their censoring times and last observed status, and use these probabilities to compute an expected rank for each subject. These expected ranks form the basis of our test statistic. Simulations demonstrate that the proposed test can improve power over the log-rank and generalized Wilcoxon tests in some settings while maintaining the nominal type 1 error rate. The method is illustrated on an amyotrophic lateral sclerosis data set.

Cumulative sojourn time in longitudinal studies: a sequential imputation method to handle missing health state data due to dropout

Missing data are ubiquitous in longitudinal studies. In this paper, we propose an imputation procedure to handle dropouts in longitudinal studies. By taking advantage of the monotone missing pattern resulting from dropouts, our imputation procedure can be carried out sequentially, which substantially reduces the computation complexity. In addition, at each step of the sequential imputation, we set up a model selection mechanism that chooses between a parametric model and a nonparametric model to impute eachmissing observation. Unlike usual model selection procedures that aim at finding a single model fitting the entire data set well, our model selection procedure is customized to find a suitable model for the prediction of each missing observation.

Misclassification can explain most apparent regression of age-related macular degeneration: results from multistate models with misclassification

PURPOSE

To investigate the impact of misclassification of age-related macular degeneration (AMD) on the baseline intensity and estimated effects of age, sex, and the Y402H variant in the complement factor H (CFH) gene on incidence, progression, and regression of AMD.

METHODS

The Beaver Dam Eye Study, a longitudinal population-based study of age-related eye diseases conducted in the city and township of Beaver Dam, Wisconsin, performed examinations every 5 years during a 20-year period (1988-1990 through 2008-2010). Study participants (N = 4379) aged 43 to 86 years at the baseline examination had retinal photographs taken at baseline and up to four subsequent examinations. Multistate models with misclassification in continuous time were used to model the effects of age, sex, and CFH genotype on incidence, progression, and regression of AMD and mortality.

RESULTS

After accounting for AMD misclassification, the occurrence of any AMD regression was rare (1%-4%), while it was relatively common (14%-21%) in models that do not account for misclassification. Failure to account for misclassification attenuated estimated age effects on incidence and progression to moderately severe early AMD and attenuated estimated CFH effects on incidence and progressions to moderately severe and severe early AMD.

CONCLUSIONS

Apparent regression of AMD can largely, if not completely, be explained by misclassification. Estimated age effects on incidence and progression to moderately severe early AMD and estimated CFH effects on incidence and progressions to moderately severe and severe early AMD were attenuated in multistate models that did not account for misclassification.

Methods for estimating kidney disease stage transition probabilities using electronic medical records

Chronic diseases are often described by stages of severity. Clinical decisions about what to do are influenced by the stage, whether a patient is progressing, and the rate of progression. For chronic kidney disease (CKD), relatively little is known about the transition rates between stages. To address this, we used electronic health records (EHR) data on a large primary care population, which should have the advantage of having both sufficient follow-up time and sample size to reliably estimate transition rates for CKD. However, EHR data have some features that threaten the validity of any analysis. In particular, the timing and frequency of laboratory values and clinical measurements are not determined a priori by research investigators, but rather, depend on many factors, including the current health of the patient. We developed an approach for estimating CKD stage transition rates using hidden Markov models (HMMs), when the level of information and observation time vary among individuals. To estimate the HMMs in a computationally manageable way, we used a "discretization" method to transform daily data into intervals of 30 days, 90 days, or 180 days. We assessed the accuracy and computation time of this method via simulation studies. We also used simulations to study the effect of informative observation times on the estimated transition rates. Our simulation results showed good performance of the method, even when missing data are non-ignorable. We applied the methods to EHR data from over 60,000 primary care patients who have chronic kidney disease (stage 2 and above). We estimated transition rates between six underlying disease states. The results were similar for men and women.

Fitting and interpreting continuous-time latent Markov models for panel data

Multistate models characterize disease processes within an individual. Clinical studies often observe the disease status of individuals at discrete time points, making exact times of transitions between disease states unknown. Such panel data pose considerable modeling challenges. Assuming the disease process progresses accordingly, a standard continuous-time Markov chain (CTMC) yields tractable likelihoods, but the assumption of exponential sojourn time distributions is typically unrealistic. More flexible semi-Markov models permit generic sojourn distributions yet yield intractable likelihoods for panel data in the presence of reversible transitions. One attractive alternative is to assume that the disease process is characterized by an underlying latent CTMC, with multiple latent states mapping to each disease state. These models retain analytic tractability due to the CTMC framework but allow for flexible, duration-dependent disease state sojourn distributions. We have developed a robust and efficient expectation-maximization algorithm in this context. Our complete data state space consists of the observed data and the underlying latent trajectory, yielding computationally efficient expectation and maximization steps. Our algorithm outperforms alternative methods measured in terms of time to convergence and robustness. We also examine the frequentist performance of latent CTMC point and interval estimates of disease process functionals based on simulated data. The performance of estimates depends on time, functional, and data-generating scenario. Finally, we illustrate the interpretive power of latent CTMC models for describing disease processes on a dataset of lung transplant patients. We hope our work will encourage wider use of these models in the biomedical setting.

A Correlated Random Effects Model for Non-homogeneous Markov Processes with Nonignorable Missingness

Life history data arising in clusters with prespecified assessment time points for patients often feature incomplete data since patients may choose to visit the clinic based on their needs. Markov process models provide a useful tool describing disease progression for life history data. The literature mainly focuses on time homogeneous process. In this paper we develop methods to deal with non-homogeneous Markov process with incomplete clustered life history data. A correlated random effects model is developed to deal with the nonignorable missingness, and a time transformation is employed to address the non-homogeneity in the transition model. Maximum likelihood estimate based on the Monte-Carlo EM algorithm is advocated for parameter estimation. Simulation studies demonstrate that the proposed method works well in many situations. We also apply this method to an Alzheimer's disease study.

State-space size considerations for disease-progression models

Markov models of disease progression are widely used to model transitions in patients' health state over time. Usually, patients' health status may be classified according to a set of ordered health states. Modelers lump together similar health states into a finite and usually small, number of health states that form the basis of a Markov chain disease-progression model. This increases the number of observations used to estimate each parameter in the transition probability matrix. However, lumping together observably distinct health states also obscures distinctions among them and may reduce the predictive power of the model. Moreover, as we demonstrate, precision in estimating the model parameters does not necessarily improve as the number of states in the model declines. This paper explores the tradeoff between lumping error introduced by grouping distinct health states and sampling error that arises when there are insufficient patient data to precisely estimate the transition probability matrix.

Effect of the Y402H variant in the complement factor H gene on the incidence and progression of age-related macular degeneration: results from multistate models applied to the Beaver Dam Eye Study

OBJECTIVES

To investigate the effect of age, sex, and the Y402H variant in the complement factor H (CFH) gene on the incidence, progression, and regression of age-related macular degeneration (AMD) as well as the effect of these factors and AMD on mortality, using multistate models.

METHODS

Analyses included 4379 persons aged 43 to 84 years at the time of the census. The status of AMD on a 5-level severity scale was graded from retinal photographs taken at up to 5 study visits between 1988 and 2010. Multistate models in continuous time were used to model the effects of age, sex, and CFH genotype on the incidence, progression, and regression of AMD and mortality.

RESULTS

The CFH Y402H genotype CC was associated, relative to genotype TT (reported as hazard ratio; 95% CI), with increased incidence of AMD (no to minimally severe early AMD, 1.98; 1.57-2.49), progression of AMD (minimally severe early to moderately severe early AMD, 1.73; 1.29-2.33; moderately severe early to severe early AMD, 1.30; 0.86-1.94; and severe early to late AMD, 1.72; 1.01-2.91) but not with regression of AMD or mortality. Late AMD was associated with increased mortality (1.37; 1.15-1.62) relative to no AMD, but earlier stages of AMD were not.

CONCLUSIONS

Using the multistate models, we show that the Y402H risk variant is associated with lifetime incidence of early AMD and progression of early to late AMD and that late AMD is associated with mortality risk.

Estimating complex multi-state misclassification rates for biopsy-measured liver fibrosis in patients with hepatitis C

For both clinical and research purposes, biopsies are used to classify liver damage known as fibrosis on an ordinal multi-state scale ranging from no damage to cirrhosis. Misclassification can arise from reading error (misreading of a specimen) or sampling error (the specimen does not accurately represent the liver). Studies of biopsy accuracy have not attempted to synthesize these two sources of error or to estimate actual misclassification rates from either source. Using data from two studies of reading error and two of sampling error, we find surprisingly large possible misclassification rates, including a greater than 50% chance of misclassification for one intermediate stage of fibrosis. We find that some readers tend to misclassify consistently low or consistently high, and some specimens tend to be misclassified low while others tend to be misclassified high. Non-invasive measures of liver fibrosis have generally been evaluated by comparison to simultaneous biopsy results, but biopsy appears to be too unreliable to be considered a gold standard. Non-invasive measures may therefore be more useful than such comparisons suggest. Both stochastic uncertainty and uncertainty about our model assumptions appear to be substantial. Improved studies of biopsy accuracy would include large numbers of both readers and specimens, greater effort to reduce or eliminate reading error in studies of sampling error, and careful estimation of misclassification rates rather than less useful quantities such as kappa statistics.

Salmonella fecal shedding and immune responses are dose- and serotype- dependent in pigs

Despite the public health importance of Salmonella infection in pigs, little is known about the associated dynamics of fecal shedding and immunity. In this study, we investigated the transitions of pigs through the states of Salmonella fecal shedding and immune response post-Salmonella inoculation as affected by the challenge dose and serotype. Continuous-time multistate Markov models were developed using published experimental data. The model for shedding had four transient states, of which two were shedding (continuous and intermittent shedding) and two non-shedding (latency and intermittent non-shedding), and one absorbing state representing permanent cessation of shedding. The immune response model had two transient states representing responses below and above the seroconversion level. The effects of two doses [low (0.65×10⁶ CFU/pig) and high (0.65×10⁹ CFU/pig)] and four serotypes (Salmonella Yoruba, Salmonella Cubana, Salmonella Typhimurium, and Salmonella Derby) on the models' transition intensities were evaluated using a proportional intensities model. Results indicated statistically significant effects of the challenge dose and serotype on the dynamics of shedding and immune response. The time spent in the specific states was also estimated. Continuous shedding was on average 10–26 days longer, while intermittent non-shedding was 2–4 days shorter, in pigs challenged with the high compared to low dose. Interestingly, among pigs challenged with the high dose, the continuous and intermittent shedding states were on average up to 10–17 and 3–4 days longer, respectively, in pigs infected with S. Cubana compared to the other three serotypes. Pigs challenged with the high dose of S. Typhimurium or S. Derby seroconverted on average up to 8–11 days faster compared to the low dose. These findings highlight that Salmonella fecal shedding and immune response following Salmonella challenge are dose- and serotype-dependent and that the detection of specific Salmonella strains and immune responses in pigs are time-sensitive.