Validation and Recalibration of Two Multivariable Prognostic Models for Survival and Independence in Acute Stroke

Dynamic updating of clinical survival prediction models in a changing environment

BACKGROUND

Over time, the performance of clinical prediction models may deteriorate due to changes in clinical management, data quality, disease risk and/or patient mix. Such prediction models must be updated in order to remain useful. In this study, we investigate dynamic model updating of clinical survival prediction models. In contrast to discrete or one-time updating, dynamic updating refers to a repeated process for updating a prediction model with new data. We aim to extend previous research which focused largely on binary outcome prediction models by concentrating on time-to-event outcomes. We were motivated by the rapidly changing environment seen during the COVID-19 pandemic where mortality rates changed over time and new treatments and vaccines were introduced.

METHODS

We illustrate three methods for dynamic model updating: Bayesian dynamic updating, recalibration, and full refitting. We use a simulation study to compare performance in a range of scenarios including changing mortality rates, predictors with low prevalence and the introduction of a new treatment. Next, the updating strategies were applied to a model for predicting 70-day COVID-19-related mortality using patient data from QResearch, an electronic health records database from general practices in the UK.

RESULTS

In simulated scenarios with mortality rates changing over time, all updating methods resulted in better calibration than not updating. Moreover, dynamic updating outperformed ad hoc updating. In the simulation scenario with a new predictor and a small updating dataset, Bayesian updating improved the C-index over not updating and refitting. In the motivating example with a rare outcome, no single updating method offered the best performance.

CONCLUSIONS

We found that a dynamic updating process outperformed one-time discrete updating in the simulations. Bayesian updating offered good performance overall, even in scenarios with new predictors and few events. Intercept recalibration was effective in scenarios with smaller sample size and changing baseline hazard. Refitting performance depended on sample size and produced abrupt changes in hazard ratio estimates between periods.

Prognostic model for predicting overall survival in patients with glioblastoma: an analysis based on the SEER database

Predicting the prognosis of glioblastoma (GBM) has always been important for improving survival. An understanding of the prognostic factors for patients with GBM can help guide treatment. Herein, we aimed to construct a prognostic model for predicting overall survival (OS) for patients with GBM. We identified 11,375 patients with pathologically confirmed GBM from the Surveillance, Epidemiology, and End Results database between 2004 and 2015. The 1-, 2-, and 3-year survival probabilities were 48.8%, 22.5%, and 13.1%, respectively. The patients were randomly divided into the training cohort (n = 8531) and the validation cohort (n = 2844). A Cox proportional risk regression model was used to analyze the prognostic factors of patients in the training cohort, and a nomogram was constructed. Then concordance indexes (C-indexes), calibration curves, and receiver operating characteristic (ROC) curves were used to assess the performance of the nomograms by internal (training cohort) and external validation (validation cohort). Log-rank test and univariate analysis showed that age, race, marital status, extent of surgical resection, chemotherapy, and radiation were the prognostic factors for patients with GBM (p < 0.05), which were used to construct nomogram. The C-index of the nomogram was 0.717 (95% confidence interval (CI), 0.710-0.724) in the training cohort, and 0.724 (95% CI, 0.713-0.735) in the validation cohort. The nomogram had a higher areas under the ROC curve value. The nomogram was well validated, which can effectively predict the OS of patients with GBM. Thus, this nomogram could be applied in clinical practice.

Development and External Validation of a Model Predicting New-Onset Chronic Uveitis at Different Disease Durations in Juvenile Idiopathic Arthritis

OBJECTIVE

To develop and externally validate a prediction model for new-onset chronic uveitis in children with juvenile idiopathic arthritis (JIA) for clinical application.

METHODS

Data from the international Pharmachild registry were used to develop a multivariable Cox proportional hazards model. Predictors were selected by backward selection, and missing values were handled by multiple imputation. The model was subsequently validated and recalibrated in 2 inception cohorts: the UK Childhood Arthritis Prospective Study (CAPS) study and the German Inception Cohort of Newly diagnosed patients with juvenile idiopathic arthritis (ICON) study. Model performance was evaluated by calibration plots and C statistics for the 2-, 4-, and 7-year risk of uveitis. A diagram and digital risk calculator were created for use in clinical practice.

RESULTS

A total of 5,393 patients were included for model development, and predictor variables were age at JIA onset (hazard ratio [HR] 0.83 [95% confidence interval (95% CI) 0.77-0.89]), ANA positivity (HR 1.59 [95% CI 1.06-2.38]), and International League of Associations for Rheumatology category of JIA (HR for oligoarthritis, psoriatic arthritis, and undifferentiated arthritis versus rheumatoid factor-negative polyarthritis 1.40 [95% CI 0.91-2.16]). Performance of the recalibrated prediction model in the validation cohorts was acceptable; calibration plots indicated good calibration and C statistics for the 7-year risk of uveitis (0.75 [95% CI 0.72-0.79] for the ICON cohort and 0.70 [95% CI 0.64-0.76] for the CAPS cohort).

CONCLUSION

We present for the first time a validated prognostic tool for easily predicting chronic uveitis risk for individual JIA patients using common clinical parameters. This model could be used by clinicians to inform patients/parents and provide guidance in choice of uveitis screening frequency and arthritis drug therapy.

Genetics in aphasia recovery

Considerable research efforts have been exerted toward understanding the mechanisms underlying recovery in aphasia. However, predictive models of spontaneous and treatment-induced recovery remain imprecise. Some of the hitherto unexplained variability in recovery may be accounted for with genetic data. A few studies have examined the effects of the BDNF val66met polymorphism on aphasia recovery, yielding mixed results. Advances in the study of stroke genetics and genetics of stroke recovery, including identification of several susceptibility genes through candidate-gene or genome-wide association studies, may have implications for the recovery of language function. The current chapter discusses both the direct and indirect evidence for a genetic basis of aphasia recovery, the implications of recent findings within the field, and potential future directions to advance understanding of the genetics-recovery associations.

Advanced cardiovascular risk prediction in the emergency department: updating a clinical prediction model - a large database study protocol

BACKGROUND

Patients presenting with chest pain represent a large proportion of attendances to emergency departments. In these patients clinicians often consider the diagnosis of acute myocardial infarction (AMI), the timely recognition and treatment of which is clinically important. Clinical prediction models (CPMs) have been used to enhance early diagnosis of AMI. The Troponin-only Manchester Acute Coronary Syndromes (T-MACS) decision aid is currently in clinical use across Greater Manchester. CPMs have been shown to deteriorate over time through calibration drift. We aim to assess potential calibration drift with T-MACS and compare methods for updating the model.

METHODS

We will use routinely collected electronic data from patients who were treated using TMACS at two large NHS hospitals. This is estimated to include approximately 14,000 patient episodes spanning June 2016 to October 2020. The primary outcome of acute myocardial infarction will be sourced from NHS Digital's admitted patient care dataset. We will assess the calibration drift of the existing model and the benefit of updating the CPM by model recalibration, model extension and dynamic updating. These models will be validated by bootstrapping and one step ahead prequential testing. We will evaluate predictive performance using calibrations plots and c-statistics. We will also examine the reclassification of predicted probability with the updated TMACS model.

DISCUSSION

CPMs are widely used in modern medicine, but are vulnerable to deteriorating calibration over time. Ongoing refinement using routinely collected electronic data will inevitably be more efficient than deriving and validating new models. In this analysis we will seek to exemplify methods for updating CPMs to protect the initial investment of time and effort. If successful, the updating methods could be used to continually refine the algorithm used within TMACS, maintaining or even improving predictive performance over time.

TRIAL REGISTRATION

ISRCTN number: ISRCTN41008456.

The Nomogram Model Predicting Overall Survival and Guiding Clinical Decision in Patients With Glioblastoma Based on the SEER Database

Background: Patients with glioblastoma have a poor prognosis. We want to develop and validate nomograms for predicting overall survival in patients with glioblastoma. Methods: Data of patients with glioblastoma diagnosed pathologically in the SEER database from 2007 to 2016 were collected by SEER*Stat software. After eliminating invalid and missing clinical information, 3,635 patients (total group) were finally identified and randomly divided into the training group (2,183 cases) and the verification group (1,452 cases). Cox proportional risk regression model was used in the training group, the verification group and the total group to analyze the prognostic factors of patients in the training group, and then the nomogram was constructed. C-indexes and calibration curves were used to evaluate the predictive value of nomogram by internal (training group data) and external validation (verification group data). Results: Cox proportional risk regression model in the training group showed that age, year of diagnosis, laterality, radiation, chemotherapy were all influential factors for prognosis of patients with glioblastoma (P < 0.05) and were all used to construct nomogram as well. The internal and external validation results of nomogram showed that the C-index of the training group was 0.729 [95% CI was (0.715, 0.743)], and the verification group was 0.734 [95% CI was (0.718, 0.750)]. The calibration curves of both groups showed good consistency. Conclusions: The proposed nomogram resulted in accurate prognostic prediction for patients with glioblastoma.

Dementia is Associated With Poorer Quality of Care and Outcomes After Stroke: An Observational Study

Background

To determine whether preexisting dementia is associated with poorer quality of care and outcomes after stroke in the acute hospital phase.

Method

This was a retrospective analysis of pooled data from the Australian Stroke Foundation national audit conducted in 2015 and 2017. Dementia status was obtained from the medical records. Processes of care to assess quality included: stroke unit care, time-dependent therapy, nursing/allied health assessments, and preparation for discharge. Outcomes included in-hospital complications, independence on discharge, and destination. Logistic regression was used to examine associations between dementia status and processes of care. Multilevel random effects logistic regression, with level defined as hospital, was used to examine associations between dementia status and outcomes.

Results

There were 683/7,070 (9.7%) audited patients with dementia included. Patients with dementia were less likely to be treated in stroke units (58.3% vs 70.6%), receive thrombolysis if an ischemic stroke (5.8% vs 11.1%), have access within 48 hours to physiotherapy (56.4% vs 69.7%) or occupational therapy (46.8% vs 55.6%), see a dietitian if problems with nutrition (64.4% vs 75.9%), or have mood assessed (2.6% vs 12.3%). Patients with dementia were more likely to receive no rehabilitation (adjusted odds ratio 1.88, 95% confidence interval 1.25, 2.83) and be discharged to residential care (adjusted odds ratio 2.36, 95% confidence interval 1.50, 3.72).

Conclusion

People with dementia received poorer quality of care and had worse outcomes after stroke. Our findings raise questions regarding equity and the need for better understanding of why the quality of care differs after stroke for people with dementia.

A parsimonious score with a free web tool for predicting disability after an ischemic stroke: the Parsifal Score

BACKGROUND

Most of the models to predict prognosis after an ischemic stroke include complex mathematical equations or too many variables, making them difficult to use in the daily clinic. We want to predict disability 3 months after an ischemic stroke in an independent patient not receiving recanalization treatment within the first 24 h, using a minimum set of variables and an easy tool to facilitate its implementation. As a secondary aim, we calculated the capacity of the score to predict an excellent/devastating outcome and mortality.

METHODS

Eight hundred and forty-four patients were evaluated. A multivariable ordinal logistic regression was used to obtain the score. The Modified Rankin Scale (mRS) was used to estimate disability at the third month. The results were replicated in another independent cohort (378 patients). The "polr" function of R was used to perform the regression, stratifying the sample into seven groups with different cutoffs (from mRS 0 to 6).

RESULTS

The Parsifal score was generated with: age, previous mRS, initial NIHSS, glycemia on admission, and dyslipidemia. This score predicts disability with an accuracy of 80-76% (discovery-replication cohorts). It has an AUC of 0.86 in the discovery and replication cohort. The specificity was 90-80% (discovery-replication cohorts); while, the sensitivity was 64-74% (discovery-replication cohorts). The prediction of an excellent or devastating outcome, as well as mortality, obtained good discrimination with AUC > 0.80.

CONCLUSIONS

The Parsifal Score is a model that predicts disability at the third month, with only five variables, with good discrimination and calibration, and being replicated in an independent cohort.

Risk of Long-Term Mortality for Complex Chronic Patients with Intracerebral Hemorrhage: A Population-Based e-Cohort Observational Study

INTRODUCTION

Over recent years there has been growing evidence of increased risk of mortality associated with hemorrhagic stroke among older patients. The main objective of this study is to propose and validate a prognostic life table for complex chronic patients after an intracerebral hemorrhage (ICH) episode in primary care settings.

METHODS

This was a multicenter and retrospective study (April 1, 2006-December 31, 2016) of a cohort from the general population presenting an episode of ICH from which a predictive model of mortality was obtained using a Cox proportional hazards regression model. In addition, Kaplan-Meier survival curves, the log-rank test, receiver operating characteristic (ROC) curves, and area under the ROC curve (AUC) were used to evaluate the ability to stratify patients according to vital prognosis. We proceeded to external validation of the model through prospective monitoring (January 1, 2013-December 31, 2017) of the population of complex chronic patients with an episode of ICH.

RESULTS

A total of 3594 people aged ≥ 65 years were identified as complex chronic patients (women 55.9%; mean age, 86.1 ± 8.4 years) of whom 161 suffered hemorrhagic stroke during the study period (January 1, 2013-December 31, 2017). The primary outcome was death from any cause within 5 years of follow-up after an ICH episode. The independent prognostic factors of mortality were age > 80 years (HR 1.048, 95% CI 1.021-1.076, p < 0.001) and HAS-BLED score (HR 1.369, 95% CI 1.057-1.774, p = 0.017). Compared to the general population, the incidence density/1000 person per year (15 vs 0.22) was significantly higher with a significantly lower annual lethality rate (17% vs 49.2%); and both the prognostic factors and the risk of stratified mortality showed different epidemiological patterns. The internal validation of the model was optimal (log-rank < 0.0001) in the general population, but its external validation was not significant in the complex chronic patient population (log-rank p = 0.104).

CONCLUSIONS

The ICH-AP is a clinical scale that can improve the prognostic prediction of mortality in primary care after an episode of ICH in the general population, but it was not significant in its external validation in a population of complex chronic patients.

TRIAL REGISTRATION

ClinicalTrials.gov ID: NCT03247049.

Deiodinases, organic anion transporter polypeptide polymorphisms and ischemic stroke outcomes

BACKGROUND

Ischemic stroke is a major cause of premature death and chronic disability worldwide, and individual variation in functional outcome is strongly influenced by genetic factors. Neuroendocrine signaling by the hypothalamic-hypophyseal-thyroid axis is a critical regulator of post-stroke pathogenesis, suggesting that allelic variants in thyroid hormone (TH) signaling can influence stroke outcome.

AIM

To examine associations between acute ischemic stroke (AIS) outcome and allelic variants of the TH metabolizing enzymes deiodinase type 1-3 (DIO1-3) and membrane transporting organic anion polypeptide C1 (OATP1C1).

METHODS

Eligible AIS patients from Lithuania (n = 248) were genotyped for ten DIO1-3 and OATP1C1 single nucleotide polymorphisms (SNPs): DIO1 rs12095080-A/G, rs11206244-C/T, and rs2235544-A/C; DIO2 rs225014-T/C and rs225015-G/A; DIO3 rs945006-T/G; OATP1C1 rs974453-G/A, rs10444412-T/C, rs10770704-C/T, and rs1515777-A/G. Functional outcome was evaluated one year after index AIS using the modified Rankin Scale. Analyses were adjusted for important confounders, including serum free triiodothyronine.

RESULTS

After adjustment for potential confounders, the major allelic (wild-type) DIO3 genotype rs945006-TT was associated with better 1-year AIS functional outcome (odds ratio [OR] = 0.25; 95% confidence interval [CI]: 0.08-0.74; p = .013), while the wild-type OATP1C1 genotype rs10770704-CC was associated with poorer outcome (OR = 2.00, 95%CI: 1.04-3.86; p = .038).

CONCLUSION

Allelic variants in thyroid axis genes may prove useful for prognosis and treatment guidance.

Sex Differences in Care and Long-Term Mortality After Stroke: Australian Stroke Clinical Registry

Introduction: There is some evidence that women receive evidence-based care less often than men, but how this influences long-term mortality after stroke is unclear. We explored this issue using data from a national stroke registry. Materials and Methods: Data are first-ever hospitalized strokes (2010-2014) in the Australian Stroke Clinical Registry from 39 hospitals linked to the national death registrations. Multilevel Poisson regression was used to estimate the women:men mortality rate ratio (MRR), with adjustment for sociodemographics, stroke severity, and processes of care (stroke unit care, intravenous thrombolysis, antihypertensive agent[s], and discharge care plan). Results: Among 14,118 events (46% females), women were 7 years older and had greater baseline severity compared to men (29% vs. 37%; p < 0.001), but there were no differences in the four processes of care available across hospitals. In the whole cohort, 1-year mortality was greater in women than men (MRR_unadjusted 1.44, 95% confidence interval [CI] 1.34-1.54). However, there were no differences after adjusting for age and stroke severity (MRR_adjusted 1.03, 95% CI 0.95-1.10). In analyses of additional processes from Queensland hospitals (n = 5224), women were less often administered aspirin ≤48 hours (61% vs. men 69%, p < 0.015). In Queensland hospitals, there were no statistically significant sex differences in 1-year mortality after adjusting for age, stroke severity, and early administration of aspirin. Conclusion: Greater mortality in women can be explained by differences in age and stroke severity. This highlights the importance of better management of risk factors in the elderly and, potentially, the need for greater access to early aspirin for women with stroke.

Is length of time in a stroke unit associated with better outcomes for patients with stroke in Australia? An observational study

OBJECTIVE

Spending at least 90% of hospital admission in a stroke unit (SU) is a recommended indicator of receiving high-quality stroke care. However, whether this makes a difference to patient outcomes is unknown. We aimed to investigate outcomes and factors associated with patients with acute stroke spending at least 90% of their admission in an SU, compared with those having less time in the SU.

DESIGN

Observational study using cross-sectional data.

SETTING

Data from hospitals which participated in the 2015 Stroke Foundation National Audit: Acute Services (Australia) and had an SU. This audit includes an organisational survey and retrospective medical record audit of approximately 40 admissions from each hospital.

PARTICIPANTS

Patients admitted to an SU during their acute admission were included.

OUTCOME MEASURES

Hospital-based patient outcomes included length of stay, independence on discharge, severe complications and discharge destination. Patient, organisational and process indicators were included in multilevel logistic modelling to determine factors associated with spending at least 90% of their admission in an SU.

RESULTS

Eighty-eight hospitals with an SU audited 2655 cases (median age 76 years, 55% male). Patients who spent at least 90% of their admission in an SU experienced: a length of stay that was 2 days shorter (coefficient -2.77, 95% CI -3.45 to -2.10), fewer severe complications (adjusted OR (aOR) 0.60, 95% CI 0.43 to 0.84) and were less often discharged to residential aged care (aOR 0.59, 95% CI 0.38 to 0.94) than those who had less time in the SU. Patients admitted to an SU within 3 hours of hospital arrival were three times more likely to spend at least 90% of their admission in an SU.

CONCLUSION

Spending at least 90% of time in an SU is a valid measure of stroke care quality as it results in improved patient outcomes. Direct admission to SUs is warranted.

Using routine HbA1c measurements in stroke and the associations of dysglycaemia with stroke outcomes

AIMS

Diabetes is a major risk factor for stroke. We aimed to investigate the prevalence of diabetes and pre-diabetes within a stroke cohort and examine the association of glycaemia status with mortality and morbidity.

METHODS

Inpatients aged ≥54 who presented with a diagnosis of stroke had a routine HbA1c measurement as part of the Austin Health Diabetes Discovery Initiative. Additional data were attained from hospital databases and Australian Stroke Clinical Registry. Outcomes included diabetes and pre-diabetes prevalence, length of stay, 6-month and in-hospital mortality, 28-day readmission rates, and 3-month modified Rankin scale score.

RESULTS

Between July 2013 and December 2015, 610 patients were studied. Of these, 31% had diabetes while 40% had pre-diabetes. Using multivariable regression analyses, the presence of diabetes was associated with higher odds of 6-month mortality (OR = 1.90, p = 0.022) and higher expected length of stay (IRR = 1.29, p = 0.004). Similarly, a higher HbA1c was associated with higher odds of 6-month mortality (OR = 1.27, p = 0.005) and higher expected length of stay (IRR = 1.08, p = 0.010).

CONCLUSIONS

71% of this cohort had diabetes or pre-diabetes. Presence of diabetes and higher HbA1c were associated with higher 6-month mortality and length of stay. Further research is necessary to determine if improved glycaemic control may improve stroke outcomes.

Quality of Acute Care and Long-Term Quality of Life and Survival

Background and Purpose—

Uncertainty exists over whether quality improvement strategies translate into better health-related quality of life (HRQoL) and survival after acute stroke. We aimed to determine the association of best practice recommended interventions and outcomes after stroke.

Methods—

Data are from the Australian Stroke Clinical Registry during 2010 to 2014. Multivariable regression was used to determine associations between 3 interventions: received acute stroke unit (ASU) care and in various combinations with prescribed antihypertensive medication at discharge, provision of a discharge care plan, and outcomes of survival and HRQoL (EuroQoL 5-dimensional questionnaire visual analogue scale) at 180 days, by stroke type. An assessment was also made of outcomes related to the number of processes patients received.

Results—

There were 17 585 stroke admissions (median age 77 years, 47% female; 81% managed in ASUs; 80% ischemic stroke) from 42 hospitals (77% metropolitan) assessed. Cumulative benefits on outcomes related to the number of care processes received by patients. ASU care was associated with a reduced likelihood of death (hazard ratio, 0.49; 95% confidence interval, 0.43–0.56) and better HRQoL (coefficient, 21.34; 95% confidence interval, 15.50–27.18) within 180 days. For those discharged from hospital, receiving ASU+antihypertensive medication provided greater 180-day survival (hazard ratio, 0.45; 95% confidence interval, 0.38–0.52) compared with ASU care alone (hazard ratio, 0.64; 95% confidence interval, 0.54–0.76). HRQoL gains were greatest for patients with intracerebral hemorrhage who received care bundles involving discharge processes (range of increase, 11%–19%).

Conclusions—

Patients with stroke who receive best practice recommended hospital care have improved long-term survival and HRQoL.