Personalized decision-making for aneurysm treatment of aneurysmal subarachnoid hemorrhage: development and validation of a clinical prediction tool

BACKGROUND

In patients with aneurysmal subarachnoid hemorrhage suitable for endovascular coiling and neurosurgical clip-reconstruction, the aneurysm treatment decision-making process could be improved by considering heterogeneity of treatment effect and durability of treatment. We aimed to develop and validate a tool to predict individualized treatment benefit of endovascular coiling compared to neurosurgical clip-reconstruction.

METHODS

We used randomized data (International Subarachnoid Aneurysm Trial, n = 2143) to develop models to predict 2-month functional outcome and to predict time-to-rebleed-or-retreatment. We modeled for heterogeneity of treatment effect by adding interaction terms of treatment with prespecified predictors and with baseline risk of the outcome. We predicted outcome with both treatments and calculated absolute treatment benefit. We described the patient characteristics of patients with ≥ 5% point difference in the predicted probability of favorable functional outcome (modified Rankin Score 0-2) and of no rebleed or retreatment within 10 years. Model performance was expressed with the c-statistic and calibration plots. We performed bootstrapping and leave-one-cluster-out cross-validation and pooled cluster-specific c-statistics with random effects meta-analysis.

RESULTS

The pooled c-statistics were 0.72 (95% CI: 0.69-0.75) for the prediction of 2-month favorable functional outcome and 0.67 (95% CI: 0.63-0.71) for prediction of no rebleed or retreatment within 10 years. We found no significant interaction between predictors and treatment. The average predicted benefit in favorable functional outcome was 6% (95% CI: 3-10%) in favor of coiling, but 11% (95% CI: 9-13%) for no rebleed or retreatment in favor of clip-reconstruction. 134 patients (6%), young and in favorable clinical condition, had negligible functional outcome benefit of coiling but had a ≥ 5% point benefit of clip-reconstruction in terms of durability of treatment.

CONCLUSIONS

We show that young patients in favorable clinical condition and without extensive vasospasm have a negligible benefit in functional outcome of endovascular coiling - compared to neurosurgical clip-reconstruction - while at the same time having a substantially lower probability of retreatment or rebleeding from neurosurgical clip-reconstruction - compared to endovascular coiling. The SHARP prediction tool ( https://sharpmodels.shinyapps.io/sharpmodels/ ) could support and incentivize a multidisciplinary discussion about aneurysm treatment decision-making by providing individualized treatment benefit estimates.

Endovascular versus neurosurgical aneurysm treatment: study protocol for the development and validation of a clinical prediction tool for individualised decision making

INTRODUCTION

Treatment decisions for aneurysmal subarachnoid haemorrhage patients should be supported by individualised predictions of the effects of aneurysm treatment. We present a study protocol and analysis plan for the development and external validation of models to predict benefit of neurosurgical versus endovascular aneurysm treatment on functional outcome and durability of treatment.

METHODS AND ANALYSIS

We will use data from the International Subarachnoid Aneurysm Trial for model development. The outcomes are functional outcome, measured with modified Rankin Scale at 12 months, and any retreatment or rebleed of the target aneurysm during follow-up. We will develop an ordinal logistic regression model and Cox regression model, considering age, World Federation of Neurological Surgeons grade, Fisher grade, vasospasm at presentation, aneurysm lumen size, aneurysm neck size, aneurysm location and time-to-aneurysm-treatment as predictors. We will test for interactions with treatment and with baseline risk and derive individualised predicted probabilities of treatment benefit. A benefit of ≥5% will be considered clinically relevant. Discriminative performance of the outcome predictions will be assessed with the c-statistic. Calibration will be assessed with calibration plots. Discriminative performance of the benefit predictions will be assessed with the c-for benefit. We will assess internal validity with bootstrapping and external validity with leave-one-out internal-external cross-validation.

ETHICS AND DISSEMINATION

The medical ethical research committee of the Erasmus MC University Medical Center Rotterdam approved the study protocol under the exemption category and waived the need for written informed consent (MEC-2020-0810). We will disseminate our results through an open-access peer-reviewed scientific publication and with a web-based clinical prediction tool.

Predictors of mortality in chronic obstructive pulmonary disease: a systematic review and meta-analysis

Background

Better insight in patients’ prognosis can help physicians to timely initiate advance care planning (ACP) discussions with patients with chronic obstructive pulmonary disease (COPD). We aimed to identify predictors of mortality.

Methods

We systematically searched databases Embase, PubMed, MEDLINE, Web of Science, and Cochrane Central in April 2020. Papers reporting on predictors or prognostic models for mortality at 3 months and up to 24 months were assessed on risk-of-bias. We performed a meta-analysis with a fixed or random-effects model, and evaluated the discriminative ability of multivariable prognostic models.

Results

We included 42 studies (49–418,251 patients); 18 studies were included in the meta-analysis. Significant predictors of mortality within 3–24 months in the random-effects model were: previous hospitalization for acute exacerbation (hazard ratio [HR] 1.97; 95% confidence interval [CI] 1.32–2.95), hospital readmission within 30 days (HR 5.01; 95% CI 2.16–11.63), cardiovascular comorbidity (HR 1.89; 95% CI 1.25–2.87), age (HR 1.48; 95% CI 1.38–1.59), male sex (HR 1.68; 95% CI 1.38–1.59), and long-term oxygen therapy (HR 1.74; 95% CI 1.10–2.73). Nineteen previously developed multicomponent prognostic models, as examined in 11 studies, mostly had moderate discriminate ability.

Conclusion

Identified predictors of mortality may aid physicians in selecting COPD patients who may benefit from ACP. However, better discriminative ability of prognostic models or development of a new prognostic model is needed for further large-scale implementation.

Registration: PROSPERO (CRD42016038494), https://www.crd.york.ac.uk/prospero/.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12890-022-01911-5.

Predictors of Mortality in Patients with Advanced Cancer-A Systematic Review and Meta-Analysis

To timely initiate advance care planning in patients with advanced cancer, physicians should identify patients with limited life expectancy. We aimed to identify predictors of mortality. To identify the relevant literature, we searched Embase, MEDLINE, Cochrane Central, Web of Science, and PubMed databases between January 2000-April 2020. Identified studies were assessed on risk-of-bias with a modified QUIPS tool. The main outcomes were predictors and prediction models of mortality within a period of 3-24 months. We included predictors that were studied in ≥2 cancer types in a meta-analysis using a fixed or random-effects model and summarized the discriminative ability of models. We included 68 studies (ranging from 42 to 66,112 patients), of which 24 were low risk-of-bias, and 39 were included in the meta-analysis. Using a fixed-effects model, the predictors of mortality were: the surprise question, performance status, cognitive impairment, (sub)cutaneous metastases, body mass index, comorbidity, serum albumin, and hemoglobin. Using a random-effects model, predictors were: disease stage IV (hazard ratio [HR] 7.58; 95% confidence interval [CI] 4.00-14.36), lung cancer (HR 2.51; 95% CI 1.24-5.06), ECOG performance status 1+ (HR 2.03; 95% CI 1.44-2.86) and 2+ (HR 4.06; 95% CI 2.36-6.98), age (HR 1.20; 95% CI 1.05-1.38), male sex (HR 1.24; 95% CI 1.14-1.36), and Charlson comorbidity score 3+ (HR 1.60; 95% CI 1.11-2.32). Thirteen studies reported on prediction models consisting of different sets of predictors with mostly moderate discriminative ability. To conclude, we identified reasonably accurate non-tumor specific predictors of mortality. Those predictors could guide in developing a more accurate prediction model and in selecting patients for advance care planning.

Imputation strategies for missing baseline neurological assessment covariates after traumatic brain injury: A CENTER-TBI study

Statistical models for outcome prediction are central to traumatic brain injury research and critical to baseline risk adjustment. Glasgow coma score (GCS) and pupil reactivity are crucial covariates in all such models but may be measured at multiple time points between the time of injury and hospital and are subject to a variable degree of unreliability and/or missingness. Imputation of missing data may be undertaken using full multiple imputation or by simple substitution of measurements from other time points. However, it is unknown which strategy is best or which time points are more predictive. We evaluated the pseudo-R² of logistic regression models (dichotomous survival) and proportional odds models (Glasgow Outcome Score—extended) using different imputation strategies on the The Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI) study dataset. Substitution strategies were easy to implement, achieved low levels of missingness (<< 10%) and could outperform multiple imputation without the need for computationally costly calculations and pooling multiple final models. While model performance was sensitive to imputation strategy, this effect was small in absolute terms and clinical relevance. A strategy of using the emergency department discharge assessments and working back in time when these were missing generally performed well. Full multiple imputation had the advantage of preserving time-dependence in the models: the pre-hospital assessments were found to be relatively unreliable predictors of survival or outcome. The predictive performance of later assessments was model-dependent. In conclusion, simple substitution strategies for imputing baseline GCS and pupil response can perform well and may be a simple alternative to full multiple imputation in many cases.

Outcome Prediction after Moderate and Severe Traumatic Brain Injury: External Validation of Two Established Prognostic Models in 1742 European Patients

The International Mission on Prognosis and Analysis of Clinical Trials in Traumatic Brain Injury (IMPACT) and Corticoid Randomisation After Significant Head injury (CRASH) prognostic models predict functional outcome after moderate and severe traumatic brain injury (TBI). We aimed to assess their performance in a contemporary cohort of patients across Europe. The Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI) core study is a prospective, observational cohort study in patients presenting with TBI and an indication for brain computed tomography. The CENTER-TBI core cohort consists of 4509 TBI patients available for analyses from 59 centers in 18 countries across Europe and Israel. The IMPACT validation cohort included 1173 patients with GCS ≤12, age ≥14, and 6-month Glasgow Outcome Scale-Extended (GOSE) available. The CRASH validation cohort contained 1742 patients with GCS ≤14, age ≥16, and 14-day mortality or 6-month GOSE available. Performance of the three IMPACT and two CRASH model variants was assessed with discrimination (area under the receiver operating characteristic curve; AUC) and calibration (comparison of observed vs. predicted outcome rates). For IMPACT, model discrimination was good, with AUCs ranging between 0.77 and 0.85 in 1173 patients and between 0.80 and 0.88 in the broader CRASH selection (n = 1742). For CRASH, AUCs ranged between 0.82 and 0.88 in 1742 patients and between 0.66 and 0.80 in the stricter IMPACT selection (n = 1173). Calibration of the IMPACT and CRASH models was generally moderate, with calibration-in-the-large and calibration slopes ranging between -2.02 and 0.61 and between 0.48 and 1.39, respectively. The IMPACT and CRASH models adequately identify patients at high risk for mortality or unfavorable outcome, which supports their use in research settings and for benchmarking in the context of quality-of-care assessment.

National Institutes of Health Stroke Scale: An Alternative Primary Outcome Measure for Trials of Acute Treatment for Ischemic Stroke

Supplemental Digital Content is available in the text.

The modified Rankin Scale (mRS) at 3 months is the most commonly used primary outcome measure in stroke treatment trials, but it lacks specificity and requires long-term follow-up interviews, which consume time and resources. An alternative may be the National Institutes of Health Stroke Scale (NIHSS), early after stroke. Our aim was to evaluate whether the NIHSS assessed within 1 week after treatment could serve as a primary outcome measure for trials of acute treatment for ischemic stroke.

Between-center and between-country differences in outcome after aneurysmal subarachnoid hemorrhage in the Subarachnoid Hemorrhage International Trialists (SAHIT) repository

OBJECTIVE

Differences in clinical outcomes between centers and countries may reflect variation in patient characteristics, diagnostic and therapeutic policies, or quality of care. The purpose of this study was to investigate the presence and magnitude of between-center and between-country differences in outcome after aneurysmal subarachnoid hemorrhage (aSAH).

METHODS

The authors analyzed data from 5972 aSAH patients enrolled in randomized clinical trials of 3 different treatments from the Subarachnoid Hemorrhage International Trialists (SAHIT) repository, including data from 179 centers and 20 countries. They used random effects logistic regression adjusted for patient characteristics and timing of aneurysm treatment to estimate between-center and between-country differences in unfavorable outcome, defined as a Glasgow Outcome Scale score of 1-3 (severe disability, vegetative state, or death) or modified Rankin Scale score of 4-6 (moderately severe disability, severe disability, or death) at 3 months. Between-center and between-country differences were quantified with the median odds ratio (MOR), which can be interpreted as the ratio of odds of unfavorable outcome between a typical high-risk and a typical low-risk center or country.

RESULTS

The proportion of patients with unfavorable outcome was 27% (n = 1599). The authors found substantial between-center differences (MOR 1.26, 95% CI 1.16-1.52), which could not be explained by patient characteristics and timing of aneurysm treatment (adjusted MOR 1.21, 95% CI 1.11-1.44). They observed no between-country differences (adjusted MOR 1.13, 95% CI 1.00-1.40).

CONCLUSIONS

Clinical outcomes after aSAH differ between centers. These differences could not be explained by patient characteristics or timing of aneurysm treatment. Further research is needed to confirm the presence of differences in outcome after aSAH between hospitals in more recent data and to investigate potential causes.

Prognosis in Moderate and Severe Traumatic Brain Injury: A Systematic Review of Contemporary Models and Validation Studies

Outcome prognostication in traumatic brain injury (TBI) is important but challenging due to heterogeneity of the disease. The aim of this systematic review is to present the current state-of-the-art on prognostic models for outcome after moderate and severe TBI and evidence on their validity. We searched for studies reporting on the development, validation or extension of prognostic models for functional outcome after TBI with Glasgow Coma Scale (GCS) ≤12 published between 2006-2018. Studies with patients age ≥14 years and evaluating a multi-variable prognostic model based on admission characteristics were included. Model discrimination was expressed with the area under the receiver operating characteristic curve (AUC), and model calibration with calibration slope and intercept. We included 58 studies describing 67 different prognostic models, comprising the development of 42 models, 149 external validations of 31 models, and 12 model extensions. The most common predictors were GCS (motor) score (n = 55), age (n = 54), and pupillary reactivity (n = 48). Model discrimination varied substantially between studies. The International Mission for Prognosis and Analysis of Clinical Trials (IMPACT) and Corticoid Randomisation After Significant Head injury (CRASH) models were developed on the largest cohorts (8509 and 10,008 patients, respectively) and were most often externally validated (n = 91), yielding AUCs ranging between 0.65-0.90 and 0.66-1.00, respectively. Model calibration was reported with a calibration intercept and slope for seven models in 53 validations, and was highly variable. In conclusion, the discriminatory validity of the IMPACT and CRASH prognostic models is supported across a range of settings. The variation in calibration, reflecting heterogeneity in reliability of predictions, motivates continuous validation and updating if clinical implementation is pursued.

Abstract TMP6: NIH Stroke Scale as the Primary Outcome Measure for Trials of Acute Treatment of Ischemic Stroke

Background: The 90-day modified Rankin Scale (mRS) is the most commonly used primary outcome measure in stroke treatment trials, but has drawbacks, including the potential loss of subjects due to prolonged follow-up. An alternative may be the NIH Stroke Scale (NIHSS) early after stroke, which is frequently used as a secondary outcome measure. The aim of this study was to evaluate whether the NIHSS assessed within one week after trial inclusion could serve as a primary outcome measure for trials of acute ischaemic stroke treatment.

Methods: We used the Prentice criteria to evaluate NIHSS 1 day and 5-7 days after trial inclusion as primary outcome measures in two trials of endovascular treatment (EVT): the positive MR CLEAN and the neutral IMS III. The four Prentice criteria evaluate a surrogate endpoint against a true endpoint (Figure). We adjusted for age, baseline NIHSS, collateral score, and time of symptom onset to randomization.

Results: The Prentice criteria were met for NIHSS at 1 day and 5-7 days in MR CLEAN (n=500). We found a significant treatment effect of EVT on the mRS and on the NIHSS at 1 day and 5-7 days. After adjustment for NIHSS at 1 day and 5-7 days, the effect of EVT on mRS decreased from common odds ratio 1·69 (95%CI: 1·22-2·33) to respectively 1·33 (95%CI: 0·95-1·85) and 1·21 (95%CI: 0·86-1·72). In IMS III (n=656) there was no treatment effect on NIHSS at 1 day (p=0.56) and 5-7 days (p=0.28), coinciding with the absence of a treatment effect on mRS.

Conclusion: NIHSS is able to replace the mRS at 90 days as a measure of treatment effect. NIHSS within one week after trial inclusion may be used as a primary outcome measure in trials of acute ischemic stroke treatment, particularly in phase II(b) trials. This could minimize loss to follow-up, and reduce trial duration and costs.

Utility-Weighted Modified Rankin Scale as Primary Outcome in Stroke Trials: A Simulation Study

Supplemental Digital Content is available in the text.

Background and Purpose—

The utility-weighted modified Rankin Scale (UW-mRS) has been proposed as a new patient-centered primary outcome in stroke trials. We aimed to describe utility weights for the mRS health states and to evaluate the statistical efficiency of the UW-mRS to detect treatment effects in stroke intervention trials.

Methods—

We used data of the 500 patients enrolled in the MR CLEAN (Multicenter Randomized Clinical Trial of Endovascular Treatment for Acute Ischemic Stroke in the Netherlands). Utility values were elicited from the EuroQol Group 5-Dimension Self-Report Questionnaire assessed at 90 days after inclusion, simultaneously with the mRS. Utility weights were determined by averaging the utilities of all patients within each mRS category. We performed simulations to evaluate statistical efficiency. The simulated treatment effect was an odds ratio of 1.65 in favor of the treatment arm, similar for all mRS cutoffs. This treatment effect was analyzed using 3 approaches: linear regression with the UW-mRS as outcome, binary logistic regression with a dichotomized mRS (0–1/2–6, 0–2/3–6, and 0–4/5–6), and proportional odds logistic regression with the ordinal mRS. The statistical power of the 3 approaches was expressed as the proportion of 10 000 simulations that resulted in a statistically significant treatment effect (P≤0.05).

Results—

The mean utility values (SD) for mRS categories 0 to 6 were: 0.95 (0.08), 0.93 (0.13), 0.83 (0.21), 0.62 (0.27), 0.42 (0.28), 0.11 (0.28), and 0 (0), respectively, but varied substantially between individual patients within each category. The UW-mRS approach was more efficient than the dichotomous approach (power 85% versus 71%) but less efficient than the ordinal approach (power 85% versus 87%).

Conclusions—

The UW-mRS as primary outcome does not capture individual variation in utility values and may reduce the statistical power of a randomized trial.

Development and validation of the Dutch Stroke Score for predicting disability and functional outcome after ischemic stroke: A tool to support efficient discharge planning

Introduction

We aimed to develop and validate a prognostic score for disability at discharge and functional outcome at three months in patients with acute ischemic stroke based on clinical information available on admission.

Patients and methods

The Dutch Stroke Score (DSS) was developed in 1227 patients with ischemic stroke included in the Paracetamol (Acetaminophen) In Stroke study. Predictors for Barthel Index (BI) at discharge (‘DSS-discharge’) and modified Rankin Scale (mRS) at three months (‘DSS-3 months’) were identified in multivariable ordinal regression. The models were internally validated with bootstrapping techniques. The DSS-3 months was externally validated in the PRomoting ACute Thrombolysis in Ischemic StrokE study (1589 patients) and the Preventive Antibiotics in Stroke Study (2107 patients). Model performance was assessed in terms of discrimination, expressed by the area under the receiver operating characteristic curve (AUC), and calibration.

Results

At model development, the strongest predictors of Barthel Index at discharge were age per decade over 60 (odds ratio = 1.55, 95% confidence interval (CI) 1.41–1.68), National Institutes of Health Stroke Scale (odds ratio = 1.24 per point, 95% CI 1.22–1.26) and diabetes (odds ratio = 1.62, 95% CI 1.32–1.91). The internally validated AUC was 0.76 (95% CI 0.75–0.79). The DSS-3 months, additionally consisting of previous stroke and atrial fibrillation, performed similarly at internal (AUC 0.75, 95% CI 0.74–0.77) and external validation (AUC 0.74 in PRomoting ACute Thrombolysis in Ischemic StrokE (95% CI 0.72–0.76) and 0.69 in Preventive Antibiotics in Stroke Study (95% CI 0.69–0.72)). Observed outcome was slightly better than predicted.

Discussion: The DSS had satisfactory performance in predicting BI at discharge and mRS at three months in ischemic stroke patients.

Conclusion

If further validated, the DSS may contribute to efficient stroke unit discharge planning alongside patients' contextual factors and therapeutic needs.