Risk modelling study for carotid endarterectomy

External Validation of Risk Prediction Models to Improve Selection of Patients for Carotid Endarterectomy

BACKGROUND AND PURPOSE

The net benefit of carotid endarterectomy (CEA) is determined partly by the risk of procedural stroke or death. Current guidelines recommend CEA if 30-day risks are <6% for symptomatic stenosis and <3% for asymptomatic stenosis. We aimed to identify prediction models for procedural stroke or death after CEA and to externally validate these models in a large registry of patients from the United States.

METHODS

We conducted a systematic search in MEDLINE and EMBASE for prediction models of procedural outcomes after CEA. We validated these models with data from patients who underwent CEA in the American College of Surgeons National Surgical Quality Improvement Program (2011-2017). We assessed discrimination using C statistics and calibration graphically. We determined the number of patients with predicted risks that exceeded recommended thresholds of procedural risks to perform CEA.

RESULTS

After screening 788 reports, 15 studies describing 17 prediction models were included. Nine were developed in populations including both asymptomatic and symptomatic patients, 2 in symptomatic and 5 in asymptomatic populations. In the external validation cohort of 26 293 patients who underwent CEA, 702 (2.7%) developed a stroke or died within 30-days. C statistics varied between 0.52 and 0.64 using all patients, between 0.51 and 0.59 using symptomatic patients, and between 0.49 to 0.58 using asymptomatic patients. The Ontario Carotid Endarterectomy Registry model that included symptomatic status, diabetes, heart failure, and contralateral occlusion as predictors, had C statistic of 0.64 and the best concordance between predicted and observed risks. This model identified 4.5% of symptomatic and 2.1% of asymptomatic patients with procedural risks that exceeded recommended thresholds.

CONCLUSIONS

Of the 17 externally validated prediction models, the Ontario Carotid Endarterectomy Registry risk model had most reliable predictions of procedural stroke or death after CEA and can inform patients about procedural hazards and help focus CEA toward patients who would benefit most from it.

Configuration of vascular services: a multiple methods research programme

Background

Vascular services is changing rapidly, having emerged as a new specialty with its own training and specialised techniques. This has resulted in the need for reconfiguration of services to provide adequate specialist provision and accessible and equitable services.

Objectives

To identify the effects of service configuration on practice, resource use and outcomes. To model potential changes in configuration. To identify and/or develop electronic data collection tools for collecting patient-reported outcome measures and other clinical information. To evaluate patient preferences for aspects of services other than health-related quality of life.

Design

This was a multiple methods study comprising multiple systematic literature reviews; the development of a new outcome measure for users of vascular services (the electronic Personal Assessment Questionnaire – Vascular) based on the reviews, qualitative studies and psychometric evaluation; a trade-off exercise to measure process utilities; Hospital Episode Statistics analysis; and the development of individual disease models and a metamodel of service configuration.

Setting

Specialist vascular inpatient services in England.

Data sources

Modelling and Hospital Episode Statistics analysis for all vascular inpatients in England from 2006 to 2018. Qualitative studies and electronic Personal Assessment Questionnaire – Vascular evaluation with vascular patients from the Sheffield area. The trade-off studies were based on a societal sample from across England.

Interventions

The data analysis, preference studies and modelling explored the effect of different potential arrangements for service provision on the resource use, workload and outcomes for all interventions in the three main areas of inpatient vascular treatment: peripheral arterial disease, abdominal aortic aneurysm and carotid artery disease. The electronic Personal Assessment Questionnaire – Vascular was evaluated as a potential tool for clinical data collection and outcome monitoring.

Main outcome measures

Systematic reviews assessed quality and psychometric properties of published outcome measures for vascular disease and the relationship between volume and outcome in vascular services. The electronic Personal Assessment Questionnaire – Vascular development considered face and construct validity, test–retest reliability and responsiveness. Models were validated using case studies from previous reconfigurations and comparisons with Hospital Episode Statistics data. Preference studies resulted in estimates of process utilities for aneurysm treatment and for travelling distances to access services.

Results

Systematic reviews provided evidence of an association between increasing volume of activity and improved outcomes for peripheral arterial disease, abdominal aortic aneurysm and carotid artery disease. Reviews of existing patient-reported outcome measures did not identify suitable condition-specific tools for incorporation in the electronic Personal Assessment Questionnaire – Vascular. Reviews of qualitative evidence, primary qualitative studies and a Delphi exercise identified the issues to be incorporated into the electronic Personal Assessment Questionnaire – Vascular, resulting in a questionnaire with one generic and three disease-specific domains. After initial item reduction, the final version has 55 items in eight scales and has acceptable psychometric properties. The preference studies showed strong preference for endovascular abdominal aortic aneurysm treatment (willingness to trade up to 0.135 quality-adjusted life-years) and for local services (up to 0.631 quality-adjusted life-years). A simulation model with a web-based interface was developed, incorporating disease-specific models for abdominal aortic aneurysm, peripheral arterial disease and carotid artery disease. This predicts the effects of specified reconfigurations on workload, resource use, outcomes and cost-effectiveness. Initial exploration suggested that further reconfiguration of services in England to accomplish high-volume centres would result in improved outcomes, within the bounds of cost-effectiveness usually considered acceptable in the NHS.

Limitations

The major source of evidence to populate the models was Hospital Episode Statistics data, which have limitations owing to the complexity of the data, deficiencies in the coding systems and variations in coding practice. The studies were not able to address all of the potential barriers to change where vascular services are not compliant with current NHS recommendations.

Conclusions

There is evidence of potential for improvement in the clinical effectiveness and cost-effectiveness of vascular services through further centralisation of sites where major vascular procedures are undertaken. Preferences for local services are strong, and this may be addressed through more integrated services, with a range of services being provided more locally. The use of a web-based tool for the collection of clinical data and patient-reported outcome measures is feasible and can provide outcome data for clinical use and service evaluation.

Future work

Further evaluation of the economic models in real-world situations where local vascular service reconfiguration is under consideration and of the barriers to change where vascular services do not meet NHS recommendations for service configuration is needed. Further work on the electronic Personal Assessment Questionnaire – Vascular is required to assess its acceptability and usefulness in clinical practice and to develop appropriate report formats for clinical use and service evaluation. Further studies to assess the implications of including non-health-related preferences for care processes, and location of services, in calculations of cost-effectiveness are required.

Study registration

This study is registered as PROSPERO CRD42016042570, CRD42016042573, CRD42016042574, CRD42016042576, CRD42016042575, CRD42014014850, CRD42015023877 and CRD42015024820.

Funding

This project was funded by the National Institute for Health Research (NIHR) Programme Grants for Applied Research programme and will be published in full in Programme Grants for Applied Research; Vol. 9, No. 5. See the NIHR Journals Library website for further project information.

High Operator and Hospital Volume Are Associated With a Decreased Risk of Death and Stroke After Carotid Revascularization: A Systematic Review and Meta-analysis

OBJECTIVE

To examine the association between operator or hospital volume and procedural outcomes of carotid revascularization.

BACKGROUND

Operator and hospital volume have been proposed as determinants of outcome after carotid endarterectomy (CEA) or carotid artery stenting (CAS). The magnitude and clinical relevance of this relationship are debated.

METHODS

We systematically searched PubMed and EMBASE until August 21, 2017. The primary outcome was procedural (30 days, in-hospital, or perioperative) death or stroke. Obtained or estimated risk estimates were pooled with a generic inverse variance random-effects model.

RESULTS

We included 87 studies. A decreased risk of death or stroke following CEA was found for high compared to low operator volume with a pooled adjusted odds ratio (OR) of 0.50 (95% confidence interval [CI] 0.28-0.87; 3 cohorts), and a pooled unadjusted relative risk (RR) of 0.59 (95% CI 0.42-0.83; 9 cohorts); for high compared to low hospital volume with a pooled adjusted OR of 0.62 (95% CI 0.42-0.90; 5 cohorts), and a pooled unadjusted RR of 0.68 (95% CI 0.51-0.92; 9 cohorts). A decreased risk of death or stroke after CAS was found for high compared to low operator volume with an adjusted OR of 0.43 (95% CI 0.20-0.95; 1 cohort), and an unadjusted RR of 0.50 (95% CI 0.32-0.79; 1 cohort); for high compared to low hospital volume with an adjusted OR of 0.46 (95% CI 0.26-0.80; 1 cohort), and no significant decreased risk in a pooled unadjusted RR of 0.72 (95% CI 0.49-1.06; 2 cohorts).

CONCLUSIONS

We found a decreased risk of procedural death and stroke after CEA and CAS for high operator and high hospital volume, indicating that aiming for a high volume may help to reduce procedural complications.

REGISTRATION

This systematic review has been registered in the international prospective registry of systematic reviews (PROSPERO): CRD42017051491.

Anesthesia type determines risk of cerebral infarction after carotid endarterectomy

OBJECTIVE

Silent and symptomatic cerebral infarctions occur in up to 34% of patients after carotid endarterectomy (CEA). This prospective study compared the risk of new brain infarctions detected by magnetic resonance imaging (MRI) in patients with internal carotid artery stenosis undergoing CEA with local anesthesia (LA) vs general anesthesia (GA).

METHODS

Consecutive patients with internal carotid artery stenosis indicated for CEA were screened at two centers. Patients without contraindication to LA or GA were randomly allocated to the LA or GA group by ZIP code randomization. Brain MRI was performed before and 24 hours after CEA. Neurologic examination was performed before and 24 hours and 30 days after surgery. The occurrence of new infarctions on the control magnetic resonance images, stroke, transient ischemic attack, and other complications was statistically evaluated.

RESULTS

Of 210 randomized patients, 105 underwent CEA with LA (67 men; mean age, 68.3 ± 8.1 years) and 105 with GA (70 men; mean age, 63.4 ± 7.5 years). New infarctions were more frequently detected on control magnetic resonance images in patients after CEA under GA compared with LA (17.1% vs 6.7%; P = .031). Stroke or transient ischemic attack occurred within 30 days of CEA in three patients under GA and in two under LA (P = 1.000). There were no significant differences between the two types of anesthesia in terms of the occurrence of other complications (14.3% for GA and 21.0% for LA; P = .277).

CONCLUSIONS

The risk of silent brain infarction after CEA as detected by MRI is higher under GA than under LA.

The Dutch Audit of Carotid Interventions: Transparency in Quality of Carotid Endarterectomy in Symptomatic Patients in the Netherlands

BACKGROUND

The Dutch Audit for Carotid Interventions (DACI) registers all patients undergoing interventions for carotid artery stenosis in the Netherlands. This study describes the design of the DACI and results of patients with a symptomatic stenosis undergoing carotid endarterectomy (CEA). It aimed to evaluate variation between hospitals in process of care and (adjusted) outcomes, as well as predictors of major stroke/death after CEA.

METHODS

All patients with a symptomatic stenosis, who underwent CEA and were registered in the DACI between 2014 and 2016 were included in this cohort. Descriptive analyses of patient characteristics, process of care, and outcomes were performed. Casemix adjusted hospital procedural outcomes as (30 day/in hospital) mortality, stroke/death, and major stroke/death, were compared with the national mean. A multivariable logistic regression model (backward elimination at p > 0.10) was used to identify predictors of major stroke/death.

RESULTS

A total of 6459 patients, registered by 52 hospitals, were included. The majority (4,832, 75%) were treated <2 weeks after their first hospital consultation, varying from 40% to 93% between hospitals. Mortality, stroke/death, and major stroke/death were, respectively, 1.1%, 3.6%, and 1.8%. Adjusted major stroke/death rates for hospital comparison varied between 0 and 6.5%. Nine hospitals performed significantly better, none performed significantly worse. Predictors of major stroke/death were sex, age, pulmonary disease, presenting neurological symptoms, and peri-operative shunt.

CONCLUSION

CEA in The Netherlands is associated with an overall low mortality and (major) stroke/death rate. Whereas the indicator time to intervention varied between hospitals, mortality and (major) stroke/death were not significantly distinctive enough to identify worse practices and therefore were unsuitable for hospital comparison in the Dutch setting. Additionally, predictors of major stroke/death at population level could be identified.

Variable life adjusted display methodology for continuous performance monitoring of carotid endarterectomy

Introduction The aim of this study was to use variable life-adjusted display (VLAD) methodology to monitor performance of six vascular surgeons undertaking carotid endarterectomy in a single institution. Materials and methods This was a prospective study with continuous analysis. A risk score model to predict 30-day stroke or death for individual patients was developed from data collected from 839 patients from 1992 to 1999. The model was used to monitor performance of six surgeons from 2000 to 2009. Individual risk factors and 30-day outcomes were analysed and VLAD plots were created for the whole unit and for each surgeon. Results Among the 941 carotid endarterectomies in the performance analysis, 28 adverse events were recorded, giving an overall stroke or death rate of 3.06%. The risk model predicted there would be 33 adverse events. There was no statistical difference between the predicted and the observed adverse events (P > 0.2, χ2 value 1.25, 4 degrees of freedom). The VLAD plot for the whole unit shows an overall net gain in operative performance, although this could have been chance variation. The individual VLAD plot showed that surgeons 1, 2, 3 and 6 to have an overall net gain in the number of successful operations. The changes observed between the surgeons was not significant (P > 0.05) suggesting chance variation only. Conclusions Performance of carotid endarterectomy can be continuously assessed using VLAD methodology for units and individual surgeons. Early identification and correction of performance variation could facilitate improved quality of care.

Systematic review of carotid artery procedures and the volume-outcome relationship in Europe

BACKGROUND

Hospitals that conduct more procedures on the carotid arteries may achieve better outcomes. In the context of ongoing reconfiguration of UK vascular services, this systematic review was conducted to evaluate the relationship between the volume of carotid procedures and outcomes, including mortality and stroke.

METHODS

Searches of electronic databases identified studies that reported the effect of hospital or clinician volume on outcomes. Reference and citation searches were also performed. Inclusion was restricted to European populations on the basis that the model of healthcare delivery is similar across Europe, but differs from that in the USA and elsewhere. Analyses of hospital and clinician volume, and carotid endarterectomy (CEA) and carotid artery stenting (CAS) were conducted separately.

RESULTS

Eleven eligible studies were identified (233 411 participants), five from the UK, two from Sweden, one each from Germany, Finland and Italy, and a combined German, Austrian and Swiss population. All studies were observational. Two large studies (179 736 patients) suggested an inverse relationship between hospital volume and mortality (number needed to treat (NNT) as low as 165), and combined mortality and stroke (NNT as low as 93), following CEA. The evidence was less clear for CAS; multiple analyses in three studies did not identify convincing evidence of an association. Limited data are available on the relationship between clinician volume and outcome in CAS; in CEA, an inverse relationship was identified by two of three small studies.

CONCLUSION

The evidence from the largest and highest-quality studies included in this review support the centralization of CEA.

MRI lesions after invasive therapy of carotid artery stenosis: a risk-modeling analysis

BACKGROUND

Diffusion-weighted imaging (DWI) abnormalities can frequently be detected after carotid endarterectomy (CEA) and carotid angioplasty with stent placement (CAS) of the carotid arteries. We looked for possible predictors for the development of DWI lesions during the intervention.

METHODS

We investigated 41 patients who underwent CAS without protection devices and 93 patients who underwent CEA. DWI studies were performed 1 day before and after the intervention.

RESULTS

Ischemic complications consisted of two strokes (2.2%) in the CEA group and one stroke (2.4%) in the CAS group. DWI lesions were detected in 28.0% of all patients after intervention. Using a multivariate regression analysis, diabetes mellitus (DM), hyperlipidemia, symptomatic stenosis, age and CAS were found to be significant predictors for the occurrence of DWI lesions.

CONCLUSIONS

DWI is an objective and highly sensitive method for monitoring interventions of the carotid arteries. Our results point to an increased risk of patients with diabetes and hyperlipidemia to develop DWI lesions during invasive therapy of the ICA.

Review: Stroke in type 2 diabetes

Stroke is the second most frequent cause of death worldwide and the most frequent cause of permanent disability. Patients with diabetes are at 1.5 to three times the risk of stroke compared with the general population. Cerebrovascular disease causes 20% of deaths in diabetic patients. Interestingly, there are some striking differences of stroke patterns between diabetic and non-diabetic subjects suffering a stroke. Even more important is the fact that diabetes dramatically increases the risk of stroke in younger subjects as well as women. These data highlight the need for detection and treatment of diabetes particularly in these patient groups. This review summarises several aspects of stroke in type 2 diabetes, focusing on differences from non-diabetic stroke.

Defining the high-risk patient for carotid endarterectomy: An analysis of the prospective National Surgical Quality Improvement Program database

BACKGROUND

Carotid endarterectomy (CEA) is the gold standard for the treatment of carotid stenosis, but carotid angioplasty and stenting has been advocated in high-risk patients. The definition of such a population has been elusive, particularly because the data are largely retrospective. Our study examined results for CEA in the National Surgical Quality Improvement Program database (both Veterans Affairs and private sector).

METHODS

National Surgical Quality Improvement Program data were gathered prospectively for all patients undergoing primary isolated CEA during the interval 2000 to 2003 at 123 Veterans Affairs and 14 private sector academic medical centers. Study end points included the 30-day occurrence of any stroke, death, or cardiac event. A variety of clinical, demographic, and operative variables were assessed with multivariate models to identify risk factors associated with the composite (stroke, death, or cardiac event) end point. Adjudication of end points was by trained nurse reviewers (previously validated).

RESULTS

A total of 13,622 CEAs were performed during the study period; 95% were on male patients, and 91% of cases were conducted within the Veterans Affairs sector. The average age was 68.6 +/- 0.1 years, and 42.1% of the population had no prior neurologic event. The composite stroke, death, or cardiac event rate was 4.0%; the stroke/death rate was 3.4%. Multivariate correlates of the composite outcome were (odds ratio, P value) as follows: deciles of age (1.13, .018), insulin-requiring diabetes (1.73, <.001), oral agent-controlled diabetes (1.39, .003), decade of pack-years smoking (1.04, >.001), history of transient ischemic attack (1.41, >.001), history of stroke (1.51, >.001), creatinine >1.5 mg/dL (1.48, >.001), hypoalbuminemia (1.49, >.001), and fourth quartile of operative time (1.44, >.001). Cardiopulmonary comorbid features did not affect the composite outcome in this model. Regional anesthesia was used in 2437 (18%) cases, with a resultant relative risk reduction for stroke (17%), death (24%), cardiac event (33%), and the composite outcome (31%; odds ratio, 0.69; P = .008).

CONCLUSIONS

Carotid endarterectomy results across a spectrum of Veterans Affairs and private sector hospitals compare favorably to contemporary studies. These data will assist in selecting patients who are at an increased risk for adverse outcomes. Use of regional anesthetic significantly reduced perioperative complications in a risk-adjusted model, thus suggesting that it is the anesthetic of choice when CEA is performed in high-risk patients.

Review of prevalence and outcome of vascular disease in patients with diabetes mellitus

OBJECTIVES

Review the literature to determine the prevalence and outcome in patients with diabetes that undergo surgery to correct carotid artery stenosis, lower extremity arterial disease, and abdominal aortic aneurysm (AAA).

DESIGN AND MATERIALS

Studies were obtained from searches over the past 15 years on the National Library of Medicine's online search engine.

RESULTS

The review demonstrated an equivalent prevalence of carotid artery stenosis requiring surgery in patients with diabetes, it favored no increase risk of post-CEA stroke, and it was split on perioperative morbidity and mortality risk. There was an increase prevalence of lower extremity arterial disease requiring surgery in patients with diabetes, it favored equivalent patency and limb salvage rates, and it was split on the morbidity and mortality risk. The review demonstrated a decrease in AAA prevalence among patients with diabetes, it found an increase in the morbidity risk, and equivalent mortality risk.

CONCLUSIONS

Stroke, graft patency, and limb salvage rates in patients with diabetes after surgery are similar to patients without diabetes; however, their risk of complications is increased after surgery and the mortality risk may be higher after CEA.

A Bayesian hierarchical approach to comparative audit for carotid surgery

OBJECTIVES

the aim of this study was to illustrate how a Bayesian hierarchical modelling approach can aid the reliable comparison of outcome rates between surgeons.

DESIGN

retrospective analysis of prospective and retrospective data.

MATERIALS

binary outcome data (death/stroke within 30 days), together with information on 15 possible risk factors specific for CEA were available on 836 CEAs performed by four vascular surgeons from 1992-99. The median patient age was 68 (range 38-86) years and 60% were men.

METHODS

the model was developed using the WinBUGS software. After adjusting for patient-level risk factors, a cross-validatory approach was adopted to identify "divergent" performance. A ranking exercise was also carried out.

RESULTS

the overall observed 30-day stroke/death rate was 3.9% (33/836). The model found diabetes, stroke and heart disease to be significant risk factors. There was no significant difference between the predicted and observed outcome rates for any surgeon (Bayesian p -value>0.05). Each surgeon had a median rank of 3 with associated 95% CI 1.0-5.0, despite the variability of observed stroke/death rate from 2.9-4.4%. After risk adjustment, there was very little residual between-surgeon variability in outcome rate.

CONCLUSIONS

Bayesian hierarchical models can help to accurately quantify the uncertainty associated with surgeons' performance and rank.

POSSUM and P-POSSUM overpredict mortality for carotid endarterectomy

OBJECTIVES

the aim was to test the predictive accuracy of POSSUM and P-POSSUM on patients undergoing CEA.

DESIGN

retrospective and prospective study.

MATERIALS

499 CEAs performed by four vascular surgeons from a single unit from 1992-99. The median age was 68 (range 38-86) and 60% were men.

METHODS

physiological parameters, operative parameters and the 30-day mortality were collected. Predicted mortality for each patient was calculated using POSSUM and P-POSSUM equations. Patients were stratified into risk groups and observed and predicted deaths were compared. Accuracy of the prediction was assessed using chi-squared analysis.

RESULTS

the observed 30-day mortality was 1.8% (9/499). The predicted deaths using POSSUM and P-POSSUM analysis were 49 and 25 respectively compared to nine observed deaths. There was significant evidence of lack of fit of both models in predicting mortality (chi-squared analysis, p<0.05).

CONCLUSIONS

POSSUM and P-POSSUM overpredict mortality and are unsuitable for comparative audit of CEA. Models developed specific for CEA might accurately predict mortality.