Comparing the predictive accuracy of frailty instruments applied to preoperative electronic health data for adults undergoing noncardiac surgery

Accuracy of frailty instruments in predicting outcomes following perioperative cardiac arrest

BACKGROUND

Frailty is associated with increased 30-day mortality and non-home discharge following perioperative cardiac arrest. We estimated the predictive accuracy of frailty when added to baseline risk prediction models.

METHODS

In this retrospective cohort study using 2015-2020 NSQIP data for 3048 patients aged 50+ undergoing non-cardiac surgery and resuscitation on post-operative day 0 (i.e., intraoperatively or postoperatively on the day of surgery), baseline models including age, sex, ASA physical status, preoperative sepsis or septic shock, and emergent surgery were compared to models that added frailty indices, either RAI or mFI-5, to predict 30-day mortality and non-home discharge. Predictive accuracy was characterized by area under the receiver operating characteristic curve (AUC-ROC), integrated calibration index (ICI), and continuous net reclassification index (NRI).

RESULTS

1786 patients (58.6%) died in the study cohort within 30 days, and 38.6% of eligible patients experienced non-home discharge. The baseline model showed good discrimination (AUC-ROC 0.77 for 30-day mortality and 0.74 for non-home discharge). AUC-ROC and ICI did not significantly change after adding frailty for 30-day mortality or non-home discharge. Adding RAI significantly improved NRI for 30-day mortality and non-home discharge; however, the magnitude was small and difficult to interpret, given other results including false positive and negative rates showing no difference in predictive accuracy.

CONCLUSIONS

Incorporating frailty did not significantly improve predictive accuracy of models for 30-day mortality and non-home discharge following perioperative resuscitation. Thus, demonstrated associations between frailty and outcomes of perioperative resuscitation may not translate into improved predictive accuracy. When engaging patients in shared decision-making regarding do-not-resuscitate orders perioperatively, providers should acknowledge uncertainty in anticipating resuscitation outcomes.

Development and multicentre validation of the FLEX score: personalised preoperative surgical risk prediction using attention-based ICD-10 and Current Procedural Terminology set embeddings

BACKGROUND

Preoperative knowledge of surgical risks can improve perioperative care and patient outcomes. However, assessments requiring clinician examination of patients or manual chart review can be too burdensome for routine use.

METHODS

We conducted a multicentre retrospective study of 243 479 adult noncardiac surgical patients at four hospitals within the Mass General Brigham (MGB) system in the USA. We developed a machine learning method using routinely collected coding and patient characteristics data from the electronic health record which predicts 30-day mortality, 30-day readmission, discharge to long-term care, and hospital length of stay.

RESULTS

Our method, the Flexible Surgical Set Embedding (FLEX) score, achieved state-of-the-art performance to identify comorbidities that significantly contribute to the risk of each adverse outcome. The contributions of comorbidities are weighted based on patient-specific context, yielding personalised risk predictions. Understanding the significant drivers of risk of adverse outcomes for each patient can inform clinicians of potential targets for intervention.

CONCLUSIONS

FLEX utilises information from a wider range of medical diagnostic and procedural codes than previously possible and can adapt to different coding practices to accurately predict adverse postoperative outcomes.

A Frailty Index to Predict Mortality, Resource Utilization and Costs in Patients Undergoing Coronary Artery Bypass Graft Surgery in Ontario

Background

People living with frailty are vulnerable to poor outcomes and incur higher health care costs after coronary artery bypass graft (CABG) surgery. Frailty-defining instruments for population-level research in the CABG setting have not been established. The objectives of the study were to develop a preoperative frailty index for CABG (pFI-C) surgery using Ontario administrative data; assess pFI-C suitability in predicting clinical and economic outcomes; and compare pFI-C predictive capabilities with other indices.

Methods

A retrospective cohort study was conducted using health administrative data of 50,682 CABG patients. The pFI-C comprised 27 frailty-related health deficits. Associations between index scores and mortality, resource use and health care costs (2022 Canadian dollars [CAD]) were assessed using multivariable regression models. Capabilities of the pFI-C in predicting mortality were evaluated using concordance statistics; goodness of fit of the models was assessed using Akakie Information Criterion.

Results

As assessed by the pFI-C, 22% of the cohort lived with frailty. The pFI-C score was strongly associated with mortality per 10% increase (odds ratio [OR], 3.04; 95% confidence interval [CI], [2.83,3.27]), and was significantly associated with resource utilization and costs. The predictive performances of the pFI-C, Charlson, and Elixhauser indices and Johns Hopkins Aggregated Diagnostic Groups were similar, and mortality models containing the pFI-C had a concordance (C)-statistic of 0.784. Cost models containing the pFI-C showed the best fit.

Conclusions

The pFI-C is predictive of mortality and associated with resource utilization and costs during the year following CABG. This index could aid in identifying a subgroup of high-risk CABG patients who could benefit from targeted perioperative health care interventions.

Study protocol for a national observational cohort investigating frailty, delirium and multimorbidity in older surgical patients: the third Sprint National Anaesthesia Project (SNAP 3)

INTRODUCTION

Older surgical patients are more likely to be living with frailty and multimorbidity and experience postoperative complications. The management of these conditions in the perioperative pathway is evolving. In order to support objective decision-making for patients, services and national guidance, accurate, contemporary data are needed to describe the impact and associations between frailty, multimorbidity and healthcare processes with patient and service-level outcomes.

METHODS AND ANALYSIS

The study is comprised of an observational cohort study of approximately 7500 patients; an organisational survey of perioperative services and a clinician survey of the unplanned, medical workload generated from older surgical patients. The cohort will consist of patients who are 60 years and older, undergoing a surgical procedure during a 5-day recruitment period in participating UK hospitals. Participants will be assessed for baseline frailty and multimorbidity; postoperative morbidity including delirium; and quality of life. Data linkage will provide additional details about individuals, their admission and mortality.The study's primary outcome is length of stay, other outcome measures include incidence of postoperative morbidity and delirium; readmission, mortality and quality of life. The cohort's incidence of frailty, multimorbidity and delirium will be estimated using 95% CIs. Their relationships with outcome measures will be examined using unadjusted and adjusted multilevel regression analyses. Choice of covariates in the adjusted models will be prespecified, based on directed acyclic graphs.A parallel study is planned to take place in Australia in 2022.

ETHICS AND DISSEMINATION

The study has received approval from the Scotland A Research Ethics Committee and Wales Research Ethics Committee 7.This work hopes to influence the development of services and guidelines. We will publish our findings in peer-reviewed journals and provide summary documents to our participants, sites, healthcare policy-makers and the public.

TRIAL REGISTRATION NUMBER

ISRCTN67043129.

Interrater reliability of the Clinical Frailty Scale in the anesthesia preadmission clinic

PURPOSE

As many as 30% of patients with frailty die, are discharged to a nursing home, or have a new disability after surgery. The 2010 United Kingdom National Confidential Enquiry into Patient Outcome and Death recommended that frailty assessment be developed and included in the routine risk assessment of older surgical patients. The Clinical Frailty Scale (CFS) is a simple, clinically-assessed frailty measure; however, few studies have investigated interrater reliability of the CFS in the surgical setting. The objective of this study was to determine the interrater reliability of frailty classification between anesthesiologists and perioperative nurses.

METHODS

We conducted a cohort study assessing interrater reliability of the CFS between perioperative nurses and anesthesiologists for elective surgical patients aged ≥ 65 yr, admitted to a large regional university-affiliated hospital in Australia between July 2020 and February 2021. Agreement was measured via Cohen's kappa.

RESULTS

Frailty assessment was conducted on 238 patients with a median [interquartile range] age of 74 [70-80] yr. Agreement was perfect between nursing and medical staff for CFS scores in 112 (47%) patients, with a further 99 (42%) differing by only one point. Interrater kappa was 0.70 (95% confidence interval, 0.63 to 0.77; P < 0.001), suggesting good agreement between anesthesiologists and perioperative nurses.

CONCLUSION

This study suggests that CFS assessment by either anesthesiologists or nursing staff is reliable across a population of patients from a range of surgical specialities, with an acceptable degree of agreement. The CFS measurement should be included in the normal preanesthesia clinic workflow.