Quantifying predictive accuracy in survival models

The Prognostic Significance of Nomogram-Based Pretreatment Inflammatory Indicators in Patients With Esophageal Squamous Cell Carcinoma Receiving Intensity-Modulated Radiotherapy

BACKGROUND

At present, there is no objective prognostic index available for patients with esophageal squamous cell carcinoma (ESCC) who underwent intensity-modulated radiotherapy (IMRT). This study is to develop a nomogram based on hematologic inflammatory indices for ESCC patients treated with IMRT.

METHODS

581 patients with ESCC receiving definitive IMRT were enrolled in our retrospective study. Of which, 434 patients with treatment-naïve ESCC in Fujian Cancer Hospital were defined as the training cohort. Additional 147 newly diagnosed ESCC patients were used as the validation cohort. Independent predictors of overall survival (OS) were employed to establish a nomogram model. The predictive ability was evaluated by time-dependent receiver operating characteristic curves, the concordance index (C-index), net reclassification index (NRI), and integrated discrimination improvement (IDI). Decision curve analysis (DCA) was performed to assess the clinical benefits of the nomogram model. The entire series was divided into 3 risk subgroups stratified by the total nomogram scores.

RESULTS

Clinical TNM staging, primary gross tumor volume, chemotherapy, neutrophil-to-lymphocyte ratio and platelet lymphocyte ratio were independent predictors of OS. Nomogram was developed incorporating these factors. Compared with the 8th American Joint Committee on Cancer (AJCC) staging, the C-index for 5-year OS (.627 and .629) and the AUC value of 5-year OS (.706 and .719) in the training and validation cohorts (respectively) were superior. Furthermore, the nomogram model presented higher NRI and IDI. DCA also demonstrated that the nomogram model provided greater clinical benefit. Finally, patients with <84.8, 84.8-151.4, and >151.4 points were categorized into low-risk, intermediate-risk, and high-risk groups. Their 5-year OS rates were 44.0%, 23.6%, and 8.9%, respectively. The C-index was .625, which was higher than the 8^th AJCC staging.

CONCLUSIONS

We have developed a nomogram model that enables risk-stratification of patients with ESCC receiving definitive IMRT. Our findings may serve as a reference for personalized treatment.

All-cause and cause-specific mortality in persons with fibromyalgia and widespread pain: An observational study in 35,248 persons with rheumatoid arthritis, non-inflammatory rheumatic disorders and clinical fibromyalgia

PURPOSE

Studies of the relation of fibromyalgia (FM) and widespread pain (WSP) to mortality have differed as to the presence or absence of an association and the extent of cause-specific mortality. However, no studies have investigated which definitions of FM and WSP associate with mortality, nor of FM mortality in other diseases. We investigated these issues and the meaning of mortality in patients with FM.

METHODS

We used Cox regression to study 35,248 rheumatic disease patients with up to 16 years of mortality follow-up in all patients and separately in those with diagnoses of rheumatoid arthritis (RA) (N = 26,458), non-inflammatory rheumatic disorders (NIRMD) (N = 5,167) and clinically diagnosed FM (N = 3,659). We applied 2016 FM criteria and other FM and WSP criteria to models adjusted for age and sex as well as to models that included a full range of covariates, including comorbid disease and functional status. We estimated the degree of explained of variance (R2) as a measure of predictive ability.

RESULTS

We found positive associations between al`l definitions of FM and WSP and all-cause mortality, with relative risks (RR)s ranging from 1.19 (95%CI 1.15-1.24) for American College of Rheumatology (ACR) 1990 WSP to 1.38 (1.31-1.46) in age and sex adjusted revised 2016 criteria (FM 2016). However, in full covariate models the FM 2016 RR reduced further to 1.15 (1.09-1.22). The association with mortality was noted with RA (1.52 (1.43-1.61)), NIRMD (1.43 (1.24-1.66)) and clinical FM (1.41 (1.14-1.75) - where 37% of FM diagnosed patients did not satisfy FM 2016 criteria. In the all-patient analyses, the age and sex explained variation (R2) was 0.255, increasing to 0.264 (4.4%) when FM 2016 criteria were added, and to 0.378 in a full covariate model. Death causes related to FM 2016 status included accidents, 1.45 (1.11-1.91); diabetes 1.78 (1.16-2,71); suicide, 3.01 (1.55-5.84) and hypertensive related disorders, 3.01 (1.55-5.84). Cancer deaths were less common 0.77 (0.68-0.88).

CONCLUSIONS

FM is weakly associated with mortality within all criteria definitions of FM and WSP examined (3.4% of explained variance), and across all diseases (RA, NIRMD, clinical FM) equally. Clinical and criteria-defined FM had different mortality outcomes. We found no evidence for a positive association of cancer and FM or WSP.

A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes

It is widely recommended that any developed-diagnostic or prognostic-prediction model is externally validated in terms of its predictive performance measured by calibration and discrimination. When multiple validations have been performed, a systematic review followed by a formal meta-analysis helps to summarize overall performance across multiple settings, and reveals under which circumstances the model performs suboptimal (alternative poorer) and may need adjustment. We discuss how to undertake meta-analysis of the performance of prediction models with either a binary or a time-to-event outcome. We address how to deal with incomplete availability of study-specific results (performance estimates and their precision), and how to produce summary estimates of the c-statistic, the observed:expected ratio and the calibration slope. Furthermore, we discuss the implementation of frequentist and Bayesian meta-analysis methods, and propose novel empirically-based prior distributions to improve estimation of between-study heterogeneity in small samples. Finally, we illustrate all methods using two examples: meta-analysis of the predictive performance of EuroSCORE II and of the Framingham Risk Score. All examples and meta-analysis models have been implemented in our newly developed R package "metamisc".