The European Network for the Study of Adrenal Tumors staging system (2015): a United States validation

OBJECTIVE

To test the ability of the 2015 modified version of the European Network for the Study of Adrenal Tumors-staging system (mENSAT) in predicting cancer specific-mortality (CSM), as well as overall mortality (OM) in adrenocortical carcinoma (ACC) patients of all stages, in a large scale, and contemporary United States cohort.

METHODS

We relied on the Surveillance, Epidemiology, and End Results (SEER) database (2004-2020) to test the accuracy and calibration of the mENSAT and subsequently compared it to the 8th edition of the American Joint Committee on Cancer-staging system (AJCC).

RESULTS

In 858 ACC patients, mENSAT accuracy was 74.7% for three-year CSM predictions and 73.8% for three-year OM predictions. The maximum departures from ideal predictions in mENSAT were +17.2% for CSM and +11.8% for OM. Conversely, AJCC accuracy was 74.5% for three-year CSM predictions and 73.5% for three-year OM predictions. The maximum departures from ideal predictions in AJCC were -6.7% for CSM and -7.1% for OM.

CONCLUSION

The accuracy of mENSAT is virtually the same as that of AJCC in predicting CSM (74.7 vs. 74.5%) and OM (73.7 vs. 73.5%). However, calibration is lower for mENSAT than for AJCC. In consequence, no obvious benefit appears to be associated with the use of mENSAT relative to AJCC in United States ACC patients.

Development and External Validation of Individualized Prediction Models for Pain Intensity Outcomes in Patients With Neck Pain, Low Back Pain, or Both in Primary Care Settings

OBJECTIVE

The purpose of this study was to develop and externally validate multivariable prediction models for future pain intensity outcomes to inform targeted interventions for patients with neck or low back pain in primary care settings.

METHODS

Model development data were obtained from a group of 679 adults with neck or low back pain who consulted a participating United Kingdom general practice. Predictors included self-report items regarding pain severity and impact from the STarT MSK Tool. Pain intensity at 2 and 6 months was modeled separately for continuous and dichotomized outcomes using linear and logistic regression, respectively. External validation of all models was conducted in a separate group of 586 patients recruited from a similar population with patients' predictor information collected both at point of consultation and 2 to 4 weeks later using self-report questionnaires. Calibration and discrimination of the models were assessed separately using STarT MSK Tool data from both time points to assess differences in predictive performance.

RESULTS

Pain intensity and patients reporting their condition would last a long time contributed most to predictions of future pain intensity conditional on other variables. On external validation, models were reasonably well calibrated on average when using tool measurements taken 2 to 4 weeks after consultation (calibration slope = 0.848 [95% CI = 0.767 to 0.928] for 2-month pain intensity score), but performance was poor using point-of-consultation tool data (calibration slope for 2-month pain intensity score of 0.650 [95% CI = 0.549 to 0.750]).

CONCLUSION

Model predictive accuracy was good when predictors were measured 2 to 4 weeks after primary care consultation, but poor when measured at the point of consultation. Future research will explore whether additional, nonmodifiable predictors improve point-of-consultation predictive performance.

IMPACT

External validation demonstrated that these individualized prediction models were not sufficiently accurate to recommend their use in clinical practice. Further research is required to improve performance through inclusion of additional nonmodifiable risk factors.

Performance of Stepwise Screening Methods in Identifying Individuals at High Risk of Type 2 Diabetes in an Iranian Population

BACKGROUND

Recent evidence recommended stepwise screening methods for identifying individuals at high risk of type 2 diabetes to be recruited in the lifestyle intervention programs for the prevention of the disease. This study aims to assess the performance of different stepwise screening methods that combine non-invasive measurements with lab-based measurements for identifying those with 5-years incident type 2 diabetes.

METHODS

3037 participants aged ≥30 years without diabetes at baseline in the Tehran Lipid and Glucose Study (TLGS) were followed. Thirty-two stepwise screening methods were developed by combining a non-invasive measurement (an anthropometric measurement (waist-to-height ratio, WtHR) or a score based on a non-invasive risk score [Australian Type 2 Diabetes Risk Assessment Tool, AUSDRISK]) with a lab-based measurement (different cut-offs of fasting plasma glucose [FPG] or predicted risk based on three lab-based prediction models [Saint Antonio, SA; Framingham Offspring Study, FOS; and the Atherosclerosis Risk in Communities, ARIC]). The validation, calibration, and usefulness of lab-based prediction models were assessed before developing the stepwise screening methods. Cut-offs were derived either based on previous studies or decision-curve analyses.

RESULTS

203 participants developed diabetes in 5 years. Lab-based risk prediction models had good discrimination power (area under the curves [AUCs]: 0.80-0.83), achieved acceptable calibration and net benefits after recalibration for population's characteristics and were useful in a wide range of risk thresholds (5%-21%). Different stepwise methods had sensitivity ranged 20%-68%, specificity 70%-98%, and positive predictive value (PPV) 14%-46%; they identified 3%-33% of the screened population eligible for preventive interventions.

CONCLUSION

Stepwise methods have acceptable performance in identifying those at high risk of incident type 2 diabetes.

Hepatocellular Carcinoma Prediction Models in Chronic Hepatitis B: A Systematic Review of 14 Models and External Validation

BACKGROUND & AIMS

The aim of our study was to characterize the performance of hepatocellular carcinoma (HCC) prediction models in chronic hepatitis B (CHB) patients through meta-analysis followed by external validation.

METHODS

We performed a systematic review and meta-analysis of current literature, followed by external validation in independent multi-center cohort with 986 patients with CHB undergoing entecavir treatment (median follow-up: 4.7 years). Model performance to predict HCC within 3, 5, 7, and 10 years was assessed using area under receiver operating characteristic curve (AUROC) and calibration index. Subgroup analysis were conducted by treatment status, cirrhotic, race and baseline alanine aminotransferase.

RESULTS

We identified 14 models with 123,885 patients (5,452 HCC cases), with REACH-B, CU-HCC, GAG-HCC, PAGE-B and mPAGE-B models being broadly externally validated. Discrimination was generally acceptable for all models, with pooled AUC ranging from 0.70 (95% CI, 0.63-0.76 for REACH-B) to 0.83 (95% CI, 0.78-0.87 for REAL-B) for 3-year, 0.68 (95% CI, 0.64-0.73 for REACH-B) to 0.81 (95% CI, 0.77-0.85 for REAL-B) for 5-year and 0.70 (95% CI, 0.58-0.80 for PAGE-B) to 0.81 (95% CI, 0.78-0.84 for REAL-B and 0.77-0.86 for AASL-HCC) for 10-year prediction. However, calibration performance was poorly reported in most studies. In external validation cohort, REAL-B showed highest discrimination with 0.76 (95% CI, 0.69-0.83) and 0.75 (95% CI, 0.70-0.81) for 3 and 5-year prediction. The REAL-B model was also well calibrated in the external validation cohort (3-year Brier score 0.066). Results were consistent in subgroup analyses.

CONCLUSIONS

In a systematic review of available HCC models, the REAL-B model exhibited best discrimination and calibration.

Update and, internal and temporal-validation of the FRANCE-2 and ACC-TAVI early-mortality prediction models for Transcatheter Aortic Valve Implantation (TAVI) using data from the Netherlands heart registration (NHR)

Background

The predictive performance of the models FRANCE-2 and ACC-TAVI for early-mortality after Transcatheter Aortic Valve Implantation (TAVI) can decline over time and can be enhanced by updating them on new populations. We aim to update and internally and temporally validate these models using a recent TAVI-cohort from the Netherlands Heart Registration (NHR).

Methods

We used data of TAVI-patients treated in 2013-2017. For each original-model, the best update-method (model-intercept, model-recalibration, or model-revision) was selected by a closed-testing procedure. We internally validated both updated models with 1000 bootstrap samples. We also updated the models on the 2013-2016 dataset and temporally validated them on the 2017-dataset. Performance measures were the Area-Under ROC-curve (AU-ROC), Brier-score, and calibration graphs.

Results

We included 6177 TAVI-patients, with 4.5% observed early-mortality. The selected update-method for FRANCE-2 was model-intercept-update. Internal validation showed an AU-ROC of 0.63 (95%CI 0.62-0.66) and Brier-score of 0.04 (0.04-0.05). Calibration graphs show that it overestimates early-mortality. In temporal-validation, the AU-ROC was 0.61 (0.53-0.67).The selected update-method for ACC-TAVI was model-revision. In internal-validation, the AU-ROC was 0.63 (0.63-0.66) and Brier-score was 0.04 (0.04-0.05). The updated ACC-TAVI calibrates well up to a probability of 20%, and subsequently underestimates early-mortality. In temporal-validation the AU-ROC was 0.65 (0.58-0.72).

Conclusion

Internal-validation of the updated models FRANCE-2 and ACC-TAVI with data from the NHR demonstrated improved performance, which was better than in external-validation studies and comparable to the original studies. In temporal-validation, ACC-TAVI outperformed FRANCE-2 because it suffered less from changes over time.