Assessing Performance and Clinical Usefulness in Prediction Models With Survival Outcomes: Practical Guidance for Cox Proportional Hazards Models

Simple blood tests to diagnose compensated advanced chronic liver disease and stratify the risk of clinically significant portal hypertension

BACKGROUND AND AIMS

Compensated advanced chronic liver disease (cACLD) identifies patients at risk for clinically significant portal hypertension (CSPH), and thus, for liver-related complications. The limited availability of liver stiffness measurements (LSM) impedes the identification of patients at risk for cACLD/CSPH outside of specialized clinics. We aimed to develop a blood-based algorithm to identify cACLD by fibrosis-4 (FIB-4) and CSPH by von Willebrand factor/platelet count ratio (VITRO).

APPROACH AND RESULTS

Patients with (suspected) compensated chronic liver disease undergoing FIB-4+LSM were included in the LSM/FIB-4 cohorts from Vienna and Salzburg. The HVPG/VITRO cohorts included patients undergoing HVPG-measurement + VITRO from Vienna and Bern.LSM/FIB-4-derivation-cohort: We included 6143 patients, of whom 211 (3.4%) developed hepatic decompensation. In all, 1724 (28.1%) had LSM ≥ 10 kPa, which corresponded to FIB-4 ≥ 1.75. Importantly, both LSM (AUROC:0.897 [95% CI:0.865-0.929]) and FIB-4 (AUROC:0.914 [95% CI:0.885-0.944]) were similarly accurate in predicting hepatic decompensation within 3 years. FIB-4 ≥ 1.75 identified patients at risk for first hepatic decompensation (5 y-cumulative incidence:7.6%), while in those <1.75, the risk was negligible (0.3%).HVPG/VITRO-derivation cohort: 247 patients of whom 202 had cACLD/FIB-4 ≥ 1.75 were included. VITRO exhibited an excellent diagnostic performance for CSPH (AUROC:0.889 [95% CI:0.844-0.934]), similar to LSM (AUROC:0.856 [95% CI:0.801-0.910], p = 0.351) and the ANTICIPATE model (AUROC:0.910 [95% CI:0.869-0.952], p = 0.498). VITRO < 1.0/ ≥ 2.5 ruled-out (sensitivity:100.0%)/ruled-in (specificity:92.4%) CSPH. The diagnostic performance was comparable to the Baveno-VII criteria.LSM/FIB-4-derivation cohort findings were externally validated in n = 1560 patients, while HVPG/VITRO-derivation-cohort findings were internally (n = 133) and externally (n = 55) validated.

CONCLUSIONS

Simple, broadly available laboratory tests (FIB-4/VITRO) facilitate cACLD detection and CSPH risk stratification in patients with (suspected) liver disease. This blood-based approach is applicable outside of specialized clinics and may promote early intervention.

Estimating Risk Factor Time Paths Among People with Type 2 Diabetes and QALY Gains from Risk Factor Management

OBJECTIVES

Most type 2 diabetes simulation models utilise equations mapping out lifetime trajectories of risk factors [e.g. glycated haemoglobin (HbA1c)]. Existing equations, using historic data or assuming constant risk factors, frequently underestimate or overestimate complication rates. Updated risk factor time path equations are needed for simulation models to more accurately predict complication rates.

AIMS

(1) Update United Kingdom Prospective Diabetes Study Outcomes Model (UKPDS-OM2) risk factor time path equations; (2) compare quality-adjusted life-years (QALYs) using original and updated equations; and (3) compare QALY gains for reference case simulations using different risk factor equations.

METHODS

Using pooled contemporary data from two randomised trials EXSCEL and TECOS (n = 28,608), we estimated: dynamic panel models of seven continuous risk factors (high-density lipoprotein cholesterol, low density lipoprotein cholesterol, HbA1c, haemoglobin, heart rate, blood pressure and body mass index); two-step models of estimated glomerular filtration rate; and survival analyses of peripheral arterial disease, atrial fibrillation and albuminuria. UKPDS-OM2-derived lifetime QALYs were extrapolated over 70 years using historical and the new risk factor equations.

RESULTS

All new risk factor equation predictions were within 95% confidence intervals of observed values, displaying good agreement between observed and estimated values. Historical risk factor time path equations predicted trial participants would accrue 9.84 QALYs, increasing to 10.98 QALYs using contemporary equations.

DISCUSSION

Incorporating updated risk factor time path equations into diabetes simulation models could give more accurate predictions of long-term health, costs, QALYs and cost-effectiveness estimates, as well as a more precise understanding of the impact of diabetes on patients' health, expenditure and quality of life.

TRIAL REGISTRATION

ClinicalTrials.gov NCT01144338 and NCT00790205.

Porto-sinusoidal vascular liver disorder with portal hypertension: Natural History and Long-Term Outcome

BACKGROUND & AIMS

Current knowledge of the natural history of patients with porto-sinusoidal vascular disorder (PSVD) is derived from small studies. The aim of the present study was to determine natural history and prognostic factors using a large multicenter cohort of PSVD patients.

METHODS

Retrospective multicentric study of PSVD patients and signs of portal hypertension (PH) prospectively registered in 27 centers.

RESULTS

587 patients were included, median age of 47 years and 38% were women. Four-hundred and one patient had an associated condition, that was graded as severe in 157. Median follow-up was 68 months. At diagnosis, 64% of patients were asymptomatic while 36% had a PH-related complication: PH-related bleeding in 112 patients; ascites in 117 and hepatic encephalopathy in 11. In those not presenting with bleeding, the incidence of first bleeding was of 15% at 5 years, with a 5-year rebleeding rate of 18%. Five-year cumulative incidence of new or worsening ascites was of 18% and of developing PVT of 16%. Fifty (8.5%) patients received a liver transplantation and 109 (19%) died, including 55 non-liver related death. Transplant-free survival was 97%, and 83% at 1 and 5 years. Variables independently associated with transplant-free survival were age, ascites, serum bilirubin, albumin and creatinine levels at diagnosis and severe associated conditions. This allowed the creation of a Nomogram that accurately predicted prognosis.

CONCLUSIONS

Prognosis of PSVD is strongly determined by the severity of the associated underlying conditions and parameters of liver and renal function.

Incorporating Individual-Level Treatment Effects and Outcome Preferences Into Personalized Blood Pressure Target Recommendations

BACKGROUND

There are no shared decision-making frameworks for selecting blood pressure (BP) targets for individuals with hypertension. This study addressed whether results from the SPRINT (Systolic Blood Pressure Intervention Trial) could be tailored to individuals using predicted risks and simulated preferences.

METHODS AND RESULTS

Among 8202 SPRINT participants, Cox models were developed and internally validated to predict each individual's absolute difference in risk from intensive versus standard BP lowering for cardiovascular events, cognitive impairment, death, and serious adverse events (AEs). Individual treatment effects were combined using simulated preference weights into a net benefit, which represents a weighted sum of risk differences across outcomes. Net benefits were compared among those above versus below the median AE risk. In simulations for which cardiovascular, cognitive, and death events had much greater weight than the AEs of BP lowering, the median net benefit was 3.3 percentage points (interquartile range [IQR], 2.0-5.7), and 100% of participants had a net benefit favoring intensive BP lowering. When simulating benefits and harms to have similar weights, the median net benefit was 0.8 percentage points (IQR, 0.2-2.2), and 87% had a positive net benefit. Compared with participants at lower risk of AEs from BP lowering, those at higher risk had a greater net benefit from intensive BP lowering despite experiencing more AEs (P<0.001 in both simulations).

CONCLUSIONS

Most SPRINT participants had a predicted net benefit that favored intensive BP lowering, but the degree of net benefit varied considerably. Tailoring BP targets using each patient's risks and preferences may provide more refined BP target recommendations.

Exploring the potential of IL-10 for risk assessment and early intervention in pediatric ALL

Acute lymphoblastic leukemia (ALL), a leading cause of childhood cancer, targets immune system B and T cells. While understanding its causes is crucial, predicting susceptibility holds immense power for early diagnosis and intervention. This study explored the potential of interleukin 10 (IL-10), a key immune regulator, as a predictive tool in Egyptian children. Investigating 100 ALL patients and 100 healthy controls, we analyzed the IL10 gene polymorphism (-1082 A/G) and serum levels. Strikingly, both the G allele and higher serum IL-10 levels were significantly associated with increased ALL risk (p < 0.05, OR > 1). Moreover, IL-10 emerged as a remarkably accurate predictor, boasting an AUC of 0.995, with a sensitivity of 97% and specificity of 96%. These findings unveil the potential of IL-10 as a powerful predictive tool for pediatric ALL in the studied Egyptian population. Identifying individuals with the GG/AG haplotype and elevated IL-10 levels could enable early intervention and potentially improve outcomes. While further validation in larger and more diverse populations is needed, this study paves the way for personalized risk assessment and potentially revolutionizes how we combat this childhood killer.

Point-of-Care Noninvasive Prediction of Liver-Related Events in Patients With Nonalcoholic Fatty Liver Disease

BACKGROUND & AIMS

Individual risk prediction of liver-related events (LRE) is needed for clinical assessment of nonalcoholic fatty liver disease (NAFLD)/nonalcoholic steatohepatitis (NASH) patients. We aimed to provide point-of-care validated liver stiffness measurement (LSM)-based risk prediction models for the development of LRE in patients with NAFLD, focusing on selecting patients for clinical trials at risk of clinical events.

METHODS

Two large multicenter cohorts were evaluated, 2638 NAFLD patients covering all LSM values as the derivation cohort and 679 more advanced patients as the validation cohort. We used Cox regression to develop and validate risk prediction models based on LSM alone, and the ANTICIPATE and ANTICIPATE-NASH models for clinically significant portal hypertension. The main outcome of the study was the rate of LRE in the first 3 years after initial assessment.

RESULTS

The 3 predictive models had similar performance in the derivation cohort with a very high discriminative value (c-statistic, 0.87-0.91). In the validation cohort, the LSM-LRE alone model had a significant inferior discrimination (c-statistic, 0.75) compared with the other 2 models, whereas the ANTICIPATE-NASH-LRE model (0.81) was significantly better than the ANTICIPATE-LRE model (0.79). In addition, the ANTICIPATE-NASH-LRE model presented very good calibration in the validation cohort (integrated calibration index, 0.016), and was better than the ANTICIPATE-LRE model.

CONCLUSIONS

The ANTICIPATE-LRE models, and especially the ANTICIPATE-NASH-LRE model, could be valuable validated clinical tools to individually assess the risk of LRE at 3 years in patients with NAFLD/NASH.

Prioritising deteriorating patients using time-to-event analysis: prediction model development and internal-external validation

BACKGROUND

Binary classification models are frequently used to predict clinical deterioration, however they ignore information on the timing of events. An alternative is to apply time-to-event models, augmenting clinical workflows by ranking patients by predicted risks. This study examines how and why time-to-event modelling of vital signs data can help prioritise deterioration assessments using lift curves, and develops a prediction model to stratify acute care inpatients by risk of clinical deterioration.

METHODS

We developed and validated a Cox regression for time to in-hospital mortality. The model used time-varying covariates to estimate the risk of clinical deterioration. Adult inpatient medical records from 5 Australian hospitals between 1 January 2019 and 31 December 2020 were used for model development and validation. Model discrimination and calibration were assessed using internal-external cross validation. A discrete-time logistic regression model predicting death within 24 h with the same covariates was used as a comparator to the Cox regression model to estimate differences in predictive performance between the binary and time-to-event outcome modelling approaches.

RESULTS

Our data contained 150,342 admissions and 1016 deaths. Model discrimination was higher for Cox regression than for discrete-time logistic regression, with cross-validated AUCs of 0.96 and 0.93, respectively, for mortality predictions within 24 h, declining to 0.93 and 0.88, respectively, for mortality predictions within 1 week. Calibration plots showed that calibration varied by hospital, but this can be mitigated by ranking patients by predicted risks.

CONCLUSION

Time-varying covariate Cox models can be powerful tools for triaging patients, which may lead to more efficient and effective care in time-poor environments when the times between observations are highly variable.

Risk of emergency hospital admission related to adverse events after antibiotic treatment in adults with a common infection: impact of COVID-19 and derivation and validation of risk prediction models

BACKGROUND

With the global challenge of antimicrobial resistance intensified during the COVID-19 pandemic, evaluating adverse events (AEs) post-antibiotic treatment for common infections is crucial. This study aims to examines the changes in incidence rates of AEs during the COVID-19 pandemic and predict AE risk following antibiotic prescriptions for common infections, considering their previous antibiotic exposure and other long-term clinical conditions.

METHODS

With the approval of NHS England, we used OpenSAFELY platform and analysed electronic health records from patients aged 18-110, prescribed antibiotics for urinary tract infection (UTI), lower respiratory tract infections (LRTI), upper respiratory tract infections (URTI), sinusitis, otitis externa, and otitis media between January 2019 and June 2023. We evaluated the temporal trends in the incidence rate of AEs for each infection, analysing monthly changes over time. The survival probability of emergency AE hospitalisation was estimated in each COVID-19 period (period 1: 1 January 2019 to 25 March 2020, period 2: 26 March 2020 to 8 March 2021, period 3: 9 March 2021 to 30 June 2023) using the Kaplan-Meier approach. Prognostic models, using Cox proportional hazards regression, were developed and validated to predict AE risk within 30 days post-prescription using the records in Period 1.

RESULTS

Out of 9.4 million patients who received antibiotics, 0.6% of UTI, 0.3% of URTI, and 0.5% of LRTI patients experienced AEs. UTI and LRTI patients demonstrated a higher risk of AEs, with a noted increase in AE incidence during the COVID-19 pandemic. Higher comorbidity and recent antibiotic use emerged as significant AE predictors. The developed models exhibited good calibration and discrimination, especially for UTIs and LRTIs, with a C-statistic above 0.70.

CONCLUSIONS

The study reveals a variable incidence of AEs post-antibiotic treatment for common infections, with UTI and LRTI patients facing higher risks. AE risks varied between infections and COVID-19 periods. These findings underscore the necessity for cautious antibiotic prescribing and call for further exploration into the intricate dynamics between antibiotic use, AEs, and the pandemic.

Post-treatment LSM rather than change during treatment predicts decompensation in patients with cACLD after HCV cure

BACKGROUND & AIMS

Baveno VII has defined a clinically significant (i.e., prognostically meaningful) decrease in liver stiffness measurement (LSM) in cACLD as a decrease of ≥20% associated with a final LSM <20 kPa or any decrease to <10 kPa. However, these rules have not yet been validated against direct clinical endpoints.

METHODS

We retrospectively analysed patients with cACLD (LSM ≥10 kPa) with paired liver stiffness measurement (LSM) before (BL) and after (FU) HCV cure by interferon-free therapies from 15 European centres. The cumulative incidence of hepatic decompensation was compared according to these criteria, considering hepatocellular carcinoma and non-liver-related death as competing risks.

RESULTS

A total of 2,335 patients followed for a median of 6 years were analysed. Median BL-LSM was 16.6 kPa with 37.1% having ≥20 kPa. After HCV cure, FU-LSM decreased to a median of 10.9 kPa (<10 kPa: 1,002 [42.9%], ≥20 kPa: 465 [19.9%]) translating into a median LSM change of -5.3 (-8.8 to -2.4) kPa corresponding to -33.9 (-48.0 to -15.9) %. Patients achieving a clinically significant decrease (65.4%) had a significantly lower risk of hepatic decompensation (subdistribution hazard ratio: 0.12, 95% CI 0.04-0.35, p <0.001). However, these risk differences were primarily driven by a negligible risk in patients with FU-LSM <10 kPa (5-year cumulative incidence: 0.3%) compared to a high risk in patients with FU-LSM ≥20 kPa (16.6%). Patients with FU-LSM 10-19.9 kPa (37.4%) also had a low risk of hepatic decompensation (5-year cumulative incidence: 1.7%), and importantly, the risk of hepatic decompensation did not differ between those with/without an LSM decrease of ≥20% (p = 0.550).

CONCLUSIONS

FU-LSM is key for risk stratification after HCV cure and should guide clinical decision making. LSM dynamics do not hold significant prognostic information in patients with FU-LSM 10-19.9 kPa, and thus, their consideration is not of sufficient incremental value in the specific context of HCV cure.

IMPACT AND IMPLICATIONS

Liver stiffness measurement (LSM) is increasingly applied as a prognostic biomarker and commonly decreases in patients with compensated advanced chronic liver disease achieving HCV cure. Although Baveno VII proposed criteria for a clinically significant decrease, little is known about the prognostic utility of LSM dynamics (changes through antiviral therapy). Interestingly, in those with a post-treatment LSM of 10-19.9 kPa, LSM dynamics did not provide incremental information, arguing against the consideration of LSM dynamics as prognostic criteria. Thus, post-treatment LSM should guide the management of patients with compensated advanced chronic liver disease achieving HCV cure.

Ten-Year Risk Equations for Incident Heart Failure in Established Atherosclerotic Cardiovascular Disease Populations

BACKGROUND

Ten-year risk equations for incident heart failure (HF) are available for the general population, but not for patients with established atherosclerotic cardiovascular disease (ASCVD), which is highly prevalent in HF cohorts. This study aimed to develop and validate 10-year risk equations for incident HF in patients with known ASCVD.

METHODS AND RESULTS

Ten-year risk equations for incident HF were developed using the United Kingdom Biobank cohort (recruitment 2006-2010) including participants with established ASCVD but free from HF at baseline. Model performance was validated using the Australian Baker Heart and Diabetes Institute Biobank cohort (recruitment 2000-2011) and compared with the performance of general population risk models. Incident HF occurred in 13.7% of the development cohort (n=31 446, median 63 years, 35% women, follow-up 10.7±2.7 years) and in 21.3% of the validation cohort (n=1659, median age 65 years, 25% women, follow-up 9.4±3.7 years). Predictors of HF included in the sex-specific models were age, body mass index, systolic blood pressure (treated or untreated), glucose (treated or untreated), cholesterol, smoking status, QRS duration, kidney disease, myocardial infarction, and atrial fibrillation. ASCVD-HF equations had good discrimination and calibration in development and validation cohorts, with superior performance to general population risk equations.

CONCLUSIONS

ASCVD-specific 10-year risk equations for HF outperform general population risk models in individuals with established ASCVD. The ASCVD-HF equations can be calculated from readily available clinical data and could facilitate screening and preventative treatment decisions in this high-risk group.

Early Breast Cancer Risk Assessment: Integrating Histopathology with Artificial Intelligence

Effective risk assessment in early breast cancer is essential for informed clinical decision-making, yet consensus on defining risk categories remains challenging. This paper explores evolving approaches in risk stratification, encompassing histopathological, immunohistochemical, and molecular biomarkers alongside cutting-edge artificial intelligence (AI) techniques. Leveraging machine learning, deep learning, and convolutional neural networks, AI is reshaping predictive algorithms for recurrence risk, thereby revolutionizing diagnostic accuracy and treatment planning. Beyond detection, AI applications extend to histological subtyping, grading, lymph node assessment, and molecular feature identification, fostering personalized therapy decisions. With rising cancer rates, it is crucial to implement AI to accelerate breakthroughs in clinical practice, benefiting both patients and healthcare providers. However, it is important to recognize that while AI offers powerful automation and analysis tools, it lacks the nuanced understanding, clinical context, and ethical considerations inherent to human pathologists in patient care. Hence, the successful integration of AI into clinical practice demands collaborative efforts between medical experts and computational pathologists to optimize patient outcomes.

Weighted metrics are required when evaluating the performance of prediction models in nested case-control studies

BACKGROUND

Nested case-control (NCC) designs are efficient for developing and validating prediction models that use expensive or difficult-to-obtain predictors, especially when the outcome is rare. Previous research has focused on how to develop prediction models in this sampling design, but little attention has been given to model validation in this context. We therefore aimed to systematically characterize the key elements for the correct evaluation of the performance of prediction models in NCC data.

METHODS

We proposed how to correctly evaluate prediction models in NCC data, by adjusting performance metrics with sampling weights to account for the NCC sampling. We included in this study the C-index, threshold-based metrics, Observed-to-expected events ratio (O/E ratio), calibration slope, and decision curve analysis. We illustrated the proposed metrics with a validation of the Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm (BOADICEA version 5) in data from the population-based Rotterdam study. We compared the metrics obtained in the full cohort with those obtained in NCC datasets sampled from the Rotterdam study, with and without a matched design.

RESULTS

Performance metrics without weight adjustment were biased: the unweighted C-index in NCC datasets was 0.61 (0.58-0.63) for the unmatched design, while the C-index in the full cohort and the weighted C-index in the NCC datasets were similar: 0.65 (0.62-0.69) and 0.65 (0.61-0.69), respectively. The unweighted O/E ratio was 18.38 (17.67-19.06) in the NCC datasets, while it was 1.69 (1.42-1.93) in the full cohort and its weighted version in the NCC datasets was 1.68 (1.53-1.84). Similarly, weighted adjustments of threshold-based metrics and net benefit for decision curves were unbiased estimates of the corresponding metrics in the full cohort, while the corresponding unweighted metrics were biased. In the matched design, the bias of the unweighted metrics was larger, but it could also be compensated by the weight adjustment.

CONCLUSIONS

Nested case-control studies are an efficient solution for evaluating the performance of prediction models that use expensive or difficult-to-obtain biomarkers, especially when the outcome is rare, but the performance metrics need to be adjusted to the sampling procedure.

Differential analysis of histopathological and genetic markers of cancer aggressiveness, and survival difference in EBV-positive and EBV-negative prostate carcinoma

Several studies have shown an association between prostate carcinoma (PCa) and Epstein-Barr virus (EBV); however, none of the studies so far have identified the histopathological and genetic markers of cancer aggressiveness associated with EBV in PCa tissues. In this study, we used previously characterized EBV-PCR-positive (n = 39) and EBV-negative (n = 60) PCa tissues to perform an IHC-based assessment of key histopathological and molecular markers of PCa aggressiveness (EMT markers, AR expression, perineural invasion, and lymphocytic infiltration characterization). Additionally, we investigated the differential expression of key oncogenes, EMT-associated genes, and PCa-specific oncomiRs, in EBV-positive and -negative tissues, using the qPCR array. Finally, survival benefit analysis was also performed in EBV-positive and EBV-negative PCa patients. The EBV-positive PCa exhibited a higher percentage (80%) of perineural invasion (PNI) compared to EBV-negative PCa (67.3%) samples. Similarly, a higher lymphocytic infiltration was observed in EBV-LMP1-positive PCa samples. The subset characterization of T and B cell lymphocytic infiltration showed a trend of higher intratumoral and tumor stromal lymphocytic infiltration in EBV-negative tissues compared with EBV-positive tissues. The logistic regression analysis showed that EBV-positive status was associated with decreased odds (OR = 0.07; p-value < 0.019) of CD3 intratumoral lymphocytic infiltration in PCa tissues. The analysis of IHC-based expression patterns of EMT markers showed comparable expression of all EMT markers, except vimentin, which showed higher expression in EBV-positive PCa tissues compared to EBV-negative PCa tissues. Furthermore, gene expression analysis showed a statistically significant difference (p < 0.05) in the expression of CDH1, AR, CHEK-2, CDKN-1B, and CDC-20 and oncomiRs miR-126, miR-152-3p, miR-452, miR-145-3p, miR-196a, miR-183-3p, and miR-146b in EBV-positive PCa tissues compared to EBV-negative PCa tissues. Overall, the survival proportion was comparable in both groups. The presence of EBV in the PCa tissues results in an increased expression of certain oncogenes, oncomiRs, and EMT marker (vimentin) and a decrease in CD3 ITL, which may be associated with the aggressive forms of PCa.

Prediction Under Interventions: Evaluation of Counterfactual Performance Using Longitudinal Observational Data

Predictions under interventions are estimates of what a person's risk of an outcome would be if they were to follow a particular treatment strategy, given their individual characteristics. Such predictions can give important input to medical decision-making. However, evaluating the predictive performance of interventional predictions is challenging. Standard ways of evaluating predictive performance do not apply when using observational data, because prediction under interventions involves obtaining predictions of the outcome under conditions that are different from those that are observed for a subset of individuals in the validation dataset. This work describes methods for evaluating counterfactual performance of predictions under interventions for time-to-event outcomes. This means we aim to assess how well predictions would match the validation data if all individuals had followed the treatment strategy under which predictions are made. We focus on counterfactual performance evaluation using longitudinal observational data, and under treatment strategies that involve sustaining a particular treatment regime over time. We introduce an estimation approach using artificial censoring and inverse probability weighting that involves creating a validation dataset mimicking the treatment strategy under which predictions are made. We extend measures of calibration, discrimination (c-index and cumulative/dynamic AUCt) and overall prediction error (Brier score) to allow assessment of counterfactual performance. The methods are evaluated using a simulation study, including scenarios in which the methods should detect poor performance. Applying our methods in the context of liver transplantation shows that our procedure allows quantification of the performance of predictions supporting crucial decisions on organ allocation.

Reproductive factors and risk of cardiovascular diseases and all-cause and cardiovascular mortality in American women: NHANES 2003-2018

BACKGROUND

The evidence regarding the association of reproductive factors with cardiovascular diseases (CVDs) is limited.

AIMS

To investigate the relationship of reproductive factors with the risk of CVDs, as well as all-cause and cardiovascular mortality.

METHODS

This study included 16,404 adults with reproductive factors from the National Health and Nutrition Examination Survey (NHANES) and followed up until 31 December 2019. Logistic models and restricted cubic spline models were used to assess the association of reproductive factors with CVDs. COX proportional hazards models and restricted cubic spline models, with adjustment for potential confounding, were employed to analyze the relation between reproductive factors and cardiovascular and all-cause death.

RESULTS

There is a nonlinear relationship between age at menarche and CVDs. Age at menopause ≤ 11(OR 1.36, 95% CI 1.10-1.69) was associated with an increased risk of CVDs compared to ages 12-13 years. Age at Menopause ≤ 44 (OR 1.69, 95% CI 1.40-2.03) was associated with increased CVDs compared to age 35-49 years. Number of pregnancies ≥ 5(OR 1.26, 95% CI 1.02-1.55) was associated with an increased risk of CVDs compared to one pregnancy. In continuous variable COX regression models, a later age at menopause (HR 0.98, 95% CI 0.97-0.99) and a longer reproductive lifespan (HR 0.98, 95% CI 0.97-0.99) were associated with a decreased risk of all-cause death. A later age at menopause (HR 0.98, 95% CI 0.97-0.99) and a longer reproductive lifespan (HR 0.98, 95% CI 0.97-0.99) were associated with a decreased risk of cardiac death.

CONCLUSIONS

Female reproductive factors are significant risk factors for CVDs American women.

Enhanced Risk Stratification for Children and Young Adults with B-Cell Acute Lymphoblastic Leukemia: A Children's Oncology Group Report

Current strategies to treat pediatric acute lymphoblastic leukemia rely on risk stratification algorithms using categorical data. We investigated whether using continuous variables assigned different weights would improve risk stratification. We developed and validated a multivariable Cox model for relapse-free survival (RFS) using information from 21199 patients. We constructed risk groups by identifying cutoffs of the COG Prognostic Index (PICOG) that maximized discrimination of the predictive model. Patients with higher PICOG have higher predicted relapse risk. The PICOG reliably discriminates patients with low vs. high relapse risk. For those with moderate relapse risk using current COG risk classification, the PICOG identifies subgroups with varying 5-year RFS. Among current COG standard-risk average patients, PICOG identifies low and intermediate risk groups with 96% and 90% RFS, respectively. Similarly, amongst current COG high-risk patients, PICOG identifies four groups ranging from 96% to 66% RFS, providing additional discrimination for future treatment stratification. When coupled with traditional algorithms, the novel PICOG can more accurately risk stratify patients, identifying groups with better outcomes who may benefit from less intensive therapy, and those who have high relapse risk needing innovative approaches for cure.

Derivation of an Annualized Claims-Based Major Adverse Cardiovascular Event Estimator in Type 2 Diabetes

Background

Major adverse cardiovascular events (MACE) are a leading cause of morbidity and mortality among adults with type 2 diabetes. Currently, available MACE prediction models have important limitations, including reliance on data that may not be routinely available, narrow focus on primary prevention, limited patient populations, and longtime horizons for risk prediction.

Objectives

The purpose of this study was to derive and internally validate a claims-based prediction model for 1-year risk of MACE in type 2 diabetes.

Methods

Using medical and pharmacy claims for adults with type 2 diabetes enrolled in commercial, Medicare Advantage, and Medicare fee-for-service plans between 2014 and 2021, we derived and internally validated the annualized claims-based MACE estimator (ACME) model to predict the risk of MACE (nonfatal acute myocardial infarction, nonfatal stroke, and all-cause mortality). The Cox proportional hazards model was composed of 30 covariates, including patient age, sex, comorbidities, and medications.

Results

The study cohort comprised 6,623,526 adults with type 2 diabetes, mean age 68.1 ± 10.6 years, 49.8% women, and 73.0% Non-Hispanic White. ACME had a concordance index of 0.74 (validation index range: 0.739-0.741). The predicted 1-year risk of the study cohort ranged from 0.4% to 99.9%, with a median risk of 3.4% (IQR: 2.3%-6.5%).

Conclusions

ACME was derived in a large usual care population, relies on routinely available data, and estimates short-term MACE risk. It can support population risk stratification at the health system and payer levels, participant identification for decentralized clinical trials of cardiovascular disease, and risk-stratified observational studies using real-world data.

Cardiovascular Risk Associated With Social Determinants of Health at Individual and Area Levels

Importance

The benefit of adding social determinants of health (SDOH) when estimating atherosclerotic cardiovascular disease (ASCVD) risk is unclear.

Objective

To examine the association of SDOH at both individual and area levels with ASCVD risks, and to assess if adding individual- and area-level SDOH to the pooled cohort equations (PCEs) or the Predicting Risk of CVD Events (PREVENT) equations improves the accuracy of risk estimates.

Design, Setting, and Participants

This cohort study included participants data from 4 large US cohort studies. Eligible participants were aged 40 to 79 years without a history of ASCVD. Baseline data were collected from 1995 to 2007; median (IQR) follow-up was 13.0 (9.3-15.0) years. Data were analyzed from September 2023 to February 2024.

Exposures

Individual- and area-level education, income, and employment status.

Main outcomes and measures

ASCVD was defined as the composite outcome of nonfatal myocardial infarction, death from coronary heart disease, and fatal or nonfatal stroke.

Results

A total of 26 316 participants were included (mean [SD] age, 61.0 [9.1] years; 15 494 women [58.9%]; 11 365 Black [43.2%], 703 Chinese American [2.7%], 1278 Hispanic [4.9%], and 12 970 White [49.3%]); 11 764 individuals (44.7%) had at least 1 adverse individual-level SDOH and 10 908 (41.5%) had at least 1 adverse area-level SDOH. A total of 2673 ASCVD events occurred during follow-up. SDOH were associated with increased risk of ASCVD at both the individual and area levels, including for low education (individual: hazard ratio [HR], 1.39 [95% CI, 1.25-1.55]; area: HR, 1.31 [95% CI, 1.20-1.42]), low income (individual: 1.35 [95% CI, 1.25-1.47]; area: HR, 1.28 [95% CI, 1.17-1.40]), and unemployment (individual: HR, 1.61 [95% CI, 1.24-2.10]; area: HR, 1.25 [95% CI, 1.14-1.37]). Adding area-level SDOH alone to the PCEs did not change model discrimination but modestly improved calibration. Furthermore, adding both individual- and area-level SDOH to the PCEs led to a modest improvement in both discrimination and calibration in non-Hispanic Black individuals (change in C index, 0.0051 [95% CI, 0.0011 to 0.0126]; change in scaled integrated Brier score [IBS], 0.396% [95% CI, 0.221% to 0.802%]), and improvement in calibration in White individuals (change in scaled IBS, 0.274% [95% CI, 0.095% to 0.665%]). Adding individual-level SDOH to the PREVENT plus area-level social deprivation index (SDI) equations did not improve discrimination but modestly improved calibration in White participants (change in scaled IBS, 0.182% [95% CI, 0.040% to 0.496%]), Black participants (0.187% [95% CI, 0.039% to 0.501%]), and women (0.289% [95% CI, 0.115% to 0.574%]).

Conclusions and Relevance

In this cohort study, both individual- and area-level SDOH were associated with ASCVD risk; adding both individual- and area-level SDOH to the PCEs modestly improved discrimination and calibration for estimating ASCVD risk for Black individuals, and adding individual-level SDOH to PREVENT plus SDI also modestly improved calibration. These findings suggest that both individual- and area-level SDOH may be considered in future development of ASCVD risk assessment tools, particularly among Black individuals.

How valid is a prescription-based multimorbidity index (Rx-risk) in predicting mortality in the Outcomes and Multimorbidity In Type 2 diabetes (OMIT) study? A nation-wide registry-based cohort study from Norway

OBJECTIVE

The prescription-based Rx-risk index has previously been developed to measure multimorbidity. We aimed to adapt and evaluate the validity of the Rx-risk index in prediction of mortality among persons with type 2 diabetes.

DESIGN

Registry-based study.

SETTING

Adults with type 2 diabetes in Norway identified within the 'Outcomes and Multimorbidity In Type 2 diabetes' cohort, with linkage to prescriptions from the Norwegian Prescription Database and mortality from the Population Registry.

PARTICIPANTS

We defined a calibration sample of 42 290 adults diagnosed with type 2 diabetes 1950-2013, and a temporal validation sample of 7085 adults diagnosed 2014-2016 to evaluate the index validity over time PRIMARY OUTCOME MEASURE: All-cause mortality METHODS: For the calibration sample, dispensed drug prescriptions in 2013 were used to define 44 morbidity categories. Weights were estimated using regression coefficients from a Cox regression model with 5 year mortality as the outcome and all morbidity categories, age and sex included as covariates. The Rx-risk index was computed as a weighted sum of morbidities. The validity of the index was evaluated using C-statistic and calibration plots.

RESULTS

In the calibration sample, mean (SD) age at start of follow-up and duration of diabetes was 63.8 (12.4) and 10.1 (7.0) years, respectively. The overall C-statistic was 0.82 and varied from 0.74 to 0.85 when stratifying on age groups, sex, level of education and country of origin. In the validation sample, mean (SD) age and duration of diabetes was 59.7 (13.0) and 2.0 (0.8) years, respectively. Despite younger age, shorter duration of diabetes and later time period, the C-index was high both in the total sample (0.84) and separately for men (0.83) and women (0.84).

CONCLUSIONS

The Rx-risk index showed good discrimination and calibration in predicting mortality and thus presents a valid tool to assess multimorbidity among persons with type 2 diabetes.

Tissue-resident memory T cell signatures from single-cell analysis associated with better melanoma prognosis

Tissue-resident memory T cells (TRM) are a specialized T cell population residing in peripheral tissues. The presence and potential impact of TRM in the tumor immune microenvironment (TIME) remain to be elucidated. Here, we systematically investigated the relationship between TRM and melanoma TIME based on multiple clinical single-cell RNA-seq datasets and developed signatures indicative of TRM infiltration. TRM infiltration is associated with longer overall survival and abundance of T cells, NK cells, M1 macrophages, and memory B cells in the TIME. A 22-gene TRM-derived risk score was further developed to effectively classify patients into low- and high-risk categories, distinguishing overall survival and immune activation, particularly in T cell-mediated responses. Altogether, our analysis suggests that TRM abundance is associated with melanoma TIME activation and patient survival, and the TRM-based machine learning model can potentially predict prognosis in melanoma patients.

The Improved Kidney Risk Score in ANCA-Associated Vasculitis for Clinical Practice and Trials

SIGNIFICANCE STATEMENT

Reliable prediction tools are needed to personalize treatment in ANCA-associated GN. More than 1500 patients were collated in an international longitudinal study to revise the ANCA kidney risk score. The score showed satisfactory performance, mimicking the original study (Harrell's C=0.779). In the development cohort of 959 patients, no additional parameters aiding the tool were detected, but replacing the GFR with creatinine identified an additional cutoff. The parameter interstitial fibrosis and tubular atrophy was modified to allow wider access, risk points were reweighted, and a fourth risk group was created, improving predictive ability (C=0.831). In the validation, the new model performed similarly well with excellent calibration and discrimination ( n =480, C=0.821). The revised score optimizes prognostication for clinical practice and trials.

BACKGROUND

Reliable prediction tools are needed to personalize treatment in ANCA-associated GN. A retrospective international longitudinal cohort was collated to revise the ANCA renal risk score.

METHODS

The primary end point was ESKD with patients censored at last follow-up. Cox proportional hazards were used to reweight risk factors. Kaplan-Meier curves, Harrell's C statistic, receiver operating characteristics, and calibration plots were used to assess model performance.

RESULTS

Of 1591 patients, 1439 were included in the final analyses, 2:1 randomly allocated per center to development and validation cohorts (52% male, median age 64 years). In the development cohort ( n =959), the ANCA renal risk score was validated and calibrated, and parameters were reinvestigated modifying interstitial fibrosis and tubular atrophy allowing semiquantitative reporting. An additional cutoff for kidney function (K) was identified, and serum creatinine replaced GFR (K0: <250 µ mol/L=0, K1: 250-450 µ mol/L=4, K2: >450 µ mol/L=11 points). The risk points for the percentage of normal glomeruli (N) and interstitial fibrosis and tubular atrophy (T) were reweighted (N0: >25%=0, N1: 10%-25%=4, N2: <10%=7, T0: none/mild or <25%=0, T1: ≥ mild-moderate or ≥25%=3 points), and four risk groups created: low (0-4 points), moderate (5-11), high (12-18), and very high (21). Discrimination was C=0.831, and the 3-year kidney survival was 96%, 79%, 54%, and 19%, respectively. The revised score performed similarly well in the validation cohort with excellent calibration and discrimination ( n =480, C=0.821).

CONCLUSIONS

The updated score optimizes clinicopathologic prognostication for clinical practice and trials.

Regularized parametric survival modeling to improve risk prediction models

We propose to combine the benefits of flexible parametric survival modeling and regularization to improve risk prediction modeling in the context of time-to-event data. Thereto, we introduce ridge, lasso, elastic net, and group lasso penalties for both log hazard and log cumulative hazard models. The log (cumulative) hazard in these models is represented by a flexible function of time that may depend on the covariates (i.e., covariate effects may be time-varying). We show that the optimization problem for the proposed models can be formulated as a convex optimization problem and provide a user-friendly R implementation for model fitting and penalty parameter selection based on cross-validation. Simulation study results show the advantage of regularization in terms of increased out-of-sample prediction accuracy and improved calibration and discrimination of predicted survival probabilities, especially when sample size was relatively small with respect to model complexity. An applied example illustrates the proposed methods. In summary, our work provides both a foundation for and an easily accessible implementation of regularized parametric survival modeling and suggests that it improves out-of-sample prediction performance.

Predicting survival in patients with 'non-high-risk' acute variceal bleeding receiving β-blockers+ligation to prevent re-bleeding

BACKGROUND & AIMS

Pre-emptive transjugular intrahepatic portosystemic shunt (TIPS) is the treatment of choice for high-risk acute variceal bleeding (AVB; i.e., Child-Turcotte-Pugh [CTP] B8-9+active bleeding/C10-13). Nevertheless, some 'non-high-risk' patients have poor outcomes despite the combination of non-selective beta-blockers and endoscopic variceal ligation for secondary prophylaxis. We investigated prognostic factors for re-bleeding and mortality in 'non-high-risk' AVB to identify subgroups who may benefit from more potent treatments (i.e., TIPS) to prevent further decompensation and mortality.

METHODS

A total of 2,225 adults with cirrhosis and variceal bleeding were prospectively recruited at 34 centres between 2011-2015; for the purpose of this study, case definitions and information on prognostic indicators at index AVB and on day 5 were further refined in low-risk patients, of whom 581 (without failure to control bleeding or contraindications to TIPS) who were managed by non-selective beta-blockers/endoscopic variceal ligation, were finally included. Patients were followed for 1 year.

RESULTS

Overall, 90 patients (15%) re-bled and 70 (12%) patients died during follow-up. Using clinical routine data, no meaningful predictors of re-bleeding were identified. However, re-bleeding (included as a time-dependent co-variable) increased mortality, even after accounting for differences in patient characteristics (adjusted cause-specific hazard ratio: 2.57; 95% CI 1.43-4.62; p = 0.002). A nomogram including CTP, creatinine, and sodium measured at baseline accurately (concordance: 0.752) stratified the risk of death.

CONCLUSION

The majority of 'non-high-risk' patients with AVB have an excellent prognosis, if treated according to current recommendations. However, about one-fifth of patients, i.e. those with CTP ≥8 and/or high creatinine levels or hyponatremia, have a considerable risk of death within 1 year of the index bleed. Future clinical trials should investigate whether elective TIPS placement reduces mortality in these patients.

IMPACT AND IMPLICATIONS

Pre-emptive transjugular intrahepatic portosystemic shunt placement improves outcomes in high-risk acute variceal bleeding; nevertheless, some 'non-high-risk' patients have poor outcomes despite the combination of non-selective beta-blockers and endoscopic variceal ligation. This is the first large-scale study investigating prognostic factors for re-bleeding and mortality in 'non-high-risk' acute variceal bleeding. While no clinically meaningful predictors were identified for re-bleeding, we developed a nomogram integrating baseline Child-Turcotte-Pugh score, creatinine, and sodium to stratify mortality risk. Our study paves the way for future clinical trials evaluating whether elective transjugular intrahepatic portosystemic shunt placement improves outcomes in presumably 'non-high-risk' patients who are identified as being at increased risk of death.

Survival after resection of malignant peripheral nerve sheath tumors: Introducing and validating a novel type-specific prognostic model

Background

This study aimed to assess the performance of currently available risk calculators in a cohort of patients with malignant peripheral nerve sheath tumors (MPNST) and to create an MPNST-specific prognostic model including type-specific predictors for overall survival (OS).

Methods

This is a retrospective multicenter cohort study of patients with MPNST from 11 secondary or tertiary centers in The Netherlands, Italy and the United States of America. All patients diagnosed with primary MPNST who underwent macroscopically complete surgical resection from 2000 to 2019 were included in this study. A multivariable Cox proportional hazard model for OS was estimated with prespecified predictors (age, grade, size, NF-1 status, triton status, depth, tumor location, and surgical margin). Model performance was assessed for the Sarculator and PERSARC calculators by examining discrimination (C-index) and calibration (calibration plots and observed-expected statistic; O/E-statistic). Internal-external cross-validation by different regions was performed to evaluate the generalizability of the model.

Results

A total of 507 patients with primary MPNSTs were included from 11 centers in 7 regions. During follow-up (median 8.7 years), 211 patients died. The C-index was 0.60 (95% CI 0.53-0.67) for both Sarculator and PERSARC. The MPNST-specific model had a pooled C-index of 0.69 (95%CI 0.65-0.73) at validation, with adequate discrimination and calibration across regions.

Conclusions

The MPNST-specific MONACO model can be used to predict 3-, 5-, and 10-year OS in patients with primary MPNST who underwent macroscopically complete surgical resection. Further validation may refine the model to inform patients and physicians on prognosis and support them in shared decision-making.

Gastric Cancer Immune Subtypes and Prognostic Modeling: Insights from Aging-Related Gene Analysis

Gastric cancer (GC) is highly heterogeneous and influenced by aging-related factors. This study aimed to improve individualized prognostic assessment of GC by identifying aging-related genes and subtypes. Immune scores of GC samples from GEO and TCGA databases were calculated using ESTIMATE and scored as high immune (IS_high) and low immune (IS_low). ssGSEA was used to analyze immune cell infiltration. Univariate Cox regression was employed to identify prognosis-related genes. LASSO regression analysis was used to construct a prognostic model. GSVA enrichment analysis was applied to determine pathways. CCK-8, wound healing, and Transwell assays tested the proliferation, migration, and invasion of the GC cell line (AGS). Cell cycle and aging were examined using flow cytometry, β-galactosidase staining, and Western blotting. Two aging-related GC subtypes were identified. Subtype 2 was characterized as lower survival probability and higher risk, along with a more immune-responsive tumor microenvironment. Three genes (IGFBP5, BCL11B, and AKR1B1) screened from aging-related genes were used to establish a prognosis model. The AUC values of the model were greater than 0.669, exhibiting strong prognostic value. In vitro, IGFBP5 overexpression in AGS cells was found to decrease viability, migration, and invasion, alter the cell cycle, and increase aging biomarkers (SA-β-galactosidase, p53, and p21). This analysis uncovered the immune characteristics of two subtypes and aging-related prognosis genes in GC. The prognostic model established for three aging-related genes (IGFBP5, BCL11B, and AKR1B1) demonstrated good prognosis performance, providing a foundation for personalized treatment strategies aimed at GC.

Dynamic updating of clinical survival prediction models in a changing environment

BACKGROUND

Over time, the performance of clinical prediction models may deteriorate due to changes in clinical management, data quality, disease risk and/or patient mix. Such prediction models must be updated in order to remain useful. In this study, we investigate dynamic model updating of clinical survival prediction models. In contrast to discrete or one-time updating, dynamic updating refers to a repeated process for updating a prediction model with new data. We aim to extend previous research which focused largely on binary outcome prediction models by concentrating on time-to-event outcomes. We were motivated by the rapidly changing environment seen during the COVID-19 pandemic where mortality rates changed over time and new treatments and vaccines were introduced.

METHODS

We illustrate three methods for dynamic model updating: Bayesian dynamic updating, recalibration, and full refitting. We use a simulation study to compare performance in a range of scenarios including changing mortality rates, predictors with low prevalence and the introduction of a new treatment. Next, the updating strategies were applied to a model for predicting 70-day COVID-19-related mortality using patient data from QResearch, an electronic health records database from general practices in the UK.

RESULTS

In simulated scenarios with mortality rates changing over time, all updating methods resulted in better calibration than not updating. Moreover, dynamic updating outperformed ad hoc updating. In the simulation scenario with a new predictor and a small updating dataset, Bayesian updating improved the C-index over not updating and refitting. In the motivating example with a rare outcome, no single updating method offered the best performance.

CONCLUSIONS

We found that a dynamic updating process outperformed one-time discrete updating in the simulations. Bayesian updating offered good performance overall, even in scenarios with new predictors and few events. Intercept recalibration was effective in scenarios with smaller sample size and changing baseline hazard. Refitting performance depended on sample size and produced abrupt changes in hazard ratio estimates between periods.

Evaluation of optimum classification measures used to define textbook outcome among patients undergoing curative-intent resection of gastric cancer

BACKGROUND

Textbook outcome (TO) is a composite measure reflecting various aspects of services provided to patients with solid malignancies. We sought to evaluate the importance of various TO components previously proposed for gastric cancer.

METHODS

Prospectively maintained electronic databases of 1,743 patients treated in two academic surgical centres were reviewed. Six candidate definitions of TO were evaluated based on their ability to accurately predict patients' prognosis by Cox proportional hazards modelling.

RESULTS

TO definition combining 10 measures corresponding to complete tumour resection with an uneventful postoperative course showed the best goodness of fit by achieving the lowest values of Akaike (AIC) and Bayesian (BIC) information criteria and the best predictive performance based on the highest value of c-index. The overall median survival was significantly longer for patients with than without textbook outcome (69.0 vs 20.1 months, P < 0.001). TO maintained its prognostic value in a multivariate model controlling for age, sex, comorbidities, treatment, and tumour related variables and was associated with a 39% lower risk of death (HR 0.61, 95%CI 0.51 - 0.73, P < 0.001). Nine variables identified as predictors of TO were used to develop a nomogram showing very good correlation between the predicted and actual probability of achieving TO. The AUC of ROC obtained from the nomogram was 0.752 (95% CI 0.727 to 0.781).

CONCLUSIONS

A uniform definition of textbook outcome provides clinically relevant prognostic information and could be used in quality improvement programs for gastric cancer patients.

Validity of the models predicting 10-year risk of cardiovascular diseases in Asia: A systematic review and prediction model meta-analysis

We aimed to review the validity of existing prediction models for cardiovascular diseases (CVDs) in Asia. In this systematic review and meta-analysis, we included studies that validated prediction models for CVD risk in the general population in Asia. Various databases, including PubMed, Web of Science conference proceedings citation index, Scopus, Global Index Medicus of the World Health Organization (WHO), and Open Access Thesis and Dissertations (OATD), were searched up to November 2022. Additional studies were identified through reference lists and related reviews. The risk of bias was assessed using the PROBAST prediction model risk of bias assessment tool. Meta-analyses were performed using the random effects model, focusing on the C-statistic as a discrimination index and the observed-to-expected ratio (OE) as a calibration index. Out of 1315 initial records, 16 studies were included, with 21 external validations of six models in Asia. The validated models consisted of Framingham models, pooled cohort equations (PCEs), SCORE, Globorisk, and WHO models, combined with the results of the first four models. The pooled C-statistic for men ranged from 0.72 (95% CI 0.70 to 0.75; PCEs) to 0.76 (95% CI 0.74 to 0.78; Framingham general CVD). In women, it varied from 0.74 (95% CI 0.22 to 0.97; SCORE) to 0.79 (95% CI 0.74 to 0.83; Framingham general CVD). The pooled OE ratio for men ranged from 0.21 (95% CI 0.018 to 2.49; Framingham CHD) to 1.11 (95%CI 0.65 to 1.89; PCEs). In women, it varied from 0.28 (95%CI 0.33 to 2.33; Framingham CHD) to 1.81 (95% CI 0.90 to 3.64; PCEs). The Framingham, PCEs, and SCORE models exhibited acceptable discrimination but poor calibration in predicting the 10-year risk of CVDs in Asia. Recalibration and updates are necessary before implementing these models in the region.

Prognostic models for heart failure in patients with type 2 diabetes: a systematic review and meta-analysis

OBJECTIVE

To provide a systematic review, critical appraisal, assessment of performance and generalisability of all the reported prognostic models for heart failure (HF) in patients with type 2 diabetes (T2D).

METHODS

We performed a literature search in Medline, Embase, Central Register of Controlled Trials, Cochrane Database of Systematic Reviews and Scopus (from inception to July 2022) and grey literature to identify any study developing and/or validating models predicting HF applicable to patients with T2D. We extracted data on study characteristics, modelling methods and measures of performance, and we performed a random-effects meta-analysis to pool discrimination in models with multiple validation studies. We also performed a descriptive synthesis of calibration and we assessed the risk of bias and certainty of evidence (high, moderate, low).

RESULTS

Fifty-five studies reporting on 58 models were identified: (1) models developed in patients with T2D for HF prediction (n=43), (2) models predicting HF developed in non-diabetic cohorts and externally validated in patients with T2D (n=3), and (3) models originally predicting a different outcome and externally validated for HF (n=12). RECODe (C-statistic=0.75 95% CI (0.72, 0.78), 95% prediction interval (PI) (0.68, 0.81); high certainty), TRS-HFDM (C-statistic=0.75 95% CI (0.69, 0.81), 95% PI (0.58, 0.87); low certainty) and WATCH-DM (C-statistic=0.70 95% CI (0.67, 0.73), 95% PI (0.63, 0.76); moderate certainty) showed the best performance. QDiabetes-HF demonstrated also good discrimination but was externally validated only once and not meta-analysed.

CONCLUSIONS

Among the prognostic models identified, four models showed promising performance and, thus, could be implemented in current clinical practice.

Validation and comparison of cardiovascular risk prediction equations in Chinese patients with Type 2 diabetes

AIMS

For patients with diabetes, the European guidelines updated the cardiovascular disease (CVD) risk prediction recommendations using diabetes-specific models with age-specific cut-offs, whereas American guidelines still advise models derived from the general population. We aimed to compare the performance of four cardiovascular risk models in diabetes populations.

METHODS AND RESULTS

Patients with diabetes from the CHERRY study, an electronic health records-based cohort study in China, were identified. Five-year CVD risk was calculated using original and recalibrated diabetes-specific models [Action in Diabetes and Vascular disease: PreterAx and diamicroN-MR Controlled Evaluation (ADVANCE) and the Hong Kong cardiovascular risk model (HK)] and general population-based models [Pooled Cohort Equations (PCE) and Prediction for Atherosclerotic cardiovascular disease Risk in China (China-PAR)]. During a median 5.8-year follow-up, 46 558 patients had 2605 CVD events. C-statistics were 0.711 [95% confidence interval: 0.693-0.729] for ADVANCE and 0.701 (0.683-0.719) for HK in men, and 0.742 (0.725-0.759) and 0.732 (0.718-0.747) in women. C-statistics were worse in two general population-based models. Recalibrated ADVANCE underestimated risk by 1.2% and 16.8% in men and women, whereas PCE underestimated risk by 41.9% and 24.2% in men and women. With the age-specific cut-offs, the overlap of the high-risk patients selected by every model pair ranged from only 22.6% to 51.2%. When utilizing the fixed cut-off at 5%, the recalibrated ADVANCE selected similar high-risk patients in men (7400) as compared to the age-specific cut-offs (7102), whereas age-specific cut-offs exhibited a reduction in the selection of high-risk patients in women (2646 under age-specific cut-offs vs. 3647 under fixed cut-off).

CONCLUSION

Diabetes-specific CVD risk prediction models showed better discrimination for patients with diabetes. High-risk patients selected by different models varied significantly. Age-specific cut-offs selected fewer patients at high CVD risk especially in women.

Fixing the Leaky Pipe: How to Improve the Uptake of Patient-Reported Outcomes-Based Prognostic and Predictive Models in Cancer Clinical Practice

PURPOSE

This discussion paper outlines challenges and proposes solutions for successfully implementing prediction models that incorporate patient-reported outcomes (PROs) in cancer practice.

METHODS

We organized a full-day multidisciplinary meeting of people with expertise in cancer care delivery, PRO collection, PRO use in prediction modeling, computing, implementation, and decision science. The discussions presented here focused on identifying challenges to the development, implementation and use of prediction models incorporating PROs, and suggesting possible solutions.

RESULTS

Specific challenges and solutions were identified across three broad areas. (1) Understanding decision making and implementation: necessitating multidisciplinary collaboration in the early stages and throughout; early stakeholder engagement to define the decision problem and ensure acceptability of PROs in prediction; understanding patient/clinician interpretation of PRO predictions and uncertainty to optimize prediction impact; striving for model integration into existing electronic health records; and early regulatory alignment. (2) Recognizing the limitations to PRO collection and their impact on prediction: incorporating validated, clinically important PROs to maximize model generalizability and clinical engagement; and minimizing missing PRO data (resulting from both structural digital exclusion and time-varying factors) to avoid exacerbating existing inequalities. (3) Statistical and modeling challenges: incorporating statistical methods to address missing data; ensuring predictive modeling recognizes complex causal relationships; and considering temporal and geographic recalibration so that model predictions reflect the relevant population.

CONCLUSION

Developing and implementing PRO-based prediction models in cancer care requires extensive multidisciplinary working from the earliest stages, recognition of implementation challenges because of PRO collection and model presentation, and robust statistical methods to manage missing data, causality, and calibration. Prediction models incorporating PROs should be viewed as complex interventions, with their development and impact assessment carried out to reflect this.

Personalised decision making to predict absolute metastatic risk in cutaneous squamous cell carcinoma: development and validation of a clinico-pathological model

Background

Cutaneous squamous cell carcinoma (cSCC) is a common skin cancer, affecting more than 2 million people worldwide yearly and metastasising in 2-5% of patients. However, current clinical staging systems do not provide estimates of absolute metastatic risk, hence missing the opportunity for more personalised treatment advice. We aimed to develop a clinico-pathological model that predicts the probability of metastasis in patients with cSCC.

Methods

Nationwide cohorts from (1) all patients with a first primary cSCC in The Netherlands in 2007-2008 and (2) all patients with a cSCC in 2013-2015 in England were used to derive nested case-control cohorts. Pathology records of primary cSCCs that originated a loco-regional or distant metastasis were identified, and these cSCCs were matched to primary cSCCs of controls without metastasis (1:1 ratio). The model was developed on the Dutch cohort (n = 390) using a weighted Cox regression model with backward selection and validated on the English cohort (n = 696). Model performance was assessed using weighted versions of the C-index, calibration metrics, and decision curve analysis; and compared to the Brigham and Women's Hospital (BWH) and the American Joint Committee on Cancer (AJCC) staging systems. Members of the multidisciplinary Skin Cancer Outcomes (SCOUT) consortium were surveyed to interpret metastatic risk cutoffs in a clinical context.

Findings

Eight out of eleven clinico-pathological variables were selected. The model showed good discriminative ability, with an optimism-corrected C-index of 0.80 (95% Confidence interval (CI) 0.75-0.85) in the development cohort and a C-index of 0.84 (95% CI 0.81-0.87) in the validation cohort. Model predictions were well-calibrated: the calibration slope was 0.96 (95% CI 0.76-1.16) in the validation cohort. Decision curve analysis showed improved net benefit compared to current staging systems, particularly for thresholds relevant for decisions on follow-up and adjuvant treatment. The model is available as an online web-based calculator (https://emc-dermatology.shinyapps.io/cscc-abs-met-risk/).

Interpretation

This validated model assigns personalised metastatic risk predictions to patients with cSCC, using routinely reported histological and patient-specific risk factors. The model can empower clinicians and healthcare systems in identifying patients with high-risk cSCC and offering personalised care/treatment and follow-up. Use of the model for clinical decision-making in different patient populations must be further investigated.

Funding

PPP Allowance made available by Health-Holland, Top Sector Life Sciences & Health, to stimulate public-private partnerships.

Call to Improve the Quality of Prediction Tools for Intrahepatic Cholangiocarcinoma Resection: A Critical Appraisal, Systematic Review, and External Validation Study

Objective

To conduct a systematic review, critical appraisal, and external validation of survival prediction tools for patients undergoing intrahepatic cholangiocarcinoma (iCCA) resection.

Summary background data

Despite the development of several survival prediction tools in recent years for patients undergoing iCCA resections, there is a lack of critical appraisal and external validation of these models.

Methods

We conducted a systematic review and critical appraisal of survival and recurrence prediction models for patients undergoing curative-intent iCCA resections. Studies were evaluated based on their model design, risk of bias, reporting, performance, and validation results. We identified the best model and externally validated it using our institution's data.

Results

This review included a total of 31 studies, consisting of 26 studies with original prediction tools and 5 studies that only conducted external validations. Among the 26, 54% of the studies conducted internal validations, 46% conducted external validations, and only 1 study scored a low risk of bias. Harrell's C-statistics ranged from 0.67 to 0.76 for internal validation and from 0.64 to 0.75 for external validation. Only 81% of the studies reported model calibration. Our external validation of the best model (Intrahepatic Cholangiocarcinoma [ICC]-Metroticket) estimated Harrell's and Uno's C-statistics of 0.67 (95% CI: 0.56-0.77) and Uno's time-dependent area under the receiver operating characteristic curve (AUC) of 0.71 (95% CI: 0.53-0.88), with a Brier score of 0.20 (95% CI: 0.15-0.26) and good calibration plots.

Conclusions

Many prediction models have been published in recent years, but their quality remains poor, and minimal methodological quality improvement has been observed. The ICC-Metroticket was selected as the best model (Uno's time-dependent AUC of 0.71) for 5-year overall survival prediction in patients undergoing curative-intent iCCA resection.

Are the MIA and MSKCC nomograms useful in selecting patients with melanoma for sentinel lymph node biopsy?

BACKGROUND AND METHODS

The Melanoma Institute of Australia (MIA) and Memorial Sloan Kettering Cancer Center (MSKCC) nomograms were developed to help guide sentinel lymph node biopsy (SLNB) decisions. Although statistically validated, whether these prediction models provide clinical benefit at National Comprehensive Cancer Network guideline-endorsed thresholds is unknown. We conducted a net benefit analysis to quantify the clinical utility of these nomograms at risk thresholds of 5%-10% compared to the alternative strategy of biopsying all patients. External validation data for MIA and MSKCC nomograms were extracted from respective published studies.

RESULTS

The MIA nomogram provided added net benefit at a risk threshold of 9% but net harm at 5%-8% and 10%. The MSKCC nomogram provided added net benefit at risk thresholds of 5% and 9%-10% but net harm at 6%-8%. When present, the magnitude of net benefit was small (1-3 net avoidable biopsies per 100 patients).

CONCLUSION

Neither model consistently provided added net benefit compared to performing SLNB for all patients.

DISCUSSION

Based on published data, use of the MIA or MSKCC nomograms as decision-making tools for SLNB at risk thresholds of 5%-10% does not clearly provide clinical benefit to patients.