Assessing the performance of risk prediction models

Identification of risk factors for attempted suicide by self-poisoning and a nomogram to predict self-poisoning suicide

Purpose

Suicide is a global concern, especially among young people. Suicide prediction models have the potential to make it easier to identify patients who are at a high risk of suicide, but they have very little predictive power when there is a positive value for suicide mortality. Therefore, the aim of the study is to uncover potential risk factors associated with suicide by self-poisoning and further to provide a trustworthy nomogram to predict self-poisoning suicide among poisoned patients.

Methods

This study prospectively enrolled 237 patients who were treated for poisoning at the Fifth Medical Center of PLA General Hospital (Beijing) between May 2021 and May 2022. Patient's basic characteristics, daily activities, mental health status, and history of psychological illnesses were gathered to examine their predictive power for self-poisoning suicide. On developing a prediction model, patients were split 8:2 into a training (n = 196) group and a validation (n = 41) group at random via computer. The training group worked on model development, while the validation group worked on model validation. In this study, the Hosmer and Lemeshow test, accuracy, and area under the curve were the primary evaluation criteria. Shapley Additive exPlanations (SHAP) was determined to evaluate feature importance. To make the prediction model easy for researchers to utilize, it was presented in nomogram format. Two risk groups of patients were identified based on the ideal cut-off value.

Results

Of all poisoned patients, 64.6% committed suicide by self-poisoning. With regard to self-poisoning attempted suicide, multivariate analysis demonstrated that female gender, smoking, generalized anxiety disorder-7 (GAD-7), and beck hopelessness scale-20 (BHS-20) were significant risk factors, whereas married status, relatively higher education level, a sedentary time of 1–3 h per day, higher sport frequency per week, higher monthly income were significant protective features. The nomogram contained each of the aforementioned nine features. In the training group, the area under curve (AUC) of the nomogram was up to 0.938 (0.904–0.972), whereas in the validation group, it reached a maximum of 0.974 (0.937–1.000). Corresponding accuracy rates were up to 0.883 and 0.927, respectively, and the P-values for the Hosmer and Lemeshow test were 0.178 and 0.346, respectively. SHAP demonstrated that the top three most important features were BHS-20, GAD-7, and marital status. Based on the best cut-off value of the nomogram (40%), patients in the high-risk group had a nearly six-time larger likelihood of committing suicide by self-poisoning than patients in the low-risk group (88.68 vs. 15.38%, P < 0.001). The dynamic nomogram was made available at the following address: https://xiaobo.shinyapps.io/Nomogramselfpoisoningsuicide/.

Conclusions

This study proposes a prediction model to stratify patients at a high risk of suicide by self-poisoning and to guide individual preventive strategies. Patients in the high-risk group require further mental health counseling to alleviate anxiety and hopelessness, healthy lifestyle like quitting smoking and exercising more, and restriction of access to poison and psychiatric drugs.

A multi-subgroup predictive model based on clinical parameters and laboratory biomarkers to predict in-hospital outcomes of plasma exchange-centered artificial liver treatment in patients with hepatitis B virus-related acute-on-chronic liver failure

Background

Postoperative risk stratification is challenging in patients with hepatitis B virus-related acute-on-chronic liver failure (HBV-ACLF) who undergo artificial liver treatment. This study characterizes patients' clinical parameters and laboratory biomarkers with different in-hospital outcomes. The purpose was to establish a multi-subgroup combined predictive model and analyze its predictive capability.

Methods

We enrolled HBV-ACLF patients who received plasma exchange (PE)-centered artificial liver support system (ALSS) therapy from May 6, 2017, to April 6, 2022. There were 110 patients who died (the death group) and 110 propensity score-matched patients who achieved satisfactory outcomes (the survivor group). We compared baseline, before ALSS, after ALSS, and change ratios of laboratory biomarkers. Outcome prediction models were established by generalized estimating equations (GEE). The discrimination was assessed using receiver operating characteristic analyses. Calibration plots compared the mean predicted probability and the mean observed outcome.

Results

We built a multi-subgroup predictive model (at admission; before ALSS; after ALSS; change ratio) to predict in-hospital outcomes of HBV-ACLF patients who received PE-centered ALSS. There were 110 patients with 363 ALSS sessions who survived and 110 who did not, and 363 ALSS sessions were analyzed. The univariate GEE models revealed that several parameters were independent risk factors. Clinical parameters and laboratory biomarkers were entered into the multivariate GEE model. The discriminative power of the multivariate GEE models was excellent, and calibration showed better agreement between the predicted and observed probabilities than the univariate models.

Conclusions

The multi-subgroup combined predictive model generated accurate prognostic information for patients undergoing HBV-ACLF patients who received PE-centered ALSS.

Mortality Prediction of the CHA2DS2-VASc Score, the HAS-BLED Score, and Their Combination in Anticoagulated Patients with Atrial Fibrillation

BACKGROUND AND OBJECTIVES

Atrial fibrillation (AF) is associated with increased mortality, predictors of which are poorly characterized. We investigated the predictive power of the commonly used CHA2DS2-VASc score (congestive heart failure, hypertension, age ≥ 75 years [doubled], diabetes, stroke/transient ischemic attack/thromboembolism [doubled], vascular disease [prior myocardial infarction, peripheral artery disease, or aortic plaque], age 65-75 years, sex category [female]), the HAS-BLED score (hypertension, abnormal renal/liver function, stroke, bleeding history or predisposition, labile international normalized ratio [INR], elderly [age ≥ 65 years], drugs/alcohol concomitantly), and their combination for mortality in AF patients.

METHODS

The PREvention oF thromboembolic events-European Registry in Atrial Fibrillation (PREFER in AF) was a prospective registry including AF patients across seven European countries. We used logistic regression to analyze the relationship between the CHA2DS2-VASc and HAS-BLED scores and outcomes, including mortality, at one year. We evaluated the performance of logistic regression models by discrimination measures (C-index and DeLong test) and calibration measures (Hosmer and Lemeshow goodness-of-fit and integrated discrimination improvement (IDI), with bootstrap techniques for internal validation.

RESULTS

In 5209 AF patients with complete information on both scores, average one-year mortality was 3.1%. We found strong gradients between stroke/systemic embolic events (SSE), major bleeding and-specifically-mortality for both CHA2DS2-VASc and HAS-BLED scores, with a similar C-statistic for event prediction. The predictive power of the models with both scores combined, removing overlapping components, was significantly enhanced (p < 0.01) compared to models including either CHA2DS2-VASc or HAS-BLED alone: for mortality, C-statistic: 0.740, compared to 0.707 for CHA2DS2-VASc or 0.646 for HAS-BLED alone. IDI analyses supported the significant improvement for the combined score model compared to separate score models for all outcomes.

CONCLUSIONS

Both the CHA2DS2-VASc and the HAS-BLED scores predict mortality similarly in patients with AF, and a combination of their components increases prediction significantly. Such combination may be useful for investigational and-possibly-also clinical purposes.