Development and external-validation of a nomogram for predicting the survival of hospitalised HIV/AIDS patients based on a large study cohort in western China

The predictive accuracy of machine learning for the risk of death in HIV patients: a systematic review and meta-analysis

BACKGROUND

Early prediction of mortality in individuals with HIV (PWH) has perpetually posed a formidable challenge. With the widespread integration of machine learning into clinical practice, some researchers endeavor to formulate models predicting the mortality risk for PWH. Nevertheless, the diverse timeframes of mortality among PWH and the potential multitude of modeling variables have cast doubt on the efficacy of the current predictive model for HIV-related deaths. To address this, we undertook a systematic review and meta-analysis, aiming to comprehensively assess the utilization of machine learning in the early prediction of HIV-related deaths and furnish evidence-based support for the advancement of artificial intelligence in this domain.

METHODS

We systematically combed through the PubMed, Cochrane, Embase, and Web of Science databases on November 25, 2023. To evaluate the bias risk in the original studies included, we employed the Predictive Model Bias Risk Assessment Tool (PROBAST). During the meta-analysis, we conducted subgroup analysis based on survival and non-survival models. Additionally, we utilized meta-regression to explore the influence of death time on the predictive value of the model for HIV-related deaths.

RESULTS

After our comprehensive review, we analyzed a total of 24 pieces of literature, encompassing data from 401,389 individuals diagnosed with HIV. Within this dataset, 23 articles specifically delved into deaths during long-term follow-ups outside hospital settings. The machine learning models applied for predicting these deaths comprised survival models (COX regression) and other non-survival models. The outcomes of the meta-analysis unveiled that within the training set, the c-index for predicting deaths among people with HIV (PWH) using predictive models stands at 0.83 (95% CI: 0.75-0.91). In the validation set, the c-index is slightly lower at 0.81 (95% CI: 0.78-0.85). Notably, the meta-regression analysis demonstrated that neither follow-up time nor the occurrence of death events significantly impacted the performance of the machine learning models.

CONCLUSIONS

The study suggests that machine learning is a viable approach for developing non-time-based predictions regarding HIV deaths. Nevertheless, the limited inclusion of original studies necessitates additional multicenter studies for thorough validation.

Development and Evaluation of a Nomogram for Predicting the Outcome of Immune Reconstitution Among HIV/AIDS Patients Receiving Antiretroviral Therapy in China

This study aims to develop and evaluate a model to predict the immune reconstitution among HIV/AIDS patients after antiretroviral therapy (ART). A total of 502 HIV/AIDS patients are randomized to the training cohort and evaluation cohort. Least absolute shrinkage and selection operator (LASSO) regression and multivariate logistic regression analysis are performed to identify the indicators and establish the nomogram for predicting the immune reconstitution. Decision curve analysis (DCA) and clinical impact curve (CIC) are used to evaluate the clinical effectiveness of the nomogram. Predictive factors included white blood cells (WBC), baseline CD4+ T-cell counts (baseline CD4), ratio of effector regulatory T cells to resting regulatory T cells (eTreg/rTreg) and low-density lipoprotein cholesterol (LDL-C) and are incorporated into the nomogram. The area under the curve (AUC) is 0.812 (95% CI, 0.767∼0.851) and 0.794 (95%CI, 0.719∼0.857) in the training cohort and evaluation cohort, respectively. The calibration curve shows a high consistency between the predicted and actual observations. Moreover, DCA and CIC indicate that the nomogram has a superior net benefit in predicting poor immune reconstitution. A simple-to-use nomogram containing four routinely collected variables is developed and internally evaluated and can be used to predict the poor immune reconstitution in HIV/AIDS patients after ART.

Machine learning-based in-hospital mortality prediction of HIV/AIDS patients with Talaromyces marneffei infection in Guangxi, China

Objective

Talaromycosis is a serious regional disease endemic in Southeast Asia. In China, Talaromyces marneffei (T. marneffei) infections is mainly concentrated in the southern region, especially in Guangxi, and cause considerable in-hospital mortality in HIV-infected individuals. Currently, the factors that influence in-hospital death of HIV/AIDS patients with T. marneffei infection are not completely clear. Existing machine learning techniques can be used to develop a predictive model to identify relevant prognostic factors to predict death and appears to be essential to reducing in-hospital mortality.

Methods

We prospectively enrolled HIV/AIDS patients with talaromycosis in the Fourth People’s Hospital of Nanning, Guangxi, from January 2012 to June 2019. Clinical features were selected and used to train four different machine learning models (logistic regression, XGBoost, KNN, and SVM) to predict the treatment outcome of hospitalized patients, and 30% internal validation was used to evaluate the performance of models. Machine learning model performance was assessed according to a range of learning metrics, including area under the receiver operating characteristic curve (AUC). The SHapley Additive exPlanations (SHAP) tool was used to explain the model.

Results

A total of 1927 HIV/AIDS patients with T. marneffei infection were included. The average in-hospital mortality rate was 13.3% (256/1927) from 2012 to 2019. The most common complications/coinfections were pneumonia (68.9%), followed by oral candida (47.5%), and tuberculosis (40.6%). Deceased patients showed higher CD4/CD8 ratios, aspartate aminotransferase (AST) levels, creatinine levels, urea levels, uric acid (UA) levels, lactate dehydrogenase (LDH) levels, total bilirubin levels, creatine kinase levels, white blood-cell counts (WBC) counts, neutrophil counts, procaicltonin levels and C-reactive protein (CRP) levels and lower CD3⁺ T-cell count, CD8⁺ T-cell count, and lymphocyte counts, platelet (PLT), high-density lipoprotein cholesterol (HDL), hemoglobin (Hb) levels than those of surviving patients. The predictive XGBoost model exhibited 0.71 sensitivity, 0.99 specificity, and 0.97 AUC in the training dataset, and our outcome prediction model provided robust discrimination in the testing dataset, showing an AUC of 0.90 with 0.69 sensitivity and 0.96 specificity. The other three models were ruled out due to poor performance. Septic shock and respiratory failure were the most important predictive features, followed by uric acid, urea, platelets, and the AST/ALT ratios.

Conclusion

The XGBoost machine learning model is a good predictor in the hospitalization outcome of HIV/AIDS patients with T. marneffei infection. The model may have potential application in mortality prediction and high-risk factor identification in the talaromycosis population.

Talaromyces marneffei can cause a fatal deeply disseminated fungal infection- talaromycosis. It is widely distributed in Southeast Asia and spreading globally, the disease is insidious and responsible for significant deaths. Clinicians need easy-to-use tools to make decisions on which patients are at a higher risk of dying after infecting T. marneffei. In this study, conducted in Southern China, we have evolved XGBoost machine learning model. 15 clinical indicators and laboratory measures were used to estimate a patient’s risk of dying in the hospital due to the T. marneffei infection. The study showed that the machine learning model has good predictive ability when tested in an internal testing population of patients. We expect that the model could help clinicians assess a patient’s risk of death in just the time of admission to help decide on early treatment timing of high-risk patients who are likely to die.

Prediction Model for the Risk of HIV Infection among MSM in China: Validation and Stability

The impact of psychosocial factors on increasing the risk of HIV infection among men who have sex with men (MSM) has attracted increasing attention. We aimed to develop and validate an integrated prediction model, especially incorporating emerging psychosocial variables, for predicting the risk of HIV infection among MSM. We surveyed and collected sociodemographic, psychosocial, and behavioral information from 547 MSM in China. The participants were split into a training set and a testing set in a 3:1 theoretical ratio. The prediction model was constructed by introducing the important variables selected with the least absolute shrinkage and selection operator (LASSO) regression, applying multivariate logistic regression, and visually assessing the risk of HIV infection through the nomogram. Receiver operating characteristic curves (ROC), Kolmogorov–Smirnov test, calibration plots, Hosmer–Lemeshow test and population stability index (PSI) were performed to test validity and stability of the model. Four of the 15 selected variables—unprotected anal intercourse, multiple sexual partners, involuntary subordination and drug use before sex—were included in the prediction model. The results indicated that the comprehensive prediction model we developed had relatively good predictive performance and stability in identifying MSM at high-risk for HIV infection, thus providing targeted interventions for high-risk MSM.

A Prognostic Model to Assess Long-Term Survival of Patients on Antiretroviral Therapy: A 15-Year Retrospective Cohort Study in Southwestern China

BACKGROUND

Because there is no assessment tool for survival of people with human immunodeficiency virus (PWH) who received antiretroviral therapy (ART) in rural southwestern China, we aimed to formulate and validate a simple-to-use model to predict long-term overall survival at the initiation of ART.

METHODS

In total, 36 268 eligible participants registered in the Guangxi autonomous region between December 2003 and December 2018 were enrolled and randomized into development and validation cohorts. Predictive variables were determined based on Cox hazard models and specialists' advice. Discrimination, calibration, and clinical utility were measured, respectively.

RESULTS

The prognostic combined 14 variables: sex, age, marital status, infectious route, opportunistic infection, acquired immunodeficiency syndrome (AIDS)-related symptoms, body mass index, CD4+ T lymphocyte count, white blood cell, platelet, hemoglobin, serum creatinine, aspartate transaminase, and total bilirubin. Age, aspartate transaminase, and serum creatinine were assigned higher risk scores than that of CD4+ T lymphocytopenia count and having opportunistic infections or AIDS-related symptoms. At 3 time points (1, 3, and 5 years), the area under the curve ranged from 0.75 to 0.81 and the Brier scores ranged from 0.03 to 0.07. The decision curve analysis showed an acceptable clinical net benefit.

CONCLUSIONS

The prognostic model incorporating routine baseline data can provide a useful tool for early risk appraisal and treatment management in ART in rural southwestern China. Moreover, our study underscores the role of non-AIDS-defining events in long-term survival in ART.