Computational models can predict response to HIV therapy without a genotype and may reduce treatment failure in different resource-limited settings

Machine learning prediction models for clinical management of blood-borne viral infections: a systematic review of current applications and future impact

BACKGROUND

Machine learning (ML) prediction models to support clinical management of blood-borne viral infections are becoming increasingly abundant in medical literature, with a number of competing models being developed for the same outcome or target population. However, evidence on the quality of these ML prediction models are limited.

OBJECTIVE

This study aimed to evaluate the development and quality of reporting of ML prediction models that could facilitate timely clinical management of blood-borne viral infections.

METHODS

We conducted narrative evidence synthesis following the synthesis without meta-analysis guidelines. We searched PubMed and Cochrane Central Register of Controlled Trials for all studies applying ML models for predicting clinical outcomes associated with hepatitis B virus (HBV), human immunodeficiency virus (HIV), or hepatitis C virus (HCV).

RESULTS

We found 33 unique ML prediction models aiming to support clinical decision making. Overall, 12 (36.4%) focused on HBV, 10 (30.3%) on HCV, 10 on HIV (30.3%) and two (6.1%) on co-infection. Among these, six (18.2%) addressed the diagnosis of infection, 16 (48.5%) the prognosis of infection, eight (24.2%) the prediction of treatment response, two (6.1%) progression through a cascade of care, and one (3.03%) focused on the choice of antiretroviral therapy (ART). Nineteen prediction models (57.6%) were developed using data from high-income countries. Evaluation of prediction models was limited to measures of performance. Detailed information on software code accessibility was often missing. Independent validation on new datasets and/or in other institutions was rarely done.

CONCLUSION

Promising approaches for ML prediction models in blood-borne viral infections were identified, but the lack of robust validation, interpretability/explainability, and poor quality of reporting hampered their clinical relevance. Our findings highlight important considerations that can inform standard reporting guidelines for ML prediction models in the future (e.g., TRIPOD-AI), and provides critical data to inform robust evaluation of the models.

2021 update to HIV-TRePS: a highly flexible and accurate system for the prediction of treatment response from incomplete baseline information in different healthcare settings

OBJECTIVES

With the goal of facilitating the use of HIV-TRePS to optimize therapy in settings with limited healthcare resources, we aimed to develop computational models to predict treatment responses accurately in the absence of commonly used baseline data.

METHODS

Twelve sets of random forest models were trained using very large, global datasets to predict either the probability of virological response (classifier models) or the absolute change in viral load in response to a new regimen (absolute models) following virological failure. Two 'standard' models were developed with all baseline variables present and 10 others developed without HIV genotype, time on therapy, CD4 count or any combination of the above.

RESULTS

The standard classifier models achieved an AUC of 0.89 in cross-validation and independent testing. Models with missing variables achieved AUC values of 0.78-0.90. The standard absolute models made predictions that correlated significantly with observed changes in viral load with a mean absolute error of 0.65 log10 copies HIV RNA/mL in cross-validation and 0.69 log10 copies HIV RNA/mL in independent testing. Models with missing variables achieved values of 0.65-0.75 log10 copies HIV RNA/mL. All models identified alternative regimens that were predicted to be effective for the vast majority of cases where the new regimen prescribed in the clinic failed. All models were significantly better predictors of treatment response than genotyping with rules-based interpretation.

CONCLUSIONS

These latest models that predict treatment responses accurately, even when a number of baseline variables are not available, are a major advance with greatly enhanced potential benefit, particularly in resource-limited settings. The only obstacle to realizing this potential is the willingness of healthcare professions to use the system.

Genotypic Methods for HIV Drug Resistance Monitoring: The Opportunities and Challenges Faced by China

AIDS is a globalized infectious disease. In 2014, UNAIDS launched a global project of "90-90-90" to end the HIV epidemic by 2030. The second and third 90 require 90% of HIV-1 infected individuals receiving antiretroviral therapy (ART) and durable virological suppression. However, wide use of ART will greatly increase the emergence and spreading of HIV drug resistance and current HIV drug resistance test (DRT) assays in China are seriously lagging behind, hindering to achieve virological suppression. Therefore, recommending an appropriate HIV DRT method is critical for HIV routine surveillance and prevention in China. In this review, we summarized the current existing HIV drug resistance genotypic testing methods around the world and discussed the advantages and disadvantages of these methods.

Predicting Virological Response to HIV Treatment Over Time: A Tool for Settings With Different Definitions of Virological Response

OBJECTIVE

Definitions of virological response vary from <50 up to 1000 copies of HIV-RNA/mL. Our previous models estimate the probability of HIV drug combinations reducing the viral load to <50 copies/mL, with no indication of whether higher thresholds of response may be achieved. Here, we describe the development of models that predict absolute viral load over time.

METHODS

Two sets of random forest models were developed using 50,270 treatment change episodes from more than 20 countries. The models estimated viral load at different time points following the introduction of a new regimen from variables including baseline viral load, CD4 count, and treatment history. One set also used genotypes in their predictions. Independent data sets were used for evaluation.

RESULTS

Both models achieved highly significant correlations between predicted and actual viral load changes (r = 0.67-0.68, mean absolute error of 0.73-0.74 log10 copies/mL). The models produced curves of virological response over time. Using failure definitions of <100, 400, or 1000 copies/mL, but not 50 copies/mL, both models were able to identify alternative regimens they predicted to be effective for the majority of cases where the new regimen prescribed in the clinic failed.

CONCLUSIONS

These models could be useful for selecting the optimum combination therapy for patients requiring a change in therapy in settings using any definition of virological response. They also give an idea of the likely response curve over time. Given that genotypes are not required, these models could be a useful addition to the HIV-TRePS system for those in resource-limited settings.

A comparative study of logistic regression based machine learning techniques for prediction of early virological suppression in antiretroviral initiating HIV patients

Background

Treatment with effective antiretroviral therapy (ART) lowers morbidity and mortality among HIV positive individuals. Effective highly active antiretroviral therapy (HAART) should lead to undetectable viral load within 6 months of initiation of therapy. Failure to achieve and maintain viral suppression may lead to development of resistance and increase the risk of viral transmission. In this paper three logistic regression based machine learning approaches are developed to predict early virological outcomes using easily measurable baseline demographic and clinical variables (age, body weight, sex, TB disease status, ART regimen, viral load, CD4 count). The predictive performance and generalizability of the approaches are compared.

Methods

The multitask temporal logistic regression (MTLR), patient specific survival prediction (PSSP) and simple logistic regression (SLR) models were developed and validated using the IDI research cohort data and predictive performance tested on an external dataset from the EFV cohort. The model calibration and discrimination plots, discriminatory measures (AUROC, F1) and overall predictive performance (brier score) were assessed.

Results

The MTLR model outperformed the PSSP and SLR models in terms of goodness of fit (RMSE = 0.053, 0.1, and 0.14 respectively), discrimination (AUROC = 0.92, 0.75 and 0.53 respectively) and general predictive performance (Brier score= 0.08, 0.19, 0.11 respectively). The predictive importance of variables varied with time after initiation of ART. The final MTLR model accurately (accuracy = 92.9%) predicted outcomes in the external (EFV cohort) dataset with satisfactory discrimination (0.878) and a low (6.9%) false positive rate.

Conclusion

Multitask Logistic regression based models are capable of accurately predicting early virological suppression using readily available baseline demographic and clinical variables and could be used to derive a risk score for use in resource limited settings.

2018 update to the HIV-TRePS system: the development of new computational models to predict HIV treatment outcomes, with or without a genotype, with enhanced usability for low-income settings

Objectives

Optimizing antiretroviral drug combination on an individual basis can be challenging, particularly in settings with limited access to drugs and genotypic resistance testing. Here we describe our latest computational models to predict treatment responses, with or without a genotype, and compare their predictive accuracy with that of genotyping.

Methods

Random forest models were trained to predict the probability of virological response to a new therapy introduced following virological failure using up to 50 000 treatment change episodes (TCEs) without a genotype and 18 000 TCEs including genotypes. Independent data sets were used to evaluate the models. This study tested the effects on model accuracy of relaxing the baseline data timing windows, the use of a new filter to exclude probable non-adherent cases and the addition of maraviroc, tipranavir and elvitegravir to the system.

Results

The no-genotype models achieved area under the receiver operator characteristic curve (AUC) values of 0.82 and 0.81 using the standard and relaxed baseline data windows, respectively. The genotype models achieved AUC values of 0.86 with the new non-adherence filter and 0.84 without. Both sets of models were significantly more accurate than genotyping with rules-based interpretation, which achieved AUC values of only 0.55-0.63, and were marginally more accurate than previous models. The models were able to identify alternative regimens that were predicted to be effective for the vast majority of cases in which the new regimen prescribed in the clinic failed.

Conclusions

These latest global models predict treatment responses accurately even without a genotype and have the potential to help optimize therapy, particularly in resource-limited settings.

Predicting CD4 count changes among patients on antiretroviral treatment: Application of data mining techniques

BACKGROUND AND OBJECTIVES

To monitor the progress of therapy and disease progression, periodic CD4 counts are required throughout the course of HIV/AIDS care and support. The demand for CD4 count measurement is increasing as ART programs expand over the last decade. This study aimed to predict CD4 count changes and to identify the predictors of CD4 count changes among patients on ART.

METHODS

A cross-sectional study was conducted at the University of Gondar Hospital from 3,104 adult patients on ART with CD4 counts measured at least twice (baseline and most recent). Data were retrieved from the HIV care clinic electronic database and patients` charts. Descriptive data were analyzed by SPSS version 20. Cross-Industry Standard Process for Data Mining (CRISP-DM) methodology was followed to undertake the study. WEKA version 3.8 was used to conduct a predictive data mining. Before building the predictive data mining models, information gain values and correlation-based Feature Selection methods were used for attribute selection. Variables were ranked according to their relevance based on their information gain values. J48, Neural Network, and Random Forest algorithms were experimented to assess model accuracies.

RESULT

The median duration of ART was 191.5 weeks. The mean CD4 count change was 243 (SD 191.14) cells per microliter. Overall, 2427 (78.2%) patients had their CD4 counts increased by at least 100 cells per microliter, while 4% had a decline from the baseline CD4 value. Baseline variables including age, educational status, CD8 count, ART regimen, and hemoglobin levels predicted CD4 count changes with predictive accuracies of J48, Neural Network, and Random Forest being 87.1%, 83.5%, and 99.8%, respectively. Random Forest algorithm had a superior performance accuracy level than both J48 and Artificial Neural Network. The precision, sensitivity and recall values of Random Forest were also more than 99%.

CONCLUSIONS

Nearly accurate prediction results were obtained using Random Forest algorithm. This algorithm could be used in a low-resource setting to build a web-based prediction model for CD4 count changes.

An update to the HIV-TRePS system: the development and evaluation of new global and local computational models to predict HIV treatment outcomes, with or without a genotype

OBJECTIVES

Optimizing antiretroviral drug combination on an individual basis in resource-limited settings is challenging because of the limited availability of drugs and genotypic resistance testing. Here, we describe our latest computational models to predict treatment responses, with or without a genotype, and compare the potential utility of global and local models as a treatment tool for South Africa.

METHODS

Global random forest models were trained to predict the probability of virological response to therapy following virological failure using 29 574 treatment change episodes (TCEs) without a genotype, 3179 of which were from South Africa and were used to develop local models. In addition, 15 130 TCEs including genotypes were used to develop another set of models. The 'no-genotype' models were tested with an independent global test set (n = 1700) plus a subset from South Africa (n = 222). The genotype models were tested with 750 independent cases.

RESULTS

The global no-genotype models achieved area under the receiver-operating characteristic curve (AUC) values of 0.82 and 0.79 with the global and South African tests sets, respectively, and the South African models achieved AUCs of 0.70 and 0.79. The genotype models achieved an AUC of 0.84. The global no-genotype models identified more alternative, locally available regimens that were predicted to be effective for cases that failed their new regimen in the South African clinics than the local models. Both sets of models were significantly more accurate predictors of outcomes than genotyping with rules-based interpretation.

CONCLUSIONS

These latest global models predict treatment responses accurately even without a genotype, out-performed the local South African models and have the potential to help optimize therapy, particularly in resource-limited settings.

Computational models as predictors of HIV treatment outcomes for the Phidisa cohort in South Africa

BACKGROUND

Selecting the optimal combination of HIV drugs for an individual in resource-limited settings is challenging because of the limited availability of drugs and genotyping.

OBJECTIVE

The evaluation as a potential treatment support tool of computational models that predict response to therapy without a genotype, using cases from the Phidisa cohort in South Africa.

METHODS

Cases from Phidisa of treatment change following failure were identified that had the following data available: baseline CD4 count and viral load, details of failing and previous antiretroviral drugs, drugs in new regimen and time to follow-up. The HIV Resistance Response Database Initiative's (RDI's) models used these data to predict the probability of a viral load < 50 copies/mL at follow-up. The models were also used to identify effective alternative combinations of three locally available drugs.

RESULTS

The models achieved accuracy (area under the receiver-operator characteristic curve) of 0.72 when predicting response to therapy, which is less accurate than for an independent global test set (0.80) but at least comparable to that of genotyping with rules-based interpretation. The models were able to identify alternative locally available three-drug regimens that were predicted to be effective in 69% of all cases and 62% of those whose new treatment failed in the clinic.

CONCLUSION

The predictive accuracy of the models for these South African patients together with the results of previous studies suggest that the RDI's models have the potential to optimise treatment selection and reduce virological failure in different patient populations, without the use of a genotype.

Estimating the dynamics and dependencies of accumulating mutations with applications to HIV drug resistance

We introduce a new model called the observed time conjunctive Bayesian network (OT-CBN) that describes the accumulation of genetic events (mutations) under partial temporal ordering constraints. Unlike other CBN models, the OT-CBN model uses sampling time points of genotypes in addition to genotypes themselves to estimate model parameters. We developed an expectation-maximization algorithm to obtain approximate maximum likelihood estimates by accounting for this additional information. In a simulation study, we show that the OT-CBN model outperforms the continuous time CBN (CT-CBN) (Beerenwinkel and Sullivant, 2009. Markov models for accumulating mutations. Biometrika 96: (3), 645-661), which does not take into account individual sampling times for parameter estimation. We also show superiority of the OT-CBN model on several datasets of HIV drug resistance mutations extracted from the Swiss HIV Cohort Study database.

A comparison of computational models with and without genotyping for prediction of response to second-line HIV therapy

OBJECTIVES

We compared the use of computational models developed with and without HIV genotype vs. genotyping itself to predict effective regimens for patients experiencing first-line virological failure.

METHODS

Two sets of models predicted virological response for 99 three-drug regimens for patients on a failing regimen of two nucleoside/nucleotide reverse transcriptase inhibitors and one nonnucleoside reverse transcriptase inhibitor in the Second-Line study. One set used viral load, CD4 count, genotype, plus treatment history and time to follow-up to make its predictions; the second set did not include genotype. Genotypic sensitivity scores were derived and the ranking of the alternative regimens compared with those of the models. The accuracy of the models and that of genotyping as predictors of the virological responses to second-line regimens were compared.

RESULTS

The rankings of alternative regimens by the two sets of models were significantly correlated in 60-69% of cases, and the rankings by the models that use a genotype and genotyping itself were significantly correlated in 60% of cases. The two sets of models identified alternative regimens that were predicted to be effective in 97% and 100% of cases, respectively. The area under the receiver-operating curve was 0.72 and 0.74 for the two sets of models, respectively, and significantly lower at 0.55 for genotyping.

CONCLUSIONS

The two sets of models performed comparably well and significantly outperformed genotyping as predictors of response. The models identified alternative regimens predicted to be effective in almost all cases. It is encouraging that models that do not require a genotype were able to predict responses to common second-line therapies in settings where genotyping is unavailable.

Personalized HIV therapy to control drug resistance

The therapy of HIV patients is characterized by both the high genomic diversity of the virus population harbored by the patient and a substantial volume of therapy options. The virus population is unique for each patient and time point. The large number of therapy options makes it difficult to select an optimal or near optimal therapy, especially with therapy-experienced patients. In the past decade, computer-based support for therapy selection, which assesses the level of viral resistance against drugs has become a mainstay for HIV patients. We discuss the properties of available systems and the perspectives of the field.

An update to the HIV-TRePS system: the development of new computational models that do not require a genotype to predict HIV treatment outcomes

OBJECTIVES

The optimal individualized selection of antiretroviral drugs in resource-limited settings is challenging because of the limited availability of drugs and genotyping. Here we describe the development of the latest computational models to predict the response to combination antiretroviral therapy without a genotype, for potential use in such settings.

METHODS

Random forest models were trained to predict the probability of a virological response to therapy (<50 copies HIV RNA/mL) following virological failure using the following data from 22,567 treatment-change episodes including 1090 from southern Africa: baseline viral load and CD4 cell count, treatment history, drugs in the new regimen, time to follow-up and follow-up viral load. The models were assessed during cross-validation and with an independent global test set of 1000 cases including 100 from southern Africa. The models' accuracy [area under the receiver-operating characteristic curve (AUC)] was evaluated and compared with genotyping using rules-based interpretation systems for those cases with genotypes available.

RESULTS

The models achieved AUCs of 0.79-0.84 (mean 0.82) during cross-validation, 0.80 with the global test set and 0.78 with the southern African subset. The AUCs were significantly lower (0.56-0.57) for genotyping.

CONCLUSIONS

The models predicted virological response to HIV therapy without a genotype as accurately as previous models that included a genotype. They were accurate for cases from southern Africa and significantly more accurate than genotyping. These models will be accessible via the online treatment support tool HIV-TRePS and have the potential to help optimize antiretroviral therapy in resource-limited settings where genotyping is not generally available.

Potential impact of a free online HIV treatment response prediction system for reducing virological failures and drug costs after antiretroviral therapy failure in a resource-limited setting

OBJECTIVE

Antiretroviral drug selection in resource-limited settings is often dictated by strict protocols as part of a public health strategy. The objective of this retrospective study was to examine if the HIV-TRePS online treatment prediction tool could help reduce treatment failure and drug costs in such settings.

METHODS

The HIV-TRePS computational models were used to predict the probability of response to therapy for 206 cases of treatment change following failure in India. The models were used to identify alternative locally available 3-drug regimens, which were predicted to be effective. The costs of these regimens were compared to those actually used in the clinic.

RESULTS

The models predicted the responses to treatment of the cases with an accuracy of 0.64. The models identified alternative drug regimens that were predicted to result in improved virological response and lower costs than those used in the clinic in 85% of the cases. The average annual cost saving was $364 USD per year (41%).

CONCLUSIONS

Computational models that do not require a genotype can predict and potentially avoid treatment failure and may reduce therapy costs. The use of such a system to guide therapeutic decision-making could confer health economic benefits in resource-limited settings.