1
|
Di Teodoro G, Siciliano F, Guarrasi V, Vandamme AM, Ghisetti V, Sönnerborg A, Zazzi M, Silvestri F, Palagi L. A graph neural network-based model with out-of-distribution robustness for enhancing antiretroviral therapy outcome prediction for HIV-1. Comput Med Imaging Graph 2025; 120:102484. [PMID: 39808870 DOI: 10.1016/j.compmedimag.2024.102484] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2024] [Revised: 11/16/2024] [Accepted: 12/23/2024] [Indexed: 01/16/2025]
Abstract
Predicting the outcome of antiretroviral therapies (ART) for HIV-1 is a pressing clinical challenge, especially when the ART includes drugs with limited effectiveness data. This scarcity of data can arise either due to the introduction of a new drug to the market or due to limited use in clinical settings, resulting in clinical dataset with highly unbalanced therapy representation. To tackle this issue, we introduce a novel joint fusion model, which combines features from a Fully Connected (FC) Neural Network and a Graph Neural Network (GNN) in a multi-modality fashion. Our model uses both tabular data about genetic sequences and a knowledge base derived from Stanford drug-resistance mutation tables, which serve as benchmark references for deducing in-vivo treatment efficacy based on the viral genetic sequence. By leveraging this knowledge base structured as a graph, the GNN component enables our model to adapt to imbalanced data distributions and account for Out-of-Distribution (OoD) drugs. We evaluated these models' robustness against OoD drugs in the test set. Our comprehensive analysis demonstrates that the proposed model consistently outperforms the FC model. These results underscore the advantage of integrating Stanford scores in the model, thereby enhancing its generalizability and robustness, but also extending its utility in contributing in more informed clinical decisions with limited data availability. The source code is available at https://github.com/federicosiciliano/graph-ood-hiv.
Collapse
Affiliation(s)
- Giulia Di Teodoro
- Sapienza University of Rome, Department of Computer Control and Management Engineering Antonio Ruberti, 00185, Rome, Italy; EuResist Network, 00152, Rome, Italy.
| | - Federico Siciliano
- Sapienza University of Rome, Department of Computer Control and Management Engineering Antonio Ruberti, 00185, Rome, Italy.
| | - Valerio Guarrasi
- Unit of Computer Systems and Bioinformatics, Department of Engineering, University Campus Bio-Medico of Rome, 00128, Rome, Italy.
| | - Anne-Mieke Vandamme
- KU Leuven, Department of Microbiology, Immunology and Transplantation, Rega Institute for Medical Research, Clinical and Epidemiological Virology, Leuven, Belgium; Center for Global Health and Tropical Medicine, Instituto de Higiene e Medicina Tropical, Universidade Nova de Lisboa, 1349-008, Lisbon, Portugal.
| | - Valeria Ghisetti
- Molecular Biology and Microbiology Unit, Amedeo di Savoia Hospital, ASL Città di Torino, 10128, Turin, Italy.
| | - Anders Sönnerborg
- Karolinska Institutet, Division of Infectious Diseases, Department of Medicine Huddinge, 14152, Stockholm, Sweden; Karolinska University Hospital, Department of Infectious Diseases, 14186, Stockholm, Sweden.
| | - Maurizio Zazzi
- Department of Medical Biotechnologies, University of Siena, 53100, Siena, Italy.
| | - Fabrizio Silvestri
- Sapienza University of Rome, Department of Computer Control and Management Engineering Antonio Ruberti, 00185, Rome, Italy.
| | - Laura Palagi
- Sapienza University of Rome, Department of Computer Control and Management Engineering Antonio Ruberti, 00185, Rome, Italy.
| |
Collapse
|
2
|
Ahmad FS, Hu TL, Adler ED, Petito LC, Wehbe RM, Wilcox JE, Mutharasan RK, Nardone B, Tadel M, Greenberg B, Yagil A, Campagnari C. Performance of risk models to predict mortality risk for patients with heart failure: evaluation in an integrated health system. Clin Res Cardiol 2024; 113:1343-1354. [PMID: 38565710 PMCID: PMC11371523 DOI: 10.1007/s00392-024-02433-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/09/2023] [Accepted: 03/05/2024] [Indexed: 04/04/2024]
Abstract
BACKGROUND Referral of patients with heart failure (HF) who are at high mortality risk for specialist evaluation is recommended. Yet, most tools for identifying such patients are difficult to implement in electronic health record (EHR) systems. OBJECTIVE To assess the performance and ease of implementation of Machine learning Assessment of RisK and EaRly mortality in Heart Failure (MARKER-HF), a machine-learning model that uses structured data that is readily available in the EHR, and compare it with two commonly used risk scores: the Seattle Heart Failure Model (SHFM) and Meta-Analysis Global Group in Chronic (MAGGIC) Heart Failure Risk Score. DESIGN Retrospective, cohort study. PARTICIPANTS Data from 6764 adults with HF were abstracted from EHRs at a large integrated health system from 1/1/10 to 12/31/19. MAIN MEASURES One-year survival from time of first cardiology or primary care visit was estimated using MARKER-HF, SHFM, and MAGGIC. Discrimination was measured by the area under the receiver operating curve (AUC). Calibration was assessed graphically. KEY RESULTS Compared to MARKER-HF, both SHFM and MAGGIC required a considerably larger amount of data engineering and imputation to generate risk score estimates. MARKER-HF, SHFM, and MAGGIC exhibited similar discriminations with AUCs of 0.70 (0.69-0.73), 0.71 (0.69-0.72), and 0.71 (95% CI 0.70-0.73), respectively. All three scores showed good calibration across the full risk spectrum. CONCLUSIONS These findings suggest that MARKER-HF, which uses readily available clinical and lab measurements in the EHR and required less imputation and data engineering than SHFM and MAGGIC, is an easier tool to identify high-risk patients in ambulatory clinics who could benefit from referral to a HF specialist.
Collapse
Affiliation(s)
- Faraz S Ahmad
- Division of Cardiology, Department of Medicine, Feinberg School of Medicine, Northwestern University, 676 North Saint Clair Street, Suite 600, Chicago, IL, 60611, USA.
- Bluhm Cardiovascular Institute Center for Artificial Intelligence, Northwestern Medicine, Chicago, IL, USA.
- Institute for Augmented Intelligence in Medicine, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA.
| | - Ted Ling Hu
- Institute for Augmented Intelligence in Medicine, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
| | - Eric D Adler
- Division of Cardiology, Department of Medicine, UC San Diego School of Medicine, La Jolla, CA, USA
| | - Lucia C Petito
- Division of Biostatistics, Department of Preventive Medicine, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
| | - Ramsey M Wehbe
- Bluhm Cardiovascular Institute Center for Artificial Intelligence, Northwestern Medicine, Chicago, IL, USA
- Division of Cardiology, Department of Medicine, Medical University of South Carolina, Charleston, SC, USA
| | - Jane E Wilcox
- Division of Cardiology, Department of Medicine, Feinberg School of Medicine, Northwestern University, 676 North Saint Clair Street, Suite 600, Chicago, IL, 60611, USA
- Bluhm Cardiovascular Institute Center for Artificial Intelligence, Northwestern Medicine, Chicago, IL, USA
| | - R Kannan Mutharasan
- Division of Cardiology, Department of Medicine, Feinberg School of Medicine, Northwestern University, 676 North Saint Clair Street, Suite 600, Chicago, IL, 60611, USA
- Bluhm Cardiovascular Institute Center for Artificial Intelligence, Northwestern Medicine, Chicago, IL, USA
| | - Beatrice Nardone
- Institute for Augmented Intelligence in Medicine, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
- Division of General Internal Medicine, Department of Medicine, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
| | - Matevz Tadel
- Physics Department, UC San Diego, La Jolla, CA, USA
| | - Barry Greenberg
- Division of Cardiology, Department of Medicine, UC San Diego School of Medicine, La Jolla, CA, USA
| | - Avi Yagil
- Physics Department, UC San Diego, La Jolla, CA, USA
| | | |
Collapse
|
3
|
Ballering AV, Olde Hartman TC, Rosmalen JG. Gender scores in epidemiological research: methods, advantages and implications. THE LANCET REGIONAL HEALTH. EUROPE 2024; 43:100962. [PMID: 38989448 PMCID: PMC11233999 DOI: 10.1016/j.lanepe.2024.100962] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/20/2024] [Revised: 05/14/2024] [Accepted: 05/30/2024] [Indexed: 07/12/2024]
Abstract
Sex and gender-related factors are strongly associated with patients' illness trajectories, underscoring their essential role in epidemiological research and healthcare. Ignoring sex and gender in research and health inevitably results in inequities between women and men in terms of detection of disease, preventative measures, and effectiveness of treatment. Historical influences, including ideas of female inferiority and conservative notions of women's health only comprising reproductive health, reinforced the perceived irrelevance of sex and gender to health. Currently, these ideas are largely abandoned and epidemiology is becoming increasingly sensitive to sex. Gender-sensitivity, however, is lagging behind. This is potentially due to lacking knowledge and awareness about the relevance of both sex and gender to health and challenges in operationalizing gender in epidemiological research. Here, we thoroughly discuss the relevance of sex and gender to health, and pay special attention to the time, place, and culture-dependent embodiment of gender. We also discuss the operationalization of gender via composite gender scores in epidemiological studies. We argue to move beyond solely using these. Rather we should consider sex and gender in the initial stages of designing a study, to facilitate relevant, reproducible, and person-centric research.
Collapse
Affiliation(s)
- Aranka V. Ballering
- University of Groningen, University Medical Center Groningen, Department of Psychiatry, P.O. Box 30.001, 9700 RB, Groningen, the Netherlands
| | - Tim C. Olde Hartman
- Department of Primary and Community Care, Radboud Institute of Medical Innovation, Radboud University Medical Center, P.O. Box 9101, 6500 HB, Nijmegen, the Netherlands
| | - Judith G.M. Rosmalen
- University of Groningen, University Medical Center Groningen, Department of Psychiatry, P.O. Box 30.001, 9700 RB, Groningen, the Netherlands
- University of Groningen, University Medical Center Groningen, Department of Internal Medicine, P.O. Box 30.001, 9700 RB, Groningen, the Netherlands
| |
Collapse
|
4
|
Liu Y, Sun S, Zhang Y, Huang X, Wang K, Qu Y, Chen X, Wu R, Zhang J, Luo J, Li Y, Wang J, Yi J. Predictive function of tumor burden-incorporated machine-learning algorithms for overall survival and their value in guiding management decisions in patients with locally advanced nasopharyngeal carcinoma. JOURNAL OF THE NATIONAL CANCER CENTER 2023; 3:295-305. [PMID: 39036668 PMCID: PMC11256522 DOI: 10.1016/j.jncc.2023.10.002] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2023] [Revised: 10/09/2023] [Accepted: 10/09/2023] [Indexed: 07/23/2024] Open
Abstract
Objective Accurate prognostic predictions and personalized decision-making on induction chemotherapy (IC) for individuals with locally advanced nasopharyngeal carcinoma (LA-NPC) remain challenging. This research examined the predictive function of tumor burden-incorporated machine-learning algorithms for overall survival (OS) and their value in guiding treatment in patients with LA-NPC. Methods Individuals with LA-NPC were reviewed retrospectively. Tumor burden signature-based OS prediction models were established using a nomogram and two machine-learning methods, the interpretable eXtreme Gradient Boosting (XGBoost) risk prediction model, and DeepHit time-to-event neural network. The models' prediction performances were compared using the concordance index (C-index) and the area under the curve (AUC). The patients were divided into two cohorts based on the risk predictions of the most successful model. The efficacy of IC combined with concurrent chemoradiotherapy was compared to that of chemoradiotherapy alone. Results The 1 221 eligible individuals, assigned to the training (n = 813) or validation (n = 408) set, showed significant respective differences in the C-indices of the XGBoost, DeepHit, and nomogram models (0.849 and 0.768, 0.811 and 0.767, 0.730 and 0.705). The training and validation sets had larger AUCs in the XGBoost and DeepHit models than the nomogram model in predicting OS (0.881 and 0.760, 0.845 and 0.776, and 0.764 and 0.729, P < 0.001). IC presented survival benefits in the XGBoost-derived high-risk but not low-risk group. Conclusion This research used machine-learning algorithms to create and verify a comprehensive model integrating tumor burden with clinical variables to predict OS and determine which patients will most likely gain from IC. This model could be valuable for delivering patient counseling and conducting clinical evaluations.
Collapse
Affiliation(s)
- Yang Liu
- Department of Radiation Oncology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100021, China
| | - Shiran Sun
- Department of Radiation Oncology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100021, China
| | - Ye Zhang
- Department of Radiation Oncology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100021, China
| | - Xiaodong Huang
- Department of Radiation Oncology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100021, China
| | - Kai Wang
- Department of Radiation Oncology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100021, China
| | - Yuan Qu
- Department of Radiation Oncology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100021, China
| | - Xuesong Chen
- Department of Radiation Oncology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100021, China
| | - Runye Wu
- Department of Radiation Oncology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100021, China
| | - Jianghu Zhang
- Department of Radiation Oncology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100021, China
| | - Jingwei Luo
- Department of Radiation Oncology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100021, China
| | - Yexiong Li
- Department of Radiation Oncology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100021, China
| | - Jingbo Wang
- Department of Radiation Oncology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100021, China
| | - Junlin Yi
- Department of Radiation Oncology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100021, China
- Department of Radiation Oncology, National Cancer Center/National Clinical Research Center for Cancer/Hebei Cancer Hospital, Chinese Academy of Medical Sciences (CAMS), Langfang 065001, China
| |
Collapse
|