1
|
Soe NN, Yu Z, Latt PM, Lee D, Ong JJ, Ge Z, Fairley CK, Zhang L. Evaluation of artificial intelligence-powered screening for sexually transmitted infections-related skin lesions using clinical images and metadata. BMC Med 2024; 22:296. [PMID: 39020355 PMCID: PMC11256573 DOI: 10.1186/s12916-024-03512-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/15/2024] [Accepted: 07/02/2024] [Indexed: 07/19/2024] Open
Abstract
BACKGROUND Sexually transmitted infections (STIs) pose a significant global public health challenge. Early diagnosis and treatment reduce STI transmission, but rely on recognising symptoms and care-seeking behaviour of the individual. Digital health software that distinguishes STI skin conditions could improve health-seeking behaviour. We developed and evaluated a deep learning model to differentiate STIs from non-STIs based on clinical images and symptoms. METHODS We used 4913 clinical images of genital lesions and metadata from the Melbourne Sexual Health Centre collected during 2010-2023. We developed two binary classification models to distinguish STIs from non-STIs: (1) a convolutional neural network (CNN) using images only and (2) an integrated model combining both CNN and fully connected neural network (FCN) using images and metadata. We evaluated the model performance by the area under the ROC curve (AUC) and assessed metadata contributions to the Image-only model. RESULTS Our study included 1583 STI and 3330 non-STI images. Common STI diagnoses were syphilis (34.6%), genital warts (24.5%) and herpes (19.4%), while most non-STIs (80.3%) were conditions such as dermatitis, lichen sclerosis and balanitis. In both STI and non-STI groups, the most frequently observed groups were 25-34 years (48.6% and 38.2%, respectively) and heterosexual males (60.3% and 45.9%, respectively). The Image-only model showed a reasonable performance with an AUC of 0.859 (SD 0.013). The Image + Metadata model achieved a significantly higher AUC of 0.893 (SD 0.018) compared to the Image-only model (p < 0.01). Out of 21 metadata, the integration of demographic and dermatological metadata led to the most significant improvement in model performance, increasing AUC by 6.7% compared to the baseline Image-only model. CONCLUSIONS The Image + Metadata model outperformed the Image-only model in distinguishing STIs from other skin conditions. Using it as a screening tool in a clinical setting may require further development and evaluation with larger datasets.
Collapse
Affiliation(s)
- Nyi N Soe
- Melbourne Sexual Health Centre, Alfred Health, 580 Swanston Street, Carlton, Melbourne, VIC, 3053, Australia
- School of Translational Medicine, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia
| | - Zhen Yu
- School of Translational Medicine, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia
| | - Phyu M Latt
- Melbourne Sexual Health Centre, Alfred Health, 580 Swanston Street, Carlton, Melbourne, VIC, 3053, Australia
- School of Translational Medicine, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia
| | - David Lee
- Melbourne Sexual Health Centre, Alfred Health, 580 Swanston Street, Carlton, Melbourne, VIC, 3053, Australia
| | - Jason J Ong
- Melbourne Sexual Health Centre, Alfred Health, 580 Swanston Street, Carlton, Melbourne, VIC, 3053, Australia
- School of Translational Medicine, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia
| | - Zongyuan Ge
- Augmented Intelligence and Multimodal analytics (AIM) for Health Lab, Faculty of Information Technology, Monash University, Melbourne, Australia
| | - Christopher K Fairley
- Melbourne Sexual Health Centre, Alfred Health, 580 Swanston Street, Carlton, Melbourne, VIC, 3053, Australia
- School of Translational Medicine, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia
| | - Lei Zhang
- Melbourne Sexual Health Centre, Alfred Health, 580 Swanston Street, Carlton, Melbourne, VIC, 3053, Australia.
- School of Translational Medicine, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia.
- Clinical Medical Research Centre, Children's Hospital of Nanjing Medical University, Nanjing, Jiangsu Province, 210008, China.
| |
Collapse
|
2
|
Ge Q, Lu X, Jiang R, Zhang Y, Zhuang X. Data mining and machine learning in HIV infection risk research: An overview and recommendations. Artif Intell Med 2024; 153:102887. [PMID: 38735156 DOI: 10.1016/j.artmed.2024.102887] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2023] [Revised: 03/07/2024] [Accepted: 04/27/2024] [Indexed: 05/14/2024]
Abstract
In the contemporary era, the applications of data mining and machine learning have permeated extensively into medical research, significantly contributing to areas such as HIV studies. By reviewing 38 articles published in the past 15 years, the study presents a roadmap based on seven different aspects, utilizing various machine learning techniques for both novice researchers and experienced researchers seeking to comprehend the current state of the art in this area. While traditional regression modeling techniques have been commonly used, researchers are increasingly adopting more advanced fully supervised machine learning and deep learning techniques, which often outperform the traditional methods in predictive performance. Additionally, the study identifies nine new open research issues and outlines possible future research plans to enhance the outcomes of HIV infection risk research. This review is expected to be an insightful guide for researchers, illuminating current practices and suggesting advancements in the field.
Collapse
Affiliation(s)
- Qiwei Ge
- Department of Epidemiology and Medical Statistics, School of Public Health, Nantong University, China
| | - Xinyu Lu
- Department of Epidemiology and Medical Statistics, School of Public Health, Nantong University, China
| | - Run Jiang
- Department of Epidemiology and Medical Statistics, School of Public Health, Nantong University, China
| | - Yuyu Zhang
- Department of Epidemiology and Medical Statistics, School of Public Health, Nantong University, China
| | - Xun Zhuang
- Department of Epidemiology and Medical Statistics, School of Public Health, Nantong University, China.
| |
Collapse
|
3
|
Volk JE, Leyden WA, Lea AN, Lee C, Donnelly MC, Krakower DS, Lee K, Liu VX, Marcus JL, Silverberg MJ. Using Electronic Health Records to Improve HIV Preexposure Prophylaxis Care: A Randomized Trial. J Acquir Immune Defic Syndr 2024; 95:362-369. [PMID: 38412047 DOI: 10.1097/qai.0000000000003376] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2023] [Accepted: 12/07/2023] [Indexed: 02/29/2024]
Abstract
BACKGROUND Preexposure prophylaxis (PrEP) use remains limited and inequitable, and strategies are needed to improve PrEP provision in primary care. METHODS We conducted a cluster randomized trial at Kaiser Permanente, San Francisco, to evaluate the effectiveness of a clinical decision support intervention guided by an electronic health record (EHR)-based HIV risk prediction model to improve PrEP provision. Primary care providers (PCPs) were randomized to usual care or intervention, with PCPs who provide care to people with HIV balanced between arms. PCPs in the intervention arm received an EHR-based staff message with prompts to discuss HIV prevention and PrEP before upcoming in-person or video visits with patients whose predicted 3-year HIV risk was above a prespecified threshold. The main study outcome was initiation of PrEP care within 90 days, defined as PrEP discussions, referrals, or prescription fills. RESULTS One hundred twenty-one PCPs had 5051 appointments with eligible patients (2580 usual care; 2471 intervention). There was a nonsignificant increase in initiation of PrEP care in the intervention arm (6.0% vs 4.5%, HR 1.32, 95% CI: 0.84 to 2.1). There was a significant interaction by HIV provider status, with an intervention HR of 2.59 (95% CI: 1.30 to 5.16) for HIV providers and 0.89 (95% CI: 0.59 to 1.35) for non-HIV providers (P-interaction <0.001). CONCLUSION An EHR-based intervention guided by an HIV risk prediction model substantially increased initiation of PrEP care among patients of PCPs who also care for people with HIV. Higher-intensity interventions may be needed to improve PrEP provision among PCPs less familiar with PrEP and HIV care.
Collapse
Affiliation(s)
- Jonathan E Volk
- Department of Infectious Diseases, Kaiser Permanente San Francisco, San Francisco, CA
| | - Wendy A Leyden
- Division of Research, Kaiser Permanente Northern California, Oakland, CA
| | - Alexandra N Lea
- Division of Research, Kaiser Permanente Northern California, Oakland, CA
| | - Catherine Lee
- Division of Research, Kaiser Permanente Northern California, Oakland, CA
| | | | - Douglas S Krakower
- Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, MA
- Division of Infectious Diseases, Beth Israel Deaconess Medical Center, Boston, MA; and
| | - Kristine Lee
- Department of Adult and Family Medicine, Kaiser Permanente San Francisco, San Francisco, CA
| | - Vincent X Liu
- Division of Research, Kaiser Permanente Northern California, Oakland, CA
| | - Julia L Marcus
- Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, MA
| | | |
Collapse
|
4
|
Soe NN, Latt PM, Yu Z, Lee D, Kim CM, Tran D, Ong JJ, Ge Z, Fairley CK, Zhang L. Clinical features-based machine learning models to separate sexually transmitted infections from other skin diagnoses. J Infect 2024; 88:106128. [PMID: 38452934 DOI: 10.1016/j.jinf.2024.106128] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2023] [Revised: 01/22/2024] [Accepted: 02/13/2024] [Indexed: 03/09/2024]
Abstract
INTRODUCTION Many sexual health services are overwhelmed and cannot cater for all the individuals who present with sexually transmitted infections (STIs). Digital health software that separates STIs from non-STIs could improve the efficiency of clinical services. We developed and evaluated a machine learning model that predicts whether patients have an STI based on their clinical features. METHODS We manually extracted 25 demographic features and clinical features from 1315 clinical records in the electronic health record system at Melbourne Sexual Health Center. We examined 16 machine learning models to predict a binary outcome of an STI or a non-STI diagnosis. We evaluated the models' performance with the area under the ROC curve (AUC), accuracy and F1-scores. RESULTS Our study included 1315 consultations, of which 36.8% (484/1315) were diagnosed with STIs and 63.2% (831/1315) had non-STI conditions. The study population predominantly consisted of heterosexual men (49.5%, 651/1315), followed by gay, bisexual and other men who have sex with men (GBMSM) (25.7%), women (21.6%) and unknown gender (3.2%). The median age was 31 years (intra-quartile range (IQR) 26-39). The top 5 performing models were CatBoost (AUC 0.912), Random Forest (AUC 0.917), LightGBM (AUC 0.907), Gradient Boosting (AUC 0.905) and XGBoost (AUC 0.900). The best model, CatBoost, achieved an accuracy of 0.837, sensitivity of 0.776, specificity of 0.831, precision of 0.782 and F1-score of 0.778. The key important features were lesion duration, type of skin lesions, age, gender, history of skin disorders, number of lesions, dysuria duration, anorectal pain and itchiness. CONCLUSIONS Our best model demonstrates a reasonable performance in distinguishing STIs from non-STIs. However, to be clinically useful, more detailed information such as clinical images, may be required to reach sufficient accuracy.
Collapse
Affiliation(s)
- Nyi Nyi Soe
- Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia; Central Clinical School, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia
| | - Phyu Mon Latt
- Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia; Central Clinical School, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia
| | - Zhen Yu
- Central Clinical School, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia; Monash e-Research Centre, Faculty of Engineering, Airdoc Research, Nvidia AI Technology Research Centre, Monash University, Melbourne, Australia
| | - David Lee
- Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia
| | - Cham-Mill Kim
- Melbourne Medical School, Faculty of Medicine, Dentistry and Health Sciences, The University of Melbourne, Melbourne, Australia
| | - Daniel Tran
- Melbourne Medical School, Faculty of Medicine, Dentistry and Health Sciences, The University of Melbourne, Melbourne, Australia
| | - Jason J Ong
- Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia; Central Clinical School, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia
| | - Zongyuan Ge
- Monash e-Research Centre, Faculty of Engineering, Airdoc Research, Nvidia AI Technology Research Centre, Monash University, Melbourne, Australia
| | - Christopher K Fairley
- Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia; Central Clinical School, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia
| | - Lei Zhang
- Clinical Medical Research Center, Children's Hospital of Nanjing Medical University, Nanjing, China; Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia; Central Clinical School, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia.
| |
Collapse
|
5
|
Nguyen TPV, Yang W, Tang Z, Xia X, Mullens AB, Dean JA, Li Y. Lightweight federated learning for STIs/HIV prediction. Sci Rep 2024; 14:6560. [PMID: 38503789 PMCID: PMC10950866 DOI: 10.1038/s41598-024-56115-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2023] [Accepted: 03/01/2024] [Indexed: 03/21/2024] Open
Abstract
This paper presents a solution that prioritises high privacy protection and improves communication throughput for predicting the risk of sexually transmissible infections/human immunodeficiency virus (STIs/HIV). The approach utilised Federated Learning (FL) to construct a model from multiple clinics and key stakeholders. FL ensured that only models were shared between clinics, minimising the risk of personal information leakage. Additionally, an algorithm was explored on the FL manager side to construct a global model that aligns with the communication status of the system. Our proposed method introduced Random Forest Federated Learning for assessing the risk of STIs/HIV, incorporating a flexible aggregation process that can be adjusted to accommodate the capacious communication system. Experimental results demonstrated the significant potential of a solution for estimating STIs/HIV risk. In comparison with recent studies, our approach yielded superior results in terms of AUC (0.97) and accuracy ( 93 % ). Despite these promising findings, a limitation of the study lies in the experiment for man's data, due to the self-reported nature of the data and sensitive content. which may be subject to participant bias. Future research could check the performance of the proposed framework in partnership with high-risk populations (e.g., men who have sex with men) to provide a more comprehensive understanding of the proposed framework's impact and ultimately aim to improve health outcomes/health service optimisation.
Collapse
Affiliation(s)
- Thi Phuoc Van Nguyen
- School of Mathematics, Physics and Computing, Centre for Health Research, University of Southern Queensland, Toowoomba Campus, Toowoomba, 4350, QLD, Australia.
| | - Wencheng Yang
- School of Mathematics, Physics and Computing, Centre for Health Research, University of Southern Queensland, Toowoomba Campus, Toowoomba, 4350, QLD, Australia
| | - Zhaohui Tang
- School of Mathematics, Physics and Computing, Centre for Health Research, University of Southern Queensland, Toowoomba Campus, Toowoomba, 4350, QLD, Australia
| | - Xiaoyu Xia
- School of Computing Technologies, RMIT University, GPO Box 2476, Melbourne, 3001, VIC, Australia
| | - Amy B Mullens
- School of Psychology and Wellbeing, Institute for Resilient Regions, Centre for Health Research, University of Southern Queensland, Ipswich Campus, Ipswich, 4305, Australia
| | - Judith A Dean
- School of Public Health, Faculty of Medicine, The University of Queensland, Herston Road, Brisbane, 4006, QLD, Australia
| | - Yan Li
- School of Mathematics, Physics and Computing, Centre for Health Research, University of Southern Queensland, Toowoomba Campus, Toowoomba, 4350, QLD, Australia
| |
Collapse
|
6
|
Latt PM, Soe NN, Xu X, Ong JJ, Chow EPF, Fairley CK, Zhang L. Identifying Individuals at High Risk for HIV and Sexually Transmitted Infections With an Artificial Intelligence-Based Risk Assessment Tool. Open Forum Infect Dis 2024; 11:ofae011. [PMID: 38440304 PMCID: PMC10911222 DOI: 10.1093/ofid/ofae011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2023] [Accepted: 01/05/2024] [Indexed: 03/06/2024] Open
Abstract
Background We have previously developed an artificial intelligence-based risk assessment tool to identify the individual risk of HIV and sexually transmitted infections (STIs) in a sexual health clinical setting. Based on this tool, this study aims to determine the optimal risk score thresholds to identify individuals at high risk for HIV/STIs. Methods Using 2008-2022 data from 216 252 HIV, 227 995 syphilis, 262 599 gonorrhea, and 320 355 chlamydia consultations at a sexual health center, we applied MySTIRisk machine learning models to estimate infection risk scores. Optimal cutoffs for determining high-risk individuals were determined using Youden's index. Results The HIV risk score cutoff for high risk was 0.56, with 86.0% sensitivity (95% CI, 82.9%-88.7%) and 65.6% specificity (95% CI, 65.4%-65.8%). Thirty-five percent of participants were classified as high risk, which accounted for 86% of HIV cases. The corresponding cutoffs were 0.49 for syphilis (sensitivity, 77.6%; 95% CI, 76.2%-78.9%; specificity, 78.1%; 95% CI, 77.9%-78.3%), 0.52 for gonorrhea (sensitivity, 78.3%; 95% CI, 77.6%-78.9%; specificity, 71.9%; 95% CI, 71.7%-72.0%), and 0.47 for chlamydia (sensitivity, 68.8%; 95% CI, 68.3%-69.4%; specificity, 63.7%; 95% CI, 63.5%-63.8%). High-risk groups identified using these thresholds accounted for 78% of syphilis, 78% of gonorrhea, and 69% of chlamydia cases. The odds of positivity were significantly higher in the high-risk group than otherwise across all infections: 11.4 (95% CI, 9.3-14.8) times for HIV, 12.3 (95% CI, 11.4-13.3) for syphilis, 9.2 (95% CI, 8.8-9.6) for gonorrhea, and 3.9 (95% CI, 3.8-4.0) for chlamydia. Conclusions Risk scores generated by the AI-based risk assessment tool MySTIRisk, together with Youden's index, are effective in determining high-risk subgroups for HIV/STIs. The thresholds can aid targeted HIV/STI screening and prevention.
Collapse
Affiliation(s)
- Phyu M Latt
- Artificial Intelligence and Modelling in Epidemiology Program, Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia
- Central Clinical School, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia
| | - Nyi N Soe
- Artificial Intelligence and Modelling in Epidemiology Program, Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia
- Central Clinical School, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia
| | - Xianglong Xu
- Artificial Intelligence and Modelling in Epidemiology Program, Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia
- School of Public Health, Shanghai University of Traditional Chinese Medicine, Shanghai, China
| | - Jason J Ong
- Central Clinical School, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia
- Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia
| | - Eric P F Chow
- Central Clinical School, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia
- Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia
- Centre for Epidemiology and Biostatistics, Melbourne School of Population and Global Health, The University of Melbourne, Melbourne, Australia
| | - Christopher K Fairley
- Central Clinical School, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia
- Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia
| | - Lei Zhang
- Artificial Intelligence and Modelling in Epidemiology Program, Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia
- Central Clinical School, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia
- Clinical Medical Research Center, Children’s Hospital of Nanjing Medical University, Nanjing, Jiangsu Province 210008, China
| |
Collapse
|
7
|
Isaacs N, Ntinga X, Keetsi T, Bhembe L, Mthembu B, Cloete A, Groenewald C. Are mHealth Interventions Effective in Improving the Uptake of Sexual and Reproductive Health Services among Adolescents? A Scoping Review. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2024; 21:165. [PMID: 38397656 PMCID: PMC10888173 DOI: 10.3390/ijerph21020165] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/15/2023] [Revised: 01/26/2024] [Accepted: 01/30/2024] [Indexed: 02/25/2024]
Abstract
Adolescents continue to face challenges to their sexual and reproductive health (SRH) both locally and internationally. Digital technologies such as the Internet, text messaging, and social media are often viewed as valuable tools for disseminating information on SRH. Mobile health, also known as mHealth, is a medical and public health practise that uses these digital technologies to communicate information. The literature has revealed that mHealth interventions have a positive outcome in delivering SRH information to adolescents. This review aimed to synthesise empirical studies that evaluate mHealth interventions and assess the extent to which these mHealth interventions promote sexual and reproductive health outcomes among young people. This scoping review reviewed the literature across four databases, including EBSCOhost, Scopus, Proquest, and Cochrane, and included 12 articles. The findings have shown that mHealth interventions are effective in enhancing sexual and reproductive health (SRH) knowledge and attitudes among young people in both low-middle and high-income countries. However, comprehensive longitudinal studies are necessary to measure the sustainability and long-term influence of mHealth interventions on behaviour. It is recommended that with artificial intelligence (AI) improvements, there is a possible path to bolstering mHealth interventions.
Collapse
Affiliation(s)
- Nazeema Isaacs
- Human Sciences Research Council, Pretoria 0001, South Africa; (X.N.); (T.K.); (B.M.); (A.C.); (C.G.)
- Impact Centre, Cape Town 8001, South Africa
| | - Xolani Ntinga
- Human Sciences Research Council, Pretoria 0001, South Africa; (X.N.); (T.K.); (B.M.); (A.C.); (C.G.)
- Centre for Community-Based Research, Durban 4001, South Africa
| | - Thabo Keetsi
- Human Sciences Research Council, Pretoria 0001, South Africa; (X.N.); (T.K.); (B.M.); (A.C.); (C.G.)
- Centre for Community-Based Research, Durban 4001, South Africa
| | - Lindelwa Bhembe
- Human Sciences Research Council, Pretoria 0001, South Africa; (X.N.); (T.K.); (B.M.); (A.C.); (C.G.)
- Centre for Community-Based Research, Durban 4001, South Africa
| | - Bongumenzi Mthembu
- Human Sciences Research Council, Pretoria 0001, South Africa; (X.N.); (T.K.); (B.M.); (A.C.); (C.G.)
- Centre for Community-Based Research, Durban 4001, South Africa
| | - Allanise Cloete
- Human Sciences Research Council, Pretoria 0001, South Africa; (X.N.); (T.K.); (B.M.); (A.C.); (C.G.)
- Public Health, Societies and Belonging (PHSB) Division, Cape Town 8001, South Africa
| | - Candice Groenewald
- Human Sciences Research Council, Pretoria 0001, South Africa; (X.N.); (T.K.); (B.M.); (A.C.); (C.G.)
- Centre for Community-Based Research, Durban 4001, South Africa
- Psychology Department, Rhodes University, Grahamstown 8001, South Africa
| |
Collapse
|
8
|
Hu M, Peng H, Zhang X, Wang L, Ren J. Building gender-specific sexually transmitted infection risk prediction models using CatBoost algorithm and NHANES data. BMC Med Inform Decis Mak 2024; 24:24. [PMID: 38267946 PMCID: PMC10809625 DOI: 10.1186/s12911-024-02426-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/04/2023] [Accepted: 01/15/2024] [Indexed: 01/26/2024] Open
Abstract
BACKGROUND AND AIMS Sexually transmitted infections (STIs) are a significant global public health challenge due to their high incidence rate and potential for severe consequences when early intervention is neglected. Research shows an upward trend in absolute cases and DALY numbers of STIs, with syphilis, chlamydia, trichomoniasis, and genital herpes exhibiting an increasing trend in age-standardized rate (ASR) from 2010 to 2019. Machine learning (ML) presents significant advantages in disease prediction, with several studies exploring its potential for STI prediction. The objective of this study is to build males-based and females-based STI risk prediction models based on the CatBoost algorithm using data from the National Health and Nutrition Examination Survey (NHANES) for training and validation, with sub-group analysis performed on each STI. The female sub-group also includes human papilloma virus (HPV) infection. METHODS The study utilized data from the National Health and Nutrition Examination Survey (NHANES) program to build males-based and females-based STI risk prediction models using the CatBoost algorithm. Data was collected from 12,053 participants aged 18 to 59 years old, with general demographic characteristics and sexual behavior questionnaire responses included as features. The Adaptive Synthetic Sampling Approach (ADASYN) algorithm was used to address data imbalance, and 15 machine learning algorithms were evaluated before ultimately selecting the CatBoost algorithm. The SHAP method was employed to enhance interpretability by identifying feature importance in the model's STIs risk prediction. RESULTS The CatBoost classifier achieved AUC values of 0.9995, 0.9948, 0.9923, and 0.9996 and 0.9769 for predicting chlamydia, genital herpes, genital warts, gonorrhea, and overall STIs infections among males. The CatBoost classifier achieved AUC values of 0.9971, 0.972, 0.9765, 1, 0.9485 and 0.8819 for predicting chlamydia, genital herpes, genital warts, gonorrhea, HPV and overall STIs infections among females. The characteristics of having sex with new partner/year, times having sex without condom/year, and the number of female vaginal sex partners/lifetime have been identified as the top three significant predictors for the overall risk of male STIs. Similarly, ever having anal sex with a man, age and the number of male vaginal sex partners/lifetime have been identified as the top three significant predictors for the overall risk of female STIs. CONCLUSIONS This study demonstrated the effectiveness of the CatBoost classifier in predicting STI risks among both male and female populations. The SHAP algorithm revealed key predictors for each infection, highlighting consistent demographic characteristics and sexual behaviors across different STIs. These insights can guide targeted prevention strategies and interventions to alleviate the impact of STIs on public health.
Collapse
Affiliation(s)
- Mengjie Hu
- Department of General Practice, First Affiliated Hospital, Zhejiang University School of Medicine, 310003, Hangzhou, China
| | - Han Peng
- Clinical Research Institute, Zhejiang Provincial People's Hospital (Affiliated People's Hospital of Hangzhou Medical College), Hangzhou, China
| | - Xuan Zhang
- Department of Cardiology, The First Affiliated Hospital, Zhejiang University School of Medicine, 310003, Hangzhou, China
| | - Lefeng Wang
- Kidney Disease Center, the First Affiliated Hospital, College of Medicine, Zhejiang University, 310003, Hangzhou, China
| | - Jingjing Ren
- Department of General Practice, First Affiliated Hospital, Zhejiang University School of Medicine, 310003, Hangzhou, China.
| |
Collapse
|
9
|
Nadarzynski T, Lunt A, Knights N, Bayley J, Llewellyn C. "But can chatbots understand sex?" Attitudes towards artificial intelligence chatbots amongst sexual and reproductive health professionals: An exploratory mixed-methods study. Int J STD AIDS 2023; 34:809-816. [PMID: 37269292 PMCID: PMC10561522 DOI: 10.1177/09564624231180777] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2023] [Accepted: 05/22/2023] [Indexed: 06/05/2023]
Abstract
BACKGROUND Artificial Intelligence (AI)-enabled chatbots can offer anonymous education about sexual and reproductive health (SRH). Understanding chatbot acceptability and feasibility allows the identification of barriers to the design and implementation. METHODS In 2020, we conducted an online survey and qualitative interviews with SRH professionals recruited online to explore the views on AI, automation and chatbots. Qualitative data were analysed thematically. RESULTS Amongst 150 respondents (48% specialist doctor/consultant), only 22% perceived chatbots as effective and 24% saw them as ineffective for SRH advice [Mean = 2.91, SD = 0.98, range: 1-5]. Overall, there were mixed attitudes towards SRH chatbots [Mean = 4.03, SD = 0.87, range: 1-7]. Chatbots were most acceptable for appointment booking, general sexual health advice and signposting, but not acceptable for safeguarding, virtual diagnosis, and emotional support. Three themes were identified: "Moving towards a 'digital' age'", "AI improving access and service efficacy", and "Hesitancy towards AI". CONCLUSIONS Half of SRH professionals were hesitant about the use of chatbots in SRH services, attributed to concerns about patient safety, and lack of familiarity with this technology. Future studies should explore the role of AI chatbots as supplementary tools for SRH promotion. Chatbot designers need to address the concerns of health professionals to increase acceptability and engagement with AI-enabled services.
Collapse
Affiliation(s)
| | - Alexandria Lunt
- Brighton and Sussex Medical School, University of Sussex, Brighton
| | | | | | - Carrie Llewellyn
- Brighton and Sussex Medical School, University of Sussex, Brighton
| |
Collapse
|
10
|
Albuquerque G, Fernandes F, Barbalho IMP, Barros DMS, Morais PSG, Morais AHF, Santos MM, Galvão-Lima LJ, Sales-Moioli AIL, Santos JPQ, Gil P, Henriques J, Teixeira C, Lima TS, Coutinho KD, Pinto TKB, Valentim RAM. Computational methods applied to syphilis: where are we, and where are we going? Front Public Health 2023; 11:1201725. [PMID: 37680278 PMCID: PMC10481400 DOI: 10.3389/fpubh.2023.1201725] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2023] [Accepted: 08/07/2023] [Indexed: 09/09/2023] Open
Abstract
Syphilis is an infectious disease that can be diagnosed and treated cheaply. Despite being a curable condition, the syphilis rate is increasing worldwide. In this sense, computational methods can analyze data and assist managers in formulating new public policies for preventing and controlling sexually transmitted infections (STIs). Computational techniques can integrate knowledge from experiences and, through an inference mechanism, apply conditions to a database that seeks to explain data behavior. This systematic review analyzed studies that use computational methods to establish or improve syphilis-related aspects. Our review shows the usefulness of computational tools to promote the overall understanding of syphilis, a global problem, to guide public policy and practice, to target better public health interventions such as surveillance and prevention, health service delivery, and the optimal use of diagnostic tools. The review was conducted according to PRISMA 2020 Statement and used several quality criteria to include studies. The publications chosen to compose this review were gathered from Science Direct, Web of Science, Springer, Scopus, ACM Digital Library, and PubMed databases. Then, studies published between 2015 and 2022 were selected. The review identified 1,991 studies. After applying inclusion, exclusion, and study quality assessment criteria, 26 primary studies were included in the final analysis. The results show different computational approaches, including countless Machine Learning algorithmic models, and three sub-areas of application in the context of syphilis: surveillance (61.54%), diagnosis (34.62%), and health policy evaluation (3.85%). These computational approaches are promising and capable of being tools to support syphilis control and surveillance actions.
Collapse
Affiliation(s)
- Gabriela Albuquerque
- Laboratory of Technological Innovation in Health, Federal University of Rio Grande do Norte, Natal, Rio Grande do Norte, Brazil
| | - Felipe Fernandes
- Laboratory of Technological Innovation in Health, Federal University of Rio Grande do Norte, Natal, Rio Grande do Norte, Brazil
| | - Ingridy M. P. Barbalho
- Laboratory of Technological Innovation in Health, Federal University of Rio Grande do Norte, Natal, Rio Grande do Norte, Brazil
| | - Daniele M. S. Barros
- Laboratory of Technological Innovation in Health, Federal University of Rio Grande do Norte, Natal, Rio Grande do Norte, Brazil
| | - Philippi S. G. Morais
- Laboratory of Technological Innovation in Health, Federal University of Rio Grande do Norte, Natal, Rio Grande do Norte, Brazil
| | - Antônio H. F. Morais
- Advanced Nucleus of Technological Innovation (NAVI), Federal Institute of Rio Grande do Norte, Natal, Rio Grande do Norte, Brazil
| | - Marquiony M. Santos
- Laboratory of Technological Innovation in Health, Federal University of Rio Grande do Norte, Natal, Rio Grande do Norte, Brazil
| | - Leonardo J. Galvão-Lima
- Laboratory of Technological Innovation in Health, Federal University of Rio Grande do Norte, Natal, Rio Grande do Norte, Brazil
| | - Ana Isabela L. Sales-Moioli
- Laboratory of Technological Innovation in Health, Federal University of Rio Grande do Norte, Natal, Rio Grande do Norte, Brazil
| | - João Paulo Q. Santos
- Advanced Nucleus of Technological Innovation (NAVI), Federal Institute of Rio Grande do Norte, Natal, Rio Grande do Norte, Brazil
| | - Paulo Gil
- Department of Informatics Engineering, Center for Informatics and Systems of the University of Coimbra, Universidade de Coimbra, Coimbra, Portugal
| | - Jorge Henriques
- Department of Informatics Engineering, Center for Informatics and Systems of the University of Coimbra, Universidade de Coimbra, Coimbra, Portugal
| | - César Teixeira
- Department of Informatics Engineering, Center for Informatics and Systems of the University of Coimbra, Universidade de Coimbra, Coimbra, Portugal
| | - Thaisa Santos Lima
- Laboratory of Technological Innovation in Health, Federal University of Rio Grande do Norte, Natal, Rio Grande do Norte, Brazil
- Ministry of Health, Esplanada dos Ministérios, Brasília, Brazil
| | - Karilany D. Coutinho
- Laboratory of Technological Innovation in Health, Federal University of Rio Grande do Norte, Natal, Rio Grande do Norte, Brazil
| | - Talita K. B. Pinto
- Laboratory of Technological Innovation in Health, Federal University of Rio Grande do Norte, Natal, Rio Grande do Norte, Brazil
| | - Ricardo A. M. Valentim
- Laboratory of Technological Innovation in Health, Federal University of Rio Grande do Norte, Natal, Rio Grande do Norte, Brazil
| |
Collapse
|
11
|
Friedman EE, Shankaran S, Devlin SA, Kishen EB, Mason JA, Sha BE, Ridgway JP. Development of a predictive model for identifying women vulnerable to HIV in Chicago. BMC Womens Health 2023; 23:313. [PMID: 37328764 PMCID: PMC10276380 DOI: 10.1186/s12905-023-02460-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2022] [Accepted: 06/03/2023] [Indexed: 06/18/2023] Open
Abstract
INTRODUCTION Researchers in the United States have created several models to predict persons most at risk for HIV. Many of these predictive models use data from all persons newly diagnosed with HIV, the majority of whom are men, and specifically men who have sex with men (MSM). Consequently, risk factors identified by these models are biased toward features that apply only to men or capture sexual behaviours of MSM. We sought to create a predictive model for women using cohort data from two major hospitals in Chicago with large opt-out HIV screening programs. METHODS We matched 48 newly diagnosed women to 192 HIV-negative women based on number of previous encounters at University of Chicago or Rush University hospitals. We examined data for each woman for the two years prior to either their HIV diagnosis or their last encounter. We assessed risk factors including demographic characteristics and clinical diagnoses taken from patient electronic medical records (EMR) using odds ratios and 95% confidence intervals. We created a multivariable logistic regression model and measured predictive power with the area under the curve (AUC). In the multivariable model, age group, race, and ethnicity were included a priori due to increased risk for HIV among specific demographic groups. RESULTS The following clinical diagnoses were significant at the bivariate level and were included in the model: pregnancy (OR 1.96 (1.00, 3.84)), hepatitis C (OR 5.73 (1.24, 26.51)), substance use (OR 3.12 (1.12, 8.65)) and sexually transmitted infections (STIs) chlamydia, gonorrhoea, or syphilis. We also a priori included demographic factors that are associated with HIV. Our final model had an AUC of 0.74 and included healthcare site, age group, race, ethnicity, pregnancy, hepatitis C, substance use, and STI diagnosis. CONCLUSIONS Our predictive model showed acceptable discrimination between those who were and were not newly diagnosed with HIV. We identified risk factors such as recent pregnancy, recent hepatitis C diagnosis, and substance use in addition to the traditionally used recent STI diagnosis that can be incorporated by health systems to detect women who are vulnerable to HIV and would benefit from preexposure prophylaxis (PrEP).
Collapse
Affiliation(s)
- Eleanor E. Friedman
- Department of Medicine, University of Chicago, 5841 S. Maryland Ave, MC 5065, Chicago, IL 60637 USA
| | | | - Samantha A. Devlin
- Department of Medicine, University of Chicago, 5841 S. Maryland Ave, MC 5065, Chicago, IL 60637 USA
| | | | - Joseph A. Mason
- Department of Medicine, University of Chicago, 5841 S. Maryland Ave, MC 5065, Chicago, IL 60637 USA
| | | | - Jessica P. Ridgway
- Department of Medicine, University of Chicago, 5841 S. Maryland Ave, MC 5065, Chicago, IL 60637 USA
| |
Collapse
|
12
|
Birri Makota RB, Musenge E. Predicting HIV infection in the decade (2005-2015) pre-COVID-19 in Zimbabwe: A supervised classification-based machine learning approach. PLOS DIGITAL HEALTH 2023; 2:e0000260. [PMID: 37285368 DOI: 10.1371/journal.pdig.0000260] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/23/2022] [Accepted: 04/24/2023] [Indexed: 06/09/2023]
Abstract
The burden of HIV and related diseases have been areas of great concern pre and post the emergence of COVID-19 in Zimbabwe. Machine learning models have been used to predict the risk of diseases, including HIV accurately. Therefore, this paper aimed to determine common risk factors of HIV positivity in Zimbabwe between the decade 2005 to 2015. The data were from three two staged population five-yearly surveys conducted between 2005 and 2015. The outcome variable was HIV status. The prediction model was fit by adopting 80% of the data for learning/training and 20% for testing/prediction. Resampling was done using the stratified 5-fold cross-validation procedure repeatedly. Feature selection was done using Lasso regression, and the best combination of selected features was determined using Sequential Forward Floating Selection. We compared six algorithms in both sexes based on the F1 score, which is the harmonic mean of precision and recall. The overall HIV prevalence for the combined dataset was 22.5% and 15.3% for females and males, respectively. The best-performing algorithm to identify individuals with a higher likelihood of HIV infection was XGBoost, with a high F1 score of 91.4% for males and 90.1% for females based on the combined surveys. The results from the prediction model identified six common features associated with HIV, with total number of lifetime sexual partners and cohabitation duration being the most influential variables for females and males, respectively. In addition to other risk reduction techniques, machine learning may aid in identifying those who might require Pre-exposure prophylaxis, particularly women who experience intimate partner violence. Furthermore, compared to traditional statistical approaches, machine learning uncovered patterns in predicting HIV infection with comparatively reduced uncertainty and, therefore, crucial for effective decision-making.
Collapse
Affiliation(s)
- Rutendo Beauty Birri Makota
- Division of Epidemiology and Biostatistics, School of Public Health, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa
| | - Eustasius Musenge
- Division of Epidemiology and Biostatistics, School of Public Health, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa
| |
Collapse
|
13
|
Borghi A, Flacco ME, Pacetti L, Orioni G, Marzola E, Cultrera R, Guerra V, Manfredini R, Gaspari V, Segala D, Corazza M. Visits to Sexually Transmitted Infection Clinics in Italy from January 2016 to November 2021: A Multicenter, Retrospective Study. J Pers Med 2023; 13:jpm13050731. [PMID: 37240901 DOI: 10.3390/jpm13050731] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2023] [Revised: 04/17/2023] [Accepted: 04/23/2023] [Indexed: 05/28/2023] Open
Abstract
There is no evidence of seasonal variation in visits to clinics dedicated to sexually transmitted infections (STIs) in Italy, nor of changes after the advent of the COVID-19 pandemic. An observational, retrospective, multicentric study was conducted to record and analyze all the visits to the STI clinics of the Dermatology Units of the University Hospitals of Ferrara and Bologna and of the Infectious Disease Unit of Ferrara, Italy, between January 2016 and November 2021. Overall, 11.733 visits were registered over a 70-month study period (63.7% males, mean age 34.5 ± 12.8 yrs). The mean number of monthly visits significantly decreased from the advent of the pandemic (136) compared to before (177). In the pre-pandemic period, visits to STI clinics increased in the autumn/winter months when compared to spring/summer, while the trend was the opposite in the pandemic period. Thus, during the pandemic, both an overall significant reduction in visits to STI clinics and a reversal in their seasonality were observed. These trends affected males and females equally. The marked decrease, mostly found in the pandemic winter months, can be linked to the "lockdown"/self-isolation ordinances and social distancing measures during the colder months, coinciding with the spread of the COVID-19 infection, which limited the opportunities for meeting and socializing.
Collapse
Affiliation(s)
- Alessandro Borghi
- Department of Medical Sciences, Section of Dermatology and Infectious Diseases, University of Ferrara, 44121 Ferrara, Italy
| | - Maria Elena Flacco
- Department of Environmental and Prevention Sciences, University of Ferrara, 44121 Ferrara, Italy
| | - Lucrezia Pacetti
- Department of Medical Sciences, Section of Dermatology and Infectious Diseases, University of Ferrara, 44121 Ferrara, Italy
| | - Gionathan Orioni
- Dermatology Unit, IRCCS Azienda Ospedaliero-Universitaria di Bologna, 40126 Bologna, Italy
- Department of Medical and Surgical Sciences, Alma Mater Studiorum University of Bologna, 40126 Bologna, Italy
| | - Elisa Marzola
- Department of Medical Sciences, Section of Dermatology and Infectious Diseases, University of Ferrara, 44121 Ferrara, Italy
| | - Rosario Cultrera
- Infectious Diseases, Department of Translational Medicine, University of Ferrara, 44121 Ferrara, Italy
| | - Valentina Guerra
- Planning and Management Control, University Hospital of Ferrara, 44124 Cona, Italy
| | - Roberto Manfredini
- University Center for Studies on Gender Medicine, Department of Medical Sciences, University of Ferrara, 44121 Ferrara, Italy
| | - Valeria Gaspari
- Dermatology Unit, IRCCS Azienda Ospedaliero-Universitaria di Bologna, 40126 Bologna, Italy
- Department of Medical and Surgical Sciences, Alma Mater Studiorum University of Bologna, 40126 Bologna, Italy
| | - Daniela Segala
- Infectious Diseases, Department of Translational Medicine, University of Ferrara, 44121 Ferrara, Italy
| | - Monica Corazza
- Department of Medical Sciences, Section of Dermatology and Infectious Diseases, University of Ferrara, 44121 Ferrara, Italy
| |
Collapse
|
14
|
Xu X, Chow EPF, Fairley CK, Chen M, Aguirre I, Goller J, Hocking J, Carvalho N, Zhang L, Ong JJ. Determinants and prediction of Chlamydia trachomatis re-testing and re-infection within 1 year among heterosexuals with chlamydia attending a sexual health clinic. Front Public Health 2023; 10:1031372. [PMID: 36711362 PMCID: PMC9880158 DOI: 10.3389/fpubh.2022.1031372] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2022] [Accepted: 12/23/2022] [Indexed: 01/14/2023] Open
Abstract
Background Chlamydia trachomatis (chlamydia) is one of the most common sexually transmitted infections (STI) globally, and re-infections are common. Current Australian guidelines recommend re-testing for chlamydia 3 months after treatment to identify possible re-infection. Patient-delivered partner therapy (PDPT) has been proposed to control chlamydia re-infection among heterosexuals. We aimed to identify determinants and the prediction of chlamydia re-testing and re-infection within 1 year among heterosexuals with chlamydia to identify potential PDPT candidates. Methods Our baseline data included 5,806 heterosexuals with chlamydia aged ≥18 years and 2,070 re-tested for chlamydia within 1 year of their chlamydia diagnosis at the Melbourne Sexual Health Center from January 2, 2015, to May 15, 2020. We used routinely collected electronic health record (EHR) variables and machine-learning models to predict chlamydia re-testing and re-infection events. We also used logistic regression to investigate factors associated with chlamydia re-testing and re-infection. Results About 2,070 (36%) of 5,806 heterosexuals with chlamydia were re-tested for chlamydia within 1 year. Among those retested, 307 (15%) were re-infected. Multivariable logistic regression analysis showed that older age (≥35 years old), female, living with HIV, being a current sex worker, patient-delivered partner therapy users, and higher numbers of sex partners were associated with an increased chlamydia re-testing within 1 year. Multivariable logistic regression analysis also showed that younger age (18-24 years), male gender, and living with HIV were associated with an increased chlamydia re-infection within 1 year. The XGBoost model was the best model for predicting chlamydia re-testing and re-infection within 1 year among heterosexuals with chlamydia; however, machine learning approaches and these self-reported answers from clients did not provide a good predictive value (AUC < 60.0%). Conclusion The low rate of chlamydia re-testing and high rate of chlamydia re-infection among heterosexuals with chlamydia highlights the need for further interventions. Better targeting of individuals more likely to be re-infected is needed to optimize the provision of PDPT and encourage the test of re-infection at 3 months.
Collapse
Affiliation(s)
- Xianglong Xu
- Department of Epidemiology and Health Statistics, School of Public Health, Shanghai University of Traditional Chinese Medicine, Shanghai, China,Melbourne Sexual Health Centre, The Alfred, Melbourne, VIC, Australia,Central Clinical School, Monash University, Melbourne, VIC, Australia
| | - Eric P. F. Chow
- Melbourne Sexual Health Centre, The Alfred, Melbourne, VIC, Australia,Central Clinical School, Monash University, Melbourne, VIC, Australia,Centre for Epidemiology and Biostatistics, Melbourne School of Population and Global Health, The University of Melbourne, Melbourne, VIC, Australia
| | - Christopher K. Fairley
- Melbourne Sexual Health Centre, The Alfred, Melbourne, VIC, Australia,Central Clinical School, Monash University, Melbourne, VIC, Australia
| | - Marcus Chen
- Melbourne Sexual Health Centre, The Alfred, Melbourne, VIC, Australia,Central Clinical School, Monash University, Melbourne, VIC, Australia
| | - Ivette Aguirre
- Melbourne Sexual Health Centre, The Alfred, Melbourne, VIC, Australia
| | - Jane Goller
- Centre for Epidemiology and Biostatistics, Melbourne School of Population and Global Health, The University of Melbourne, Melbourne, VIC, Australia
| | - Jane Hocking
- Centre for Epidemiology and Biostatistics, Melbourne School of Population and Global Health, The University of Melbourne, Melbourne, VIC, Australia
| | - Natalie Carvalho
- Centre for Health Policy, Melbourne School of Population and Global Health, The University of Melbourne, Melbourne, VIC, Australia
| | - Lei Zhang
- Melbourne Sexual Health Centre, The Alfred, Melbourne, VIC, Australia,Central Clinical School, Monash University, Melbourne, VIC, Australia,China Australia Joint Research Center for Infectious Diseases, School of Public Health, Xi'an Jiaotong University Health Science Centre, Xi'an, Shaanxi, China,*Correspondence: Lei Zhang ✉
| | - Jason J. Ong
- Melbourne Sexual Health Centre, The Alfred, Melbourne, VIC, Australia,Central Clinical School, Monash University, Melbourne, VIC, Australia,Faculty of Infectious and Tropical Diseases, London School of Hygiene and Tropical Medicine, London, United Kingdom,Jason J. Ong ✉
| |
Collapse
|
15
|
Majam M, Segal B, Fieggen J, Smith E, Hermans L, Singh L, Phatsoane M, Arora L, Lalla-Edward S. Utility of a machine-guided tool for assessing risk behaviour associated with contracting HIV in three sites in South Africa. INFORMATICS IN MEDICINE UNLOCKED 2023; 37:101192. [PMID: 36911795 PMCID: PMC9993399 DOI: 10.1016/j.imu.2023.101192] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2022] [Revised: 02/03/2023] [Accepted: 02/04/2023] [Indexed: 02/11/2023] Open
Abstract
Introduction Digital data collection and the associated mobile health technologies have allowed for the recent exploration of artificial intelligence as a tool for combatting the HIV epidemic. Machine learning has been found to be useful both in HIV risk prediction and as a decision support tool for guiding pre-exposure prophylaxis (PrEP) treatment. This paper reports data from two sequential studies evaluating the viability of using machine learning to predict the susceptibility of adults to HIV infection using responses from a digital survey deployed in a high burden, low-resource setting. Methods 1036 and 593 participants were recruited across two trials. The first trial was a cross-sectional study in one location and the second trial was a cohort study across three trial sites. The data from the studies were merged, partitioned using standard techniques, and then used to train and evaluate multiple different machine learning models and select and evaluate a final model. Variable importance estimates were calculated using the PIMP and SHAP methodologies. Results Characteristics associated with HIV were consistent across both studies. Overall, HIV positive patients had a higher median age (34 [IQR: 29-39] vs 26 [IQR 22-33], p < 0.001), and were more likely to be female (155/703 [22%] vs 107/927 [12%], p < 0.001). HIV positive participants also had more commonly gone a year or more since their last HIV test (183/262 [70%] vs 540/1368 [39%], p < 0.001) and were less likely to report consistent condom usage (113/262 [43%] vs 758/1368 [55%], p < 0.001). Patients who reported TB symptoms were more likely to be HIV positive. The trained models had accuracy values (AUROCs) ranging from 78.5% to 82.8%. A boosted tree model performed best with a sensitivity of 84% (95% CI 72-92), specificity of 71% (95% CI 67-76), and a negative predictive value of 95% (95% CI 93-96) in a hold-out dataset. Age, duration since last HIV test, and number of male sexual partners were consistently three of the four most important variables across both variable importance estimates. Conclusions This study has highlighted the synergies present between mobile health and machine learning in HIV. It has been demonstrated that a viable ML model can be built using digital survey data from an low-middle income setting with potential utility in directing health resources.
Collapse
Affiliation(s)
- M. Majam
- Ezintsha, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa
| | - B. Segal
- Phithos Technologies, Johannesburg, South Africa
| | - J. Fieggen
- Phithos Technologies, Johannesburg, South Africa
| | - Eli Smith
- Phithos Technologies, Johannesburg, South Africa
| | - L. Hermans
- Ezintsha, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa
- Department of Microbiology, University Medical Center Utrecht, Utrecht, the Netherlands
- Infectious Diseases Unit, Department of Medicine, University of Cape Town, Cape Town, South Africa
| | - L. Singh
- Ezintsha, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa
| | - M. Phatsoane
- Ezintsha, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa
| | - L. Arora
- Phithos Technologies, Johannesburg, South Africa
| | - S.T. Lalla-Edward
- Ezintsha, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa
- Corresponding author. Ezintsha, Building C, Sunnyside Office Park, 32 Princess of Wales Terrace, Parktown, Gauteng, 2000, South Africa.
| |
Collapse
|
16
|
Fieggen J, Smith E, Arora L, Segal B. The role of machine learning in HIV risk prediction. FRONTIERS IN REPRODUCTIVE HEALTH 2022; 4:1062387. [PMID: 36619681 PMCID: PMC9815547 DOI: 10.3389/frph.2022.1062387] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2022] [Accepted: 12/05/2022] [Indexed: 12/24/2022] Open
Abstract
Despite advances in reducing HIV-related mortality, persistently high HIV incidence rates are undermining global efforts to end the epidemic by 2030. The UNAIDS Fast-track targets as well as other preventative strategies, such as pre-exposure prophylaxis, have been identified as priority areas to reduce the ongoing transmission threatening to undermine recent progress. Accurate and granular risk prediction is critical for these campaigns but is often lacking in regions where the burden is highest. Owing to their ability to capture complex interactions between data, machine learning and artificial intelligence algorithms have proven effective at predicting the risk of HIV infection in both high resource and low resource settings. However, interpretability of these algorithms presents a challenge to the understanding and adoption of these algorithms. In this perspectives article, we provide an introduction to machine learning and discuss some of the important considerations when choosing the variables used in model development and when evaluating the performance of different machine learning algorithms, as well as the role emerging tools such as Shapely Additive Explanations may play in helping understand and decompose these models in the context of HIV. Finally, we discuss some of the potential public health and clinical use cases for such decomposed risk assessment models in directing testing and preventative interventions including pre-exposure prophylaxis, as well as highlight the potential integration synergies with algorithms that predict the risk of sexually transmitted infections and tuberculosis.
Collapse
Affiliation(s)
- Joshua Fieggen
- School of Public Health and Family Medicine, Faculty of Health Sciences, University of Cape Town, Cape Town, South Africa,Phithos Technologies, Johannesburg, South Africa,Correspondence: Joshua Fieggen ;
| | - Eli Smith
- Phithos Technologies, Johannesburg, South Africa
| | | | - Bradley Segal
- Phithos Technologies, Johannesburg, South Africa,Department of Biomedical Engineering, University of the Witwatersrand, Johannesburg, South Africa
| |
Collapse
|
17
|
Xu X, Yu Z, Ge Z, Chow EPF, Bao Y, Ong JJ, Li W, Wu J, Fairley CK, Zhang L. Web-Based Risk Prediction Tool for an Individual's Risk of HIV and Sexually Transmitted Infections Using Machine Learning Algorithms: Development and External Validation Study. J Med Internet Res 2022; 24:e37850. [PMID: 36006685 PMCID: PMC9459839 DOI: 10.2196/37850] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2022] [Revised: 04/13/2022] [Accepted: 07/28/2022] [Indexed: 12/05/2022] Open
Abstract
Background HIV and sexually transmitted infections (STIs) are major global public health concerns. Over 1 million curable STIs occur every day among people aged 15 years to 49 years worldwide. Insufficient testing or screening substantially impedes the elimination of HIV and STI transmission. Objective The aim of our study was to develop an HIV and STI risk prediction tool using machine learning algorithms. Methods We used clinic consultations that tested for HIV and STIs at the Melbourne Sexual Health Centre between March 2, 2015, and December 31, 2018, as the development data set (training and testing data set). We also used 2 external validation data sets, including data from 2019 as external “validation data 1” and data from January 2020 and January 2021 as external “validation data 2.” We developed 34 machine learning models to assess the risk of acquiring HIV, syphilis, gonorrhea, and chlamydia. We created an online tool to generate an individual’s risk of HIV or an STI. Results The important predictors for HIV and STI risk were gender, age, men who reported having sex with men, number of casual sexual partners, and condom use. Our machine learning–based risk prediction tool, named MySTIRisk, performed at an acceptable or excellent level on testing data sets (area under the curve [AUC] for HIV=0.78; AUC for syphilis=0.84; AUC for gonorrhea=0.78; AUC for chlamydia=0.70) and had stable performance on both external validation data from 2019 (AUC for HIV=0.79; AUC for syphilis=0.85; AUC for gonorrhea=0.81; AUC for chlamydia=0.69) and data from 2020-2021 (AUC for HIV=0.71; AUC for syphilis=0.84; AUC for gonorrhea=0.79; AUC for chlamydia=0.69). Conclusions Our web-based risk prediction tool could accurately predict the risk of HIV and STIs for clinic attendees using simple self-reported questions. MySTIRisk could serve as an HIV and STI screening tool on clinic websites or digital health platforms to encourage individuals at risk of HIV or an STI to be tested or start HIV pre-exposure prophylaxis. The public can use this tool to assess their risk and then decide if they would attend a clinic for testing. Clinicians or public health workers can use this tool to identify high-risk individuals for further interventions.
Collapse
Affiliation(s)
- Xianglong Xu
- Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia.,Central Clinical School, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia.,China Australia Joint Research Center for Infectious Diseases, Xi'an Jiaotong University Health Science Centre, Xi'an, China
| | - Zhen Yu
- Central Clinical School, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia.,China Australia Joint Research Center for Infectious Diseases, Xi'an Jiaotong University Health Science Centre, Xi'an, China.,Monash e-Research Centre, Faculty of Engineering, Airdoc Research, Nvidia AI Technology Research Centre, Monash University, Melbourne, Australia
| | - Zongyuan Ge
- Monash e-Research Centre, Faculty of Engineering, Airdoc Research, Nvidia AI Technology Research Centre, Monash University, Melbourne, Australia
| | - Eric P F Chow
- Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia.,Central Clinical School, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia.,Centre for Epidemiology and Biostatistics, Melbourne School of Population and Global Health, University of Melbourne, Melbourne, Australia
| | - Yining Bao
- China Australia Joint Research Center for Infectious Diseases, Xi'an Jiaotong University Health Science Centre, Xi'an, China
| | - Jason J Ong
- Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia.,Central Clinical School, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia.,China Australia Joint Research Center for Infectious Diseases, Xi'an Jiaotong University Health Science Centre, Xi'an, China
| | - Wei Li
- School of Public Health, Southeast University, Nanjing, China
| | - Jinrong Wu
- Research Centre for Data Analytics and Cognition, La Trobe University, Melbourne, Australia
| | - Christopher K Fairley
- Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia.,Central Clinical School, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia.,China Australia Joint Research Center for Infectious Diseases, Xi'an Jiaotong University Health Science Centre, Xi'an, China
| | - Lei Zhang
- Melbourne Sexual Health Centre, Alfred Health, Melbourne, Australia.,Central Clinical School, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia.,China Australia Joint Research Center for Infectious Diseases, Xi'an Jiaotong University Health Science Centre, Xi'an, China
| |
Collapse
|