1
|
Mohsen F, Al-Absi HRH, Yousri NA, El Hajj N, Shah Z. A scoping review of artificial intelligence-based methods for diabetes risk prediction. NPJ Digit Med 2023; 6:197. [PMID: 37880301 PMCID: PMC10600138 DOI: 10.1038/s41746-023-00933-5] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2023] [Accepted: 09/25/2023] [Indexed: 10/27/2023] Open
Abstract
The increasing prevalence of type 2 diabetes mellitus (T2DM) and its associated health complications highlight the need to develop predictive models for early diagnosis and intervention. While many artificial intelligence (AI) models for T2DM risk prediction have emerged, a comprehensive review of their advancements and challenges is currently lacking. This scoping review maps out the existing literature on AI-based models for T2DM prediction, adhering to the PRISMA extension for Scoping Reviews guidelines. A systematic search of longitudinal studies was conducted across four databases, including PubMed, Scopus, IEEE-Xplore, and Google Scholar. Forty studies that met our inclusion criteria were reviewed. Classical machine learning (ML) models dominated these studies, with electronic health records (EHR) being the predominant data modality, followed by multi-omics, while medical imaging was the least utilized. Most studies employed unimodal AI models, with only ten adopting multimodal approaches. Both unimodal and multimodal models showed promising results, with the latter being superior. Almost all studies performed internal validation, but only five conducted external validation. Most studies utilized the area under the curve (AUC) for discrimination measures. Notably, only five studies provided insights into the calibration of their models. Half of the studies used interpretability methods to identify key risk predictors revealed by their models. Although a minority highlighted novel risk predictors, the majority reported commonly known ones. Our review provides valuable insights into the current state and limitations of AI-based models for T2DM prediction and highlights the challenges associated with their development and clinical integration.
Collapse
Affiliation(s)
- Farida Mohsen
- College of Science and Engineering, Hamad Bin Khalifa University, Qatar Foundation, 34110, Doha, Qatar
| | - Hamada R H Al-Absi
- College of Science and Engineering, Hamad Bin Khalifa University, Qatar Foundation, 34110, Doha, Qatar
| | - Noha A Yousri
- Genetic Medicine, Weill Cornell Medicine-Qatar, Qatar Foundation, Doha, Qatar
- College of Health and Life Sciences, Hamad Bin Khalifa University, Qatar Foundation, 34110, Doha, Qatar
- Computer and Systems Engineering, Alexandria University, Alexandria, Egypt
| | - Nady El Hajj
- College of Science and Engineering, Hamad Bin Khalifa University, Qatar Foundation, 34110, Doha, Qatar
- College of Health and Life Sciences, Hamad Bin Khalifa University, Qatar Foundation, 34110, Doha, Qatar
| | - Zubair Shah
- College of Science and Engineering, Hamad Bin Khalifa University, Qatar Foundation, 34110, Doha, Qatar.
| |
Collapse
|
2
|
Abdulazeem H, Whitelaw S, Schauberger G, Klug SJ. A systematic review of clinical health conditions predicted by machine learning diagnostic and prognostic models trained or validated using real-world primary health care data. PLoS One 2023; 18:e0274276. [PMID: 37682909 PMCID: PMC10491005 DOI: 10.1371/journal.pone.0274276] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2022] [Accepted: 08/29/2023] [Indexed: 09/10/2023] Open
Abstract
With the advances in technology and data science, machine learning (ML) is being rapidly adopted by the health care sector. However, there is a lack of literature addressing the health conditions targeted by the ML prediction models within primary health care (PHC) to date. To fill this gap in knowledge, we conducted a systematic review following the PRISMA guidelines to identify health conditions targeted by ML in PHC. We searched the Cochrane Library, Web of Science, PubMed, Elsevier, BioRxiv, Association of Computing Machinery (ACM), and IEEE Xplore databases for studies published from January 1990 to January 2022. We included primary studies addressing ML diagnostic or prognostic predictive models that were supplied completely or partially by real-world PHC data. Studies selection, data extraction, and risk of bias assessment using the prediction model study risk of bias assessment tool were performed by two investigators. Health conditions were categorized according to international classification of diseases (ICD-10). Extracted data were analyzed quantitatively. We identified 106 studies investigating 42 health conditions. These studies included 207 ML prediction models supplied by the PHC data of 24.2 million participants from 19 countries. We found that 92.4% of the studies were retrospective and 77.3% of the studies reported diagnostic predictive ML models. A majority (76.4%) of all the studies were for models' development without conducting external validation. Risk of bias assessment revealed that 90.8% of the studies were of high or unclear risk of bias. The most frequently reported health conditions were diabetes mellitus (19.8%) and Alzheimer's disease (11.3%). Our study provides a summary on the presently available ML prediction models within PHC. We draw the attention of digital health policy makers, ML models developer, and health care professionals for more future interdisciplinary research collaboration in this regard.
Collapse
Affiliation(s)
- Hebatullah Abdulazeem
- Chair of Epidemiology, Department of Sport and Health Sciences, Technical University of Munich (TUM), Munich, Germany
| | - Sera Whitelaw
- Faculty of Medicine and Health Sciences, McGill University, Montreal, Quebec, Canada
| | - Gunther Schauberger
- Chair of Epidemiology, Department of Sport and Health Sciences, Technical University of Munich (TUM), Munich, Germany
| | - Stefanie J. Klug
- Chair of Epidemiology, Department of Sport and Health Sciences, Technical University of Munich (TUM), Munich, Germany
| |
Collapse
|
3
|
Jeong K, Mallard AR, Coombe L, Ward J. Artificial intelligence and prediction of cardiometabolic disease: Systematic review of model performance and potential benefits in indigenous populations. Artif Intell Med 2023; 139:102534. [PMID: 37100506 DOI: 10.1016/j.artmed.2023.102534] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2021] [Revised: 11/20/2022] [Accepted: 03/21/2023] [Indexed: 03/30/2023]
Abstract
BACKGROUND Indigenous peoples often have higher rates of morbidity and mortality associated with cardiometabolic disease (CMD) than non-Indigenous people and this may be even more so in urban areas. The use of electronic health records and expansion of computing power has led to mainstream use of artificial intelligence (AI) to predict the onset of disease in primary health care (PHC) settings. However, it is unknown if AI and in particular machine learning is used for risk prediction of CMD in Indigenous peoples. METHODS We searched peer-reviewed literature using terms associated with AI machine learning, PHC, CMD, and Indigenous peoples. RESULTS We identified 13 suitable studies for inclusion in this review. Median total number of participants was 19,270 (range 911-2,994,837). The most common algorithms used in machine learning in this setting were support vector machine, random forest, and decision tree learning. Twelve studies used the area under the receiver operating characteristic curve (AUC) to measure performance. Two studies reported an AUC of >0.9. Six studies had an AUC score between 0.9 and 0.8, 4 studies had an AUC score between 0.8 and 0.7. 1 study reported an AUC score between 0.7 and 0.6. Risk of bias was observed in 10 (77 %) studies. CONCLUSION AI machine learning and risk prediction models show moderate to excellent discriminatory ability over traditional statistical models in predicting CMD. This technology could help address the needs of urban Indigenous peoples by predicting CMD early and more rapidly than conventional methods.
Collapse
|
4
|
Yan Q, Shan S, Sun M, Zhao F, Yang Y, Li Y. A Social Media Infodemic-Based Prediction Model for the Number of Severe and Critical COVID-19 Patients in the Lockdown Area. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:ijerph19138109. [PMID: 35805766 PMCID: PMC9266038 DOI: 10.3390/ijerph19138109] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/13/2022] [Revised: 06/20/2022] [Accepted: 06/28/2022] [Indexed: 02/01/2023]
Abstract
Accurately predicting the number of severe and critical COVID-19 patients is critical for the treatment and control of the epidemic. Social media data have gained great popularity and widespread application in various research domains. The viral-related infodemic outbreaks have occurred alongside the COVID-19 outbreak. This paper aims to discover trustworthy sources of social media data to improve the prediction performance of severe and critical COVID-19 patients. The innovation of this paper lies in three aspects. First, it builds an improved prediction model based on machine learning. This model helps predict the number of severe and critical COVID-19 patients on a specific urban or regional scale. The effectiveness of the prediction model, shown as accuracy and satisfactory robustness, is verified by a case study of the lockdown in Hubei Province. Second, it finds the transition path of the impact of social media data for predicting the number of severe and critical COVID-19 patients. Third, this paper provides a promising and powerful model for COVID-19 prevention and control. The prediction model can help medical organizations to realize a prediction of COVID-19 severe and critical patients in multi-stage with lead time in specific areas. This model can guide the Centers for Disease Control and Prevention and other clinic institutions to expand the monitoring channels and research methods concerning COVID-19 by using web-based social media data. The model can also facilitate optimal scheduling of medical resources as well as prevention and control policy formulation.
Collapse
Affiliation(s)
- Qi Yan
- School of Economics and Management, Beihang University, Beijing 100191, China; (S.S.); (M.S.); (F.Z.); (Y.Y.); (Y.L.)
- Beijing Key Laboratory of Emergency Support Simulation Technologies for City Operation, Beijing 100191, China
- Correspondence:
| | - Siqing Shan
- School of Economics and Management, Beihang University, Beijing 100191, China; (S.S.); (M.S.); (F.Z.); (Y.Y.); (Y.L.)
- Beijing Key Laboratory of Emergency Support Simulation Technologies for City Operation, Beijing 100191, China
| | - Menghan Sun
- School of Economics and Management, Beihang University, Beijing 100191, China; (S.S.); (M.S.); (F.Z.); (Y.Y.); (Y.L.)
- Beijing Key Laboratory of Emergency Support Simulation Technologies for City Operation, Beijing 100191, China
| | - Feng Zhao
- School of Economics and Management, Beihang University, Beijing 100191, China; (S.S.); (M.S.); (F.Z.); (Y.Y.); (Y.L.)
- Beijing Key Laboratory of Emergency Support Simulation Technologies for City Operation, Beijing 100191, China
| | - Yangzi Yang
- School of Economics and Management, Beihang University, Beijing 100191, China; (S.S.); (M.S.); (F.Z.); (Y.Y.); (Y.L.)
- Beijing Key Laboratory of Emergency Support Simulation Technologies for City Operation, Beijing 100191, China
| | - Yinong Li
- School of Economics and Management, Beihang University, Beijing 100191, China; (S.S.); (M.S.); (F.Z.); (Y.Y.); (Y.L.)
- Beijing Key Laboratory of Emergency Support Simulation Technologies for City Operation, Beijing 100191, China
| |
Collapse
|
5
|
A novel kernel based approach to arbitrary length symbolic data with application to type 2 diabetes risk. Sci Rep 2022; 12:4985. [PMID: 35322076 PMCID: PMC8943170 DOI: 10.1038/s41598-022-08757-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/09/2021] [Accepted: 03/07/2022] [Indexed: 11/08/2022] Open
Abstract
Predictive modeling of clinical data is fraught with challenges arising from the manner in which events are recorded. Patients typically fall ill at irregular intervals and experience dissimilar intervention trajectories. This results in irregularly sampled and uneven length data which poses a problem for standard multivariate tools. The alternative of feature extraction into equal-length vectors via methods like Bag-of-Words (BoW) potentially discards useful information. We propose an approach based on a kernel framework in which data is maintained in its native form: discrete sequences of symbols. Kernel functions derived from the edit distance between pairs of sequences may then be utilized in conjunction with support vector machines to classify the data. Our method is evaluated in the context of the prediction task of determining patients likely to develop type 2 diabetes following an earlier episode of elevated blood pressure of 130/80 mmHg. Kernels combined via multi kernel learning achieved an F1-score of 0.96, outperforming classification with SVM 0.63, logistic regression 0.63, Long Short Term Memory 0.61 and Multi-Layer Perceptron 0.54 applied to a BoW representation of the data. We achieved an F1-score of 0.97 on MKL on external dataset. The proposed approach is consequently able to overcome limitations associated with feature-based classification in the context of clinical data.
Collapse
|
6
|
Vehi J, Mujahid O, Contreras I. Aim and Diabetes. Artif Intell Med 2022. [DOI: 10.1007/978-3-030-64573-1_158] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
7
|
Gautier T, Ziegler LB, Gerber MS, Campos-Náñez E, Patek SD. Artificial intelligence and diabetes technology: A review. Metabolism 2021; 124:154872. [PMID: 34480920 DOI: 10.1016/j.metabol.2021.154872] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/18/2021] [Revised: 07/27/2021] [Accepted: 08/28/2021] [Indexed: 12/15/2022]
Abstract
Artificial intelligence (AI) is widely discussed in the popular literature and is portrayed as impacting many aspects of human life, both in and out of the workplace. The potential for revolutionizing healthcare is significant because of the availability of increasingly powerful computational platforms and methods, along with increasingly informative sources of patient data, both in and out of clinical settings. This review aims to provide a realistic assessment of the potential for AI in understanding and managing diabetes, accounting for the state of the art in the methodology and medical devices that collect data, process data, and act accordingly. Acknowledging that many conflicting definitions of AI have been put forth, this article attempts to characterize the main elements of the field as they relate to diabetes, identifying the main perspectives and methods that can (i) affect basic understanding of the disease, (ii) affect understanding of risk factors (genetic, clinical, and behavioral) of diabetes development, (iii) improve diagnosis, (iv) improve understanding of the arc of disease (progression and personal/societal impact), and finally (v) improve treatment.
Collapse
Affiliation(s)
- Thibault Gautier
- Dexcom/TypeZero, 946 Grady Avenue, Suite 203, Charlottesville, VA 22903, United States of America.
| | - Leah B Ziegler
- Dexcom/TypeZero, 946 Grady Avenue, Suite 203, Charlottesville, VA 22903, United States of America
| | - Matthew S Gerber
- Dexcom/TypeZero, 946 Grady Avenue, Suite 203, Charlottesville, VA 22903, United States of America
| | - Enrique Campos-Náñez
- Dexcom/TypeZero, 946 Grady Avenue, Suite 203, Charlottesville, VA 22903, United States of America
| | - Stephen D Patek
- Dexcom/TypeZero, 946 Grady Avenue, Suite 203, Charlottesville, VA 22903, United States of America
| |
Collapse
|
8
|
Wie profitieren Menschen mit Diabetes von Big Data und künstlicher Intelligenz? DIABETOLOGE 2021. [DOI: 10.1007/s11428-021-00818-9] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/11/2022]
|
9
|
Abbasgholizadeh Rahimi S, Légaré F, Sharma G, Archambault P, Zomahoun HTV, Chandavong S, Rheault N, T Wong S, Langlois L, Couturier Y, Salmeron JL, Gagnon MP, Légaré J. Application of Artificial Intelligence in Community-Based Primary Health Care: Systematic Scoping Review and Critical Appraisal. J Med Internet Res 2021; 23:e29839. [PMID: 34477556 PMCID: PMC8449300 DOI: 10.2196/29839] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2021] [Revised: 05/29/2021] [Accepted: 05/31/2021] [Indexed: 12/27/2022] Open
Abstract
BACKGROUND Research on the integration of artificial intelligence (AI) into community-based primary health care (CBPHC) has highlighted several advantages and disadvantages in practice regarding, for example, facilitating diagnosis and disease management, as well as doubts concerning the unintended harmful effects of this integration. However, there is a lack of evidence about a comprehensive knowledge synthesis that could shed light on AI systems tested or implemented in CBPHC. OBJECTIVE We intended to identify and evaluate published studies that have tested or implemented AI in CBPHC settings. METHODS We conducted a systematic scoping review informed by an earlier study and the Joanna Briggs Institute (JBI) scoping review framework and reported the findings according to PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analysis-Scoping Reviews) reporting guidelines. An information specialist performed a comprehensive search from the date of inception until February 2020, in seven bibliographic databases: Cochrane Library, MEDLINE, EMBASE, Web of Science, Cumulative Index to Nursing and Allied Health Literature (CINAHL), ScienceDirect, and IEEE Xplore. The selected studies considered all populations who provide and receive care in CBPHC settings, AI interventions that had been implemented, tested, or both, and assessed outcomes related to patients, health care providers, or CBPHC systems. Risk of bias was assessed using the Prediction Model Risk of Bias Assessment Tool (PROBAST). Two authors independently screened the titles and abstracts of the identified records, read the selected full texts, and extracted data from the included studies using a validated extraction form. Disagreements were resolved by consensus, and if this was not possible, the opinion of a third reviewer was sought. A third reviewer also validated all the extracted data. RESULTS We retrieved 22,113 documents. After the removal of duplicates, 16,870 documents were screened, and 90 peer-reviewed publications met our inclusion criteria. Machine learning (ML) (41/90, 45%), natural language processing (NLP) (24/90, 27%), and expert systems (17/90, 19%) were the most commonly studied AI interventions. These were primarily implemented for diagnosis, detection, or surveillance purposes. Neural networks (ie, convolutional neural networks and abductive networks) demonstrated the highest accuracy, considering the given database for the given clinical task. The risk of bias in diagnosis or prognosis studies was the lowest in the participant category (4/49, 4%) and the highest in the outcome category (22/49, 45%). CONCLUSIONS We observed variabilities in reporting the participants, types of AI methods, analyses, and outcomes, and highlighted the large gap in the effective development and implementation of AI in CBPHC. Further studies are needed to efficiently guide the development and implementation of AI interventions in CBPHC settings.
Collapse
Affiliation(s)
- Samira Abbasgholizadeh Rahimi
- Department of Family Medicine, Faculty of Medicine and Health Sciences, McGill University, Montreal, QC, Canada.,Mila-Quebec AI Institute, Montreal, QC, Canada
| | - France Légaré
- Department of Family Medicine and Emergency Medicine, Université Laval, Quebec City, QC, Canada.,VITAM - Centre de recherche en santé durable, Université Laval, Quebec City, QC, Canada
| | - Gauri Sharma
- Faculty of Engineering, Dayalbagh Educational Institute, Agra, India
| | - Patrick Archambault
- Department of Family Medicine and Emergency Medicine, Université Laval, Quebec City, QC, Canada.,VITAM - Centre de recherche en santé durable, Université Laval, Quebec City, QC, Canada
| | - Herve Tchala Vignon Zomahoun
- VITAM - Centre de recherche en santé durable, Université Laval, Quebec City, QC, Canada.,Quebec SPOR-Support Unit, Quebec City, QC, Canada
| | - Sam Chandavong
- Faculty of Science and Engineering, Université Laval, Quebec City, QC, Canada
| | - Nathalie Rheault
- VITAM - Centre de recherche en santé durable, Université Laval, Quebec City, QC, Canada.,Quebec SPOR-Support Unit, Quebec City, QC, Canada
| | - Sabrina T Wong
- School of Nursing, University of British Columbia, Vancouver, BC, Canada.,Center for Health Services and Policy Research, University of British Columbia, Vancouver, BC, Canada
| | - Lyse Langlois
- Department of Industrial Relations, Université Laval, Quebec City, QC, Canada.,OBVIA - Quebec International Observatory on the social impacts of AI and digital technology, Quebec City, QC, Canada
| | - Yves Couturier
- School of Social Work, University of Sherbrooke, Sherbrooke, QC, Canada
| | - Jose L Salmeron
- Department of Data Science, University Pablo de Olavide, Seville, Spain
| | | | - Jean Légaré
- Arthritis Alliance of Canada, Montreal, QC, Canada
| |
Collapse
|
10
|
Alhassan Z, Watson M, Budgen D, Alshammari R, Alessa A, Al Moubayed N. Improving Current Glycated Hemoglobin Prediction in Adults: Use of Machine Learning Algorithms With Electronic Health Records. JMIR Med Inform 2021; 9:e25237. [PMID: 34028357 PMCID: PMC8185616 DOI: 10.2196/25237] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/23/2020] [Revised: 01/05/2021] [Accepted: 04/22/2021] [Indexed: 01/30/2023] Open
Abstract
Background Predicting the risk of glycated hemoglobin (HbA1c) elevation can help identify patients with the potential for developing serious chronic health problems, such as diabetes. Early preventive interventions based upon advanced predictive models using electronic health records data for identifying such patients can ultimately help provide better health outcomes. Objective Our study investigated the performance of predictive models to forecast HbA1c elevation levels by employing several machine learning models. We also examined the use of patient electronic health record longitudinal data in the performance of the predictive models. Explainable methods were employed to interpret the decisions made by the black box models. Methods This study employed multiple logistic regression, random forest, support vector machine, and logistic regression models, as well as a deep learning model (multilayer perceptron) to classify patients with normal (<5.7%) and elevated (≥5.7%) levels of HbA1c. We also integrated current visit data with historical (longitudinal) data from previous visits. Explainable machine learning methods were used to interrogate the models and provide an understanding of the reasons behind the decisions made by the models. All models were trained and tested using a large data set from Saudi Arabia with 18,844 unique patient records. Results The machine learning models achieved promising results for predicting current HbA1c elevation risk. When coupled with longitudinal data, the machine learning models outperformed the multiple logistic regression model used in the comparative study. The multilayer perceptron model achieved an accuracy of 83.22% for the area under receiver operating characteristic curve when used with historical data. All models showed a close level of agreement on the contribution of random blood sugar and age variables with and without longitudinal data. Conclusions This study shows that machine learning models can provide promising results for the task of predicting current HbA1c levels (≥5.7% or less). Using patients’ longitudinal data improved the performance and affected the relative importance for the predictors used. The models showed results that are consistent with comparable studies.
Collapse
Affiliation(s)
- Zakhriya Alhassan
- Department of Computer Science, Durham University, Durham, United Kingdom.,College of Computer Science and Engineering, University of Jeddah, Jeddah, Saudi Arabia
| | - Matthew Watson
- Department of Computer Science, Durham University, Durham, United Kingdom
| | - David Budgen
- Department of Computer Science, Durham University, Durham, United Kingdom
| | - Riyad Alshammari
- National Center for Artificial Intelligence, Saudi Data and Artificial Intelligence Authority, Riyadh, Saudi Arabia
| | - Ali Alessa
- Department of Information Technology Programs, Institute of Public Administration, Riyadh, Saudi Arabia
| | - Noura Al Moubayed
- Department of Computer Science, Durham University, Durham, United Kingdom
| |
Collapse
|
11
|
Ravaut M, Harish V, Sadeghi H, Leung KK, Volkovs M, Kornas K, Watson T, Poutanen T, Rosella LC. Development and Validation of a Machine Learning Model Using Administrative Health Data to Predict Onset of Type 2 Diabetes. JAMA Netw Open 2021; 4:e2111315. [PMID: 34032855 PMCID: PMC8150694 DOI: 10.1001/jamanetworkopen.2021.11315] [Citation(s) in RCA: 31] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/20/2020] [Accepted: 04/01/2021] [Indexed: 11/14/2022] Open
Abstract
Importance Systems-level barriers to diabetes care could be improved with population health planning tools that accurately discriminate between high- and low-risk groups to guide investments and targeted interventions. Objective To develop and validate a population-level machine learning model for predicting type 2 diabetes 5 years before diabetes onset using administrative health data. Design, Setting, and Participants This decision analytical model study used linked administrative health data from the diverse, single-payer health system in Ontario, Canada, between January 1, 2006, and December 31, 2016. A gradient boosting decision tree model was trained on data from 1 657 395 patients, validated on 243 442 patients, and tested on 236 506 patients. Costs associated with each patient were estimated using a validated costing algorithm. Data were analyzed from January 1, 2006, to December 31, 2016. Exposures A random sample of 2 137 343 residents of Ontario without type 2 diabetes was obtained at study start time. More than 300 features from data sets capturing demographic information, laboratory measurements, drug benefits, health care system interactions, social determinants of health, and ambulatory care and hospitalization records were compiled over 2-year patient medical histories to generate quarterly predictions. Main Outcomes and Measures Discrimination was assessed using the area under the receiver operating characteristic curve statistic, and calibration was assessed visually using calibration plots. Feature contribution was assessed with Shapley values. Costs were estimated in 2020 US dollars. Results This study trained a gradient boosting decision tree model on data from 1 657 395 patients (12 900 257 instances; 6 666 662 women [51.7%]). The developed model achieved a test area under the curve of 80.26 (range, 80.21-80.29), demonstrated good calibration, and was robust to sex, immigration status, area-level marginalization with regard to material deprivation and race/ethnicity, and low contact with the health care system. The top 5% of patients predicted as high risk by the model represented 26% of the total annual diabetes cost in Ontario. Conclusions and Relevance In this decision analytical model study, a machine learning model approach accurately predicted the incidence of diabetes in the population using routinely collected health administrative data. These results suggest that the model could be used to inform decision-making for population health planning and diabetes prevention.
Collapse
Affiliation(s)
- Mathieu Ravaut
- Layer 6 AI, Toronto, Ontario, Canada
- Department of Computer Science, University of Toronto, Toronto, Ontario, Canada
| | - Vinyas Harish
- Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada
- Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine, University of Toronto, Toronto, Ontario, Canada
- Vector Institute, Toronto, Ontario, Canada
| | | | | | | | - Kathy Kornas
- Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada
| | - Tristan Watson
- Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada
- Institute of Clinical Evaluative Sciences (ICES), Toronto, Ontario, Canada
| | | | - Laura C. Rosella
- Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada
- Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
- Temerty Centre for Artificial Intelligence Research and Education in Medicine, University of Toronto, Toronto, Ontario, Canada
- Vector Institute, Toronto, Ontario, Canada
- Institute of Clinical Evaluative Sciences (ICES), Toronto, Ontario, Canada
- Institute for Better Health, Trillium Health Partners, Mississauga, Ontario, Canada
| |
Collapse
|
12
|
Ravaut M, Sadeghi H, Leung KK, Volkovs M, Kornas K, Harish V, Watson T, Lewis GF, Weisman A, Poutanen T, Rosella L. Predicting adverse outcomes due to diabetes complications with machine learning using administrative health data. NPJ Digit Med 2021; 4:24. [PMID: 33580109 PMCID: PMC7881135 DOI: 10.1038/s41746-021-00394-8] [Citation(s) in RCA: 36] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2020] [Accepted: 01/11/2021] [Indexed: 02/07/2023] Open
Abstract
Across jurisdictions, government and health insurance providers hold a large amount of data from patient interactions with the healthcare system. We aimed to develop a machine learning-based model for predicting adverse outcomes due to diabetes complications using administrative health data from the single-payer health system in Ontario, Canada. A Gradient Boosting Decision Tree model was trained on data from 1,029,366 patients, validated on 272,864 patients, and tested on 265,406 patients. Discrimination was assessed using the AUC statistic and calibration was assessed visually using calibration plots overall and across population subgroups. Our model predicting three-year risk of adverse outcomes due to diabetes complications (hyper/hypoglycemia, tissue infection, retinopathy, cardiovascular events, amputation) included 700 features from multiple diverse data sources and had strong discrimination (average test AUC = 77.7, range 77.7-77.9). Through the design and validation of a high-performance model to predict diabetes complications adverse outcomes at the population level, we demonstrate the potential of machine learning and administrative health data to inform health planning and healthcare resource allocation for diabetes management.
Collapse
Affiliation(s)
- Mathieu Ravaut
- Layer 6 AI, Toronto, ON, Canada
- Department of Computer Science, University of Toronto, Toronto, ON, Canada
| | | | | | | | - Kathy Kornas
- Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
| | - Vinyas Harish
- Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
- MD/PhD Program, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
| | - Tristan Watson
- Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
- ICES, Toronto, ON, Canada
| | - Gary F Lewis
- Department of Medicine, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
- Department of Physiology, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
| | - Alanna Weisman
- Lunenfeld-Tanenbaum Research Institute, Mt. Sinai Hospital, Toronto, ON, Canada
- Division of Endocrinology and Metabolism, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
| | | | - Laura Rosella
- Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada.
- ICES, Toronto, ON, Canada.
- Vector Institute, Toronto, ON, Canada.
- Institute for Better Health, Trillium Health Partners, Mississauga, ON, Canada.
- Department of Laboratory Medicine & Pathology, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada.
| |
Collapse
|
13
|
Abstract
Machine learning shows enormous potential in facilitating decision-making regarding kidney diseases. With the development of data preservation and processing, as well as the advancement of machine learning algorithms, machine learning is expected to make remarkable breakthroughs in nephrology. Machine learning models have yielded many preliminaries to moderate and several excellent achievements in the fields, including analysis of renal pathological images, diagnosis and prognosis of chronic kidney diseases and acute kidney injury, as well as management of dialysis treatments. However, it is just scratching the surface of the field; at the same time, machine learning and its applications in renal diseases are facing a number of challenges. In this review, we discuss the application status, challenges and future prospects of machine learning in nephrology to help people further understand and improve the capacity for prediction, detection, and care quality in kidney diseases.
Collapse
|
14
|
Aim and Diabetes. Artif Intell Med 2021. [DOI: 10.1007/978-3-030-58080-3_158-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
15
|
Wu Y, Hu H, Cai J, Chen R, Zuo X, Cheng H, Yan D. Machine Learning for Predicting the 3-Year Risk of Incident Diabetes in Chinese Adults. Front Public Health 2021; 9:626331. [PMID: 34268283 PMCID: PMC8275929 DOI: 10.3389/fpubh.2021.626331] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2020] [Accepted: 05/21/2021] [Indexed: 02/05/2023] Open
Abstract
Purpose: We aimed to establish and validate a risk assessment system that combines demographic and clinical variables to predict the 3-year risk of incident diabetes in Chinese adults. Methods: A 3-year cohort study was performed on 15,928 Chinese adults without diabetes at baseline. All participants were randomly divided into a training set (n = 7,940) and a validation set (n = 7,988). XGBoost method is an effective machine learning technique used to select the most important variables from candidate variables. And we further established a stepwise model based on the predictors chosen by the XGBoost model. The area under the receiver operating characteristic curve (AUC), decision curve and calibration analysis were used to assess discrimination, clinical use and calibration of the model, respectively. The external validation was performed on a cohort of 11,113 Japanese participants. Result: In the training and validation sets, 148 and 145 incident diabetes cases occurred. XGBoost methods selected the 10 most important variables from 15 candidate variables. Fasting plasma glucose (FPG), body mass index (BMI) and age were the top 3 important variables. And we further established a stepwise model and a prediction nomogram. The AUCs of the stepwise model were 0.933 and 0.910 in the training and validation sets, respectively. The Hosmer-Lemeshow test showed a perfect fit between the predicted diabetes risk and the observed diabetes risk (p = 0.068 for the training set, p = 0.165 for the validation set). Decision curve analysis presented the clinical use of the stepwise model and there was a wide range of alternative threshold probability spectrum. And there were almost no the interactions between these predictors (most P-values for interaction >0.05). Furthermore, the AUC for the external validation set was 0.830, and the Hosmer-Lemeshow test for the external validation set showed no statistically significant difference between the predicted diabetes risk and observed diabetes risk (P = 0.824). Conclusion: We established and validated a risk assessment system for characterizing the 3-year risk of incident diabetes.
Collapse
Affiliation(s)
- Yang Wu
- Department of Endocrinology, The First Affiliated Hospital of Shenzhen University, Shenzhen, China
- Department of Endocrinology, Shenzhen Second People's Hospital, Shenzhen, China
- Shenzhen University Health Science Center, Shenzhen, China
| | - Haofei Hu
- Shenzhen University Health Science Center, Shenzhen, China
- Department of Nephrology, The First Affiliated Hospital of Shenzhen University, Shenzhen, China
- Department of Nephrology, Shenzhen Second People's Hospital, Shenzhen, China
| | - Jinlin Cai
- Department of Endocrinology, The First Affiliated Hospital of Shenzhen University, Shenzhen, China
- Department of Endocrinology, Shenzhen Second People's Hospital, Shenzhen, China
- Shantou University Medical College, Shantou, China
| | - Runtian Chen
- Department of Endocrinology, The First Affiliated Hospital of Shenzhen University, Shenzhen, China
- Department of Endocrinology, Shenzhen Second People's Hospital, Shenzhen, China
- Shenzhen University Health Science Center, Shenzhen, China
| | - Xin Zuo
- Department of Endocrinology, The Third People's Hospital of Shenzhen, Shenzhen, China
| | - Heng Cheng
- Department of Endocrinology, The Third People's Hospital of Shenzhen, Shenzhen, China
| | - Dewen Yan
- Department of Endocrinology, The First Affiliated Hospital of Shenzhen University, Shenzhen, China
- Department of Endocrinology, Shenzhen Second People's Hospital, Shenzhen, China
- Shenzhen University Health Science Center, Shenzhen, China
- *Correspondence: Dewen Yan
| |
Collapse
|
16
|
Cascini F, De Giovanni N, Inserra I, Santaroni F, Laura L. A data-driven methodology to discover similarities between cocaine samples. Sci Rep 2020; 10:15976. [PMID: 32994485 PMCID: PMC7525495 DOI: 10.1038/s41598-020-72652-w] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2020] [Accepted: 08/27/2020] [Indexed: 11/10/2022] Open
Abstract
Machine learning has been used for distinct purposes in the science field but no applications on illegal drug have been done before. This study proposes a new web-based system for cocaine classification, profiling relations and comparison, that is capable of producing meaningful output based on a large amount of chemical profiling’s data. In particular, the Profiling Relations In Drug trafficking in Europe (PRIDE) system, offers several advantages to intelligence actions across Europe. Thus, it provides a standardized, broad methodology which uses machine learning algorithms to classify and compare drug profiles, highlight how similar drug samples are, and how probable it is that they share a common origin, batch, or preparation process. We evaluated the proposed algorithms using precision and recall metrics and analyzed the quality of predictions performed by the algorithms, with respect to our gold standard. In our experiments, we reached a value of 88% for F0.5-measure, 91% for precision, and 78% for recall, confirming our main hypothesis: machine learning can learn and be applied to have an automatic classification of cocaine profiles.
Collapse
Affiliation(s)
- Fidelia Cascini
- Department of Life Sciences and Public Health, Università Cattolica del Sacro Cuore, 00168, Rome, Italy.
| | - Nadia De Giovanni
- Fondazione Policlinico Agostino Gemelli IRCCS, Largo Agostino Gemelli 8, 00168, Rome, Italy
| | - Ilaria Inserra
- Department of Life Sciences and Public Health, Università Cattolica del Sacro Cuore, 00168, Rome, Italy
| | - Federico Santaroni
- Department of Computer, Control, and Management Engineering Antonio Ruberti (DIAG), Sapienza University of Rome, 00186, Rome, Italy
| | - Luigi Laura
- International Telematic University Uninettuno of Rome, Rome, Italy
| |
Collapse
|
17
|
Ljubic B, Hai AA, Stanojevic M, Diaz W, Polimac D, Pavlovski M, Obradovic Z. Predicting complications of diabetes mellitus using advanced machine learning algorithms. J Am Med Inform Assoc 2020; 27:1343-1351. [PMID: 32869093 PMCID: PMC7647294 DOI: 10.1093/jamia/ocaa120] [Citation(s) in RCA: 25] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2020] [Revised: 05/17/2020] [Accepted: 05/28/2020] [Indexed: 12/14/2022] Open
Abstract
OBJECTIVE We sought to predict if patients with type 2 diabetes mellitus (DM2) would develop 10 selected complications. Accurate prediction of complications could help with more targeted measures that would prevent or slow down their development. MATERIALS AND METHODS Experiments were conducted on the Healthcare Cost and Utilization Project State Inpatient Databases of California for the period of 2003 to 2011. Recurrent neural network (RNN) long short-term memory (LSTM) and RNN gated recurrent unit (GRU) deep learning methods were designed and compared with random forest and multilayer perceptron traditional models. Prediction accuracy of selected complications were compared on 3 settings corresponding to minimum number of hospitalizations between diabetes diagnosis and the diagnosis of complications. RESULTS The diagnosis domain was used for experiments. The best results were achieved with RNN GRU model, followed by RNN LSTM model. The prediction accuracy achieved with RNN GRU model was between 73% (myocardial infarction) and 83% (chronic ischemic heart disease), while accuracy of traditional models was between 66% - 76%. DISCUSSION The number of hospitalizations was an important factor for the prediction accuracy. Experiments with 4 hospitalizations achieved significantly better accuracy than with 2 hospitalizations. To achieve improved accuracy deep learning models required training on at least 1000 patients and accuracy significantly dropped if training datasets contained 500 patients. The prediction accuracy of complications decreases over time period. Considering individual complications, the best accuracy was achieved on depressive disorder and chronic ischemic heart disease. CONCLUSIONS The RNN GRU model was the best choice for electronic medical record type of data, based on the achieved results.
Collapse
Affiliation(s)
- Branimir Ljubic
- Center for Data Analytics and Biomedical Informatics, Temple University, Philadelphia, Pennsylvania, USA
| | - Ameen Abdel Hai
- Center for Data Analytics and Biomedical Informatics, Temple University, Philadelphia, Pennsylvania, USA
| | - Marija Stanojevic
- Center for Data Analytics and Biomedical Informatics, Temple University, Philadelphia, Pennsylvania, USA
| | - Wilson Diaz
- Center for Data Analytics and Biomedical Informatics, Temple University, Philadelphia, Pennsylvania, USA
| | - Daniel Polimac
- Center for Data Analytics and Biomedical Informatics, Temple University, Philadelphia, Pennsylvania, USA
| | - Martin Pavlovski
- Center for Data Analytics and Biomedical Informatics, Temple University, Philadelphia, Pennsylvania, USA
| | - Zoran Obradovic
- Center for Data Analytics and Biomedical Informatics, Temple University, Philadelphia, Pennsylvania, USA
| |
Collapse
|