1
|
Crump RT, Mohammed E, Biglarbeiki M, Eshragh M, Shakeri E, Siljedal GJ, Far B, Weis E. Artificial intelligence in the classification and segmentation of fundus images with choroidal nevi. CANADIAN JOURNAL OF OPHTHALMOLOGY 2024:S0008-4182(24)00211-4. [PMID: 39151894 DOI: 10.1016/j.jcjo.2024.07.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/15/2024] [Revised: 05/24/2024] [Accepted: 07/02/2024] [Indexed: 08/19/2024]
Abstract
OBJECTIVE The purpose of this study is to summarize the results from 3 experimental studies into the use of artificial intelligence to classify and segment colour fundus images with choroidal nevi. STUDY DESIGN This study is based on a secondary analysis of colour fundus images taken of patients receiving usual clinical care from the Alberta Ocular Brachytherapy Program. METHODS High-resolution colour fundus images were labeled by experienced ocular oncologists. In experimental study 1, four pre-trained models (ResNet 50, VGG-19, VGG-16, and AlexNet) were evaluated for their ability to classify images based on the presence of choroidal nevi. In experimental study 2, the performance of 3 patch-based models to classify images based on the presence of choroidal nevi were compared. In experimental study 3, four convolutional neural network models were developed to segment the images. In experimental studies 1 and 2, performance was measured using accuracy, precision, recall, F1 score, and AUC. In experimental study 3, IoU and Dice measures were used to evaluate performance. RESULTS A total of 591 labelled colour fundus images were used for analysis. In experimental study 1, VGG-16 showed the best accuracy, AUC, and recall, but lower precision in classifying images. In experimental study 2, the patched approached enhanced with artifact and contrast outperformed the others in classifying images. In experimental study 3, a voting-based Ensemble model excelled in segmenting the part of images with nevi. CONCLUSIONS It is feasible to train AI models to identify choroidal nevi in colour fundus images.
Collapse
Affiliation(s)
- R Trafford Crump
- Cumming School of Medicine, University of Calgary, Calgary, AB; Schulich School of Engineering, University of Calgary, Calgary, AB.
| | - Emad Mohammed
- Faculty of Science, Wilfrid Laurier University, Waterloo, ON
| | | | | | - Esmaeil Shakeri
- Schulich School of Engineering, University of Calgary, Calgary, AB
| | | | - Behrouz Far
- Schulich School of Engineering, University of Calgary, Calgary, AB
| | - Ezekiel Weis
- Cumming School of Medicine, University of Calgary, Calgary, AB; Faculty of Medicine and Dentistry, University of Alberta, Edmonton, AB
| |
Collapse
|
2
|
Salih AM, Galazzo IB, Gkontra P, Rauseo E, Lee AM, Lekadir K, Radeva P, Petersen SE, Menegaz G. A review of evaluation approaches for explainable AI with applications in cardiology. Artif Intell Rev 2024; 57:240. [PMID: 39132011 PMCID: PMC11315784 DOI: 10.1007/s10462-024-10852-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/03/2024] [Indexed: 08/13/2024]
Abstract
Explainable artificial intelligence (XAI) elucidates the decision-making process of complex AI models and is important in building trust in model predictions. XAI explanations themselves require evaluation as to accuracy and reasonableness and in the context of use of the underlying AI model. This review details the evaluation of XAI in cardiac AI applications and has found that, of the studies examined, 37% evaluated XAI quality using literature results, 11% used clinicians as domain-experts, 11% used proxies or statistical analysis, with the remaining 43% not assessing the XAI used at all. We aim to inspire additional studies within healthcare, urging researchers not only to apply XAI methods but to systematically assess the resulting explanations, as a step towards developing trustworthy and safe models. Supplementary Information The online version contains supplementary material available at 10.1007/s10462-024-10852-w.
Collapse
Affiliation(s)
- Ahmed M. Salih
- William Harvey Research Institute, NIHR Barts Biomedical Research Centre, Queen Mary University of London, Charterhouse Square, London, EC1M 6BQ UK
- Department of Population Health Sciences, University of Leicester, University Rd, Leicester, LE1 7RH UK
- Department of Computer Science, University of Zakho, Duhok road, Zakho, Kurdistan Iraq
| | - Ilaria Boscolo Galazzo
- Department of Engineering for Innovative Medicine, University of Verona, S. Francesco, 22, 37129 Verona, Italy
| | - Polyxeni Gkontra
- Artificial Intelligence in Medicine Lab (BCN-AIM), Departament de Matemàtiques i Informàtica, Universitat de Barcelona, Gran Via de les Corts Catalanes, 585, 08007 Barcelona, Spain
| | - Elisa Rauseo
- William Harvey Research Institute, NIHR Barts Biomedical Research Centre, Queen Mary University of London, Charterhouse Square, London, EC1M 6BQ UK
| | - Aaron Mark Lee
- William Harvey Research Institute, NIHR Barts Biomedical Research Centre, Queen Mary University of London, Charterhouse Square, London, EC1M 6BQ UK
| | - Karim Lekadir
- Artificial Intelligence in Medicine Lab (BCN-AIM), Departament de Matemàtiques i Informàtica, Universitat de Barcelona, Gran Via de les Corts Catalanes, 585, 08007 Barcelona, Spain
- Institució Catalana de Recerca i Estudis Avançats (ICREA), Passeig Lluís Companys 23, Barcelona, Spain
| | - Petia Radeva
- Departament de Matemàtiques i Informàtica, Universitat de Barcelona, Gran Via de les Corts Catalanes, 585, 08007 Barcelona, Spain
| | - Steffen E. Petersen
- William Harvey Research Institute, NIHR Barts Biomedical Research Centre, Queen Mary University of London, Charterhouse Square, London, EC1M 6BQ UK
- Barts Heart Centre, St Bartholomew’s Hospital, Barts Health NHS Trust, West Smithfield, London, UK
- Health Data Research, London, UK
- Alan Turing Institute, London, UK
| | - Gloria Menegaz
- Department of Engineering for Innovative Medicine, University of Verona, S. Francesco, 22, 37129 Verona, Italy
| |
Collapse
|
3
|
Young DL, Hannum SM, Engels R, Colantuoni E, Friedman LA, Hoyer EH. Dynamic Prediction of Post-Acute Care Needs for Hospitalized Medicine Patients. J Am Med Dir Assoc 2024; 25:104939. [PMID: 38387858 DOI: 10.1016/j.jamda.2024.01.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2023] [Revised: 10/05/2023] [Accepted: 01/10/2024] [Indexed: 02/24/2024]
Abstract
OBJECTIVES Use patient demographic and clinical characteristics at admission and time-varying in-hospital measures of patient mobility to predict patient post-acute care (PAC) discharge. DESIGN Retrospective cohort analysis of electronic medical records. SETTING AND PARTICIPANTS Patients admitted to the two participating Hospitals from November 2016 through December 2019 with ≥72 hours in a general medicine service. METHODS Discharge location (PAC vs home) was the primary outcome, and 2 time-varying measures of patient mobility, Activity Measure for Post-Acute Care (AM-PAC) Mobility "6-clicks" and Johns Hopkins Highest Level of Mobility, were the primary predictors. Other predictors included demographic and clinical characteristics. For each day of hospitalization, we predicted discharge to PAC using the demographic and clinical characteristics and most recent mobility data within a random forest (RF) for survival, longitudinal, and multivariate (RF-SLAM) data. A regression tree for the daily predicted probabilities of discharge to PAC was constructed to represent a global summary of the RF. RESULTS There were 23,090 total patients and compared to PAC, those discharged home were younger (64 vs 71), had shorter length of stay (5 vs 8 days), higher AM-PAC at admission (43 vs 32), and average AM-PAC throughout hospitalization (45 vs 35). AM-PAC was the most important predictor, followed by age, and whether the patient lives alone. The area under the hospital day-specific receiver operating characteristic curve ranged from 0.76 to 0.79 during the first 5 days. The global summary tree explained 75% of the variation in predicted probabilities for PAC from the RF. Sensitivity (75%), specificity (70%), and accuracy (72%) were maximized at a PAC probability threshold of 40%. CONCLUSIONS AND IMPLICATIONS Daily assessment of patient mobility should be part of routine practice to help inform care planning by hospital teams. Our prediction model could be used as a valuable tool by multidisciplinary teams in the discharge planning process.
Collapse
Affiliation(s)
- Daniel L Young
- Department of Physical Therapy, University of Nevada, Las Vegas, Las Vegas, NV, USA; Department of Physical Medicine and Rehabilitation, Johns Hopkins University, Baltimore, MD, USA.
| | - Susan M Hannum
- Department of Health, Behavior and Society, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
| | - Rebecca Engels
- Division of Hospital Medicine, Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Elizabeth Colantuoni
- Department of Biostatistics, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, USA
| | - Lisa Aronson Friedman
- Division of Pulmonary and Critical Care Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA; Outcomes After Critical Illness and Surgery (OACIS) Group, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Erik H Hoyer
- Department of Physical Medicine and Rehabilitation, Johns Hopkins University, Baltimore, MD, USA; Division of Hospital Medicine, Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| |
Collapse
|
4
|
Wongvibulsin S, Adamson AS. Deep learning for Mpox: Advances, challenges, and opportunities. MED 2023; 4:283-284. [PMID: 37178679 PMCID: PMC10176662 DOI: 10.1016/j.medj.2023.04.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2023] [Revised: 04/10/2023] [Accepted: 04/10/2023] [Indexed: 05/15/2023]
Abstract
Although deep-learning algorithms in dermatology have shown promise in diagnosing skin cancers, less is known about potential applications for the diagnosis of infectious diseases. In a recent publication in Nature Medicine, Thieme et al. develop a deep-learning algorithm to classify skin lesions from Mpox virus (MPXV) infections.1.
Collapse
Affiliation(s)
- Shannon Wongvibulsin
- Division of Dermatology, David Geffen School of Medicine at the University of California, Los Angeles, CA, USA
| | - Adewole S Adamson
- Division of Dermatology, Dell Medical School at the University of Texas at Austin, Austin, TX, USA.
| |
Collapse
|
5
|
Chen R, Fu Y, Yi X, Pei Q, Zai H, Chen BT. Application of Radiomics in Predicting Treatment Response to Neoadjuvant Chemoradiotherapy in Locally Advanced Rectal Cancer: Strategies and Challenges. JOURNAL OF ONCOLOGY 2022; 2022:1590620. [PMID: 36471884 PMCID: PMC9719428 DOI: 10.1155/2022/1590620] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/20/2022] [Revised: 10/30/2022] [Accepted: 11/09/2022] [Indexed: 08/01/2023]
Abstract
Neoadjuvant chemoradiotherapy (nCRT) followed by total mesorectal excision is the standard treatment for locally advanced rectal cancer (LARC). A noninvasive preoperative prediction method should greatly assist in the evaluation of response to nCRT and for the development of a personalized strategy for patients with LARC. Assessment of nCRT relies on imaging and radiomics can extract valuable quantitative data from medical images. In this review, we examined the status of radiomic application for assessing response to nCRT in patients with LARC and indicated a potential direction for future research.
Collapse
Affiliation(s)
- Rui Chen
- Department of Radiology, Xiangya Hospital, Central South University, Changsha 410008, Hunan, China
| | - Yan Fu
- Department of Radiology, Xiangya Hospital, Central South University, Changsha 410008, Hunan, China
| | - Xiaoping Yi
- Department of Radiology, Xiangya Hospital, Central South University, Changsha 410008, Hunan, China
| | - Qian Pei
- Department of Radiology, Xiangya Hospital, Central South University, Changsha 410008, Hunan, China
| | - Hongyan Zai
- Department of Radiology, Xiangya Hospital, Central South University, Changsha 410008, Hunan, China
| | - Bihong T. Chen
- Department of Diagnostic Radiology, City of Hope National Medical Center, Duarte, CA, USA
| |
Collapse
|
6
|
Loch AA, Lopes-Rocha AC, Ara A, Gondim JM, Cecchi GA, Corcoran CM, Mota NB, Argolo FC. Ethical Implications of the Use of Language Analysis Technologies for the Diagnosis and Prediction of Psychiatric Disorders. JMIR Ment Health 2022; 9:e41014. [PMID: 36318266 PMCID: PMC9667377 DOI: 10.2196/41014] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/13/2022] [Revised: 09/09/2022] [Accepted: 10/04/2022] [Indexed: 11/05/2022] Open
Abstract
Recent developments in artificial intelligence technologies have come to a point where machine learning algorithms can infer mental status based on someone's photos and texts posted on social media. More than that, these algorithms are able to predict, with a reasonable degree of accuracy, future mental illness. They potentially represent an important advance in mental health care for preventive and early diagnosis initiatives, and for aiding professionals in the follow-up and prognosis of their patients. However, important issues call for major caution in the use of such technologies, namely, privacy and the stigma related to mental disorders. In this paper, we discuss the bioethical implications of using such technologies to diagnose and predict future mental illness, given the current scenario of swiftly growing technologies that analyze human language and the online availability of personal information given by social media. We also suggest future directions to be taken to minimize the misuse of such important technologies.
Collapse
Affiliation(s)
- Alexandre Andrade Loch
- Institute of Psychiatry, University of Sao Paulo, Sao Paulo, Brazil.,Instituto Nacional de Biomarcadores em Neuropsiquiatria, Conselho Nacional de Desenvolvimento Científico e Tecnológico, Brazilia, Brazil
| | | | - Anderson Ara
- Departamento de Estatística, Universidade Federal do Paraná, Curitiba, Brazil
| | | | - Guillermo A Cecchi
- IBM Thomas J. Watson Research Center, Yorktown Heights, NY, United States
| | | | - Natália Bezerra Mota
- Instituto de Psiquiatria, Universidade Federal do Rio de Janeiro, Rio de Janeiro, Brazil.,Research Department at Motrix Lab, Motrix, Rio de Janeiro, Brazil
| | - Felipe C Argolo
- Institute of Psychiatry, University of Sao Paulo, Sao Paulo, Brazil
| |
Collapse
|
7
|
Chen H, Gomez C, Huang CM, Unberath M. Explainable medical imaging AI needs human-centered design: guidelines and evidence from a systematic review. NPJ Digit Med 2022; 5:156. [PMID: 36261476 PMCID: PMC9581990 DOI: 10.1038/s41746-022-00699-2] [Citation(s) in RCA: 34] [Impact Index Per Article: 17.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2022] [Accepted: 09/29/2022] [Indexed: 11/16/2022] Open
Abstract
Transparency in Machine Learning (ML), often also referred to as interpretability or explainability, attempts to reveal the working mechanisms of complex models. From a human-centered design perspective, transparency is not a property of the ML model but an affordance, i.e., a relationship between algorithm and users. Thus, prototyping and user evaluations are critical to attaining solutions that afford transparency. Following human-centered design principles in highly specialized and high stakes domains, such as medical image analysis, is challenging due to the limited access to end users and the knowledge imbalance between those users and ML designers. To investigate the state of transparent ML in medical image analysis, we conducted a systematic review of the literature from 2012 to 2021 in PubMed, EMBASE, and Compendex databases. We identified 2508 records and 68 articles met the inclusion criteria. Current techniques in transparent ML are dominated by computational feasibility and barely consider end users, e.g. clinical stakeholders. Despite the different roles and knowledge of ML developers and end users, no study reported formative user research to inform the design and development of transparent ML models. Only a few studies validated transparency claims through empirical user evaluations. These shortcomings put contemporary research on transparent ML at risk of being incomprehensible to users, and thus, clinically irrelevant. To alleviate these shortcomings in forthcoming research, we introduce the INTRPRT guideline, a design directive for transparent ML systems in medical image analysis. The INTRPRT guideline suggests human-centered design principles, recommending formative user research as the first step to understand user needs and domain requirements. Following these guidelines increases the likelihood that the algorithms afford transparency and enable stakeholders to capitalize on the benefits of transparent ML.
Collapse
Affiliation(s)
- Haomin Chen
- Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA
| | - Catalina Gomez
- Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA
| | - Chien-Ming Huang
- Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA
| | - Mathias Unberath
- Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA.
| |
Collapse
|
8
|
Polo Friz H, Esposito V, Marano G, Primitz L, Bovio A, Delgrossi G, Bombelli M, Grignaffini G, Monza G, Boracchi P. Machine learning and LACE index for predicting 30-day readmissions after heart failure hospitalization in elderly patients. Intern Emerg Med 2022; 17:1727-1737. [PMID: 35661313 DOI: 10.1007/s11739-022-02996-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/04/2021] [Accepted: 04/20/2022] [Indexed: 11/05/2022]
Abstract
Machine learning (ML) techniques may improve readmission prediction performance in heart failure (HF) patients. This study aimed to assess the ability of ML algorithms to predict unplanned all-cause 30-day readmissions in HF elderly patients, and to compare them with conventional LACE (Length of hospitalization, Acuity, Comorbidities, Emergency department visits) index. All patients aged ≥ 65 years discharged alive between 2010 and 2019 after a hospitalization for acute HF were included in this retrospective cohort study. We applied MICE (Multivariate Imputation via Chained Equations) method to obtain a balanced, fully valued dataset and LASSO (Least Absolute Shrinkage and Selection Operator) algorithm to get the most significant features. Training (80% of records) and test (20%) cohorts were randomly selected. Study population: 3079 patients, 394 (12.8%) presented at least one readmission within 30 days, and 2685 (87.2%) did not. In the test cohort AUCs (IC95%) of XGBoost, Ada Boost Classifier, Random forest, and Gradient Boosting, and LACE Index were: 0.803 (0.734-0.872), 0.782 (0.711-0.854), 0.776 (0.703-0.848), 0.786 (0.715-0.857), and 0.504 (0.414-0.594), respectively, for predicting readmissions. A SHAP analysis was performed to offer a breakdown of the ML variables associated with readmission. Positive and negative predicting values estimates of the different ML models and LACE index were also provided, for several values of readmission rate prevalence. Among elderly patients, the rate of all-cause unplanned 30-day readmissions after hospitalization due to an acute HF was high. ML models performed better than the conventional LACE index for predicting readmissions. ML models can be proposed as promising tools for the identification of subjects at high risk of hospitalization in this clinical setting, enabling care teams to target interventions for improving overall clinical outcomes.
Collapse
Affiliation(s)
- Hernan Polo Friz
- Internal Medicine, Medical Department, Vimercate Hospital, Azienda Socio Sanitaria Territoriale (ASST) della Brianza, Via Santi Cosma e Damiano 10, 20871, Vimercate, MB, Italy.
| | | | - Giuseppe Marano
- Department of Biomedical and Clinical Sciences "L. Sacco", University of Milan, Milan, Italy
| | - Laura Primitz
- Internal Medicine, Medical Department, Vimercate Hospital, Azienda Socio Sanitaria Territoriale (ASST) della Brianza, Via Santi Cosma e Damiano 10, 20871, Vimercate, MB, Italy
| | | | | | - Michele Bombelli
- Internal Medicine, Medical Department, Desio Hospital, ASST della Brianza, Desio, Italy
| | - Guido Grignaffini
- Director for Health and Social Care, ASST della Brianza, Vimercate, Italy
| | | | - Patrizia Boracchi
- Department of Biomedical and Clinical Sciences "L. Sacco", University of Milan, Milan, Italy
| |
Collapse
|
9
|
Constantinescu G, Schulze M, Peitzsch M, Hofmockel T, Scholl UI, Williams TA, Lenders JW, Eisenhofer G. Integration of artificial intelligence and plasma steroidomics with laboratory information management systems: application to primary aldosteronism. Clin Chem Lab Med 2022; 60:1929-1937. [DOI: 10.1515/cclm-2022-0470] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2022] [Accepted: 06/28/2022] [Indexed: 12/11/2022]
Abstract
Abstract
Objectives
Mass spectrometry-based steroidomics combined with machine learning (ML) provides a potentially powerful approach in endocrine diagnostics, but is hampered by limitations in the conveyance of results and interpretations to clinicians. We address this shortcoming by integration of the two technologies with a laboratory information management systems (LIMS) model.
Methods
The approach involves integration of ML algorithm-derived models with commercially available mathematical programming software and a web-based LIMS prototype. To illustrate clinical utility, the process was applied to plasma steroidomics data from 22 patients tested for primary aldosteronism (PA).
Results
Once mass spectrometry data are uploaded into the system, automated processes enable generation of interpretations of steroid profiles from ML models. Generated reports include plasma concentrations of steroids in relation to age- and sex-specific reference intervals along with results of ML models and narrative interpretations that cover probabilities of PA. If PA is predicted, reports include probabilities of unilateral disease and mutations of KCNJ5 known to be associated with successful outcomes of adrenalectomy. Preliminary results, with no overlap in probabilities of disease among four patients with and 18 without PA and correct classification of all four patients with unilateral PA including three of four with KCNJ5 mutations, illustrate potential utility of the approach to guide diagnosis and subtyping of patients with PA.
Conclusions
The outlined process for integrating plasma steroidomics data and ML with LIMS may facilitate improved diagnostic-decision-making when based on higher-dimensional data otherwise difficult to interpret. The approach is relevant to other diagnostic applications involving ML.
Collapse
Affiliation(s)
- Georgiana Constantinescu
- Department of Internal Medicine III , University Hospital “Carl Gustav Carus”, Technische Universität Dresden , Dresden , Germany
- Grigore T. Popa University of Medicine and Pharmacy , Iasi , Romania
| | - Manuel Schulze
- Department of Distributed and Data Intensive Computing , Center for Information Services and High Performance Computing (ZIH), Technische Universität Dresden , Dresden , Germany
| | - Mirko Peitzsch
- Institute of Clinical Chemistry and Laboratory Medicine, University Hospital “Carl Gustav Carus”, Technische Universität Dresden , Dresden , Germany
| | - Thomas Hofmockel
- Department of Radiology , University Hospital “Carl Gustav Carus”, Technische Universität Dresden , Dresden , Germany
| | - Ute I. Scholl
- Berlin Institute of Health at Charité – Universitätsmedizin Berlin, Center of Functional Genomics , Berlin , Germany
| | - Tracy Ann Williams
- Medizinische Klinik und Poliklinik IV, Klinikum der Universität, Ludwig-Maximilians-Universität München , Munich , Germany
- Department of Medical Sciences, Division of Internal Medicine and Hypertension , University of Turin , Turin , Italy
| | - Jacques W.M. Lenders
- Department of Internal Medicine III , University Hospital “Carl Gustav Carus”, Technische Universität Dresden , Dresden , Germany
- Department of Internal Medicine , Radboud University Medical Centre , Nijmegen , The Netherlands
| | - Graeme Eisenhofer
- Department of Internal Medicine III , University Hospital “Carl Gustav Carus”, Technische Universität Dresden , Dresden , Germany
- Institute of Clinical Chemistry and Laboratory Medicine, University Hospital “Carl Gustav Carus”, Technische Universität Dresden , Dresden , Germany
| |
Collapse
|
10
|
Gama F, Tyskbo D, Nygren J, Barlow J, Reed J, Svedberg P. Implementation Frameworks for Artificial Intelligence Translation Into Health Care Practice: Scoping Review. J Med Internet Res 2022; 24:e32215. [PMID: 35084349 PMCID: PMC8832266 DOI: 10.2196/32215] [Citation(s) in RCA: 40] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2021] [Revised: 09/02/2021] [Accepted: 12/27/2021] [Indexed: 01/22/2023] Open
Abstract
Background Significant efforts have been made to develop artificial intelligence (AI) solutions for health care improvement. Despite the enthusiasm, health care professionals still struggle to implement AI in their daily practice. Objective This paper aims to identify the implementation frameworks used to understand the application of AI in health care practice. Methods A scoping review was conducted using the Cochrane, Evidence Based Medicine Reviews, Embase, MEDLINE, and PsycINFO databases to identify publications that reported frameworks, models, and theories concerning AI implementation in health care. This review focused on studies published in English and investigating AI implementation in health care since 2000. A total of 2541 unique publications were retrieved from the databases and screened on titles and abstracts by 2 independent reviewers. Selected articles were thematically analyzed against the Nilsen taxonomy of implementation frameworks, and the Greenhalgh framework for the nonadoption, abandonment, scale-up, spread, and sustainability (NASSS) of health care technologies. Results In total, 7 articles met all eligibility criteria for inclusion in the review, and 2 articles included formal frameworks that directly addressed AI implementation, whereas the other articles provided limited descriptions of elements influencing implementation. Collectively, the 7 articles identified elements that aligned with all the NASSS domains, but no single article comprehensively considered the factors known to influence technology implementation. New domains were identified, including dependency on data input and existing processes, shared decision-making, the role of human oversight, and ethics of population impact and inequality, suggesting that existing frameworks do not fully consider the unique needs of AI implementation. Conclusions This literature review demonstrates that understanding how to implement AI in health care practice is still in its early stages of development. Our findings suggest that further research is needed to provide the knowledge necessary to develop implementation frameworks to guide the future implementation of AI in clinical practice and highlight the opportunity to draw on existing knowledge from the field of implementation science.
Collapse
Affiliation(s)
- Fábio Gama
- School of Business, Innovation and Sustainability, Halmstad University, Halmstad, Sweden.,School of Administration and Economic Science, Santa Catarina State University, Florianópolis, Brazil
| | - Daniel Tyskbo
- School of Health and Welfare, Halmstad University, Halmstad, Sweden
| | - Jens Nygren
- School of Health and Welfare, Halmstad University, Halmstad, Sweden
| | - James Barlow
- Centre for Health Economics and Policy Innovation, Imperial College Business School, London, United Kingdom
| | - Julie Reed
- School of Health and Welfare, Halmstad University, Halmstad, Sweden
| | - Petra Svedberg
- School of Health and Welfare, Halmstad University, Halmstad, Sweden
| |
Collapse
|
11
|
Ibrahim ZM, Bean D, Searle T, Qian L, Wu H, Shek A, Kraljevic Z, Galloway J, Norton S, Teo JT, Dobson RJ. A Knowledge Distillation Ensemble Framework for Predicting Short- and Long-Term Hospitalization Outcomes From Electronic Health Records Data. IEEE J Biomed Health Inform 2022; 26:423-435. [PMID: 34129509 DOI: 10.1109/jbhi.2021.3089287] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
The ability to perform accurate prognosis is crucial for proactive clinical decision making, informed resource management and personalised care. Existing outcome prediction models suffer from a low recall of infrequent positive outcomes. We present a highly-scalable and robust machine learning framework to automatically predict adversity represented by mortality and ICU admission and readmission from time-series of vital signs and laboratory results obtained within the first 24 hours of hospital admission. The stacked ensemble platform comprises two components: a) an unsupervised LSTM Autoencoder that learns an optimal representation of the time-series, using it to differentiate the less frequent patterns which conclude with an adverse event from the majority patterns that do not, and b) a gradient boosting model, which relies on the constructed representation to refine prediction by incorporating static features. The model is used to assess a patient's risk of adversity and provides visual justifications of its prediction. Results of three case studies show that the model outperforms existing platforms in ICU and general ward settings, achieving average Precision-Recall Areas Under the Curve (PR-AUCs) of 0.891 (95% CI: 0.878-0.939) for mortality and 0.908 (95% CI: 0.870-0.935) in predicting ICU admission and readmission.
Collapse
|
12
|
Wongvibulsin S, Garibaldi BT, Antar AAR, Wen J, Wang MC, Gupta A, Bollinger R, Xu Y, Wang K, Betz JF, Muschelli J, Bandeen-Roche K, Zeger SL, Robinson ML. Development of Severe COVID-19 Adaptive Risk Predictor (SCARP), a Calculator to Predict Severe Disease or Death in Hospitalized Patients With COVID-19. Ann Intern Med 2021; 174:777-785. [PMID: 33646849 PMCID: PMC7934337 DOI: 10.7326/m20-6754] [Citation(s) in RCA: 31] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
BACKGROUND Predicting the clinical trajectory of individual patients hospitalized with coronavirus disease 2019 (COVID-19) is challenging but necessary to inform clinical care. The majority of COVID-19 prognostic tools use only data present upon admission and do not incorporate changes occurring after admission. OBJECTIVE To develop the Severe COVID-19 Adaptive Risk Predictor (SCARP) (https://rsconnect.biostat.jhsph.edu/covid_trajectory/), a novel tool that can provide dynamic risk predictions for progression from moderate disease to severe illness or death in patients with COVID-19 at any time within the first 14 days of their hospitalization. DESIGN Retrospective observational cohort study. SETTINGS Five hospitals in Maryland and Washington, D.C. PATIENTS Patients who were hospitalized between 5 March and 4 December 2020 with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) confirmed by nucleic acid test and symptomatic disease. MEASUREMENTS A clinical registry for patients hospitalized with COVID-19 was the primary data source; data included demographic characteristics, admission source, comorbid conditions, time-varying vital signs, laboratory measurements, and clinical severity. Random forest for survival, longitudinal, and multivariate (RF-SLAM) data analysis was applied to predict the 1-day and 7-day risks for progression to severe disease or death for any given day during the first 14 days of hospitalization. RESULTS Among 3163 patients admitted with moderate COVID-19, 228 (7%) became severely ill or died in the next 24 hours; an additional 355 (11%) became severely ill or died in the next 7 days. The area under the receiver-operating characteristic curve (AUC) for 1-day risk predictions for progression to severe disease or death was 0.89 (95% CI, 0.88 to 0.90) and 0.89 (CI, 0.87 to 0.91) during the first and second weeks of hospitalization, respectively. The AUC for 7-day risk predictions for progression to severe disease or death was 0.83 (CI, 0.83 to 0.84) and 0.87 (CI, 0.86 to 0.89) during the first and second weeks of hospitalization, respectively. LIMITATION The SCARP tool was developed by using data from a single health system. CONCLUSION Using the predictive power of RF-SLAM and longitudinal data from more than 3000 patients hospitalized with COVID-19, an interactive tool was developed that rapidly and accurately provides the probability of an individual patient's progression to severe illness or death on the basis of readily available clinical information. PRIMARY FUNDING SOURCE Hopkins inHealth and COVID-19 Administrative Supplement for the HHS Region 3 Treatment Center from the Office of the Assistant Secretary for Preparedness and Response.
Collapse
Affiliation(s)
- Shannon Wongvibulsin
- Johns Hopkins University School of Medicine, Baltimore, Maryland (S.W., B.T.G., A.A.A., A.G., R.B., M.L.R.)
| | - Brian T Garibaldi
- Johns Hopkins University School of Medicine, Baltimore, Maryland (S.W., B.T.G., A.A.A., A.G., R.B., M.L.R.)
| | - Annukka A R Antar
- Johns Hopkins University School of Medicine, Baltimore, Maryland (S.W., B.T.G., A.A.A., A.G., R.B., M.L.R.)
| | - Jiyang Wen
- Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland (J.W., M.W., J.F.B., J.M., K.B., S.L.Z.)
| | - Mei-Cheng Wang
- Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland (J.W., M.W., J.F.B., J.M., K.B., S.L.Z.)
| | - Amita Gupta
- Johns Hopkins University School of Medicine, Baltimore, Maryland (S.W., B.T.G., A.A.A., A.G., R.B., M.L.R.)
| | - Robert Bollinger
- Johns Hopkins University School of Medicine, Baltimore, Maryland (S.W., B.T.G., A.A.A., A.G., R.B., M.L.R.)
| | - Yanxun Xu
- Johns Hopkins University, Baltimore, Maryland (Y.X., K.W.)
| | - Kunbo Wang
- Johns Hopkins University, Baltimore, Maryland (Y.X., K.W.)
| | - Joshua F Betz
- Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland (J.W., M.W., J.F.B., J.M., K.B., S.L.Z.)
| | - John Muschelli
- Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland (J.W., M.W., J.F.B., J.M., K.B., S.L.Z.)
| | - Karen Bandeen-Roche
- Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland (J.W., M.W., J.F.B., J.M., K.B., S.L.Z.)
| | - Scott L Zeger
- Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland (J.W., M.W., J.F.B., J.M., K.B., S.L.Z.)
| | - Matthew L Robinson
- Johns Hopkins University School of Medicine, Baltimore, Maryland (S.W., B.T.G., A.A.A., A.G., R.B., M.L.R.)
| |
Collapse
|
13
|
Flaus A, Amat J, Prevot N, Olagne L, Descamps L, Bouvet C, Barres B, Valla C, Mathieu S, Andre M, Soubrier M, Merlin C, Kelly A, Chanchou M, Cachin F. Decision Tree With Only Two Musculoskeletal Sites to Diagnose Polymyalgia Rheumatica Using [ 18F]FDG PET-CT. Front Med (Lausanne) 2021; 8:646974. [PMID: 33681267 PMCID: PMC7928279 DOI: 10.3389/fmed.2021.646974] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2020] [Accepted: 01/28/2021] [Indexed: 12/25/2022] Open
Abstract
Introduction: The aim of this study was to find the best ordered combination of two FDG positive musculoskeletal sites with a machine learning algorithm to diagnose polymyalgia rheumatica (PMR) vs. other rheumatisms in a cohort of patients with inflammatory rheumatisms. Methods: This retrospective study included 140 patients who underwent [18F]FDG PET-CT and whose final diagnosis was inflammatory rheumatism. The cohort was randomized, stratified on the final diagnosis into a training and a validation cohort. FDG uptake of 17 musculoskeletal sites was evaluated visually and set positive if uptake was at least equal to that of the liver. A decision tree classifier was trained and validated to find the best combination of two positives sites to diagnose PMR. Diagnosis performances were measured first, for each musculoskeletal site, secondly for combination of two positive sites and thirdly using the decision tree created with machine learning. Results: 55 patients with PMR and 85 patients with other inflammatory rheumatisms were included. Musculoskeletal sites, used either individually or in combination of two, were highly imbalanced to diagnose PMR with a high specificity and a low sensitivity. The machine learning algorithm identified an optimal ordered combination of two sites to diagnose PMR. This required a positive interspinous bursa or, if negative, a positive trochanteric bursa. Following the decision tree, sensitivity and specificity to diagnose PMR were respectively 73.2 and 87.5% in the training cohort and 78.6 and 80.1% in the validation cohort. Conclusion: Ordered combination of two visually positive sites leads to PMR diagnosis with an accurate sensitivity and specificity vs. other rheumatisms in a large cohort of patients with inflammatory rheumatisms.
Collapse
Affiliation(s)
- Anthime Flaus
- Department of Nuclear Medicine, Saint-Etienne University Hospital, University of Saint-Etienne, Saint-Etienne, France
| | - Julie Amat
- Department of Nuclear Medicine, Jean Perrin Oncology Institute of Clermont-Ferrand, Clermont-Ferrand, France
| | - Nathalie Prevot
- Department of Nuclear Medicine, Saint-Etienne University Hospital, University of Saint-Etienne, Saint-Etienne, France.,Institut national de la santé et de la recherche médicale, U 1059 Sainbiose, Université Jean Monnet, Saint-Etienne, France
| | - Louis Olagne
- Department of Internal Medicine, Gabriel Montpied University Hospital, University of Clermont-Ferrand, Clermont-Ferrand, France
| | - Lucie Descamps
- Department of Rheumatology, Gabriel Montpied University Hospital, University of Clermont-Ferrand, Clermont-Ferrand, France
| | - Clément Bouvet
- Department of Nuclear Medicine, Jean Perrin Oncology Institute of Clermont-Ferrand, Clermont-Ferrand, France
| | - Bertrand Barres
- Department of Nuclear Medicine, Jean Perrin Oncology Institute of Clermont-Ferrand, Clermont-Ferrand, France
| | - Clémence Valla
- Department of Nuclear Medicine, Jean Perrin Oncology Institute of Clermont-Ferrand, Clermont-Ferrand, France
| | - Sylvain Mathieu
- Department of Rheumatology, Gabriel Montpied University Hospital, University of Clermont-Ferrand, Clermont-Ferrand, France
| | - Marc Andre
- Department of Internal Medicine, Gabriel Montpied University Hospital, University of Clermont-Ferrand, Clermont-Ferrand, France
| | - Martin Soubrier
- Department of Rheumatology, Gabriel Montpied University Hospital, University of Clermont-Ferrand, Clermont-Ferrand, France
| | - Charles Merlin
- Department of Nuclear Medicine, Jean Perrin Oncology Institute of Clermont-Ferrand, Clermont-Ferrand, France
| | - Antony Kelly
- Department of Nuclear Medicine, Jean Perrin Oncology Institute of Clermont-Ferrand, Clermont-Ferrand, France
| | - Marion Chanchou
- Department of Nuclear Medicine, Jean Perrin Oncology Institute of Clermont-Ferrand, Clermont-Ferrand, France
| | - Florent Cachin
- Department of Nuclear Medicine, Jean Perrin Oncology Institute of Clermont-Ferrand, Clermont-Ferrand, France
| |
Collapse
|
14
|
Wu KC, Wongvibulsin S, Tao S, Ashikaga H, Stillabower M, Dickfeld TM, Marine JE, Weiss RG, Tomaselli GF, Zeger SL. Baseline and Dynamic Risk Predictors of Appropriate Implantable Cardioverter Defibrillator Therapy. J Am Heart Assoc 2020; 9:e017002. [PMID: 33023350 PMCID: PMC7763383 DOI: 10.1161/jaha.120.017002] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/08/2023]
Abstract
Background Current approaches fail to separate patients at high versus low risk for ventricular arrhythmias owing to overreliance on a snapshot left ventricular ejection fraction measure. We used statistical machine learning to identify important cardiac imaging and time-varying risk predictors. Methods and Results Three hundred eighty-two cardiomyopathy patients (left ventricular ejection fraction ≤35%) underwent cardiac magnetic resonance before primary prevention implantable cardioverter defibrillator insertion. The primary end point was appropriate implantable cardioverter defibrillator discharge or sudden death. Patient characteristics; serum biomarkers of inflammation, neurohormonal status, and injury; and cardiac magnetic resonance-measured left ventricle and left atrial indices and myocardial scar burden were assessed at baseline. Time-varying covariates comprised interval heart failure hospitalizations and left ventricular ejection fractions. A random forest statistical method for survival, longitudinal, and multivariable outcomes incorporating baseline and time-varying variables was compared with (1) Seattle Heart Failure model scores and (2) random forest survival and Cox regression models incorporating baseline characteristics with and without imaging variables. Age averaged 57±13 years with 28% women, 66% white, 51% ischemic, and follow-up time of 5.9±2.3 years. The primary end point (n=75) occurred at 3.3±2.4 years. Random forest statistical method for survival, longitudinal, and multivariable outcomes with baseline and time-varying predictors had the highest area under the receiver operating curve, median 0.88 (95% CI, 0.75-0.96). Top predictors comprised heart failure hospitalization, left ventricle scar, left ventricle and left atrial volumes, left atrial function, and interleukin-6 level; heart failure accounted for 67% of the variation explained by the prediction, imaging 27%, and interleukin-6 2%. Serial left ventricular ejection fraction was not a significant predictor. Conclusions Hospitalization for heart failure and baseline cardiac metrics substantially improve ventricular arrhythmic risk prediction.
Collapse
Affiliation(s)
- Katherine C Wu
- Department of Medicine Division of Cardiology Johns Hopkins University School of Medicine Baltimore MD
| | - Shannon Wongvibulsin
- Department of Biomedical Engineering and School of Medicine Johns Hopkins University Baltimore MD
| | - Susumu Tao
- Department of Medicine Division of Cardiology Johns Hopkins University School of Medicine Baltimore MD
| | - Hiroshi Ashikaga
- Department of Medicine Division of Cardiology Johns Hopkins University School of Medicine Baltimore MD.,Department of Biomedical Engineering and School of Medicine Johns Hopkins University Baltimore MD
| | | | - Timm M Dickfeld
- Department of Medicine University of Maryland Medical Systems Baltimore MD
| | - Joseph E Marine
- Department of Medicine Division of Cardiology Johns Hopkins University School of Medicine Baltimore MD
| | - Robert G Weiss
- Department of Medicine Division of Cardiology Johns Hopkins University School of Medicine Baltimore MD.,The Russell H. Morgan Department of Radiology and Radiological Science Johns Hopkins University School of Medicine Baltimore MD
| | | | - Scott L Zeger
- Department of Biostatistics Johns Hopkins Bloomberg School of Public Health Baltimore MD
| |
Collapse
|