1
|
Moreno Mendez R, Marín A, Ferrando JR, Rissi Castro G, Cepeda Madrigal S, Agostini G, Catalan Serra P. Artificial Intelligence Applied to Forced Spirometry in Primary Care. OPEN RESPIRATORY ARCHIVES 2024; 6:100313. [PMID: 38828405 PMCID: PMC11137334 DOI: 10.1016/j.opresp.2024.100313] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2023] [Accepted: 02/19/2024] [Indexed: 06/05/2024] Open
Abstract
Introduction This study aims to create an artificial intelligence (AI) based machine learning (ML) model capable of predicting a spirometric obstructive pattern using variables with the highest predictive power derived from an active case-finding program for COPD in primary care. Material and methods A total of 1190 smokers, aged 30-80 years old with no prior history of respiratory disease, underwent spirometry with bronchodilation. The sample was analyzed using AI tools. Based on an exploratory data analysis (EDA), independent variables (according to mutual information analysis) were trained using a gradient boosting algorithm (GBT) and validated through cross-validation. Results With an area under the curve close to unity, the model predicted a spirometric obstructive pattern using variables with the highest predictive power: FEV1_theoretical_pre values. Sensitivity: 93%. Positive predictive value: 94%. Specificity: 97%. Negative predictive value: 96%. Accuracy: 95%. Precision: 94%. Conclusion An ML model can predict the presence of an obstructive pattern in spirometry in a primary care smoking population with no prior diagnosis of respiratory disease using the FEV1_theoretical_pre values with an accuracy and precision exceeding 90%. Further studies including clinical data and strategies for integrating AI into clinical workflow are needed.
Collapse
Affiliation(s)
| | | | | | | | | | - Gabriela Agostini
- Otolaryngology Department, Vida Clinic, Santa Cruz de Tenerife, Spain
| | - Pablo Catalan Serra
- Department of Internal Medicine, Kristiansund Hospital, Møre og Romsdal, Norway
| |
Collapse
|
2
|
Ahmad F, Muhmood T. Clinical translation of nanomedicine with integrated digital medicine and machine learning interventions. Colloids Surf B Biointerfaces 2024; 241:114041. [PMID: 38897022 DOI: 10.1016/j.colsurfb.2024.114041] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2024] [Revised: 06/11/2024] [Accepted: 06/13/2024] [Indexed: 06/21/2024]
Abstract
Nanomaterials based therapeutics transform the ways of disease prevention, diagnosis and treatment with increasing sophistications in nanotechnology at a breakneck pace, but very few could reach to the clinic due to inconsistencies in preclinical studies followed by regulatory hinderances. To tackle this, integrating the nanomedicine discovery with digital medicine provide technologies as tools of specific biological activity measurement. Hence, overcome the redundancies in nanomedicine discovery by the on-site data acquisition and analytics through integrating intelligent sensors and artificial intelligence (AI) or machine learning (ML). Integrated AI/ML wearable sensors directly gather clinically relevant biochemical information from the subject's body and process data for physicians to make right clinical decision(s) in a time and cost-effective way. This review summarizes insights and recommend the infusion of actionable big data computation enabled sensors in burgeoning field of nanomedicine at academia, research institutes, and pharmaceutical industries, with a potential of clinical translation. Furthermore, many blind spots are present in modern clinically relevant computation, one of which could prevent ML-guided low-cost new nanomedicine development from being successfully translated into the clinic was also discussed.
Collapse
Affiliation(s)
- Farooq Ahmad
- State Key Laboratory of Chemistry and Utilization of Carbon Based Energy Resources, College of Chemistry, Xinjiang University, Urumqi 830017, China.
| | - Tahir Muhmood
- International Iberian Nanotechnology Laboratory (INL), Avenida Mestre José Veiga, Braga 4715-330, Portugal.
| |
Collapse
|
3
|
Lundin RM, Falcao VP, Kannangara S, Eakin CW, Abdar M, O'Neill J, Khosravi A, Eyre H, Nahavandi S, Loo C, Berk M. Machine Learning in Electroconvulsive Therapy: A Systematic Review. J ECT 2024:00124509-990000000-00167. [PMID: 38857315 DOI: 10.1097/yct.0000000000001009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 06/12/2024]
Abstract
ABSTRACT Despite years of research, we are still not able to reliably predict who might benefit from electroconvulsive therapy (ECT) treatment. As we exhaust what is possible using traditional statistical analysis, ECT remains a good candidate for machine learning approaches due to the large data sets with data captured through electroencephalography (EEG) and other objective measures. A systematic review of 6 databases led to the full-text examination of 26 articles using machine learning approaches in examining data predicting response to ECT treatment. The identified articles used a wide variety of data types covering structural and functional imaging data (n = 15), clinical data (n = 5), a combination of clinical and imaging data (n = 2), EEG (n = 3), and social media posts (n = 1). The clinical indications in which response prediction was assessed were depression (n = 21) and psychosis (n = 4). Changes in multiple anatomical regions in the brain were identified as holding a predictive value for response to ECT. These primarily centered on the limbic system and associated networks. Clinical features predicting good response to ECT in depression included shorter duration, lower severity, higher medication dose, psychotic features, low cortisol levels, and positive family history. It has also been possible to predict the likelihood of relapse of readmission with psychosis after ECT treatment, including a better response if higher transfer entropy was calculated from EEG signals. A transdisciplinary approach with an international consortium collecting a wide range of retrospective and prospective data may help to refine and extend these outcomes and translate them into clinical practice.
Collapse
Affiliation(s)
| | | | | | - Charles W Eakin
- From the Mental Health, Drug and Alcohol Services, Barwon Health
| | - Moloud Abdar
- Institute for Intelligent Systems Research and Innovation, Deakin University, Geelong, Victoria, Australia
| | - John O'Neill
- Waikato District Health Board, Hamilton, New Zealand
| | - Abbas Khosravi
- Institute for Intelligent Systems Research and Innovation, Deakin University, Geelong, Victoria, Australia
| | | | - Saeid Nahavandi
- Institute for Intelligent Systems Research and Innovation, Deakin University, Geelong, Victoria, Australia
| | | | - Michael Berk
- IMPACT - the Institute for Mental and Physical Health and Clinical Translation, School of Medicine, Deakin University, Geelong, Victoria, Australia
| |
Collapse
|
4
|
Fabreti-Oliveira RA, Nascimento E, de Melo Santos LH, de Oliveira Santos MR, Veloso AA. Predicting kidney allograft survival with explainable machine learning. Transpl Immunol 2024; 85:102057. [PMID: 38797338 DOI: 10.1016/j.trim.2024.102057] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2024] [Revised: 05/19/2024] [Accepted: 05/23/2024] [Indexed: 05/29/2024]
Abstract
INTRODUCTION Despite significant progress over the last decades in the survival of kidney allografts, several risk factors remain contributing to worsening kidney function or even loss of transplants. We aimed to evaluate a new machine learning method to identify these variables which may predict the early graft loss in kidney transplant patients and to assess their usefulness for improving clinical decisions. MATERIAL AND METHODS A retrospective cohort study was carried out with 627 kidney transplant patients followed at least three months. All these data were pre-processed, and their selected features were used to develop an automatically working a machine learning algorithm; this algorithm was then applied for training and parameterization of the model; and finally, the tested model was then used for the analysis of patients' features that were the most impactful for the prediction of clinical outcomes. Our models were evaluated using the Area Under the Curve (AUC), and the SHapley Additive exPlanations (SHAP) algorithm was used to interpret its predictions. RESULTS The final selected model achieved a precision of 0.81, a sensitivity of 0.61, a specificity of 0.89, and an AUC value of 0.84. In our model, serum creatinine levels of kidney transplant patients, evaluated at the hospital discharge, proved to be the most important factor in the decision-making for the allograft loss. Patients with a weight equivalent to a BMI closer to the normal range prior to a kidney transplant are less likely to experience graft loss compared to patients with a BMI below the normal range. The age of patients at transplantation and Polyomavirus (BKPyV) infection had significant impact on clinical outcomes in our model. CONCLUSIONS Our algorithm suggests that the main characteristics that impacted early allograft loss were serum creatinine levels at the hospital discharge, as well as the pre-transplant values such as body weight, age of patients, and their BKPyV infection. We propose that machine learning tools can be developed to effectively assist medical decision-making in kidney transplantation.
Collapse
Affiliation(s)
- Raquel A Fabreti-Oliveira
- Artificial Intelligence Laboratory, Departament of Computer Sciences, Federal University of Minas Gerais, Belo Horizonte, Minas Gerais, Brazil; Faculty of Medical Sciences of Minas Gerais, Belo Horizonte, Minas Gerais, Brazil; IMUNOLAB - Laboratory of Histocompatibility, Belo Horizonte, Minas Gerais, Brazil.
| | - Evaldo Nascimento
- IMUNOLAB - Laboratory of Histocompatibility, Belo Horizonte, Minas Gerais, Brazil; Faculty of Hospital Santa Casa, Belo Horizonte, Minas Gerais, Brazil.
| | - Luiz Henrique de Melo Santos
- Artificial Intelligence Laboratory, Departament of Computer Sciences, Federal University of Minas Gerais, Belo Horizonte, Minas Gerais, Brazil
| | | | - Adriano Alonso Veloso
- Artificial Intelligence Laboratory, Departament of Computer Sciences, Federal University of Minas Gerais, Belo Horizonte, Minas Gerais, Brazil.
| |
Collapse
|
5
|
Jeong E, Woo Shin Y, Byun JI, Sunwoo JS, Roascio M, Mattioli P, Giorgetti L, Famà F, Arnulfo G, Arnaldi D, Kim HJ, Jung KY. EEG-based machine learning models for the prediction of phenoconversion time and subtype in isolated rapid eye movement sleep behavior disorder. Sleep 2024; 47:zsae031. [PMID: 38330231 DOI: 10.1093/sleep/zsae031] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2023] [Revised: 01/20/2024] [Indexed: 02/10/2024] Open
Abstract
STUDY OBJECTIVES Isolated rapid eye movement sleep behavior disorder (iRBD) is a prodromal stage of α-synucleinopathies and eventually phenoconverts to overt neurodegenerative diseases including Parkinson's disease (PD), dementia with Lewy bodies (DLB), and multiple system atrophy (MSA). Associations of baseline resting-state electroencephalography (EEG) with phenoconversion have been reported. In this study, we aimed to develop machine learning models to predict phenoconversion time and subtype using baseline EEG features in patients with iRBD. METHODS At baseline, resting-state EEG and neurological assessments were performed on patients with iRBD. Calculated EEG features included spectral power, weighted phase lag index, and Shannon entropy. Three models were used for survival prediction, and four models were used for α-synucleinopathy subtype prediction. The models were externally validated using data from a different institution. RESULTS A total of 236 iRBD patients were followed up for up to 8 years (mean 3.5 years), and 31 patients converted to α-synucleinopathies (16 PD, 9 DLB, 6 MSA). The best model for survival prediction was the random survival forest model with an integrated Brier score of 0.114 and a concordance index of 0.775. The K-nearest neighbor model was the best model for subtype prediction with an area under the receiver operating characteristic curve of 0.901. Slowing of the EEG was an important feature for both models. CONCLUSIONS Machine learning models using baseline EEG features can be used to predict phenoconversion time and its subtype in patients with iRBD. Further research including large sample data from many countries is needed to make a more robust model.
Collapse
Affiliation(s)
- El Jeong
- Interdisciplinary Program in Bioengineering, College of Engineering, Seoul National University, Seoul, South Korea
| | - Yong Woo Shin
- Department of Neurology, Seoul National University Hospital, Seoul National University College of Medicine, Seoul, South Korea
| | - Jung-Ick Byun
- Department of Neurology, Kyung Hee University Hospital at Gangdong, Seoul, South Korea
| | - Jun-Sang Sunwoo
- Department of Neurology, Kangbuk Samsung Hospital, Seoul, South Korea
| | - Monica Roascio
- Department of Informatics, Bioengineering, Robotics and System engineering (DIBRIS), University of Genoa, Genoa, Italy
- RAISE (Robotics and AI for Socio-economic Empowerment) Ecosystem, Genoa, Italy
| | - Pietro Mattioli
- Department of Neuroscience (DINOGMI), University of Genoa, Genoa, Italy
- Neurophysiopathology Unit, IRCCS Ospedale Policlinico San Martino, Genoa, Italy
| | - Laura Giorgetti
- Department of Neuroscience (DINOGMI), University of Genoa, Genoa, Italy
| | - Francesco Famà
- Department of Neuroscience (DINOGMI), University of Genoa, Genoa, Italy
- Neurophysiopathology Unit, IRCCS Ospedale Policlinico San Martino, Genoa, Italy
| | - Gabriele Arnulfo
- Department of Informatics, Bioengineering, Robotics and System engineering (DIBRIS), University of Genoa, Genoa, Italy
- RAISE (Robotics and AI for Socio-economic Empowerment) Ecosystem, Genoa, Italy
| | - Dario Arnaldi
- Department of Neuroscience (DINOGMI), University of Genoa, Genoa, Italy
- Neurophysiopathology Unit, IRCCS Ospedale Policlinico San Martino, Genoa, Italy
| | - Han-Joon Kim
- Department of Neurology, Seoul National University Hospital, Seoul National University College of Medicine, Seoul, South Korea
| | - Ki-Young Jung
- Seoul National University Hospital, Seoul, South Korea
- Seoul National University Medical Research Center Neuroscience Research Institute, Sensory Organ Research Institute, Seoul National University College of Medicine, Seoul, South Korea
| |
Collapse
|
6
|
Ahmed M, Stone ML, Stidham RW. Artificial Intelligence and IBD: Where are We Now and Where Will We Be in the Future? Curr Gastroenterol Rep 2024; 26:137-144. [PMID: 38411898 DOI: 10.1007/s11894-024-00918-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/19/2024] [Indexed: 02/28/2024]
Abstract
PURPOSE OF REVIEW Artificial intelligence (AI) is quickly demonstrating the ability to address problems and challenges in the care of IBD. This review with commentary will highlight today's advancements in AI applications for IBD in image analysis, understanding text, and replicating clinical knowledge and experience. RECENT FINDINGS Advancements in machine learning methods, availability of high-performance computing, and increasing digitization of medical data are providing opportunities for AI to assist in IBD care. Multiple groups have demonstrated the ability of AI to replicate expert endoscopic scoring in IBD, with expansion into automated capsule endoscopy, enterography, and histologic interpretations. Further, AI image analysis is being used to develop new endoscopic scoring with more granularity and detail than is possible using conventional methods. Advancements in natural language processing are proving to reduce laborious tasks required in the care of IBD, including documentation, information searches, and chart review. Finally, large language models and chatbots that can understand language and generate human-like replies are beginning to exhibit clinical intelligence that will revolutionize how we deliver IBD care. Today, AI is being deployed to replicate expert judgement in specific tasks where disagreement, subjectivity, and bias are common. However, the near future will herald contributions of AI doing what we cannot, including new detailed measures of IBD, enhanced analysis of images, and perhaps even fully automating care. As we speculate on future technologic capabilities that may improve how we care for IBD, this review will also consider how we will implement and fairly use AI in practice.
Collapse
Affiliation(s)
- Mehwish Ahmed
- Division of Gastroenterology, Department of Internal Medicine, Michigan Medicine, Ann Arbor, MI, USA
| | - Molly L Stone
- Division of Gastroenterology, Department of Internal Medicine, Michigan Medicine, Ann Arbor, MI, USA
| | - Ryan W Stidham
- Division of Gastroenterology, Department of Internal Medicine, Michigan Medicine, Ann Arbor, MI, USA.
- Department of Computational Medicine and Bioinformatics, University of Michigan, 3912 Taubman Center, 1500 East Medical Center Drive, Ann Arbor, MI, 48109, USA.
- Michigan Institute for Data Science (MIDAS), University of Michigan, Ann Arbor, MI, USA.
| |
Collapse
|
7
|
Shara N, Mirabal-Beltran R, Talmadge B, Falah N, Ahmad M, Dempers R, Crovatt S, Eisenberg S, Anderson K. Use of Machine Learning for Early Detection of Maternal Cardiovascular Conditions: Retrospective Study Using Electronic Health Record Data. JMIR Cardio 2024; 8:e53091. [PMID: 38648629 DOI: 10.2196/53091] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2023] [Revised: 03/08/2024] [Accepted: 03/11/2024] [Indexed: 04/25/2024] Open
Abstract
BACKGROUND Cardiovascular conditions (eg, cardiac and coronary conditions, hypertensive disorders of pregnancy, and cardiomyopathies) were the leading cause of maternal mortality between 2017 and 2019. The United States has the highest maternal mortality rate of any high-income nation, disproportionately impacting those who identify as non-Hispanic Black or Hispanic. Novel clinical approaches to the detection and diagnosis of cardiovascular conditions are therefore imperative. Emerging research is demonstrating that machine learning (ML) is a promising tool for detecting patients at increased risk for hypertensive disorders during pregnancy. However, additional studies are required to determine how integrating ML and big data, such as electronic health records (EHRs), can improve the identification of obstetric patients at higher risk of cardiovascular conditions. OBJECTIVE This study aimed to evaluate the capability and timing of a proprietary ML algorithm, Healthy Outcomes for all Pregnancy Experiences-Cardiovascular-Risk Assessment Technology (HOPE-CAT), to detect maternal-related cardiovascular conditions and outcomes. METHODS Retrospective data from the EHRs of a large health care system were investigated by HOPE-CAT in a virtual server environment. Deidentification of EHR data and standardization enabled HOPE-CAT to analyze data without pre-existing biases. The ML algorithm assessed risk factors selected by clinical experts in cardio-obstetrics, and the algorithm was iteratively trained using relevant literature and current standards of risk identification. After refinement of the algorithm's learned risk factors, risk profiles were generated for every patient including a designation of standard versus high risk. The profiles were individually paired with clinical outcomes pertaining to cardiovascular pregnancy conditions and complications, wherein a delta was calculated between the date of the risk profile and the actual diagnosis or intervention in the EHR. RESULTS In total, 604 pregnancies resulting in birth had records or diagnoses that could be compared against the risk profile; the majority of patients identified as Black (n=482, 79.8%) and aged between 21 and 34 years (n=509, 84.4%). Preeclampsia (n=547, 90.6%) was the most common condition, followed by thromboembolism (n=16, 2.7%) and acute kidney disease or failure (n=13, 2.2%). The average delta was 56.8 (SD 69.7) days between the identification of risk factors by HOPE-CAT and the first date of diagnosis or intervention of a related condition reported in the EHR. HOPE-CAT showed the strongest performance in early risk detection of myocardial infarction at a delta of 65.7 (SD 81.4) days. CONCLUSIONS This study provides additional evidence to support ML in obstetrical patients to enhance the early detection of cardiovascular conditions during pregnancy. ML can synthesize multiday patient presentations to enhance provider decision-making and potentially reduce maternal health disparities.
Collapse
Affiliation(s)
- Nawar Shara
- MedStar Health Research Institute, Hyattesville, MD, United States
- Georgetown-Howard Universities Center for Clinical and Translational Science, Washington, DC, DC, United States
| | | | | | - Noor Falah
- MedStar Health Research Institute, Hyattesville, MD, United States
| | - Maryam Ahmad
- MedStar Health Research Institute, Hyattesville, MD, United States
| | | | | | | | - Kelley Anderson
- School of Nursing, Georgetown University, Washington, DC, United States
| |
Collapse
|
8
|
Park C, Loza-Avalos SE, Harvey J, Hirschkorn C, Dultz LA, Dumas RP, Sanders D, Chowdhry V, Starr A, Cripps M. A Real-Time Automated Machine Learning Algorithm for Predicting Mortality in Trauma Patients: Survey Says it's Ready for Prime-Time. Am Surg 2024; 90:655-661. [PMID: 37848176 DOI: 10.1177/00031348231207299] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2023]
Abstract
BACKGROUND Though artificial intelligence ("AI") has been increasingly applied to patient care, many of these predictive models are retrospective and not readily available for real-time decision-making. This survey-based study aims to evaluate implementation of a new, validated mortality risk calculator (Parkland Trauma Index of Mortality, "PTIM") embedded in our electronic healthrecord ("EHR") that calculates hourly predictions of mortality with high sensitivity and specificity. METHODS This is a prospective, survey-based study performed at a level 1 trauma center. An anonymous survey was sent to surgical providers and regarding PTIM implementation. The PTIM score evaluates 23 variables including Glasgow Coma Score (GCS), vital signs, and laboratory data. RESULTS Of the 40 completed surveys, 35 reported using PTIM in decision-making. Prior to reviewing PTIM, providers identified perceived top 3 predictors of mortality, including GCS (22/38, 58%), age (18/35, 47%), and maximum heart rate (17/35, 45%). Most providers reported the PTIM assisted their treatment decisions (27/35, 77%) and timing of operative intervention (23/35, 66%). Many providers agreed that PTIM integrated into rounds and patient assessment (22/36, 61%) and that it improved efficiency in assessing patients' potential mortality (21/36, 58%). CONCLUSIONS Artificial intelligence algorithms are mostly retrospective and lag in real-time prediction of mortality. To our knowledge, this is the first real-time, automated algorithm predicting mortality in trauma patients. In this small survey-based study, we found PTIM assists in decision-making, timing of intervention, and improves accuracy in assessing mortality. Next steps include evaluating the short- and long-term impact on patient outcomes.
Collapse
Affiliation(s)
- Caroline Park
- Division of Burns, Trauma and Acute Care Surgery, Department of Surgery, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Sandra E Loza-Avalos
- Division of Burns, Trauma and Acute Care Surgery, Department of Surgery, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Jalen Harvey
- Division of Burns, Trauma and Acute Care Surgery, Department of Surgery, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | | | - Linda A Dultz
- Division of Burns, Trauma and Acute Care Surgery, Department of Surgery, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Ryan P Dumas
- Division of Burns, Trauma and Acute Care Surgery, Department of Surgery, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Drew Sanders
- Department of Orthopedic Surgery, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | | | - Adam Starr
- Department of Orthopedic Surgery, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Michael Cripps
- Division of Burns, Trauma and Acute Care Surgery, Department of Surgery, University of Texas Southwestern Medical Center, Dallas, TX, USA
| |
Collapse
|
9
|
Vadalà G, Ambrosio L, Denaro V. Safety and Complications Related to Emerging Technologies. Neurospine 2024; 21:6-7. [PMID: 38569626 PMCID: PMC10992665 DOI: 10.14245/ns.2448162.081] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/05/2024] Open
Affiliation(s)
- Gianluca Vadalà
- Operative Research Unit of Orthopaedic and Trauma Surgery, Fondazione Policlinico Universitario Campus Bio-Medico, Rome, Italy
- Research Unit of Orthopaedic and Trauma Surgery, Department of Medicine and Surgery, Università Campus Bio-Medico di Roma, Rome, Italy
| | - Luca Ambrosio
- Operative Research Unit of Orthopaedic and Trauma Surgery, Fondazione Policlinico Universitario Campus Bio-Medico, Rome, Italy
- Research Unit of Orthopaedic and Trauma Surgery, Department of Medicine and Surgery, Università Campus Bio-Medico di Roma, Rome, Italy
| | - Vincenzo Denaro
- Operative Research Unit of Orthopaedic and Trauma Surgery, Fondazione Policlinico Universitario Campus Bio-Medico, Rome, Italy
| |
Collapse
|
10
|
Evans RP, Bryant LD, Russell G, Absolom K. Trust and acceptability of data-driven clinical recommendations in everyday practice: A scoping review. Int J Med Inform 2024; 183:105342. [PMID: 38266426 DOI: 10.1016/j.ijmedinf.2024.105342] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2023] [Revised: 12/08/2023] [Accepted: 01/14/2024] [Indexed: 01/26/2024]
Abstract
BACKGROUND Increasing attention is being given to the analysis of large health datasets to derive new clinical decision support systems (CDSS). However, few data-driven CDSS are being adopted into clinical practice. Trust in these tools is believed to be fundamental for acceptance and uptake but to date little attention has been given to defining or evaluating trust in clinical settings. OBJECTIVES A scoping review was conducted to explore how and where acceptability and trustworthiness of data-driven CDSS have been assessed from the health professional's perspective. METHODS Medline, Embase, PsycInfo, Web of Science, Scopus, ACM Digital, IEEE Xplore and Google Scholar were searched in March 2022 using terms expanded from: "data-driven" AND "clinical decision support" AND "acceptability". Included studies focused on healthcare practitioner-facing data-driven CDSS, relating directly to clinical care. They included trust or a proxy as an outcome, or in the discussion. The preferred reporting items for systematic reviews and meta-analyses extension for scoping reviews (PRISMA-ScR) is followed in the reporting of this review. RESULTS 3291 papers were screened, with 85 primary research studies eligible for inclusion. Studies covered a diverse range of clinical specialisms and intended contexts, but hypothetical systems (24) outnumbered those in clinical use (18). Twenty-five studies measured trust, via a wide variety of quantitative, qualitative and mixed methods. A further 24 discussed themes of trust without it being explicitly evaluated, and from these, themes of transparency, explainability, and supporting evidence were identified as factors influencing healthcare practitioner trust in data-driven CDSS. CONCLUSION There is a growing body of research on data-driven CDSS, but few studies have explored stakeholder perceptions in depth, with limited focused research on trustworthiness. Further research on healthcare practitioner acceptance, including requirements for transparency and explainability, should inform clinical implementation.
Collapse
Affiliation(s)
- Ruth P Evans
- University of Leeds, Woodhouse Lane, Leeds LS2 9JT, UK.
| | | | - Gregor Russell
- Bradford District Care Trust, Bradford, New Mill, Victoria Rd, BD18 3LD, UK.
| | - Kate Absolom
- University of Leeds, Woodhouse Lane, Leeds LS2 9JT, UK.
| |
Collapse
|
11
|
Morley J, Hamilton N, Floridi L. Selling NHS patient data. BMJ 2024; 384:q420. [PMID: 38387965 DOI: 10.1136/bmj.q420] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/24/2024]
Affiliation(s)
- Jessica Morley
- Digital Ethics Center, Yale University, New Haven, CT, USA
| | | | - Luciano Floridi
- Digital Ethics Center, Yale University, New Haven, CT, USA
- Department of Legal Studies, University of Bologna, Bologna, Italy
| |
Collapse
|
12
|
Rahman A, Debnath T, Kundu D, Khan MSI, Aishi AA, Sazzad S, Sayduzzaman M, Band SS. Machine learning and deep learning-based approach in smart healthcare: Recent advances, applications, challenges and opportunities. AIMS Public Health 2024; 11:58-109. [PMID: 38617415 PMCID: PMC11007421 DOI: 10.3934/publichealth.2024004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2023] [Accepted: 12/18/2023] [Indexed: 04/16/2024] Open
Abstract
In recent years, machine learning (ML) and deep learning (DL) have been the leading approaches to solving various challenges, such as disease predictions, drug discovery, medical image analysis, etc., in intelligent healthcare applications. Further, given the current progress in the fields of ML and DL, there exists the promising potential for both to provide support in the realm of healthcare. This study offered an exhaustive survey on ML and DL for the healthcare system, concentrating on vital state of the art features, integration benefits, applications, prospects and future guidelines. To conduct the research, we found the most prominent journal and conference databases using distinct keywords to discover scholarly consequences. First, we furnished the most current along with cutting-edge progress in ML-DL-based analysis in smart healthcare in a compendious manner. Next, we integrated the advancement of various services for ML and DL, including ML-healthcare, DL-healthcare, and ML-DL-healthcare. We then offered ML and DL-based applications in the healthcare industry. Eventually, we emphasized the research disputes and recommendations for further studies based on our observations.
Collapse
Affiliation(s)
- Anichur Rahman
- Department of CSE, National Institute of Textile Engineering and Research (NITER), Constituent Institute of the University of Dhaka, Savar, Dhaka-1350
- Department of CSE, Mawlana Bhashani Science and Technology University, Tangail, Bangladesh
| | - Tanoy Debnath
- Department of CSE, Mawlana Bhashani Science and Technology University, Tangail, Bangladesh
- Department of CSE, Green University of Bangladesh, 220/D, Begum Rokeya Sarani, Dhaka -1207, Bangladesh
| | - Dipanjali Kundu
- Department of CSE, National Institute of Textile Engineering and Research (NITER), Constituent Institute of the University of Dhaka, Savar, Dhaka-1350
| | - Md. Saikat Islam Khan
- Department of CSE, Mawlana Bhashani Science and Technology University, Tangail, Bangladesh
| | - Airin Afroj Aishi
- Department of Computing and Information System, Daffodil International University, Savar, Dhaka, Bangladesh
| | - Sadia Sazzad
- Department of CSE, National Institute of Textile Engineering and Research (NITER), Constituent Institute of the University of Dhaka, Savar, Dhaka-1350
| | - Mohammad Sayduzzaman
- Department of CSE, National Institute of Textile Engineering and Research (NITER), Constituent Institute of the University of Dhaka, Savar, Dhaka-1350
| | - Shahab S. Band
- Department of Information Management, International Graduate School of Artificial Intelligence, National Yunlin University of Science and Technology, Taiwan
| |
Collapse
|
13
|
Evans HG, Murphy MF, Foy R, Dhiman P, Green L, Kotze A, von Neree L, Palmer AJ, Robinson SE, Shah A, Tomini F, Trompeter S, Warnakulasuriya S, Wong WK, Stanworth SJ. Harnessing the potential of data-driven strategies to optimise transfusion practice. Br J Haematol 2024; 204:74-85. [PMID: 37964471 DOI: 10.1111/bjh.19158] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2023] [Revised: 09/24/2023] [Accepted: 10/03/2023] [Indexed: 11/16/2023]
Abstract
No one doubts the significant variation in the practice of transfusion medicine. Common examples are the variability in transfusion thresholds and the use of tranexamic acid for surgery with likely high blood loss despite evidence-based standards. There is a long history of applying different strategies to address this variation, including education, clinical guidelines, audit and feedback, but the effectiveness and cost-effectiveness of these initiatives remains unclear. Advances in computerised decision support systems and the application of novel electronic capabilities offer alternative approaches to improving transfusion practice. In England, the National Institute for Health and Care Research funded a Blood and Transplant Research Unit (BTRU) programme focussing on 'A data-enabled programme of research to improve transfusion practices'. The overarching aim of the BTRU is to accelerate the development of data-driven methods to optimise the use of blood and transfusion alternatives, and to integrate them within routine practice to improve patient outcomes. One particular area of focus is implementation science to address variation in practice.
Collapse
Affiliation(s)
- H G Evans
- NIHR Blood and Transplant Research Unit in Data Driven Transfusion Practice, Radcliffe Department of Medicine, University of Oxford, Oxford, UK
| | - M F Murphy
- NIHR Blood and Transplant Research Unit in Data Driven Transfusion Practice, Radcliffe Department of Medicine, University of Oxford, Oxford, UK
- Oxford University Hospitals NHS Foundation Trust, John Radcliffe Hospital, Oxford, UK
- NHS Blood and Transplant, John Radcliffe Hospital, Oxford, UK
| | - R Foy
- Leeds Institute of Health Sciences, University of Leeds, Leeds, UK
| | - P Dhiman
- Centre for Statistics in Medicine, Botnar Research Centre, Oxford, UK
| | - L Green
- Blizard Institute, Queen Mary University of London, London, UK
- Barts Health NHS Trust, London, UK
- NHS Blood and Transplant, London, UK
| | - A Kotze
- Leeds Teaching Hospitals, Leeds, UK
| | - L von Neree
- University College London Hospitals NHS Foundation Trust, London, UK
| | - A J Palmer
- Nuffield Orthopaedic Centre, Oxford University NHS Foundation Trust, Oxford, UK
| | - S E Robinson
- Guy's and St Thomas' NHS Foundation Trust, London, UK
| | - A Shah
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | - F Tomini
- Queen Mary University of London, London, UK
| | - S Trompeter
- University College London Hospitals NHS Foundation Trust, London, UK
- University College London, London, UK
| | - S Warnakulasuriya
- University College London Hospitals NHS Foundation Trust, London, UK
- University College London, London, UK
| | - W K Wong
- Cambridge University Hospitals NHS Foundation Trust, Cambridge, UK
| | - S J Stanworth
- NIHR Blood and Transplant Research Unit in Data Driven Transfusion Practice, Radcliffe Department of Medicine, University of Oxford, Oxford, UK
- Oxford University Hospitals NHS Foundation Trust, John Radcliffe Hospital, Oxford, UK
- NHS Blood and Transplant, John Radcliffe Hospital, Oxford, UK
| |
Collapse
|
14
|
Maynard S, Farrington J, Alimam S, Evans H, Li K, Wong WK, Stanworth SJ. Machine learning in transfusion medicine: A scoping review. Transfusion 2024; 64:162-184. [PMID: 37950535 DOI: 10.1111/trf.17582] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2023] [Revised: 09/25/2023] [Accepted: 09/27/2023] [Indexed: 11/12/2023]
Affiliation(s)
- Suzanne Maynard
- Medical Sciences Division, Radcliffe Department of Medicine, University of Oxford, Oxford, UK
- NIHR Blood and Transplant Research Unit in Data Driven Transfusion Practice, Nuffield Division of Clinical Laboratory Sciences, Radcliffe Department of Medicine, University of Oxford, Oxford, UK
- NHSBT and Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Joseph Farrington
- Institute of Health Informatics, University College London, London, UK
| | - Samah Alimam
- Haematology Department, University College London Hospitals NHS Foundation Trust, London, UK
| | - Hayley Evans
- NIHR Blood and Transplant Research Unit in Data Driven Transfusion Practice, Nuffield Division of Clinical Laboratory Sciences, Radcliffe Department of Medicine, University of Oxford, Oxford, UK
| | - Kezhi Li
- Institute of Health Informatics, University College London, London, UK
| | - Wai Keong Wong
- Director of Digital, Cambridge University Hospitals NHS Foundation Trust, Cambridge, UK
| | - Simon J Stanworth
- Medical Sciences Division, Radcliffe Department of Medicine, University of Oxford, Oxford, UK
- NIHR Blood and Transplant Research Unit in Data Driven Transfusion Practice, Nuffield Division of Clinical Laboratory Sciences, Radcliffe Department of Medicine, University of Oxford, Oxford, UK
- NHSBT and Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| |
Collapse
|
15
|
Sumner J, Lim HW, Chong LS, Bundele A, Mukhopadhyay A, Kayambu G. Artificial intelligence in physical rehabilitation: A systematic review. Artif Intell Med 2023; 146:102693. [PMID: 38042593 DOI: 10.1016/j.artmed.2023.102693] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2022] [Revised: 10/25/2023] [Accepted: 10/29/2023] [Indexed: 12/04/2023]
Abstract
BACKGROUND Physical disabilities become more common with advancing age. Rehabilitation restores function, maintaining independence for longer. However, the poor availability and accessibility of rehabilitation limits its clinical impact. Artificial Intelligence (AI) guided interventions have improved many domains of healthcare, but whether rehabilitation can benefit from AI remains unclear. METHODS We conducted a systematic review of AI-supported physical rehabilitation technology tested in the clinical setting to understand: 1) availability of AI-supported physical rehabilitation technology; 2) its clinical effect; 3) and the barriers and facilitators to implementation. We searched in MEDLINE, EMBASE, CINAHL, Science Citation Index (Web of Science), CIRRIE (now NARIC), and OpenGrey. RESULTS We identified 9054 articles and included 28 projects. AI solutions spanned five categories: App-based systems, robotic devices that replace function, robotic devices that restore function, gaming systems and wearables. We identified five randomised controlled trials (RCTs), which evaluated outcomes relating to physical function, activity, pain, and health-related quality of life. The clinical effects were inconsistent. Implementation barriers included technology literacy, reliability, and user fatigue. Enablers included greater access to rehabilitation programmes, remote monitoring of progress, reduction in manpower requirements and lower cost. CONCLUSION Application of AI in physical rehabilitation is a growing field, but clinical effects have yet to be studied rigorously. Developers must strive to conduct robust clinical evaluations in the real-world setting and appraise post implementation experiences.
Collapse
Affiliation(s)
- Jennifer Sumner
- Medical Affairs - Research Innovation & Enterprise, Alexandra Hospital, National University Health System, Singapore.
| | - Hui Wen Lim
- Medical Affairs - Research Innovation & Enterprise, Alexandra Hospital, National University Health System, Singapore
| | - Lin Siew Chong
- Medical Affairs - Research Innovation & Enterprise, Alexandra Hospital, National University Health System, Singapore
| | - Anjali Bundele
- Medical Affairs - Research Innovation & Enterprise, Alexandra Hospital, National University Health System, Singapore
| | - Amartya Mukhopadhyay
- Yong Loo Lin School of Medicine, Department of Medicine, National University of Singapore, Singapore; Medical Affairs - Research Innovation & Enterprise, Alexandra Hospital, National University Health System, Singapore; Division of Respiratory and Critical Care Medicine, Department of Medicine, National University Hospital, Singapore
| | - Geetha Kayambu
- Department of Rehabilitation, National University Hospital, Singapore
| |
Collapse
|
16
|
Aminizadeh S, Heidari A, Toumaj S, Darbandi M, Navimipour NJ, Rezaei M, Talebi S, Azad P, Unal M. The applications of machine learning techniques in medical data processing based on distributed computing and the Internet of Things. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 241:107745. [PMID: 37579550 DOI: 10.1016/j.cmpb.2023.107745] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Revised: 07/15/2023] [Accepted: 08/02/2023] [Indexed: 08/16/2023]
Abstract
Medical data processing has grown into a prominent topic in the latest decades with the primary goal of maintaining patient data via new information technologies, including the Internet of Things (IoT) and sensor technologies, which generate patient indexes in hospital data networks. Innovations like distributed computing, Machine Learning (ML), blockchain, chatbots, wearables, and pattern recognition can adequately enable the collection and processing of medical data for decision-making in the healthcare era. Particularly, to assist experts in the disease diagnostic process, distributed computing is beneficial by digesting huge volumes of data swiftly and producing personalized smart suggestions. On the other side, the current globe is confronting an outbreak of COVID-19, so an early diagnosis technique is crucial to lowering the fatality rate. ML systems are beneficial in aiding radiologists in examining the incredible amount of medical images. Nevertheless, they demand a huge quantity of training data that must be unified for processing. Hence, developing Deep Learning (DL) confronts multiple issues, such as conventional data collection, quality assurance, knowledge exchange, privacy preservation, administrative laws, and ethical considerations. In this research, we intend to convey an inclusive analysis of the most recent studies in distributed computing platform applications based on five categorized platforms, including cloud computing, edge, fog, IoT, and hybrid platforms. So, we evaluated 27 articles regarding the usage of the proposed framework, deployed methods, and applications, noting the advantages, drawbacks, and the applied dataset and screening the security mechanism and the presence of the Transfer Learning (TL) method. As a result, it was proved that most recent research (about 43%) used the IoT platform as the environment for the proposed architecture, and most of the studies (about 46%) were done in 2021. In addition, the most popular utilized DL algorithm was the Convolutional Neural Network (CNN), with a percentage of 19.4%. Hence, despite how technology changes, delivering appropriate therapy for patients is the primary aim of healthcare-associated departments. Therefore, further studies are recommended to develop more functional architectures based on DL and distributed environments and better evaluate the present healthcare data analysis models.
Collapse
Affiliation(s)
| | - Arash Heidari
- Department of Computer Engineering, Tabriz Branch, Islamic Azad University, Tabriz, Iran; Department of Software Engineering, Haliç University, Istanbul, Turkiye.
| | - Shiva Toumaj
- Urmia University of Medical Sciences, Urmia, Iran
| | - Mehdi Darbandi
- Department of Electrical and Electronic Engineering, Eastern Mediterranean University, Gazimagusa 99628, Turkiye
| | - Nima Jafari Navimipour
- Department of Computer Engineering, Kadir Has University, Istanbul, Turkiye; Future Technology Research Center, National Yunlin University of Science and Technology, Douliou, Yunlin 64002, Taiwan.
| | - Mahsa Rezaei
- Tabriz University of Medical Sciences, Faculty of Surgery, Tabriz, Iran
| | - Samira Talebi
- Department of Computer Science, University of Texas at San Antonio, TX, USA
| | - Poupak Azad
- Department of Computer Science, University of Manitoba, Winnipeg, Canada
| | - Mehmet Unal
- Department of Computer Engineering, Nisantasi University, Istanbul, Turkiye
| |
Collapse
|
17
|
Soria Bretones C, Roncero Parra C, Cascón J, Borja AL, Mateo Sotos J. Automatic identification of schizophrenia employing EEG records analyzed with deep learning algorithms. Schizophr Res 2023; 261:36-46. [PMID: 37690170 DOI: 10.1016/j.schres.2023.09.010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/26/2022] [Revised: 07/24/2023] [Accepted: 09/04/2023] [Indexed: 09/12/2023]
Abstract
Electroencephalography is a method of detecting and analyzing electrical activity in the brain. This electrical activity can be recorded and processed to aid in the clinical diagnosis of mental disorders. In this study, a novel system for classifying schizophrenia patients from EEG recordings is presented. The developed algorithm decomposes the EEG signals into a system of radial basis functions using the method of fuzzy means. This decomposition helps to obtain the information from the various electrodes of the EEG and allows separating between healthy controls and patients with schizophrenia. The proposed method has been compared with classical machine learning algorithms, such as, K-Nearest Neighbor, Adaboost, Support Vector Machine, and Bayesian Linear Discriminant Analysis. The results show that the proposed method obtains the highest values in terms of balanced accuracy, recall, precision and F1 score, close to 93 % in all cases. The model developed in this study can be implemented in brain activity analysis systems that help in the prediction of patients with schizophrenia.
Collapse
Affiliation(s)
| | - Carlos Roncero Parra
- Departamento de Sistema Informáticos, Universidad de Castilla-La Mancha, 02071 Albacete, Spain
| | - Joaquín Cascón
- Departamento de Ingeniería Eléctrica, Electrónica, Automática y Comunicaciones, Universidad de Castilla-La Mancha, 02071 Albacete, Spain; Expert Group in Medical Analysis, Instituto de Tecnología, Construcción y Telecomunicaciones, Universidad de Castilla-La Mancha, 16071 Cuenca, Spain
| | - Alejandro L Borja
- Departamento de Ingeniería Eléctrica, Electrónica, Automática y Comunicaciones, Universidad de Castilla-La Mancha, 02071 Albacete, Spain.
| | - Jorge Mateo Sotos
- Departamento de Ingeniería Eléctrica, Electrónica, Automática y Comunicaciones, Universidad de Castilla-La Mancha, 02071 Albacete, Spain; Expert Group in Medical Analysis, Instituto de Tecnología, Construcción y Telecomunicaciones, Universidad de Castilla-La Mancha, 16071 Cuenca, Spain
| |
Collapse
|
18
|
Malorgio A, Henckert D, Schweiger G, Braun J, Zacharowski K, Raimann FJ, Piekarski F, Meybohm P, Hottenrott S, Froehlich C, Spahn DR, Noethiger CB, Tscholl DW, Roche TR. Using Visual Patient to Show Vital Sign Predictions, a Computer-Based Mixed Quantitative and Qualitative Simulation Study. Diagnostics (Basel) 2023; 13:3281. [PMID: 37892102 PMCID: PMC10606017 DOI: 10.3390/diagnostics13203281] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2023] [Revised: 10/16/2023] [Accepted: 10/19/2023] [Indexed: 10/29/2023] Open
Abstract
BACKGROUND Machine learning can analyze vast amounts of data and make predictions for events in the future. Our group created machine learning models for vital sign predictions. To transport the information of these predictions without numbers and numerical values and make them easily usable for human caregivers, we aimed to integrate them into the Philips Visual-Patient-avatar, an avatar-based visualization of patient monitoring. METHODS We conducted a computer-based simulation study with 70 participants in 3 European university hospitals. We validated the vital sign prediction visualizations by testing their identification by anesthesiologists and intensivists. Each prediction visualization consisted of a condition (e.g., low blood pressure) and an urgency (a visual indication of the timespan in which the condition is expected to occur). To obtain qualitative user feedback, we also conducted standardized interviews and derived statements that participants later rated in an online survey. RESULTS The mixed logistic regression model showed 77.9% (95% CI 73.2-82.0%) correct identification of prediction visualizations (i.e., condition and urgency both correctly identified) and 93.8% (95% CI 93.7-93.8%) for conditions only (i.e., without considering urgencies). A total of 49 out of 70 participants completed the online survey. The online survey participants agreed that the prediction visualizations were fun to use (32/49, 65.3%), and that they could imagine working with them in the future (30/49, 61.2%). They also agreed that identifying the urgencies was difficult (32/49, 65.3%). CONCLUSIONS This study found that care providers correctly identified >90% of the conditions (i.e., without considering urgencies). The accuracy of identification decreased when considering urgencies in addition to conditions. Therefore, in future development of the technology, we will focus on either only displaying conditions (without urgencies) or improving the visualizations of urgency to enhance usability for human users.
Collapse
Affiliation(s)
- Amos Malorgio
- Institute of Anesthesiology, University Hospital Zurich, University of Zurich, 8091 Zurich, Switzerland; (A.M.); (D.H.); (G.S.); (D.R.S.); (C.B.N.); (D.W.T.)
| | - David Henckert
- Institute of Anesthesiology, University Hospital Zurich, University of Zurich, 8091 Zurich, Switzerland; (A.M.); (D.H.); (G.S.); (D.R.S.); (C.B.N.); (D.W.T.)
| | - Giovanna Schweiger
- Institute of Anesthesiology, University Hospital Zurich, University of Zurich, 8091 Zurich, Switzerland; (A.M.); (D.H.); (G.S.); (D.R.S.); (C.B.N.); (D.W.T.)
| | - Julia Braun
- Departments of Epidemiology and Biostatistics, Epidemiology, Biostatistics and Prevention Institute, University of Zurich, 8001 Zurich, Switzerland;
| | - Kai Zacharowski
- Department of Anesthesiology, Intensive Care Medicine, and Pain Therapy, University Hospital Frankfurt, Goethe University Frankfurt, 60323 Frankfurt, Germany; (K.Z.); (F.J.R.); (F.P.)
| | - Florian J. Raimann
- Department of Anesthesiology, Intensive Care Medicine, and Pain Therapy, University Hospital Frankfurt, Goethe University Frankfurt, 60323 Frankfurt, Germany; (K.Z.); (F.J.R.); (F.P.)
| | - Florian Piekarski
- Department of Anesthesiology, Intensive Care Medicine, and Pain Therapy, University Hospital Frankfurt, Goethe University Frankfurt, 60323 Frankfurt, Germany; (K.Z.); (F.J.R.); (F.P.)
| | - Patrick Meybohm
- Department of Anesthesiology, Intensive Care, Emergency, and Pain Medicine, University Hospital Wuerzburg, 97070 Wuerzburg, Germany; (P.M.); (S.H.); (C.F.)
| | - Sebastian Hottenrott
- Department of Anesthesiology, Intensive Care, Emergency, and Pain Medicine, University Hospital Wuerzburg, 97070 Wuerzburg, Germany; (P.M.); (S.H.); (C.F.)
| | - Corinna Froehlich
- Department of Anesthesiology, Intensive Care, Emergency, and Pain Medicine, University Hospital Wuerzburg, 97070 Wuerzburg, Germany; (P.M.); (S.H.); (C.F.)
| | - Donat R. Spahn
- Institute of Anesthesiology, University Hospital Zurich, University of Zurich, 8091 Zurich, Switzerland; (A.M.); (D.H.); (G.S.); (D.R.S.); (C.B.N.); (D.W.T.)
| | - Christoph B. Noethiger
- Institute of Anesthesiology, University Hospital Zurich, University of Zurich, 8091 Zurich, Switzerland; (A.M.); (D.H.); (G.S.); (D.R.S.); (C.B.N.); (D.W.T.)
| | - David W. Tscholl
- Institute of Anesthesiology, University Hospital Zurich, University of Zurich, 8091 Zurich, Switzerland; (A.M.); (D.H.); (G.S.); (D.R.S.); (C.B.N.); (D.W.T.)
| | - Tadzio R. Roche
- Institute of Anesthesiology, University Hospital Zurich, University of Zurich, 8091 Zurich, Switzerland; (A.M.); (D.H.); (G.S.); (D.R.S.); (C.B.N.); (D.W.T.)
| |
Collapse
|
19
|
Ahmed MI, Spooner B, Isherwood J, Lane M, Orrock E, Dennison A. A Systematic Review of the Barriers to the Implementation of Artificial Intelligence in Healthcare. Cureus 2023; 15:e46454. [PMID: 37927664 PMCID: PMC10623210 DOI: 10.7759/cureus.46454] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/03/2023] [Indexed: 11/07/2023] Open
Abstract
Artificial intelligence (AI) is expected to improve healthcare outcomes by facilitating early diagnosis, reducing the medical administrative burden, aiding drug development, personalising medical and oncological management, monitoring healthcare parameters on an individual basis, and allowing clinicians to spend more time with their patients. In the post-pandemic world where there is a drive for efficient delivery of healthcare and manage long waiting times for patients to access care, AI has an important role in supporting clinicians and healthcare systems to streamline the care pathways and provide timely and high-quality care for the patients. Despite AI technologies being used in healthcare for some decades, and all the theoretical potential of AI, the uptake in healthcare has been uneven and slower than anticipated and there remain a number of barriers, both overt and covert, which have limited its incorporation. This literature review highlighted barriers in six key areas: ethical, technological, liability and regulatory, workforce, social, and patient safety barriers. Defining and understanding the barriers preventing the acceptance and implementation of AI in the setting of healthcare will enable clinical staff and healthcare leaders to overcome the identified hurdles and incorporate AI technologies for the benefit of patients and clinical staff.
Collapse
Affiliation(s)
- Molla Imaduddin Ahmed
- Paediatric Respiratory Medicine, University Hospitals of Leicester NHS Trust, Leicester, GBR
| | - Brendan Spooner
- Intensive Care and Anaesthesia, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, GBR
| | - John Isherwood
- Hepatobiliary and Pancreatic Surgery, University Hospitals of Leicester NHS Trust, Leicester, GBR
| | - Mark Lane
- Ophthalmology, Birmingham and Midland Eye Centre, Birmingham, GBR
| | - Emma Orrock
- Head of Clinical Senates, East and West Midlands Clinical Senate, Leicester, GBR
| | - Ashley Dennison
- Hepatobiliary and Pancreatic Surgery, University Hospitals of Leicester NHS Trust, Leicester, GBR
| |
Collapse
|
20
|
Shimada K, Tsuneto S. Novel method for predicting nonvisible symptoms using machine learning in cancer palliative care. Sci Rep 2023; 13:12088. [PMID: 37495739 PMCID: PMC10371999 DOI: 10.1038/s41598-023-39119-0] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2022] [Accepted: 07/20/2023] [Indexed: 07/28/2023] Open
Abstract
End-of-life patients with cancer may find expressing their symptoms difficult if they can no longer communicate verbally because of deteriorating health. In this study, we assessed these symptoms using machine learning, which has excellent predictive capabilities and has recently been applied in healthcare. We performed a retrospective clinical survey involving 213 patients with cancer from August 2015 to August 2016. We divided the reported symptoms into two groups-visible and nonvisible symptoms. We used decision tree analysis, an analytical machine learning method that organizes and analyzes information in the form of a tree diagram to visually represent the information structure. Our machine learning model used patient background data and visible symptoms to predict nonvisible symptoms: pain, dyspnea, fatigue, drowsiness, anxiety, delirium, inadequate informed consent, and spiritual issues. The highest and/or lowest values for prediction accuracy, sensitivity, and specificity were 88.0%/55.5%, 84.9%/3.3%, and 96.7%/24.1%, respectively. This work will facilitate better assessment and management of symptoms in patients with cancer. This study was the first to predict nonvisible symptoms using decision tree analyses for patients with cancer receiving palliative care. Notably, applications based on our results may assess symptoms to the same extent as healthcare professionals.
Collapse
Affiliation(s)
- Kazuki Shimada
- Department of Palliative Medicine, Kyoto University Hospital, 54 Kawaharacho, Shogoin, Sakyo-ku, Kyoto, 606-8507, Japan.
| | - Satoru Tsuneto
- Department of Palliative Medicine, Graduate School of Medicine, Kyoto University, Kyoto University Hospital, 53 Kawaharacho, Shogoin, Sakyo-ku, Kyoto, 606-8507, Japan
| |
Collapse
|
21
|
Verma AA, Pou-Prom C, McCoy LG, Murray J, Nestor B, Bell S, Mourad O, Fralick M, Friedrich J, Ghassemi M, Mamdani M. Developing and Validating a Prediction Model For Death or Critical Illness in Hospitalized Adults, an Opportunity for Human-Computer Collaboration. Crit Care Explor 2023; 5:e0897. [PMID: 37151895 PMCID: PMC10155889 DOI: 10.1097/cce.0000000000000897] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/09/2023] Open
Abstract
Hospital early warning systems that use machine learning (ML) to predict clinical deterioration are increasingly being used to aid clinical decision-making. However, it is not known how ML predictions complement physician and nurse judgment. Our objective was to train and validate a ML model to predict patient deterioration and compare model predictions with real-world physician and nurse predictions. DESIGN Retrospective and prospective cohort study. SETTING Academic tertiary care hospital. PATIENTS Adult general internal medicine hospitalizations. MEASUREMENTS AND MAIN RESULTS We developed and validated a neural network model to predict in-hospital death and ICU admission in 23,528 hospitalizations between April 2011 and April 2019. We then compared model predictions with 3,374 prospectively collected predictions from nurses, residents, and attending physicians about their own patients in 960 hospitalizations between April 30, and August 28, 2019. ML model predictions achieved clinician-level accuracy for predicting ICU admission or death (ML median F1 score 0.32 [interquartile range (IQR) 0.30-0.34], AUC 0.77 [IQ 0.76-0.78]; clinicians median F1-score 0.33 [IQR 0.30-0.35], AUC 0.64 [IQR 0.63-0.66]). ML predictions were more accurate than clinicians for ICU admission. Of all ICU admissions and deaths, 36% occurred in hospitalizations where the model and clinicians disagreed. Combining human and model predictions detected 49% of clinical deterioration events, improving sensitivity by 16% compared with clinicians alone and 24% compared with the model alone while maintaining a positive predictive value of 33%, thus keeping false alarms at a clinically acceptable level. CONCLUSIONS ML models can complement clinician judgment to predict clinical deterioration in hospital. These findings demonstrate important opportunities for human-computer collaboration to improve prognostication and personalized medicine in hospital.
Collapse
Affiliation(s)
- Amol A Verma
- St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Department of Medicine, University of Toronto, Toronto, ON, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada
| | - Chloe Pou-Prom
- St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Liam G McCoy
- Department of Medicine, University of Toronto, Toronto, ON, Canada
| | - Joshua Murray
- St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Bret Nestor
- Department of Computer Science, University of Toronto, Toronto, ON, Canada
- Vector Institute, Toronto, ON, Canada
| | - Shirley Bell
- St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Ophyr Mourad
- St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Department of Medicine, University of Toronto, Toronto, ON, Canada
| | - Michael Fralick
- Department of Medicine, University of Toronto, Toronto, ON, Canada
- Sinai Health System, Toronto, ON, Canada
| | - Jan Friedrich
- St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Department of Medicine, University of Toronto, Toronto, ON, Canada
| | - Marzyeh Ghassemi
- Vector Institute, Toronto, ON, Canada
- Massachusetts Institute of Technology, Cambridge, MA
| | - Muhammad Mamdani
- St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Department of Medicine, University of Toronto, Toronto, ON, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada
- Vector Institute, Toronto, ON, Canada
- Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
22
|
Rust LOH, Gorham TJ, Bambach S, Bode RS, Maa T, Hoffman JM, Rust SW. The Deterioration Risk Index: Developing and Piloting a Machine Learning Algorithm to Reduce Pediatric Inpatient Deterioration. Pediatr Crit Care Med 2023; 24:322-333. [PMID: 36735282 DOI: 10.1097/pcc.0000000000003186] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
Abstract
OBJECTIVES Develop and deploy a disease cohort-based machine learning algorithm for timely identification of hospitalized pediatric patients at risk for clinical deterioration that outperforms our existing situational awareness program. DESIGN Retrospective cohort study. SETTING Nationwide Children's Hospital, a freestanding, quaternary-care, academic children's hospital in Columbus, OH. PATIENTS All patients admitted to inpatient units participating in the preexisting situational awareness program from October 20, 2015, to December 31, 2019, excluding patients over 18 years old at admission and those with a neonatal ICU stay during their hospitalization. INTERVENTIONS We developed separate algorithms for cardiac, malignancy, and general cohorts via lasso-regularized logistic regression. Candidate model predictors included vital signs, supplemental oxygen, nursing assessments, early warning scores, diagnoses, lab results, and situational awareness criteria. Model performance was characterized in clinical terms and compared with our previous situational awareness program based on a novel retrospective validation approach. Simulations with frontline staff, prior to clinical implementation, informed user experience and refined interdisciplinary workflows. Model implementation was piloted on cardiology and hospital medicine units in early 2021. MEASUREMENTS AND MAIN RESULTS The Deterioration Risk Index (DRI) was 2.4 times as sensitive as our existing situational awareness program (sensitivities of 53% and 22%, respectively; p < 0.001) and required 2.3 times fewer alarms per detected event (121 DRI alarms per detected event vs 276 for existing program). Notable improvements were a four-fold sensitivity gain for the cardiac diagnostic cohort (73% vs 18%; p < 0.001) and a three-fold gain (81% vs 27%; p < 0.001) for the malignancy diagnostic cohort. Postimplementation pilot results over 18 months revealed a 77% reduction in deterioration events (three events observed vs 13.1 expected, p = 0.001). CONCLUSIONS The etiology of pediatric inpatient deterioration requires acknowledgement of the unique pathophysiology among cardiology and oncology patients. Selection and weighting of diverse candidate risk factors via machine learning can produce a more sensitive early warning system for clinical deterioration. Leveraging preexisting situational awareness platforms and accounting for operational impacts of model implementation are key aspects to successful bedside translation.
Collapse
Affiliation(s)
- Laura O H Rust
- Division of Clinical Informatics, Department of Pediatrics, The Ohio State University College of Medicine, Nationwide Children's Hospital, Columbus, OH
- Division of Emergency Medicine, Department of Pediatrics, The Ohio State University College of Medicine, Nationwide Children's Hospital, Columbus, OH
- Center for Clinical Excellence, Nationwide Children's Hospital, Columbus, OH
- Department of Pediatrics, The Ohio State University College of Medicine, Columbus, OH
- Information Technology Research & Innovation, Nationwide Children's Hospital, Columbus, OH
- Division of Hospital Pediatrics, Department of Pediatrics, The Ohio State University College of Medicine, Nationwide Children's Hospital, Columbus, OH
- Division of Pediatric Critical Care, Department of Pediatrics, The Ohio State University College of Medicine, Nationwide Children's Hospital, Columbus, OH
| | - Tyler J Gorham
- Information Technology Research & Innovation, Nationwide Children's Hospital, Columbus, OH
| | - Sven Bambach
- Information Technology Research & Innovation, Nationwide Children's Hospital, Columbus, OH
| | - Ryan S Bode
- Center for Clinical Excellence, Nationwide Children's Hospital, Columbus, OH
- Department of Pediatrics, The Ohio State University College of Medicine, Columbus, OH
- Division of Hospital Pediatrics, Department of Pediatrics, The Ohio State University College of Medicine, Nationwide Children's Hospital, Columbus, OH
| | - Tensing Maa
- Center for Clinical Excellence, Nationwide Children's Hospital, Columbus, OH
- Department of Pediatrics, The Ohio State University College of Medicine, Columbus, OH
- Division of Pediatric Critical Care, Department of Pediatrics, The Ohio State University College of Medicine, Nationwide Children's Hospital, Columbus, OH
| | - Jeffrey M Hoffman
- Division of Clinical Informatics, Department of Pediatrics, The Ohio State University College of Medicine, Nationwide Children's Hospital, Columbus, OH
- Division of Emergency Medicine, Department of Pediatrics, The Ohio State University College of Medicine, Nationwide Children's Hospital, Columbus, OH
- Center for Clinical Excellence, Nationwide Children's Hospital, Columbus, OH
- Department of Pediatrics, The Ohio State University College of Medicine, Columbus, OH
- Information Technology Research & Innovation, Nationwide Children's Hospital, Columbus, OH
- Division of Hospital Pediatrics, Department of Pediatrics, The Ohio State University College of Medicine, Nationwide Children's Hospital, Columbus, OH
- Division of Pediatric Critical Care, Department of Pediatrics, The Ohio State University College of Medicine, Nationwide Children's Hospital, Columbus, OH
| | - Steven W Rust
- Information Technology Research & Innovation, Nationwide Children's Hospital, Columbus, OH
| |
Collapse
|
23
|
Iacobelli F, Yang A, Tom L, Leung IS, Crissman J, Salgado R, Simon M. Predicting Social Determinants of Health in Patient Navigation: Case Study. JMIR Form Res 2023; 7:e42683. [PMID: 36976634 PMCID: PMC10131925 DOI: 10.2196/42683] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2022] [Revised: 01/12/2023] [Accepted: 02/07/2023] [Indexed: 02/10/2023] Open
Abstract
BACKGROUND Patient navigation (PN) programs have demonstrated efficacy in improving health outcomes for marginalized populations across a range of clinical contexts by addressing barriers to health care, including social determinants of health (SDoHs). However, it can be challenging for navigators to identify SDoHs by asking patients directly because of many factors, including patients' reluctance to disclose information, communication barriers, and the variable resources and experience levels of patient navigators. Navigators could benefit from strategies that augment their ability to gather SDoH data. Machine learning can be leveraged as one of these strategies to identify SDoH-related barriers. This could further improve health outcomes, particularly in underserved populations. OBJECTIVE In this formative study, we explored novel machine learning-based approaches to predict SDoHs in 2 Chicago area PN studies. In the first approach, we applied machine learning to data that include comments and interaction details between patients and navigators, whereas the second approach augmented patients' demographic information. This paper presents the results of these experiments and provides recommendations for data collection and the application of machine learning techniques more generally to the problem of predicting SDoHs. METHODS We conducted 2 experiments to explore the feasibility of using machine learning to predict patients' SDoHs using data collected from PN research. The machine learning algorithms were trained on data collected from 2 Chicago area PN studies. In the first experiment, we compared several machine learning algorithms (logistic regression, random forest, support vector machine, artificial neural network, and Gaussian naive Bayes) to predict SDoHs from both patient demographics and navigator's encounter data over time. In the second experiment, we used multiclass classification with augmented information, such as transportation time to a hospital, to predict multiple SDoHs for each patient. RESULTS In the first experiment, the random forest classifier achieved the highest accuracy among the classifiers tested. The overall accuracy to predict SDoHs was 71.3%. In the second experiment, multiclass classification effectively predicted a few patients' SDoHs based purely on demographic and augmented data. The best accuracy of these predictions overall was 73%. However, both experiments yielded high variability in individual SDoH predictions and correlations that become salient among SDoHs. CONCLUSIONS To our knowledge, this study is the first approach to applying PN encounter data and multiclass learning algorithms to predict SDoHs. The experiments discussed yielded valuable lessons, including the awareness of model limitations and bias, planning for standardization of data sources and measurement, and the need to identify and anticipate the intersectionality and clustering of SDoHs. Although our focus was on predicting patients' SDoHs, machine learning can have a broad range of applications in the field of PN, from tailoring intervention delivery (eg, supporting PN decision-making) to informing resource allocation for measurement, and PN supervision.
Collapse
Affiliation(s)
- Francisco Iacobelli
- Department of Computer Science, Northeastern Illinois University, Chicago, IL, United States
- Center for Advancing Safety of Machine Intelligence, Northwestern University, Evanston, IL, United States
| | - Anna Yang
- Center for Health Equity Transformation, Feinberg School of Medicine Chicago, Northwestern University, Chicago, IL, United States
- Department of Obstetrics and Gynecology, Feinberg School of Medicine Chicago, Northwestern University, Chicago, IL, United States
| | - Laura Tom
- Center for Health Equity Transformation, Feinberg School of Medicine Chicago, Northwestern University, Chicago, IL, United States
- Department of Obstetrics and Gynecology, Feinberg School of Medicine Chicago, Northwestern University, Chicago, IL, United States
| | - Ivy S Leung
- Center for Health Equity Transformation, Feinberg School of Medicine Chicago, Northwestern University, Chicago, IL, United States
- Department of Obstetrics and Gynecology, Feinberg School of Medicine Chicago, Northwestern University, Chicago, IL, United States
| | - John Crissman
- Department of Computer Science, Northeastern Illinois University, Chicago, IL, United States
| | - Rufino Salgado
- Department of Computer Science, Northeastern Illinois University, Chicago, IL, United States
| | - Melissa Simon
- Center for Health Equity Transformation, Feinberg School of Medicine Chicago, Northwestern University, Chicago, IL, United States
- Department of Obstetrics and Gynecology, Feinberg School of Medicine Chicago, Northwestern University, Chicago, IL, United States
- Robert H. Lurie Comprehensive Cancer Center, Feinberg School of Medicine Chicago, Northwestern University, Chicago, IL, United States
| |
Collapse
|
24
|
Wu A, Xue P, Abulizi G, Tuerxun D, Rezhake R, Qiao Y. Artificial intelligence in colposcopic examination: A promising tool to assist junior colposcopists. Front Med (Lausanne) 2023; 10:1060451. [PMID: 37056736 PMCID: PMC10088560 DOI: 10.3389/fmed.2023.1060451] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2022] [Accepted: 02/08/2023] [Indexed: 03/17/2023] Open
Abstract
IntroductionWell-trained colposcopists are in huge shortage worldwide, especially in low-resource areas. Here, we aimed to evaluate the Colposcopic Artificial Intelligence Auxiliary Diagnostic System (CAIADS) to detect abnormalities based on digital colposcopy images, especially focusing on its role in assisting junior colposcopist to correctly identify the lesion areas where biopsy should be performed.Materials and methodsThis is a hospital-based retrospective study, which recruited the women who visited colposcopy clinics between September 2021 to January 2022. A total of 366 of 1,146 women with complete medical information recorded by a senior colposcopist and valid histology results were included. Anonymized colposcopy images were reviewed by CAIADS and a junior colposcopist separately, and the junior colposcopist reviewed the colposcopy images with CAIADS results (named CAIADS-Junior). The diagnostic accuracy and biopsy efficiency of CAIADS and CAIADS-Junior were assessed in detecting cervical intraepithelial neoplasia grade 2 or worse (CIN2+), CIN3+, and cancer in comparison with the senior and junior colposcipists. The factors influencing the accuracy of CAIADS were explored.ResultsFor CIN2 + and CIN3 + detection, CAIADS showed a sensitivity at ~80%, which was not significantly lower than the sensitivity achieved by the senior colposcopist (for CIN2 +: 80.6 vs. 91.3%, p = 0.061 and for CIN3 +: 80.0 vs. 90.0%, p = 0.189). The sensitivity of the junior colposcopist was increased significantly with the assistance of CAIADS (for CIN2 +: 95.1 vs. 79.6%, p = 0.002 and for CIN3 +: 97.1 vs. 85.7%, p = 0.039) and was comparable to those of the senior colposcopists (for CIN2 +: 95.1 vs. 91.3%, p = 0.388 and for CIN3 +: 97.1 vs. 90.0%, p = 0.125). In detecting cervical cancer, CAIADS achieved the highest sensitivity at 100%. For all endpoints, CAIADS showed the highest specificity (55–64%) and positive predictive values compared to both senior and junior colposcopists. When CIN grades became higher, the average biopsy numbers decreased for the subspecialists and CAIADS required a minimum number of biopsies to detect per case (2.2–2.6 cut-points). Meanwhile, the biopsy sensitivity of the junior colposcopist was the lowest, but the CAIADS-assisted junior colposcopist achieved a higher biopsy sensitivity.ConclusionColposcopic Artificial Intelligence Auxiliary Diagnostic System could assist junior colposcopists to improve diagnostic accuracy and biopsy efficiency, which might be a promising solution to improve the quality of cervical cancer screening in low-resource settings.
Collapse
Affiliation(s)
- Aiyuan Wu
- The Affiliated Cancer Hospital of Xinjiang Medical University, Urumqi, China
| | - Peng Xue
- School of Population Medicine and Public Health, Peking Union Medical College, Chinese Academy of Medical Sciences, Beijing, China
| | - Guzhalinuer Abulizi
- The Affiliated Cancer Hospital of Xinjiang Medical University, Urumqi, China
| | - Dilinuer Tuerxun
- The Affiliated Cancer Hospital of Xinjiang Medical University, Urumqi, China
| | - Remila Rezhake
- The Affiliated Cancer Hospital of Xinjiang Medical University, Urumqi, China
- *Correspondence: Remila Rezhake,
| | - Youlin Qiao
- The Affiliated Cancer Hospital of Xinjiang Medical University, Urumqi, China
- School of Population Medicine and Public Health, Peking Union Medical College, Chinese Academy of Medical Sciences, Beijing, China
- Youlin Qiao,
| |
Collapse
|
25
|
Fowler GE, Blencowe NS, Hardacre C, Callaway MP, Smart NJ, Macefield R. Artificial intelligence as a diagnostic aid in cross-sectional radiological imaging of surgical pathology in the abdominopelvic cavity: a systematic review. BMJ Open 2023; 13:e064739. [PMID: 36878659 PMCID: PMC9990659 DOI: 10.1136/bmjopen-2022-064739] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 03/08/2023] Open
Abstract
OBJECTIVES There is emerging use of artificial intelligence (AI) models to aid diagnostic imaging. This review examined and critically appraised the application of AI models to identify surgical pathology from radiological images of the abdominopelvic cavity, to identify current limitations and inform future research. DESIGN Systematic review. DATA SOURCES Systematic database searches (Medline, EMBASE, Cochrane Central Register of Controlled Trials) were performed. Date limitations (January 2012 to July 2021) were applied. ELIGIBILITY CRITERIA Primary research studies were considered for eligibility using the PIRT (participants, index test(s), reference standard and target condition) framework. Only publications in the English language were eligible for inclusion in the review. DATA EXTRACTION AND SYNTHESIS Study characteristics, descriptions of AI models and outcomes assessing diagnostic performance were extracted by independent reviewers. A narrative synthesis was performed in accordance with the Synthesis Without Meta-analysis guidelines. Risk of bias was assessed (Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2)). RESULTS Fifteen retrospective studies were included. Studies were diverse in surgical specialty, the intention of the AI applications and the models used. AI training and test sets comprised a median of 130 (range: 5-2440) and 37 (range: 10-1045) patients, respectively. Diagnostic performance of models varied (range: 70%-95% sensitivity, 53%-98% specificity). Only four studies compared the AI model with human performance. Reporting of studies was unstandardised and often lacking in detail. Most studies (n=14) were judged as having overall high risk of bias with concerns regarding applicability. CONCLUSIONS AI application in this field is diverse. Adherence to reporting guidelines is warranted. With finite healthcare resources, future endeavours may benefit from targeting areas where radiological expertise is in high demand to provide greater efficiency in clinical care. Translation to clinical practice and adoption of a multidisciplinary approach should be of high priority. PROSPERO REGISTRATION NUMBER CRD42021237249.
Collapse
Affiliation(s)
- George E Fowler
- NIHR Bristol Biomedical Research Centre, Population Health Sciences, Bristol Medical School. University of Bristol, Bristol, UK
| | - Natalie S Blencowe
- NIHR Bristol Biomedical Research Centre, Population Health Sciences, Bristol Medical School. University of Bristol, Bristol, UK
| | - Conor Hardacre
- Bristol Medical School, University of Bristol, Bristol, UK
| | - Mark P Callaway
- Department of Clinical Radiology, University Hospital Bristol and Weston NHS Foundation Trust, Bristol, UK
| | - Neil J Smart
- Exeter Surgical Health Services Research Unit (HeSRU), Royal Devon and Exeter NHS Foundation Trust, Exeter, UK
| | - Rhiannon Macefield
- NIHR Bristol Biomedical Research Centre, Population Health Sciences, Bristol Medical School. University of Bristol, Bristol, UK
| |
Collapse
|
26
|
Lu SC, Knafl M, Turin A, Offodile AC, Ravi V, Sidey-Gibbons C. Machine Learning Models Using Routinely Collected Clinical Data Offer Robust and Interpretable Predictions of 90-Day Unplanned Acute Care Use for Cancer Immunotherapy Patients. JCO Clin Cancer Inform 2023; 7:e2200123. [PMID: 37001039 PMCID: PMC10281452 DOI: 10.1200/cci.22.00123] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2022] [Revised: 11/23/2022] [Accepted: 01/20/2023] [Indexed: 04/03/2023] Open
Abstract
PURPOSE Clinical management of patients receiving immune checkpoint inhibitors (ICIs) could be informed using accurate predictive tools to identify patients at risk of short-term acute care utilization (ACU). We used routinely collected data to develop and assess machine learning (ML) algorithms to predict unplanned ACU within 90 days of ICI treatment initiation. METHODS We used aggregated electronic health record data from 7,960 patients receiving ICI treatments to train and assess eight ML algorithms. We developed the models using pre-SARS-COV-19 COVID-19 data generated between January 2016 and February 2020. We validated our algorithms using data collected between March 2020 and June 2022 (peri-COVID-19 sample). We assessed performance using area under the receiver operating characteristic curves (AUROC), sensitivity, specificity, and calibration plots. We derived intuitive explanations of predictions using variable importance and Shapley additive explanation analyses. We assessed the marginal performance of ML models compared with that of univariate and multivariate logistic regression (LR) models. RESULTS Most algorithms significantly outperformed the univariate and multivariate LR models. The extreme gradient boosting trees (XGBT) algorithm demonstrated the best overall performance (AUROC, 0.70; sensitivity, 0.53; specificity, 0.74) on the peri-COVID-19 sample. The algorithm performance was stable across both pre- and peri-COVID-19 samples, as well as ICI regimen and cancer groups. Type of ICI agents, oxygen saturation, diastolic blood pressure, albumin level, platelet count, immature granulocytes, absolute monocyte, chloride level, red cell distribution width, and alcohol intake were the top 10 key predictors used by the XGBT algorithm. CONCLUSION Machine learning algorithms trained using routinely collected data outperformed traditional statistical models when predicting 90-day ACU. The XGBT algorithm has the potential to identify high-ACU risk patients and enable preventive interventions to avoid ACU.
Collapse
Affiliation(s)
- Sheng-Chieh Lu
- The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Mark Knafl
- The University of Texas MD Anderson Cancer Center, Houston, TX
| | | | | | - Vinod Ravi
- The University of Texas MD Anderson Cancer Center, Houston, TX
| | | |
Collapse
|
27
|
Ahmad HK, Milne MR, Buchlak QD, Ektas N, Sanderson G, Chamtie H, Karunasena S, Chiang J, Holt X, Tang CHM, Seah JCY, Bottrell G, Esmaili N, Brotchie P, Jones C. Machine Learning Augmented Interpretation of Chest X-rays: A Systematic Review. Diagnostics (Basel) 2023; 13:diagnostics13040743. [PMID: 36832231 PMCID: PMC9955112 DOI: 10.3390/diagnostics13040743] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2023] [Revised: 02/13/2023] [Accepted: 02/14/2023] [Indexed: 02/18/2023] Open
Abstract
Limitations of the chest X-ray (CXR) have resulted in attempts to create machine learning systems to assist clinicians and improve interpretation accuracy. An understanding of the capabilities and limitations of modern machine learning systems is necessary for clinicians as these tools begin to permeate practice. This systematic review aimed to provide an overview of machine learning applications designed to facilitate CXR interpretation. A systematic search strategy was executed to identify research into machine learning algorithms capable of detecting >2 radiographic findings on CXRs published between January 2020 and September 2022. Model details and study characteristics, including risk of bias and quality, were summarized. Initially, 2248 articles were retrieved, with 46 included in the final review. Published models demonstrated strong standalone performance and were typically as accurate, or more accurate, than radiologists or non-radiologist clinicians. Multiple studies demonstrated an improvement in the clinical finding classification performance of clinicians when models acted as a diagnostic assistance device. Device performance was compared with that of clinicians in 30% of studies, while effects on clinical perception and diagnosis were evaluated in 19%. Only one study was prospectively run. On average, 128,662 images were used to train and validate models. Most classified less than eight clinical findings, while the three most comprehensive models classified 54, 72, and 124 findings. This review suggests that machine learning devices designed to facilitate CXR interpretation perform strongly, improve the detection performance of clinicians, and improve the efficiency of radiology workflow. Several limitations were identified, and clinician involvement and expertise will be key to driving the safe implementation of quality CXR machine learning systems.
Collapse
Affiliation(s)
- Hassan K. Ahmad
- Annalise.ai, Sydney, NSW 2000, Australia
- Department of Emergency Medicine, Royal North Shore Hospital, Sydney, NSW 2065, Australia
- Correspondence:
| | | | - Quinlan D. Buchlak
- Annalise.ai, Sydney, NSW 2000, Australia
- School of Medicine, University of Notre Dame Australia, Sydney, NSW 2007, Australia
- Department of Neurosurgery, Monash Health, Melbourne, VIC 3168, Australia
| | | | | | | | | | - Jason Chiang
- Annalise.ai, Sydney, NSW 2000, Australia
- Department of General Practice, University of Melbourne, Melbourne, VIC 3010, Australia
- Westmead Applied Research Centre, University of Sydney, Sydney, NSW 2006, Australia
| | | | | | - Jarrel C. Y. Seah
- Annalise.ai, Sydney, NSW 2000, Australia
- Department of Radiology, Alfred Health, Melbourne, VIC 3004, Australia
| | | | - Nazanin Esmaili
- School of Medicine, University of Notre Dame Australia, Sydney, NSW 2007, Australia
- Faculty of Engineering and Information Technology, University of Technology Sydney, Sydney, NSW 2007, Australia
| | - Peter Brotchie
- Annalise.ai, Sydney, NSW 2000, Australia
- Department of Radiology, St Vincent’s Health Australia, Melbourne, VIC 3065, Australia
| | - Catherine Jones
- Annalise.ai, Sydney, NSW 2000, Australia
- I-MED Radiology Network, Brisbane, QLD 4006, Australia
- School of Public and Preventive Health, Monash University, Clayton, VIC 3800, Australia
- Department of Clinical Imaging Science, University of Sydney, Sydney, NSW 2006, Australia
| |
Collapse
|
28
|
Eysenbach G, Kleib M, Norris C, O'Rourke HM, Montgomery C, Douma M. The Use and Structure of Emergency Nurses' Triage Narrative Data: Scoping Review. JMIR Nurs 2023; 6:e41331. [PMID: 36637881 PMCID: PMC9883744 DOI: 10.2196/41331] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2022] [Revised: 11/24/2022] [Accepted: 11/28/2022] [Indexed: 11/30/2022] Open
Abstract
BACKGROUND Emergency departments use triage to ensure that patients with the highest level of acuity receive care quickly and safely. Triage is typically a nursing process that is documented as structured and unstructured (free text) data. Free-text triage narratives have been studied for specific conditions but never reviewed in a comprehensive manner. OBJECTIVE The objective of this paper was to identify and map the academic literature that examines triage narratives. The paper described the types of research conducted, identified gaps in the research, and determined where additional review may be warranted. METHODS We conducted a scoping review of unstructured triage narratives. We mapped the literature, described the use of triage narrative data, examined the information available on the form and structure of narratives, highlighted similarities among publications, and identified opportunities for future research. RESULTS We screened 18,074 studies published between 1990 and 2022 in CINAHL, MEDLINE, Embase, Cochrane, and ProQuest Central. We identified 0.53% (96/18,074) of studies that directly examined the use of triage nurses' narratives. More than 12 million visits were made to 2438 emergency departments included in the review. In total, 82% (79/96) of these studies were conducted in the United States (43/96, 45%), Australia (31/96, 32%), or Canada (5/96, 5%). Triage narratives were used for research and case identification, as input variables for predictive modeling, and for quality improvement. Overall, 31% (30/96) of the studies offered a description of the triage narrative, including a list of the keywords used (27/96, 28%) or more fulsome descriptions (such as word counts, character counts, abbreviation, etc; 7/96, 7%). We found limited use of reporting guidelines (8/96, 8%). CONCLUSIONS The breadth of the identified studies suggests that there is widespread routine collection and research use of triage narrative data. Despite the use of triage narratives as a source of data in studies, the narratives and nurses who generate them are poorly described in the literature, and data reporting is inconsistent. Additional research is needed to describe the structure of triage narratives, determine the best use of triage narratives, and improve the consistent use of triage-specific data reporting guidelines. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID) RR2-10.1136/bmjopen-2021-055132.
Collapse
Affiliation(s)
| | - Manal Kleib
- Faculty of Nursing, University of Alberta, Edmonton, AB, Canada
| | - Colleen Norris
- Faculty of Nursing, University of Alberta, Edmonton, AB, Canada
| | | | | | - Matthew Douma
- School of Nursing, Midwifery and Health Systems, University College Dublin, Dublin, Ireland
| |
Collapse
|
29
|
Oh W, Nadkarni GN. Federated Learning in Health care Using Structured Medical Data. ADVANCES IN KIDNEY DISEASE AND HEALTH 2023; 30:4-16. [PMID: 36723280 PMCID: PMC10208416 DOI: 10.1053/j.akdh.2022.11.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/20/2023]
Abstract
The success of machine learning-based studies is largely subjected to accessing a large amount of data. However, accessing such data is typically not feasible within a single health system/hospital. Although multicenter studies are the most effective way to access a vast amount of data, sharing data outside the institutes involves legal, business, and technical challenges. Federated learning (FL) is a newly proposed machine learning framework for multicenter studies, tackling data-sharing issues across participant institutes. The promise of FL is simple. FL facilitates multicenter studies without losing data access control and allows the construction of a global model by aggregating local models trained from participant institutes. This article reviewed recently published studies that utilized FL in clinical studies with structured medical data. In addition, challenges and open questions in FL in clinical studies with structured medical data were discussed.
Collapse
Affiliation(s)
- Wonsuk Oh
- Charles Bronfman Institute for Personalized Medicine, Icahn School of Medicine at Mount Sinai, New York, NY
| | - Girish N Nadkarni
- Charles Bronfman Institute for Personalized Medicine, Icahn School of Medicine at Mount Sinai, New York, NY; Division of Data-Driven and Digital Medicine, Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, NY; Division of Nephrology, Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, NY.
| |
Collapse
|
30
|
Optical spectroscopy and chemometrics in intraoperative tumor margin assessment. Trends Analyt Chem 2023. [DOI: 10.1016/j.trac.2023.116955] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/03/2023]
|
31
|
Bannister JJ, Wilms M, Aponte JD, Katz DC, Klein OD, Bernier FPJ, Spritz RA, Hallgrímsson B, Forkert ND. Detecting 3D syndromic faces as outliers using unsupervised normalizing flow models. Artif Intell Med 2022; 134:102425. [PMID: 36462895 PMCID: PMC10949379 DOI: 10.1016/j.artmed.2022.102425] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2022] [Revised: 10/08/2022] [Accepted: 10/09/2022] [Indexed: 12/13/2022]
Abstract
Many genetic syndromes are associated with distinctive facial features. Several computer-assisted methods have been proposed that make use of facial features for syndrome diagnosis. Training supervised classifiers, the most common approach for this purpose, requires large, comprehensive, and difficult to collect databases of syndromic facial images. In this work, we use unsupervised, normalizing flow-based manifold and density estimation models trained entirely on unaffected subjects to detect syndromic 3D faces as statistical outliers. Furthermore, we demonstrate a general, user-friendly, gradient-based interpretability mechanism that enables clinicians and patients to understand model inferences. 3D facial surface scans of 2471 unaffected subjects and 1629 syndromic subjects representing 262 different genetic syndromes were used to train and evaluate the models. The flow-based models outperformed unsupervised comparison methods, with the best model achieving an ROC-AUC of 86.3% on a challenging, age and sex diverse data set. In addition to highlighting the viability of outlier-based syndrome screening tools, our methods generalize and extend previously proposed outlier scores for 3D face-based syndrome detection, resulting in improved performance for unsupervised syndrome detection.
Collapse
Affiliation(s)
- Jordan J Bannister
- Biomedical Engineering Graduate Program, University of Calgary, Calgary, AB, Canada.
| | - Matthias Wilms
- Department of Pediatrics, Department of Community Health Sciences, Alberta Children's Hospital Research Institute, University of Calgary, Calgary, AB, Canada
| | - J David Aponte
- Department of Cell Biology and Anatomy, University of Calgary, 2500 University Dr NW, Calgary, AB, Canada
| | - David C Katz
- Department of Cell Biology and Anatomy, University of Calgary, 2500 University Dr NW, Calgary, AB, Canada
| | - Ophir D Klein
- Program in Craniofacial Biology, Department of Orofacial Sciences, University of California, San Francisco, CA, USA
| | - Francois P J Bernier
- Department of Medical Genetics, Alberta Children's Hospital Research Institute, University of Calgary, Calgary, AB, Canada
| | - Richard A Spritz
- Department of Pediatrics, University of Colorado School of Medicine, Aurora, CO, USA
| | - Benedikt Hallgrímsson
- Department of Cell Biology and Anatomy, University of Calgary, 2500 University Dr NW, Calgary, AB, Canada
| | - Nils D Forkert
- Department of Radiology, Alberta Children's Hospital Research Institute, Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada
| |
Collapse
|
32
|
Parkash O, Siddiqui ATS, Jiwani U, Rind F, Padhani ZA, Rizvi A, Hoodbhoy Z, Das JK. Diagnostic accuracy of artificial intelligence for detecting gastrointestinal luminal pathologies: A systematic review and meta-analysis. Front Med (Lausanne) 2022; 9:1018937. [PMID: 36405592 PMCID: PMC9672666 DOI: 10.3389/fmed.2022.1018937] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2022] [Accepted: 10/03/2022] [Indexed: 11/06/2022] Open
Abstract
Background Artificial Intelligence (AI) holds considerable promise for diagnostics in the field of gastroenterology. This systematic review and meta-analysis aims to assess the diagnostic accuracy of AI models compared with the gold standard of experts and histopathology for the diagnosis of various gastrointestinal (GI) luminal pathologies including polyps, neoplasms, and inflammatory bowel disease. Methods We searched PubMed, CINAHL, Wiley Cochrane Library, and Web of Science electronic databases to identify studies assessing the diagnostic performance of AI models for GI luminal pathologies. We extracted binary diagnostic accuracy data and constructed contingency tables to derive the outcomes of interest: sensitivity and specificity. We performed a meta-analysis and hierarchical summary receiver operating characteristic curves (HSROC). The risk of bias was assessed using Quality Assessment for Diagnostic Accuracy Studies-2 (QUADAS-2) tool. Subgroup analyses were conducted based on the type of GI luminal disease, AI model, reference standard, and type of data used for analysis. This study is registered with PROSPERO (CRD42021288360). Findings We included 73 studies, of which 31 were externally validated and provided sufficient information for inclusion in the meta-analysis. The overall sensitivity of AI for detecting GI luminal pathologies was 91.9% (95% CI: 89.0–94.1) and specificity was 91.7% (95% CI: 87.4–94.7). Deep learning models (sensitivity: 89.8%, specificity: 91.9%) and ensemble methods (sensitivity: 95.4%, specificity: 90.9%) were the most commonly used models in the included studies. Majority of studies (n = 56, 76.7%) had a high risk of selection bias while 74% (n = 54) studies were low risk on reference standard and 67% (n = 49) were low risk for flow and timing bias. Interpretation The review suggests high sensitivity and specificity of AI models for the detection of GI luminal pathologies. There is a need for large, multi-center trials in both high income countries and low- and middle- income countries to assess the performance of these AI models in real clinical settings and its impact on diagnosis and prognosis. Systematic review registration [https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=288360], identifier [CRD42021288360].
Collapse
Affiliation(s)
- Om Parkash
- Department of Medicine, Aga Khan University, Karachi, Pakistan
| | | | - Uswa Jiwani
- Center of Excellence in Women and Child Health, Aga Khan University, Karachi, Pakistan
| | - Fahad Rind
- Head and Neck Oncology, The Ohio State University, Columbus, OH, United States
| | - Zahra Ali Padhani
- Institute for Global Health and Development, Aga Khan University, Karachi, Pakistan
| | - Arjumand Rizvi
- Center of Excellence in Women and Child Health, Aga Khan University, Karachi, Pakistan
| | - Zahra Hoodbhoy
- Department of Pediatrics and Child Health, Aga Khan University, Karachi, Pakistan
| | - Jai K. Das
- Institute for Global Health and Development, Aga Khan University, Karachi, Pakistan
- Department of Pediatrics and Child Health, Aga Khan University, Karachi, Pakistan
- *Correspondence: Jai K. Das,
| |
Collapse
|
33
|
Cross K, Harding K. Risk profiling in the prevention and treatment of chronic wounds using artificial intelligence. Int Wound J 2022; 19:1283-1285. [PMID: 36131590 PMCID: PMC9493230 DOI: 10.1111/iwj.13952] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/24/2022] [Indexed: 12/13/2022] Open
|
34
|
Pou-Prom C, Murray J, Kuzulugil S, Mamdani M, Verma AA. From compute to care: Lessons learned from deploying an early warning system into clinical practice. Front Digit Health 2022; 4:932123. [PMID: 36133802 PMCID: PMC9483018 DOI: 10.3389/fdgth.2022.932123] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2022] [Accepted: 08/08/2022] [Indexed: 12/03/2022] Open
Abstract
Background Deploying safe and effective machine learning models is essential to realize the promise of artificial intelligence for improved healthcare. Yet, there remains a large gap between the number of high-performing ML models trained on healthcare data and the actual deployment of these models. Here, we describe the deployment of CHARTwatch, an artificial intelligence-based early warning system designed to predict patient risk of clinical deterioration. Methods We describe the end-to-end infrastructure that was developed to deploy CHARTwatch and outline the process from data extraction to communicating patient risk scores in real-time to physicians and nurses. We then describe the various challenges that were faced in deployment, including technical issues (e.g., unstable database connections), process-related challenges (e.g., changes in how a critical lab is measured), and challenges related to deploying a clinical system in the middle of a pandemic. We report various measures to quantify the success of the deployment: model performance, adherence to workflows, and infrastructure uptime/downtime. Ultimately, success is driven by end-user adoption and impact on relevant clinical outcomes. We assess our deployment process by evaluating how closely we followed existing guidance for good machine learning practice (GMLP) and identify gaps that are not addressed in this guidance. Results The model demonstrated strong and consistent performance in real-time in the first 19 months after deployment (AUC 0.76) as in the silent deployment heldout test data (AUC 0.79). The infrastructure remained online for >99% of time in the first year of deployment. Our deployment adhered to all 10 aspects of GMLP guiding principles. Several steps were crucial for deployment but are not mentioned or are missing details in the GMLP principles, including the need for a silent testing period, the creation of robust downtime protocols, and the importance of end-user engagement. Evaluation for impacts on clinical outcomes and adherence to clinical protocols is underway. Conclusion We deployed an artificial intelligence-based early warning system to predict clinical deterioration in hospital. Careful attention to data infrastructure, identifying problems in a silent testing period, close monitoring during deployment, and strong engagement with end-users were critical for successful deployment.
Collapse
Affiliation(s)
- Chloé Pou-Prom
- Data Science and Advanced Analytics, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Correspondence: Chloé Pou-Prom
| | - Joshua Murray
- Department of Statistics, University of Toronto, Toronto, ON, Canada
| | - Sebnem Kuzulugil
- Data Science and Advanced Analytics, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
| | - Muhammad Mamdani
- Data Science and Advanced Analytics, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Department of Medicine, Faculty of Medicine, University of Toronto, Toronto, ON, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada
- Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
- Vector Institute, Toronto, ON, Canada
- Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, ON, Canada
| | - Amol A. Verma
- Data Science and Advanced Analytics, St. Michael's Hospital, Unity Health Toronto, Toronto, ON, Canada
- Department of Medicine, Faculty of Medicine, University of Toronto, Toronto, ON, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
35
|
Plana D, Shung DL, Grimshaw AA, Saraf A, Sung JJY, Kann BH. Randomized Clinical Trials of Machine Learning Interventions in Health Care: A Systematic Review. JAMA Netw Open 2022; 5:e2233946. [PMID: 36173632 PMCID: PMC9523495 DOI: 10.1001/jamanetworkopen.2022.33946] [Citation(s) in RCA: 47] [Impact Index Per Article: 23.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/16/2022] Open
Abstract
IMPORTANCE Despite the potential of machine learning to improve multiple aspects of patient care, barriers to clinical adoption remain. Randomized clinical trials (RCTs) are often a prerequisite to large-scale clinical adoption of an intervention, and important questions remain regarding how machine learning interventions are being incorporated into clinical trials in health care. OBJECTIVE To systematically examine the design, reporting standards, risk of bias, and inclusivity of RCTs for medical machine learning interventions. EVIDENCE REVIEW In this systematic review, the Cochrane Library, Google Scholar, Ovid Embase, Ovid MEDLINE, PubMed, Scopus, and Web of Science Core Collection online databases were searched and citation chasing was done to find relevant articles published from the inception of each database to October 15, 2021. Search terms for machine learning, clinical decision-making, and RCTs were used. Exclusion criteria included implementation of a non-RCT design, absence of original data, and evaluation of nonclinical interventions. Data were extracted from published articles. Trial characteristics, including primary intervention, demographics, adherence to the CONSORT-AI reporting guideline, and Cochrane risk of bias were analyzed. FINDINGS Literature search yielded 19 737 articles, of which 41 RCTs involved a median of 294 participants (range, 17-2488 participants). A total of 16 RCTS (39%) were published in 2021, 21 (51%) were conducted at single sites, and 15 (37%) involved endoscopy. No trials adhered to all CONSORT-AI standards. Common reasons for nonadherence were not assessing poor-quality or unavailable input data (38 trials [93%]), not analyzing performance errors (38 [93%]), and not including a statement regarding code or algorithm availability (37 [90%]). Overall risk of bias was high in 7 trials (17%). Of 11 trials (27%) that reported race and ethnicity data, the median proportion of participants from underrepresented minority groups was 21% (range, 0%-51%). CONCLUSIONS AND RELEVANCE This systematic review found that despite the large number of medical machine learning-based algorithms in development, few RCTs for these technologies have been conducted. Among published RCTs, there was high variability in adherence to reporting standards and risk of bias and a lack of participants from underrepresented minority groups. These findings merit attention and should be considered in future RCT design and reporting.
Collapse
Affiliation(s)
| | - Dennis L Shung
- Department of Medicine, Yale University, New Haven, Connecticut
| | - Alyssa A Grimshaw
- Harvey Cushing/John Hay Whitney Medical Library, Yale University, New Haven, Connecticut
| | - Anurag Saraf
- Department of Radiation Oncology, Massachusetts General Hospital, Boston, Massachusetts
| | - Joseph J Y Sung
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore
| | - Benjamin H Kann
- Artificial Intelligence in Medicine Program, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| |
Collapse
|
36
|
Vistisen ST, Pollard TJ, Harris S, Lauritsen SM. Artificial intelligence in the clinical setting: Towards actual implementation of reliable outcome predictions. Eur J Anaesthesiol 2022; 39:729-732. [PMID: 35919024 DOI: 10.1097/eja.0000000000001696] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Affiliation(s)
- Simon Tilma Vistisen
- From the Institute of Clinical Medicine, Aarhus University (STV), Department of Anaesthesiology & Intensive Care, Aarhus University Hospital, Denmark (STV), Laboratory for Computational Physiology, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA (TJP), Department of Critical Care, University College London Hospital and Institute of Health Informatics, University College London, UK (SH) and Enversion A/S, Denmark (SML)
| | | | | | | |
Collapse
|
37
|
Coombs L, Orlando A, Wang X, Shaw P, Rich AS, Lakhtakia S, Titchener K, Adamson B, Miksad RA, Mooney K. A machine learning framework supporting prospective clinical decisions applied to risk prediction in oncology. NPJ Digit Med 2022; 5:117. [PMID: 35974092 PMCID: PMC9380664 DOI: 10.1038/s41746-022-00660-3] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2021] [Accepted: 07/21/2022] [Indexed: 11/10/2022] Open
Abstract
We present a general framework for developing a machine learning (ML) tool that supports clinician assessment of patient risk using electronic health record-derived real-world data and apply the framework to a quality improvement use case in an oncology setting to identify patients at risk for a near-term (60 day) emergency department (ED) visit who could potentially be eligible for a home-based acute care program. Framework steps include defining clinical quality improvement goals, model development and validation, bias assessment, retrospective and prospective validation, and deployment in clinical workflow. In the retrospective analysis for the use case, 8% of patient encounters were associated with a high risk (pre-defined as predicted probability ≥20%) for a near-term ED visit by the patient. Positive predictive value (PPV) and negative predictive value (NPV) for future ED events was 26% and 91%, respectively. Odds ratio (OR) of ED visit (high- vs. low-risk) was 3.5 (95% CI: 3.4–3.5). The model appeared to be calibrated across racial, gender, and ethnic groups. In the prospective analysis, 10% of patients were classified as high risk, 76% of whom were confirmed by clinicians as eligible for home-based acute care. PPV and NPV for future ED events was 22% and 95%, respectively. OR of ED visit (high- vs. low-risk) was 5.4 (95% CI: 2.6–11.0). The proposed framework for an ML-based tool that supports clinician assessment of patient risk is a stepwise development approach; we successfully applied the framework to an ED visit risk prediction use case.
Collapse
Affiliation(s)
- Lorinda Coombs
- Huntsman Cancer Institute, University of Utah, Salt Lake City, UT, USA.,University of North Carolina-Chapel Hill, Lineberger Cancer Institute, Chapel Hill, NC, USA
| | | | | | | | | | | | - Karen Titchener
- Huntsman Cancer Institute, University of Utah, Salt Lake City, UT, USA
| | | | | | - Kathi Mooney
- Huntsman Cancer Institute, University of Utah, Salt Lake City, UT, USA
| |
Collapse
|
38
|
Blasi F, Caiani EG, Cereda MG, Donetti D, Montorsi M, Panella V, Panina G, Pelagalli F, Speroni E. Six Drivers to Face the XXI Century Challenges and Build the New Healthcare System: "La Salute in Movimento" Manifesto. Front Public Health 2022; 10:876625. [PMID: 35844841 PMCID: PMC9277183 DOI: 10.3389/fpubh.2022.876625] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2022] [Accepted: 06/07/2022] [Indexed: 11/13/2022] Open
Abstract
The aging of the population, the burden of chronic diseases, possible new pandemics are among the challenges for healthcare in the XXI century. To face them, technological innovations and the national recovery and resilience plan within the European Union can represent opportunities to implement changes and renovate the current healthcare system in Italy, in an effort to guarantee equal access to health services. Considering such scenario, a panel of Italian experts gathered in a multidisciplinary Think Tank to discuss possible design of concepts at the basis of a new healthcare system. These ideas were summarized in a manifesto with six drivers for change: vision, governance, competence, intelligence, humanity and relationship. Each driver was linked to an action to actively move toward a new healthcare system based on trust between science, citizens and institutions.
Collapse
Affiliation(s)
- Francesco Blasi
- Department of Pathophysiology and Transplantation, Università degli Studi di Milano, Milan, Italy.,Internal Medicine Department and Respiratory Unit, Fondazione IRCCS Cà Granda Ospedale Maggiore Policlinico di Milano, Milan, Italy
| | - Enrico Gianluca Caiani
- Department of Electronics, Information and Biomedical Engineering Department, Politecnico di Milano, Milan, Italy
| | | | | | - Marco Montorsi
- Department of Biomedical Sciences, Humanitas University, Pieve Emanuele, Italy.,Department of Surgery, Humanitas Research Hospital IRCCS, Milan, Italy
| | | | | | | | | |
Collapse
|
39
|
The advanced design of bioleaching process for metal recovery: A machine learning approach. Sep Purif Technol 2022. [DOI: 10.1016/j.seppur.2022.120919] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
|
40
|
Kompa B, Hakim JB, Palepu A, Kompa KG, Smith M, Bain PA, Woloszynek S, Painter JL, Bate A, Beam AL. Artificial Intelligence Based on Machine Learning in Pharmacovigilance: A Scoping Review. Drug Saf 2022; 45:477-491. [PMID: 35579812 PMCID: PMC9883349 DOI: 10.1007/s40264-022-01176-1] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/13/2022] [Indexed: 02/07/2023]
Abstract
INTRODUCTION Artificial intelligence based on machine learning has made large advancements in many fields of science and medicine but its impact on pharmacovigilance is yet unclear. OBJECTIVE The present study conducted a scoping review of the use of artificial intelligence based on machine learning to understand how it is used for pharmacovigilance tasks, characterize differences with other fields, and identify opportunities to improve pharmacovigilance through the use of machine learning. DESIGN The PubMed, Embase, Web of Science, and IEEE Xplore databases were searched to identify articles pertaining to the use of machine learning in pharmacovigilance published from the year 2000 to September 2021. After manual screening of 7744 abstracts, a total of 393 papers met the inclusion criteria for further analysis. Extraction of key data on study design, data sources, sample size, and machine learning methodology was performed. Studies with the characteristics of good machine learning practice were defined and manual review focused on identifying studies that fulfilled these criteria and results that showed promise. RESULTS The majority of studies (53%) were focused on detecting safety signals using traditional statistical methods. Of the studies that used more recent machine learning methods, 61% used off-the-shelf techniques with minor modifications. Temporal analysis revealed that newer methods such as deep learning have shown increased use in recent years. We found only 42 studies (10%) that reflect current best practices and trends in machine learning. In the subset of 154 papers that focused on data intake and ingestion, 30 (19%) were found to incorporate the same best practices. CONCLUSION Advances from artificial intelligence have yet to fully penetrate pharmacovigilance, although recent studies show signs that this may be changing.
Collapse
Affiliation(s)
- Benjamin Kompa
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
- CAUSALab, Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - Joe B Hakim
- Department of Health Sciences and Technology, Harvard-MIT, Cambridge, MA, USA
| | - Anil Palepu
- Department of Health Sciences and Technology, Harvard-MIT, Cambridge, MA, USA
| | | | - Michael Smith
- Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - Paul A Bain
- Countway Library of Medicine, Harvard Medical School, Boston, MA, USA
| | | | | | - Andrew Bate
- GlaxoSmithKline, Brentford, UK
- Department of Non-Communicable Disease Epidemiology, London School of Hygiene and Tropical Medicine, University of London, London, UK
- Department of Medicine, NYU Grossman School of Medicine, New York, NY, USA
| | - Andrew L Beam
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA.
- CAUSALab, Harvard T.H. Chan School of Public Health, Boston, MA, USA.
- Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, MA, USA.
| |
Collapse
|
41
|
Heo J, Yoo J, Lee H, Lee IH, Kim JS, Park E, Kim YD, Nam HS. Prediction of Hidden Coronary Artery Disease Using Machine Learning in Patients With Acute Ischemic Stroke. Neurology 2022; 99:e55-e65. [PMID: 35470135 DOI: 10.1212/wnl.0000000000200576] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2021] [Accepted: 03/02/2022] [Indexed: 11/15/2022] Open
Abstract
BACKGROUND AND OBJECTIVES A machine learning technique for identifying hidden coronary artery disease (CAD) might be useful. We developed and validated machine learning models to predict patients with hidden CAD and assess long-term outcomes in patients with acute ischemic stroke. METHODS Multidetector coronary computed tomography was performed for patients without known history of CAD. Primary outcomes were defined as having any degree of CAD and having obstructive CAD (≥50% stenosis). Demographic variables, risk factors, laboratory results, Trial of ORG 10172 in Acute Stroke Treatment (TOAST) classification, NIH Stroke Scale score, blood pressure, and carotid artery stenosis were used to develop and validate machine learning models to predict CAD. Area under the receiver operating characteristic curves (AUC) was calculated for performance analysis, and Kaplan-Meier and Cox survival analyses of long-term outcomes were performed. Major adverse cardiovascular events (MACE) were defined as ischemic stroke, myocardial infarction, unstable angina, urgent coronary revascularization, and cardiovascular mortality. RESULTS Overall, 1,710 patients were included for the training dataset and 348 patients for the validation dataset. An Extreme Gradient Boosting model was developed to predict any degree of CAD, which showed an AUC of 0.763 (95% CI 0.711-0.814) on validation. A logistic regression model was used to predict obstructive CAD and had an AUC of 0.714 (95% CI 0.692-0.799). During the first 5 years of follow-up, MACE occurred more frequently when predicted of any CAD (P = 0.022) or obstructive CAD (P < 0.001). Cox proportional analysis showed that the hazard ratio of MACE was 1.5 (95% CI 1.1-2.2; P = 0.016) when predicted of any CAD, whereas it was 1.9 (95% CI 1.3-2.6; P < 0.001) for obstructive CAD. DISCUSSION We demonstrated that machine learning may help identify hidden CAD in patients with acute ischemic stroke. Long-term outcomes were also associated with prediction results. CLASSIFICATION OF EVIDENCE This study provides Class II evidence that in patients with acute ischemic stroke with CAD risk factors but no known history of CAD, a machine learning model predicts CAD on multidetector coronary computed tomography with an AUC of 0.763 (95% CI 0.711-0.814).
Collapse
Affiliation(s)
- JoonNyung Heo
- Department of Neurology, Yonsei University College of Medicine, Seoul, Korea
| | - Joonsang Yoo
- Department of Neurology, Yonsei University College of Medicine, Yongin Severance Hospital, Yongin, Korea
| | - Hyungwoo Lee
- Department of Neurology, Yonsei University College of Medicine, Seoul, Korea
| | - Il Hyung Lee
- Department of Neurology, Yonsei University College of Medicine, Seoul, Korea
| | - Jung-Sun Kim
- Division of Cardiology, Department of Internal Medicine, Yonsei University College of Medicine, Seoul, Korea
| | - Eunjeong Park
- Integrative Research Center for Cerebrovascular and Cardiovascular Diseases, Seoul, Korea
| | - Young Dae Kim
- Department of Neurology, Yonsei University College of Medicine, Seoul, Korea
| | - Hyo Suk Nam
- Department of Neurology, Yonsei University College of Medicine, Seoul, Korea
| |
Collapse
|
42
|
Chu H, Moon S, Park J, Bak S, Ko Y, Youn BY. The Use of Artificial Intelligence in Complementary and Alternative Medicine: A Systematic Scoping Review. Front Pharmacol 2022; 13:826044. [PMID: 35431917 PMCID: PMC9011141 DOI: 10.3389/fphar.2022.826044] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2021] [Accepted: 03/01/2022] [Indexed: 01/04/2023] Open
Abstract
Background: The development of artificial intelligence (AI) in the medical field has been growing rapidly. As AI models have been introduced in complementary and alternative medicine (CAM), a systematized review must be performed to understand its current status. Objective: To categorize and seek the current usage of AI in CAM. Method: A systematic scoping review was conducted based on the method proposed by the Joanna Briggs Institute. The three databases, PubMed, Embase, and Cochrane Library, were used to find studies regarding AI and CAM. Only English studies from 2000 were included. Studies without mentioning either AI techniques or CAM modalities were excluded along with the non-peer-reviewed studies. A broad-range search strategy was applied to locate all relevant studies. Results: A total of 32 studies were identified, and three main categories were revealed: 1) acupuncture treatment, 2) tongue and lip diagnoses, and 3) herbal medicine. Other CAM modalities were music therapy, meditation, pulse diagnosis, and TCM syndromes. The majority of the studies utilized AI models to predict certain patterns and find reliable computerized models to assist physicians. Conclusion: Although the results from this review have shown the potential use of AI models in CAM, future research ought to focus on verifying and validating the models by performing a large-scale clinical trial to better promote AI in CAM in the era of digital health.
Collapse
Affiliation(s)
- Hongmin Chu
- Daecheong Public Health Subcenter, Incheon, South Korea
| | - Seunghwan Moon
- Department of Global Public Health and Korean Medicine Management, Graduate School, Kyung Hee University, Seoul, South Korea
| | - Jeongsu Park
- Department of College of Korean Medicine, Wonkwang University, Iksan, South Korea
| | - Seongjun Bak
- Department of College of Korean Medicine, Wonkwang University, Iksan, South Korea
| | - Youme Ko
- National Institute for Korean Medicine Development (NIKOM), Seoul, South Korea
| | - Bo-Young Youn
- Department of Preventive Medicine, College of Korean Medicine, Kyung Hee University, Seoul, South Korea
- *Correspondence: Bo-Young Youn,
| |
Collapse
|
43
|
Hakkoum H, Abnane I, Idri A. Interpretability in the medical field: A systematic mapping and review study. Appl Soft Comput 2022. [DOI: 10.1016/j.asoc.2021.108391] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022]
|
44
|
Stokes K, Castaldo R, Federici C, Pagliara S, Maccaro A, Cappuccio F, Fico G, Salvatore M, Franzese M, Pecchia L. The use of artificial intelligence systems in diagnosis of pneumonia via signs and symptoms: A systematic review. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2021.103325] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/09/2023]
|
45
|
Rubinger L, Gazendam A, Ekhtiari S, Bhandari M. Machine learning and artificial intelligence in research and healthcare ✰,✰✰. Injury 2022:S0020-1383(22)00076-6. [PMID: 35135685 DOI: 10.1016/j.injury.2022.01.046] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/03/2021] [Accepted: 01/29/2022] [Indexed: 02/02/2023]
Abstract
Artificial intelligence (AI) is a broad term referring to the application of computational algorithms that can analyze large data sets to classify, predict, or gain useful conclusions. Under the umbrella of AI is machine learning (ML). ML is the process of building or learning statistical models using previously observed real world data to predict outcomes, or categorize observations based on 'training' provided by humans. These predictions are then applied to future data, all the while folding in the new data into its perpetually improving and calibrated statistical model. The future of AI and ML in healthcare research is exciting and expansive. AI and ML are becoming cornerstones in the medical and healthcare-research domains and are integral in our continued processing and capitalization of robust patient EMR data. Considerations for the use and application of ML in healthcare settings include assessing the quality of data inputs and decision-making that serve as the foundations of the ML model, ensuring the end-product is interpretable, transparent, and ethical concerns are considered throughout the development process. The current and future applications of ML include improving the quality and quantity of data collected from EMRs to improve registry data, utilizing these robust datasets to improve and standardized research protocols and outcomes, clinical decision-making applications, natural language processing and improving the fundamentals of value-based care, to name only a few.
Collapse
Affiliation(s)
- Luc Rubinger
- Division of Orthopaedics, Department of Surgery, McMaster University, Hamilton, ON Canada; Centre for Evidence-Based Orthopaedics, 293 Wellington St. N, Suite 110, Hamilton, ON L8L 8E7 Canada.
| | - Aaron Gazendam
- Division of Orthopaedics, Department of Surgery, McMaster University, Hamilton, ON Canada; Centre for Evidence-Based Orthopaedics, 293 Wellington St. N, Suite 110, Hamilton, ON L8L 8E7 Canada
| | - Seper Ekhtiari
- Division of Orthopaedics, Department of Surgery, McMaster University, Hamilton, ON Canada; Centre for Evidence-Based Orthopaedics, 293 Wellington St. N, Suite 110, Hamilton, ON L8L 8E7 Canada
| | - Mohit Bhandari
- Division of Orthopaedics, Department of Surgery, McMaster University, Hamilton, ON Canada; Centre for Evidence-Based Orthopaedics, 293 Wellington St. N, Suite 110, Hamilton, ON L8L 8E7 Canada
| |
Collapse
|
46
|
Morley J, Murphy L, Mishra A, Joshi I, Karpathakis K. Governing Data and Artificial Intelligence for Health Care: Developing an International Understanding. JMIR Form Res 2022; 6:e31623. [PMID: 35099403 PMCID: PMC8844981 DOI: 10.2196/31623] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2021] [Revised: 10/12/2021] [Accepted: 10/13/2021] [Indexed: 01/04/2023] Open
Abstract
Background Although advanced analytical techniques falling under the umbrella heading of artificial intelligence (AI) may improve health care, the use of AI in health raises safety and ethical concerns. There are currently no internationally recognized governance mechanisms (policies, ethical standards, evaluation, and regulation) for developing and using AI technologies in health care. A lack of international consensus creates technical and social barriers to the use of health AI while potentially hampering market competition. Objective The aim of this study is to review current health data and AI governance mechanisms being developed or used by Global Digital Health Partnership (GDHP) member countries that commissioned this research, identify commonalities and gaps in approaches, identify examples of best practices, and understand the rationale for policies. Methods Data were collected through a scoping review of academic literature and a thematic analysis of policy documents published by selected GDHP member countries. The findings from this data collection and the literature were used to inform semistructured interviews with key senior policy makers from GDHP member countries exploring their countries’ experience of AI-driven technologies in health care and associated governance and inform a focus group with professionals working in international health and technology to discuss the themes and proposed policy recommendations. Policy recommendations were developed based on the aggregated research findings. Results As this is an empirical research paper, we primarily focused on reporting the results of the interviews and the focus group. Semistructured interviews (n=10) and a focus group (n=6) revealed 4 core areas for international collaborations: leadership and oversight, a whole systems approach covering the entire AI pipeline from data collection to model deployment and use, standards and regulatory processes, and engagement with stakeholders and the public. There was a broad range of maturity in health AI activity among the participants, with varying data infrastructure, application of standards across the AI life cycle, and strategic approaches to both development and deployment. A demand for further consistency at the international level and policies was identified to support a robust innovation pipeline. In total, 13 policy recommendations were developed to support GDHP member countries in overcoming core AI governance barriers and establishing common ground for international collaboration. Conclusions AI-driven technology research and development for health care outpaces the creation of supporting AI governance globally. International collaboration and coordination on AI governance for health care is needed to ensure coherent solutions and allow countries to support and benefit from each other’s work. International bodies and initiatives have a leading role to play in the international conversation, including the production of tools and sharing of practical approaches to the use of AI-driven technologies for health care.
Collapse
Affiliation(s)
- Jessica Morley
- Oxford Internet Institute, University of Oxford, Oxford, United Kingdom
| | | | - Abhishek Mishra
- Uehiro Centre for Practical Ethics, University of Oxford, Oxford, United Kingdom
| | | | - Kassandra Karpathakis
- Harvard T.H. Chan School of Public Health, Harvard University, Boston, MA, United States
| |
Collapse
|
47
|
Gama F, Tyskbo D, Nygren J, Barlow J, Reed J, Svedberg P. Implementation Frameworks for Artificial Intelligence Translation Into Health Care Practice: Scoping Review. J Med Internet Res 2022; 24:e32215. [PMID: 35084349 PMCID: PMC8832266 DOI: 10.2196/32215] [Citation(s) in RCA: 32] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2021] [Revised: 09/02/2021] [Accepted: 12/27/2021] [Indexed: 01/22/2023] Open
Abstract
Background Significant efforts have been made to develop artificial intelligence (AI) solutions for health care improvement. Despite the enthusiasm, health care professionals still struggle to implement AI in their daily practice. Objective This paper aims to identify the implementation frameworks used to understand the application of AI in health care practice. Methods A scoping review was conducted using the Cochrane, Evidence Based Medicine Reviews, Embase, MEDLINE, and PsycINFO databases to identify publications that reported frameworks, models, and theories concerning AI implementation in health care. This review focused on studies published in English and investigating AI implementation in health care since 2000. A total of 2541 unique publications were retrieved from the databases and screened on titles and abstracts by 2 independent reviewers. Selected articles were thematically analyzed against the Nilsen taxonomy of implementation frameworks, and the Greenhalgh framework for the nonadoption, abandonment, scale-up, spread, and sustainability (NASSS) of health care technologies. Results In total, 7 articles met all eligibility criteria for inclusion in the review, and 2 articles included formal frameworks that directly addressed AI implementation, whereas the other articles provided limited descriptions of elements influencing implementation. Collectively, the 7 articles identified elements that aligned with all the NASSS domains, but no single article comprehensively considered the factors known to influence technology implementation. New domains were identified, including dependency on data input and existing processes, shared decision-making, the role of human oversight, and ethics of population impact and inequality, suggesting that existing frameworks do not fully consider the unique needs of AI implementation. Conclusions This literature review demonstrates that understanding how to implement AI in health care practice is still in its early stages of development. Our findings suggest that further research is needed to provide the knowledge necessary to develop implementation frameworks to guide the future implementation of AI in clinical practice and highlight the opportunity to draw on existing knowledge from the field of implementation science.
Collapse
Affiliation(s)
- Fábio Gama
- School of Business, Innovation and Sustainability, Halmstad University, Halmstad, Sweden.,School of Administration and Economic Science, Santa Catarina State University, Florianópolis, Brazil
| | - Daniel Tyskbo
- School of Health and Welfare, Halmstad University, Halmstad, Sweden
| | - Jens Nygren
- School of Health and Welfare, Halmstad University, Halmstad, Sweden
| | - James Barlow
- Centre for Health Economics and Policy Innovation, Imperial College Business School, London, United Kingdom
| | - Julie Reed
- School of Health and Welfare, Halmstad University, Halmstad, Sweden
| | - Petra Svedberg
- School of Health and Welfare, Halmstad University, Halmstad, Sweden
| |
Collapse
|
48
|
Harris S, Bonnici T, Keen T, Lilaonitkul W, White MJ, Swanepoel N. Clinical deployment environments: Five pillars of translational machine learning for health. Front Digit Health 2022; 4:939292. [PMID: 36060542 PMCID: PMC9437594 DOI: 10.3389/fdgth.2022.939292] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2022] [Accepted: 07/25/2022] [Indexed: 01/14/2023] Open
Abstract
Machine Learning for Health (ML4H) has demonstrated efficacy in computer imaging and other self-contained digital workflows, but has failed to substantially impact routine clinical care. This is no longer because of poor adoption of Electronic Health Records Systems (EHRS), but because ML4H needs an infrastructure for development, deployment and evaluation within the healthcare institution. In this paper, we propose a design pattern called a Clinical Deployment Environment (CDE). We sketch the five pillars of the CDE: (1) real world development supported by live data where ML4H teams can iteratively build and test at the bedside (2) an ML-Ops platform that brings the rigour and standards of continuous deployment to ML4H (3) design and supervision by those with expertise in AI safety (4) the methods of implementation science that enable the algorithmic insights to influence the behaviour of clinicians and patients and (5) continuous evaluation that uses randomisation to avoid bias but in an agile manner. The CDE is intended to answer the same requirements that bio-medicine articulated in establishing the translational medicine domain. It envisions a transition from "real-world" data to "real-world" development.
Collapse
Affiliation(s)
- Steve Harris
- Institute of Health Informatics, University College London, London, United Kingdom
- Department of Critical Care, University College London Hospital, London, United Kingdom
- Correspondence: Steve Harris
| | - Tim Bonnici
- Institute of Health Informatics, University College London, London, United Kingdom
- Department of Critical Care, University College London Hospital, London, United Kingdom
| | - Thomas Keen
- Institute of Health Informatics, University College London, London, United Kingdom
| | - Watjana Lilaonitkul
- Institute of Health Informatics, University College London, London, United Kingdom
| | - Mark J. White
- Digital Healthcare, University College London Hospital, London, United Kingdom
| | - Nel Swanepoel
- Centre for Advanced Research Computing, University College London, London, United Kingdom
| |
Collapse
|
49
|
Sánchez-Salmerón R, Gómez-Urquiza JL, Albendín-García L, Correa-Rodríguez M, Martos-Cabrera MB, Velando-Soriano A, Suleiman-Martos N. Machine learning methods applied to triage in emergency services: A systematic review. Int Emerg Nurs 2021; 60:101109. [PMID: 34952482 DOI: 10.1016/j.ienj.2021.101109] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2021] [Revised: 08/23/2021] [Accepted: 10/22/2021] [Indexed: 12/23/2022]
Abstract
BACKGROUND In emergency services is important to accurately assess and classify symptoms, which may be improved with the help of technology. One mechanism that could help and improve predictions from health records or patient flow is machine learning (ML). AIM To analyse the effectiveness of ML systems in triage for making predictions at the emergency department in comparison with other triage scales/scores. METHODS Following the PRISMA recommendations, a systematic review was conducted using CINAHL, Cochrane, Cuiden, Medline and Scopus databases with the search equation "Machine learning AND triage AND emergency". RESULTS Eleven studies were identified. The studies show that the use of ML methods consistently predict important outcomes like mortality, critical care outcomes and admission, and the need for hospitalization in comparison with scales like Emergency Severity Index or others. Among the ML models considered, XGBoost and Deep Neural Networks obtained the highest levels of prediction accuracy, while Logistic Regression performed obtained the worst values. CONCLUSIONS Machine learning methods can be a good instrument for helping triage process with the prediction of important emergency variables like mortality or the need for critical care or hospitalization.
Collapse
Affiliation(s)
| | - José L Gómez-Urquiza
- Faculty of Health Sciences, University of Granada, Avenida de la Ilustración N. 60, 18016 Granada, Spain.
| | - Luis Albendín-García
- Faculty of Health Sciences, University of Granada, Avenida de la Ilustración N. 60, 18016 Granada, Spain.
| | - María Correa-Rodríguez
- Faculty of Health Sciences, University of Granada, Avenida de la Ilustración N. 60, 18016 Granada, Spain.
| | - María Begoña Martos-Cabrera
- San Cecilio Clinical University Hospital, Andalusian Health Service, Avenida del Conocimiento s/n, 18016 Granada, Spain.
| | - Almudena Velando-Soriano
- San Cecilio Clinical University Hospital, Andalusian Health Service, Avenida del Conocimiento s/n, 18016 Granada, Spain.
| | - Nora Suleiman-Martos
- Faculty of Health Sciences, Ceuta University Campus, University of Granada, C/Cortadura del Valle SN, 51001 Ceuta, Spain.
| |
Collapse
|
50
|
Pettit RW, Fullem R, Cheng C, Amos CI. Artificial intelligence, machine learning, and deep learning for clinical outcome prediction. Emerg Top Life Sci 2021; 5:ETLS20210246. [PMID: 34927670 PMCID: PMC8786279 DOI: 10.1042/etls20210246] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2021] [Revised: 12/03/2021] [Accepted: 12/07/2021] [Indexed: 12/12/2022]
Abstract
AI is a broad concept, grouping initiatives that use a computer to perform tasks that would usually require a human to complete. AI methods are well suited to predict clinical outcomes. In practice, AI methods can be thought of as functions that learn the outcomes accompanying standardized input data to produce accurate outcome predictions when trialed with new data. Current methods for cleaning, creating, accessing, extracting, augmenting, and representing data for training AI clinical prediction models are well defined. The use of AI to predict clinical outcomes is a dynamic and rapidly evolving arena, with new methods and applications emerging. Extraction or accession of electronic health care records and combining these with patient genetic data is an area of present attention, with tremendous potential for future growth. Machine learning approaches, including decision tree methods of Random Forest and XGBoost, and deep learning techniques including deep multi-layer and recurrent neural networks, afford unique capabilities to accurately create predictions from high dimensional, multimodal data. Furthermore, AI methods are increasing our ability to accurately predict clinical outcomes that previously were difficult to model, including time-dependent and multi-class outcomes. Barriers to robust AI-based clinical outcome model deployment include changing AI product development interfaces, the specificity of regulation requirements, and limitations in ensuring model interpretability, generalizability, and adaptability over time.
Collapse
Affiliation(s)
- Rowland W. Pettit
- Institute for Clinical and Translational Research, Baylor College of Medicine, Houston, TX, U.S.A
| | - Robert Fullem
- Department of Molecular and Human Genetics, Baylor College of Medicine, Houston, TX, U.S.A
| | - Chao Cheng
- Institute for Clinical and Translational Research, Baylor College of Medicine, Houston, TX, U.S.A
- Section of Epidemiology and Population Sciences, Department of Medicine, Baylor College of Medicine, Houston, TX, U.S.A
| | - Christopher I. Amos
- Institute for Clinical and Translational Research, Baylor College of Medicine, Houston, TX, U.S.A
- Section of Epidemiology and Population Sciences, Department of Medicine, Baylor College of Medicine, Houston, TX, U.S.A
- Dan L Duncan Comprehensive Cancer Center, Baylor College of Medicine, Houston, TX, U.S.A
| |
Collapse
|