51
|
Didi I, Alliot JM, Dumas PY, Vergez F, Tavitian S, Largeaud L, Bidet A, Rieu JB, Luquet I, Lechevalier N, Delabesse E, Sarry A, De Grande AC, Bérard E, Pigneux A, Récher C, Simoncini D, Bertoli S. Artificial intelligence-based prediction models for acute myeloid leukemia using real-life data: A DATAML registry study. Leuk Res 2024; 136:107437. [PMID: 38215555 DOI: 10.1016/j.leukres.2024.107437] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2023] [Revised: 12/19/2023] [Accepted: 01/05/2024] [Indexed: 01/14/2024]
Abstract
We designed artificial intelligence-based prediction models (AIPM) using 52 diagnostic variables from 3687 patients included in the DATAML registry treated with intensive chemotherapy (IC, N = 3030) or azacitidine (AZA, N = 657) for an acute myeloid leukemia (AML). A neural network called multilayer perceptron (MLP) achieved a prediction accuracy for overall survival (OS) of 68.5% and 62.1% in the IC and AZA cohorts, respectively. The Boruta algorithm could select the most important variables for prediction without decreasing accuracy. Thirteen features were retained with this algorithm in the IC cohort: age, cytogenetic risk, white blood cells count, LDH, platelet count, albumin, MPO expression, mean corpuscular volume, CD117 expression, NPM1 mutation, AML status (de novo or secondary), multilineage dysplasia and ASXL1 mutation; and 7 variables in the AZA cohort: blood blasts, serum ferritin, CD56, LDH, hemoglobin, CD13 and disseminated intravascular coagulation (DIC). We believe that AIPM could help hematologists to deal with the huge amount of data available at diagnosis, enabling them to have an OS estimation and guide their treatment choice. Our registry-based AIPM could offer a large real-life dataset with original and exhaustive features and select a low number of diagnostic features with an equivalent accuracy of prediction, more appropriate to routine practice.
Collapse
Affiliation(s)
| | | | - Pierre-Yves Dumas
- Centre Hospitalier Universitaire de Bordeaux, Service d'Hématologie Clinique et de Thérapie Cellulaire, Bordeaux, France; Université de Bordeaux, Bordeaux, France; Institut National de la Santé et de la Recherche Médicale, U1035 Bordeaux, France
| | - François Vergez
- Centre Hospitalo-Universitaire de Toulouse, Institut Universitaire du Cancer Toulouse-Oncopole, Laboratoire d'hématologie, Toulouse, France; Centre de Recherches en Cancérologie de Toulouse, Université Toulouse 3 Paul Sabatier, Toulouse, France
| | - Suzanne Tavitian
- Centre Hospitalo-Universitaire de Toulouse, Institut Universitaire du Cancer de Toulouse-Oncopole, Service d'hématologie, Toulouse, France
| | - Laëtitia Largeaud
- Centre Hospitalo-Universitaire de Toulouse, Institut Universitaire du Cancer Toulouse-Oncopole, Laboratoire d'hématologie, Toulouse, France; Centre de Recherches en Cancérologie de Toulouse, Université Toulouse 3 Paul Sabatier, Toulouse, France
| | - Audrey Bidet
- CHU Bordeaux, Laboratoire d'Hématologie Biologique, F-33000 Bordeaux, France
| | - Jean-Baptiste Rieu
- Centre Hospitalo-Universitaire de Toulouse, Institut Universitaire du Cancer Toulouse-Oncopole, Laboratoire d'hématologie, Toulouse, France
| | - Isabelle Luquet
- Centre Hospitalo-Universitaire de Toulouse, Institut Universitaire du Cancer Toulouse-Oncopole, Laboratoire d'hématologie, Toulouse, France
| | - Nicolas Lechevalier
- CHU Bordeaux, Laboratoire d'Hématologie Biologique, F-33000 Bordeaux, France
| | - Eric Delabesse
- Centre Hospitalo-Universitaire de Toulouse, Institut Universitaire du Cancer Toulouse-Oncopole, Laboratoire d'hématologie, Toulouse, France; Centre de Recherches en Cancérologie de Toulouse, Université Toulouse 3 Paul Sabatier, Toulouse, France
| | - Audrey Sarry
- Centre Hospitalo-Universitaire de Toulouse, Institut Universitaire du Cancer de Toulouse-Oncopole, Service d'hématologie, Toulouse, France
| | - Anne-Charlotte De Grande
- Centre Hospitalier Universitaire de Bordeaux, Service d'Hématologie Clinique et de Thérapie Cellulaire, Bordeaux, France
| | - Emilie Bérard
- Department of Epidemiology, Health Economics and Public Health, UMR 1295 CERPOP, University of Toulouse, INSERM, UPS, Toulouse University Hospital (CHU), Toulouse, France
| | - Arnaud Pigneux
- Centre Hospitalier Universitaire de Bordeaux, Service d'Hématologie Clinique et de Thérapie Cellulaire, Bordeaux, France; Université de Bordeaux, Bordeaux, France
| | - Christian Récher
- Centre de Recherches en Cancérologie de Toulouse, Université Toulouse 3 Paul Sabatier, Toulouse, France; Centre Hospitalo-Universitaire de Toulouse, Institut Universitaire du Cancer de Toulouse-Oncopole, Service d'hématologie, Toulouse, France
| | - David Simoncini
- IRIT UMR 5505-CNRS, Université Toulouse I Capitole, Toulouse, France
| | - Sarah Bertoli
- Centre de Recherches en Cancérologie de Toulouse, Université Toulouse 3 Paul Sabatier, Toulouse, France; Centre Hospitalo-Universitaire de Toulouse, Institut Universitaire du Cancer de Toulouse-Oncopole, Service d'hématologie, Toulouse, France.
| |
Collapse
|
52
|
Kolasa K, Admassu B, Hołownia-Voloskova M, Kędzior KJ, Poirrier JE, Perni S. Systematic reviews of machine learning in healthcare: a literature review. Expert Rev Pharmacoecon Outcomes Res 2024; 24:63-115. [PMID: 37955147 DOI: 10.1080/14737167.2023.2279107] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2023] [Accepted: 10/31/2023] [Indexed: 11/14/2023]
Abstract
INTRODUCTION The increasing availability of data and computing power has made machine learning (ML) a viable approach to faster, more efficient healthcare delivery. METHODS A systematic literature review (SLR) of published SLRs evaluating ML applications in healthcare settings published between1 January 2010 and 27 March 2023 was conducted. RESULTS In total 220 SLRs covering 10,462 ML algorithms were reviewed. The main application of AI in medicine related to the clinical prediction and disease prognosis in oncology and neurology with the use of imaging data. Accuracy, specificity, and sensitivity were provided in 56%, 28%, and 25% SLRs respectively. Internal and external validation was reported in 53% and less than 1% of the cases respectively. The most common modeling approach was neural networks (2,454 ML algorithms), followed by support vector machine and random forest/decision trees (1,578 and 1,522 ML algorithms, respectively). EXPERT OPINION The review indicated considerable reporting gaps in terms of the ML's performance, both internal and external validation. Greater accessibility to healthcare data for developers can ensure the faster adoption of ML algorithms into clinical practice.
Collapse
Affiliation(s)
- Katarzyna Kolasa
- Division of Health Economics and Healthcare Management, Kozminski University, Warsaw, Poland
| | - Bisrat Admassu
- Division of Health Economics and Healthcare Management, Kozminski University, Warsaw, Poland
| | | | | | | | | |
Collapse
|
53
|
Adhikari K, Naik N, Hameed BZ, Raghunath SK, Somani BK. Exploring the Ethical, Legal, and Social Implications of ChatGPT in Urology. Curr Urol Rep 2024; 25:1-8. [PMID: 37735339 DOI: 10.1007/s11934-023-01185-2] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/05/2023] [Indexed: 09/23/2023]
Abstract
PURPOSE OF THE REVIEW ChatGPT is programmed to generate responses based on pattern recognition. With this vast popularity and exponential growth, the question arises of moral issues, security and legitimacy. In this review article, we aim to analyze the ethical and legal implications of using ChatGPT in Urology and explore potential solutions addressing these concerns. RECENT FINDINGS There are many potential applications of ChatGPT in urology, and the extent to which it might improve healthcare may cause a profound shift in the way we deliver our services to patients and the overall healthcare system. This encompasses diagnosis and treatment planning, clinical workflow, patient education, augmenting consultations, and urological research. The ethical and legal considerations include patient autonomy and informed consent, privacy and confidentiality, bias and fairness, human oversight and accountability, trust and transparency, liability and malpractice, intellectual property rights, and regulatory framework. The application of ChatGPT in urology has shown great potential to improve patient care and assist urologists in various aspects of clinical practice, research, and education. Complying with data security and privacy regulations, and ensuring human oversight and accountability are some potential solutions to these legal and ethical concerns. Overall, the benefits and risks of using ChatGPT in urology must be weighed carefully, and a cautious approach must be taken to ensure that its use aligns with human values and advances patient care ethically and responsibly.
Collapse
Affiliation(s)
- Kinju Adhikari
- Department of Urology, HCG Cancer Centre, Bangaluru, India
| | - Nithesh Naik
- Department of Mechanical and Industrial Engineering, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, Karnataka, India
| | - Bm Zeeshan Hameed
- Department of Urology, Father Muller Medical College, Mangalore, Karnataka, India
| | - S K Raghunath
- Department of Urology, HCG Cancer Centre, Bangaluru, India
| | - Bhaskar K Somani
- Department of Urology, University Hospital Southampton NHS Trust, Southampton, SO16 6YD, UK.
| |
Collapse
|
54
|
Lou SS, Liu Y, Cohen ME, Ko CY, Hall BL, Kannampallil T. National Multi-Institutional Validation of a Surgical Transfusion Risk Prediction Model. J Am Coll Surg 2024; 238:99-105. [PMID: 37737660 DOI: 10.1097/xcs.0000000000000874] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/23/2023]
Abstract
BACKGROUND Accurate estimation of surgical transfusion risk is important for many aspects of surgical planning, yet few methods for estimating are available for estimating such risk. There is a need for reliable validated methods for transfusion risk stratification to support effective perioperative planning and resource stewardship. STUDY DESIGN This study was conducted using the American College of Surgeons NSQIP datafile from 2019. S-PATH performance was evaluated at each contributing hospital, with and without hospital-specific model tuning. Linear regression was used to assess the relationship between hospital characteristics and area under the receiver operating characteristic (AUROC) curve. RESULTS A total of 1,000,927 surgical cases from 414 hospitals were evaluated. Aggregate AUROC was 0.910 (95% CI 0.904 to 0.916) without model tuning and 0.925 (95% CI 0.919 to 0.931) with model tuning. AUROC varied across individual hospitals (median 0.900, interquartile range 0.849 to 0.944), but no statistically significant relationships were found between hospital-level characteristics studied and model AUROC. CONCLUSIONS S-PATH demonstrated excellent discriminative performance, although there was variation across hospitals that was not well-explained by hospital-level characteristics. These results highlight the S-PATH's viability as a generalizable surgical transfusion risk prediction tool.
Collapse
Affiliation(s)
- Sunny S Lou
- From the Department of Anesthesiology, Washington University School of Medicine, St Louis, MO (Lou, Kannampallil)
| | - Yaoming Liu
- Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, IL (Liu, Ko, Hall, Cohen)
| | - Mark E Cohen
- Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, IL (Liu, Ko, Hall, Cohen)
| | - Clifford Y Ko
- Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, IL (Liu, Ko, Hall, Cohen)
- Department of Surgery, David Geffen School of Medicine, University of California Los Angeles, and the VA Greater Los Angeles Health System, Los Angeles, CA (Ko)
| | - Bruce L Hall
- Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, IL (Liu, Ko, Hall, Cohen)
- Department of Surgery, Washington University School of Medicine; Center for Health Policy and the Olin Business School at Washington University in St Louis; John Cochran Veterans Affairs Medical Center; and BJC Healthcare, St Louis, MO (Hall)
| | - Thomas Kannampallil
- From the Department of Anesthesiology, Washington University School of Medicine, St Louis, MO (Lou, Kannampallil)
| |
Collapse
|
55
|
Xie J, Zhong W, Yang R, Wang L, Zhen X. Discriminative fusion of moments-aligned latent representation of multimodality medical data. Phys Med Biol 2023; 69:015015. [PMID: 38052076 DOI: 10.1088/1361-6560/ad1271] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2023] [Accepted: 12/05/2023] [Indexed: 12/07/2023]
Abstract
Fusion of multimodal medical data provides multifaceted, disease-relevant information for diagnosis or prognosis prediction modeling. Traditional fusion strategies such as feature concatenation often fail to learn hidden complementary and discriminative manifestations from high-dimensional multimodal data. To this end, we proposed a methodology for the integration of multimodality medical data by matching their moments in a latent space, where the hidden, shared information of multimodal data is gradually learned by optimization with multiple feature collinearity and correlation constrains. We first obtained the multimodal hidden representations by learning mappings between the original domain and shared latent space. Within this shared space, we utilized several relational regularizations, including data attribute preservation, feature collinearity and feature-task correlation, to encourage learning of the underlying associations inherent in multimodal data. The fused multimodal latent features were finally fed to a logistic regression classifier for diagnostic prediction. Extensive evaluations on three independent clinical datasets have demonstrated the effectiveness of the proposed method in fusing multimodal data for medical prediction modeling.
Collapse
Affiliation(s)
- Jincheng Xie
- School of Biomedical Engineering, Southern Medical University, Guangzhou, Guangdong 510515, People's Republic of China
| | - Weixiong Zhong
- School of Biomedical Engineering, Southern Medical University, Guangzhou, Guangdong 510515, People's Republic of China
| | - Ruimeng Yang
- Department of Radiology, the Second Affiliated Hospital, School of Medicine, South China University of Technology, Guangzhou, Guangdong, 510180, People's Republic of China
| | - Linjing Wang
- Radiotherapy Center, Affiliated Cancer Hospital & Institute of Guangzhou Medical University, Guangzhou, Guangdong 510095, People's Republic of China
| | - Xin Zhen
- School of Biomedical Engineering, Southern Medical University, Guangzhou, Guangdong 510515, People's Republic of China
| |
Collapse
|
56
|
Staes CJ, Beck AC, Chalkidis G, Scheese CH, Taft T, Guo JW, Newman MG, Kawamoto K, Sloss EA, McPherson JP. Design of an interface to communicate artificial intelligence-based prognosis for patients with advanced solid tumors: a user-centered approach. J Am Med Inform Assoc 2023; 31:174-187. [PMID: 37847666 PMCID: PMC10746322 DOI: 10.1093/jamia/ocad201] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2023] [Revised: 09/18/2023] [Accepted: 10/02/2023] [Indexed: 10/19/2023] Open
Abstract
OBJECTIVES To design an interface to support communication of machine learning (ML)-based prognosis for patients with advanced solid tumors, incorporating oncologists' needs and feedback throughout design. MATERIALS AND METHODS Using an interdisciplinary user-centered design approach, we performed 5 rounds of iterative design to refine an interface, involving expert review based on usability heuristics, input from a color-blind adult, and 13 individual semi-structured interviews with oncologists. Individual interviews included patient vignettes and a series of interfaces populated with representative patient data and predicted survival for each treatment decision point when a new line of therapy (LoT) was being considered. Ongoing feedback informed design decisions, and directed qualitative content analysis of interview transcripts was used to evaluate usability and identify enhancement requirements. RESULTS Design processes resulted in an interface with 7 sections, each addressing user-focused questions, supporting oncologists to "tell a story" as they discuss prognosis during a clinical encounter. The iteratively enhanced interface both triggered and reflected design decisions relevant when attempting to communicate ML-based prognosis, and exposed misassumptions. Clinicians requested enhancements that emphasized interpretability over explainability. Qualitative findings confirmed that previously identified issues were resolved and clarified necessary enhancements (eg, use months not days) and concerns about usability and trust (eg, address LoT received elsewhere). Appropriate use should be in the context of a conversation with an oncologist. CONCLUSION User-centered design, ongoing clinical input, and a visualization to communicate ML-related outcomes are important elements for designing any decision support tool enabled by artificial intelligence, particularly when communicating prognosis risk.
Collapse
Affiliation(s)
- Catherine J Staes
- College of Nursing, University of Utah, Salt Lake City, UT 84112, United States
- Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, UT 84108, United States
| | - Anna C Beck
- Department of Internal Medicine, Huntsman Cancer Institute, University of Utah, Salt Lake City, UT 84112, United States
| | - George Chalkidis
- Healthcare IT Research Department, Center for Digital Services, Hitachi Ltd., Tokyo, Japan
| | - Carolyn H Scheese
- College of Nursing, University of Utah, Salt Lake City, UT 84112, United States
- Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, UT 84108, United States
| | - Teresa Taft
- Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, UT 84108, United States
| | - Jia-Wen Guo
- College of Nursing, University of Utah, Salt Lake City, UT 84112, United States
- Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, UT 84108, United States
| | - Michael G Newman
- Department of Population Sciences, Huntsman Cancer Institute, Salt Lake City, UT 84112, United States
| | - Kensaku Kawamoto
- Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, UT 84108, United States
| | - Elizabeth A Sloss
- College of Nursing, University of Utah, Salt Lake City, UT 84112, United States
| | - Jordan P McPherson
- Department of Pharmacotherapy, College of Pharmacy, University of Utah, Salt Lake City, UT 84108, United States
- Department of Pharmacy, Huntsman Cancer Institute, Salt Lake City, UT 84112, United States
| |
Collapse
|
57
|
Barreto TDO, Veras NVR, Cardoso PH, Fernandes FRDS, Medeiros LPDS, Bezerra MV, de Andrade FMQ, Pinheiro CDO, Sánchez-Gendriz I, Silva GJPC, Rodrigues LF, de Morais AHF, dos Santos JPQ, Paiva JC, de Andrade IGM, Valentim RADM. Artificial intelligence applied to analyzes during the pandemic: COVID-19 beds occupancy in the state of Rio Grande do Norte, Brazil. Front Artif Intell 2023; 6:1290022. [PMID: 38145230 PMCID: PMC10748397 DOI: 10.3389/frai.2023.1290022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2023] [Accepted: 11/17/2023] [Indexed: 12/26/2023] Open
Abstract
The COVID-19 pandemic is already considered one of the biggest global health crises. In Rio Grande do Norte, a Brazilian state, the RegulaRN platform was the health information system used to regulate beds for patients with COVID-19. This article explored machine learning and deep learning techniques with RegulaRN data in order to identify the best models and parameters to predict the outcome of a hospitalized patient. A total of 25,366 bed regulations for COVID-19 patients were analyzed. The data analyzed comes from the RegulaRN Platform database from April 2020 to August 2022. From these data, the nine most pertinent characteristics were selected from the twenty available, and blank or inconclusive data were excluded. This was followed by the following steps: data pre-processing, database balancing, training, and test. The results showed better performance in terms of accuracy (84.01%), precision (79.57%), and F1-score (81.00%) for the Multilayer Perceptron model with Stochastic Gradient Descent optimizer. The best results for recall (84.67%), specificity (84.67%), and ROC-AUC (91.6%) were achieved by Root Mean Squared Propagation. This study compared different computational methods of machine and deep learning whose objective was to classify bed regulation data for patients with COVID-19 from the RegulaRN Platform. The results have made it possible to identify the best model to help health professionals during the process of regulating beds for patients with COVID-19. The scientific findings of this article demonstrate that the computational methods used applied through a digital health solution, can assist in the decision-making of medical regulators and government institutions in situations of public health crisis.
Collapse
Affiliation(s)
- Tiago de Oliveira Barreto
- Laboratory of Technological Innovation in Health (LAIS), Federal University of Rio Grande do Norte (UFRN), Natal, Rio Grande do Norte, Brazil
| | - Nícolas Vinícius Rodrigues Veras
- Laboratory of Technological Innovation in Health (LAIS), Federal University of Rio Grande do Norte (UFRN), Natal, Rio Grande do Norte, Brazil
- Advanced Nucleus of Technological Innovation (NAVI), Federal Institute of Rio Grande do Norte (IFRN), Natal, Rio Grande do Norte, Brazil
| | - Pablo Holanda Cardoso
- Laboratory of Technological Innovation in Health (LAIS), Federal University of Rio Grande do Norte (UFRN), Natal, Rio Grande do Norte, Brazil
- Advanced Nucleus of Technological Innovation (NAVI), Federal Institute of Rio Grande do Norte (IFRN), Natal, Rio Grande do Norte, Brazil
| | - Felipe Ricardo dos Santos Fernandes
- Laboratory of Technological Innovation in Health (LAIS), Federal University of Rio Grande do Norte (UFRN), Natal, Rio Grande do Norte, Brazil
| | | | - Maria Valéria Bezerra
- Secretary of Public Health of Rio Grande do Norte, Natal, Rio Grande do Norte, Brazil
| | | | | | - Ignacio Sánchez-Gendriz
- Laboratory of Technological Innovation in Health (LAIS), Federal University of Rio Grande do Norte (UFRN), Natal, Rio Grande do Norte, Brazil
| | - Gleyson José Pinheiro Caldeira Silva
- Laboratory of Technological Innovation in Health (LAIS), Federal University of Rio Grande do Norte (UFRN), Natal, Rio Grande do Norte, Brazil
- Advanced Nucleus of Technological Innovation (NAVI), Federal Institute of Rio Grande do Norte (IFRN), Natal, Rio Grande do Norte, Brazil
| | - Leandro Farias Rodrigues
- Brazilian Company of Hospital Services (EBSERH), University Hospital of Pelotas, Federal University of Pelotas (UFPel), Pelotas, Rio Grande do Sul, Brazil
| | - Antonio Higor Freire de Morais
- Laboratory of Technological Innovation in Health (LAIS), Federal University of Rio Grande do Norte (UFRN), Natal, Rio Grande do Norte, Brazil
- Advanced Nucleus of Technological Innovation (NAVI), Federal Institute of Rio Grande do Norte (IFRN), Natal, Rio Grande do Norte, Brazil
| | - João Paulo Queiroz dos Santos
- Laboratory of Technological Innovation in Health (LAIS), Federal University of Rio Grande do Norte (UFRN), Natal, Rio Grande do Norte, Brazil
- Advanced Nucleus of Technological Innovation (NAVI), Federal Institute of Rio Grande do Norte (IFRN), Natal, Rio Grande do Norte, Brazil
| | - Jailton Carlos Paiva
- Laboratory of Technological Innovation in Health (LAIS), Federal University of Rio Grande do Norte (UFRN), Natal, Rio Grande do Norte, Brazil
- Advanced Nucleus of Technological Innovation (NAVI), Federal Institute of Rio Grande do Norte (IFRN), Natal, Rio Grande do Norte, Brazil
| | - Ion Garcia Mascarenhas de Andrade
- Laboratory of Technological Innovation in Health (LAIS), Federal University of Rio Grande do Norte (UFRN), Natal, Rio Grande do Norte, Brazil
| | | |
Collapse
|
58
|
Antão J, de Mast J, Marques A, Franssen FME, Spruit MA, Deng Q. Demystification of artificial intelligence for respiratory clinicians managing patients with obstructive lung diseases. Expert Rev Respir Med 2023; 17:1207-1219. [PMID: 38270524 DOI: 10.1080/17476348.2024.2302940] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2023] [Accepted: 01/04/2024] [Indexed: 01/26/2024]
Abstract
INTRODUCTION Asthma and chronic obstructive pulmonary disease (COPD) are leading causes of morbidity and mortality worldwide. Despite all available diagnostics and treatments, these conditions pose a significant individual, economic and social burden. Artificial intelligence (AI) promises to support clinical decision-making processes by optimizing diagnosis and treatment strategies of these heterogeneous and complex chronic respiratory diseases. Its capabilities extend to predicting exacerbation risk, disease progression and mortality, providing healthcare professionals with valuable insights for more effective care. Nevertheless, the knowledge gap between respiratory clinicians and data scientists remains a major constraint for wide application of AI and may hinder future progress. This narrative review aims to bridge this gap and encourage AI deployment by explaining its methodology and added value in asthma and COPD diagnosis and treatment. AREAS COVERED This review offers an overview of the fundamental concepts of AI and machine learning, outlines the key steps in building a model, provides examples of their applicability in asthma and COPD care, and discusses barriers to their implementation. EXPERT OPINION Machine learning can advance our understanding of asthma and COPD, enabling personalized therapy and better outcomes. Further research and validation are needed to ensure the development of clinically meaningful and generalizable models.
Collapse
Affiliation(s)
- Joana Antão
- Lab3R - Respiratory Research and Rehabilitation Laboratory, School of Health Sciences, University of Aveiro (ESSUA), Aveiro, Portugal
- iBiMED - Institute of Biomedicine, Department of Medical Sciences, University of Aveiro, Aveiro, Portugal
- Department of Research and Development, Ciro, Horn, The Netherlands
- Department of Respiratory Medicine, Maastricht University Medical Centre, NUTRIM School of Nutrition and Translational Research in Metabolism, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands
| | - Jeroen de Mast
- Economics and Business, University of Amsterdam, Amsterdam, The Netherlands
| | - Alda Marques
- Lab3R - Respiratory Research and Rehabilitation Laboratory, School of Health Sciences, University of Aveiro (ESSUA), Aveiro, Portugal
- iBiMED - Institute of Biomedicine, Department of Medical Sciences, University of Aveiro, Aveiro, Portugal
| | - Frits M E Franssen
- Department of Research and Development, Ciro, Horn, The Netherlands
- Department of Respiratory Medicine, Maastricht University Medical Centre, NUTRIM School of Nutrition and Translational Research in Metabolism, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands
| | - Martijn A Spruit
- Department of Research and Development, Ciro, Horn, The Netherlands
- Department of Respiratory Medicine, Maastricht University Medical Centre, NUTRIM School of Nutrition and Translational Research in Metabolism, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands
| | - Qichen Deng
- Department of Research and Development, Ciro, Horn, The Netherlands
- Department of Respiratory Medicine, Maastricht University Medical Centre, NUTRIM School of Nutrition and Translational Research in Metabolism, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands
| |
Collapse
|
59
|
Hasan SU, Siddiqui MAR. Diagnostic accuracy of smartphone-based artificial intelligence systems for detecting diabetic retinopathy: A systematic review and meta-analysis. Diabetes Res Clin Pract 2023; 205:110943. [PMID: 37805002 DOI: 10.1016/j.diabres.2023.110943] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/12/2023] [Revised: 07/28/2023] [Accepted: 10/05/2023] [Indexed: 10/09/2023]
Abstract
AIMS Diabetic retinopathy (DR) is a major cause of blindness globally, early detection is critical to prevent vision loss. Traditional screening that, rely on human experts are, however, costly, and time-consuming. The purpose of this systematic review is to assess the diagnostic accuracy of smartphone-based artificial intelligence(AI) systems for DR detection. METHODS Literature review was conducted on MEDLINE, Embase, Scopus, CINAHL Plus, and Cochrane from inception to December 2022. We included diagnostic test accuracy studies evaluating the use of smartphone-based AI algorithms for DR screening in patients with diabetes, with expert human grader as the reference standard. Random-effects model was used to pool sensitivity and specificity. Any DR(ADR) and referable DR(RDR) were analyzed separately. RESULTS Out of 968 identified articles, six diagnostic test accuracy studies met our inclusion criteria, comprising 3,931 patients. Four of these studies used the Medios AI algorithm. The pooled sensitivity and specificity for diagnosis of ADR were 88 % and 91.5 % respectively and for diagnosis of RDR were 98.2 % and 81.2 % respectively. The overall risk of bias across the studies was low. CONCLUSIONS Smartphone-based AI algorithms show high diagnostic accuracy for detecting DR. However, more high-quality comparative studies are needed to evaluate the effectiveness in real-world clinical settings.
Collapse
Affiliation(s)
- S Umar Hasan
- Department of Ophthalmology and Visual Sciences, Aga Khan University Hospital, National Stadium Road, Karachi, Pakistan
| | - M A Rehman Siddiqui
- Department of Ophthalmology and Visual Sciences, Aga Khan University Hospital, National Stadium Road, Karachi, Pakistan.
| |
Collapse
|
60
|
Dang T, Spathis D, Ghosh A, Mascolo C. Human-centred artificial intelligence for mobile health sensing: challenges and opportunities. ROYAL SOCIETY OPEN SCIENCE 2023; 10:230806. [PMID: 38026044 PMCID: PMC10646451 DOI: 10.1098/rsos.230806] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/09/2023] [Accepted: 10/23/2023] [Indexed: 12/01/2023]
Abstract
Advances in wearable sensing and mobile computing have enabled the collection of health and well-being data outside of traditional laboratory and hospital settings, paving the way for a new era of mobile health. Meanwhile, artificial intelligence (AI) has made significant strides in various domains, demonstrating its potential to revolutionize healthcare. Devices can now diagnose diseases, predict heart irregularities and unlock the full potential of human cognition. However, the application of machine learning (ML) to mobile health sensing poses unique challenges due to noisy sensor measurements, high-dimensional data, sparse and irregular time series, heterogeneity in data, privacy concerns and resource constraints. Despite the recognition of the value of mobile sensing, leveraging these datasets has lagged behind other areas of ML. Furthermore, obtaining quality annotations and ground truth for such data is often expensive or impractical. While recent large-scale longitudinal studies have shown promise in leveraging wearable sensor data for health monitoring and prediction, they also introduce new challenges for data modelling. This paper explores the challenges and opportunities of human-centred AI for mobile health, focusing on key sensing modalities such as audio, location and activity tracking. We discuss the limitations of current approaches and propose potential solutions.
Collapse
Affiliation(s)
- Ting Dang
- University of Cambridge, Cambridge, UK
- Nokia Bell Labs, Cambridge, UK
| | - Dimitris Spathis
- University of Cambridge, Cambridge, UK
- Nokia Bell Labs, Cambridge, UK
| | - Abhirup Ghosh
- University of Cambridge, Cambridge, UK
- University of Birmingham, Birmingham, UK
| | | |
Collapse
|
61
|
Spiero I, Schuit E, Wijers O, Hoebers F, Langendijk J, Leeuwenberg A. Comparing supervised and semi-supervised machine learning approaches in NTCP modeling to predict complications in head and neck cancer patients. Clin Transl Radiat Oncol 2023; 43:100677. [PMID: 37822705 PMCID: PMC10562149 DOI: 10.1016/j.ctro.2023.100677] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2023] [Revised: 08/01/2023] [Accepted: 09/14/2023] [Indexed: 10/13/2023] Open
Abstract
Background and purpose Head and neck cancer (HNC) patients treated with radiotherapy often suffer from radiation-induced toxicities. Normal Tissue Complication Probability (NTCP) modeling can be used to determine the probability to develop these toxicities based on patient, tumor, treatment and dose characteristics. Since the currently used NTCP models are developed using supervised methods that discard unlabeled patient data, we assessed whether the addition of unlabeled patient data by using semi-supervised modeling would gain predictive performance. Materials and methods The semi-supervised method of self-training was compared to supervised regression methods with and without prior multiple imputation by chained equation (MICE). The models were developed for the most common toxicity outcomes in HNC patients, xerostomia (dry mouth) and dysphagia (difficulty swallowing), measured at six months after treatment, in a development cohort of 750 HNC patients. The models were externally validated in a validation cohort of 395 HNC patients. Model performance was assessed by discrimination and calibration. Results MICE and self-training did not improve performance in terms of discrimination or calibration at external validation compared to current regression models. In addition, the relative performance of the different models did not change upon a decrease in the amount of (labeled) data available for model development. Models using ridge regression outperformed the logistic models for the dysphagia outcome. Conclusion Since there was no apparent gain in the addition of unlabeled patient data by using the semi-supervised method of self-training or MICE, the supervised regression models would still be preferred in current NTCP modeling for HNC patients.
Collapse
Affiliation(s)
- I. Spiero
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
| | - E. Schuit
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
| | - O.B. Wijers
- Radiotherapeutic Institute Friesland, Leeuwarden, the Netherlands
| | - F.J.P. Hoebers
- Department of Radiation Oncology (Maastro), GROW School for Oncology and Reproduction, Maastricht University Medical Centre+, Maastricht, the Netherlands
| | - J.A. Langendijk
- Department of Radiation Oncology, University of Groningen, University Medical Center Groningen, Groningen, the Netherlands
| | - A.M. Leeuwenberg
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
| |
Collapse
|
62
|
Saenz AD, Harned Z, Banerjee O, Abràmoff MD, Rajpurkar P. Autonomous AI systems in the face of liability, regulations and costs. NPJ Digit Med 2023; 6:185. [PMID: 37803209 PMCID: PMC10558567 DOI: 10.1038/s41746-023-00929-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2023] [Accepted: 09/14/2023] [Indexed: 10/08/2023] Open
Abstract
Autonomous AI systems in medicine promise improved outcomes but raise concerns about liability, regulation, and costs. With the advent of large-language models, which can understand and generate medical text, the urgency for addressing these concerns increases as they create opportunities for more sophisticated autonomous AI systems. This perspective explores the liability implications for physicians, hospitals, and creators of AI technology, as well as the evolving regulatory landscape and payment models. Physicians may be favored in malpractice cases if they follow rigorously validated AI recommendations. However, AI developers may face liability for failing to adhere to industry-standard best practices during development and implementation. The evolving regulatory landscape, led by the FDA, seeks to ensure transparency, evaluation, and real-world monitoring of AI systems, while payment models such as MPFS, NTAP, and commercial payers adapt to accommodate them. The widespread adoption of autonomous AI systems can potentially streamline workflows and allow doctors to concentrate on the human aspects of healthcare.
Collapse
Affiliation(s)
- Agustina D Saenz
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
- Department of Medicine, Brigham and Women's Hospital, Boston, MA, USA
- Department of Medicine, Harvard Medical School, Boston, MA, USA
| | - Zach Harned
- Center for Artificial Intelligence in Medicine & Imaging, Stanford University, Palo Alto, CA, USA
- Fenwick & West LLP, Mountain View, CA, USA
| | - Oishi Banerjee
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Michael D Abràmoff
- Department of Ophthalmology and Visual Sciences, University of Iowa, Iowa City, IA, USA
- Department of Biomedical Engineering, University of Iowa, Iowa City, IA, USA
- Department of Electrical and Computer Engineering, University of Iowa, Iowa City, IA, USA
- Digital Diagnostics Inc, Coralville, IA, USA
| | - Pranav Rajpurkar
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA.
| |
Collapse
|
63
|
Cao Y, Kunaprayoon D, Ren L. Interpretable AI-assisted clinical decision making (CDM) for dose prescription in radiosurgery of brain metastases. Radiother Oncol 2023; 187:109842. [PMID: 37543055 PMCID: PMC11195016 DOI: 10.1016/j.radonc.2023.109842] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2023] [Revised: 07/19/2023] [Accepted: 07/27/2023] [Indexed: 08/07/2023]
Abstract
PURPOSE AI modeling physicians' clinical decision-making (CDM) can improve the efficiency and accuracy of clinical practice or serve as a surrogate to provide initial consultations to patients seeking secondary opinions. In this study, we developed an interpretable AI model that predicts dose fractionation for patients receiving radiation therapy for brain metastases with an interpretation of its decision-making process. MATERIALS/METHODS 152 patients with brain metastases treated by radiosurgery from 2017 to 2021 were obtained. CT images and target and organ-at-risk (OAR) contours were extracted. Eight non-image clinical parameters were also extracted and digitized, including age, the number of brain metastasis, ECOG performance status, presence of symptoms, sequencing with surgery (pre- or post-operative radiation therapy), de novo vs. re-treatment, primary cancer type, and metastasis to other sites. 3D convolutional neural networks (CNN) architectures with encoding paths were built based on the CT data and clinical parameters to capture three inputs: (1) Tumor size, shape, and location; (2) The spatial relationship between tumors and OARs; (3) The clinical parameters. The models fuse the features extracted from these three inputs at the decision-making level to learn the input independently to predict dose prescription. Models with different independent paths were developed, including models combining two independent paths (IM-2), three independent paths (IM-3), and ten independent paths (IM-10) at the decision-making level. A class activation score and relative weighting were calculated for each input path during the model prediction to represent the role of each input in the decision-making process, providing an interpretation of the model prediction. The actual prescription in the record was used as ground truth for model training. The model performance was assessed by 19-fold cross-validation, with each fold consisting of randomly selected 128 training, 16 validation, and 8 testing subjects. RESULT The dose prescriptions of 152 patient cases included 48 cases with 1 × 24 Gy, 48 cases with 1 × 20-22 Gy, 32 cases with 3 × 9 Gy, and 24 cases with 5 × 6 Gy prescribed by 8 physicians. IM-2 achieved slightly superior performance than IM-3 and IM-10, with 131 (86%) patients classified correctly and 21 (14%) patients misclassified. IM-10 provided the most interpretability with a relative weighting for each input: target (34%), the relationship between target and OAR (35%), ECOG (6%), re-treatment (6%), metastasis to other sites (6%), number of brain metastases (3%), symptomatic (3%), pre/post-surgery (3%), primary cancer type (2%), age (2%), reflecting the importance of the inputs in decision making. The importance ranking of inputs interpreted from the model also matched closely with a physician's own ranking in the decision process. CONCLUSION Interpretable CNN models were successfully developed to use CT images and non-image clinical parameters to predict dose prescriptions for brain metastases patients treated by radiosurgery. Models showed high prediction accuracy while providing an interpretation of the decision process, which was validated by the physician. Such interpretability makes the model more transparent, which is crucial for the future clinical adoption of the models in routine practice for CDM assistance.
Collapse
Affiliation(s)
- Yufeng Cao
- Department of Radiation Oncology, University of Maryland, Baltimore, Maryland, USA
| | - Dan Kunaprayoon
- Department of Radiation Oncology, University of Maryland, Baltimore, Maryland, USA
| | - Lei Ren
- Department of Radiation Oncology, University of Maryland, Baltimore, Maryland, USA.
| |
Collapse
|
64
|
Meijs C, Handoko ML, Savarese G, Vernooij RWM, Vaartjes I, Banerjee A, Koudstaal S, Brugts JJ, Asselbergs FW, Uijl A. Discovering Distinct Phenotypical Clusters in Heart Failure Across the Ejection Fraction Spectrum: a Systematic Review. Curr Heart Fail Rep 2023; 20:333-349. [PMID: 37477803 PMCID: PMC10589200 DOI: 10.1007/s11897-023-00615-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 06/27/2023] [Indexed: 07/22/2023]
Abstract
REVIEW PURPOSE This systematic review aims to summarise clustering studies in heart failure (HF) and guide future clinical trial design and implementation in routine clinical practice. FINDINGS 34 studies were identified (n = 19 in HF with preserved ejection fraction (HFpEF)). There was significant heterogeneity invariables and techniques used. However, 149/165 described clusters could be assigned to one of nine phenotypes: 1) young, low comorbidity burden; 2) metabolic; 3) cardio-renal; 4) atrial fibrillation (AF); 5) elderly female AF; 6) hypertensive-comorbidity; 7) ischaemic-male; 8) valvular disease; and 9) devices. There was room for improvement on important methodological topics for all clustering studies such as external validation and transparency of the modelling process. The large overlap between the phenotypes of the clustering studies shows that clustering is a robust approach for discovering clinically distinct phenotypes. However, future studies should invest in a phenotype model that can be implemented in routine clinical practice and future clinical trial design. HF = heart failure, EF = ejection fraction, HFpEF = heart failure with preserved ejection fraction, HFrEF = heart failure with reduced ejection fraction, CKD = chronic kidney disease, AF = atrial fibrillation, IHD = ischaemic heart disease, CAD = coronary artery disease, ICD = implantable cardioverter-defibrillator, CRT = cardiac resynchronization therapy, NT-proBNP = N-terminal pro b-type natriuretic peptide, BMI = Body Mass Index, COPD = Chronic obstructive pulmonary disease.
Collapse
Affiliation(s)
- Claartje Meijs
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
- Helmholtz Zentrum München GmbH - German Research Center for Environmental Health, Institute of Computational Biology, Neuherberg, Germany
| | - M Louis Handoko
- Department of Cardiology, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam Cardiovascular Sciences, Amsterdam, the Netherlands
| | - Gianluigi Savarese
- Division of Cardiology, Department of Medicine, Karolinska Institutet, Stockholm, Sweden
| | - Robin W M Vernooij
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
- Department of Nephrology and Hypertension, University Medical Centre Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Ilonca Vaartjes
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
| | - Amitava Banerjee
- Health Data Research UK London, Institute for Health Informatics, University College London, London, UK
| | - Stefan Koudstaal
- Department of Cardiology, Green Heart Hospital, Gouda, the Netherlands
| | - Jasper J Brugts
- Department of Cardiology, Thoraxcenter, Erasmus MC University Medical Center, Rotterdam, The Netherlands
| | - Folkert W Asselbergs
- Health Data Research UK London, Institute for Health Informatics, University College London, London, UK
- Department of Cardiology, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, The Netherlands
| | - Alicia Uijl
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands.
- Division of Cardiology, Department of Medicine, Karolinska Institutet, Stockholm, Sweden.
- Department of Cardiology, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, The Netherlands.
| |
Collapse
|
65
|
Radiya K, Joakimsen HL, Mikalsen KØ, Aahlin EK, Lindsetmo RO, Mortensen KE. Performance and clinical applicability of machine learning in liver computed tomography imaging: a systematic review. Eur Radiol 2023; 33:6689-6717. [PMID: 37171491 PMCID: PMC10511359 DOI: 10.1007/s00330-023-09609-w] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2022] [Revised: 02/02/2023] [Accepted: 02/06/2023] [Indexed: 05/13/2023]
Abstract
OBJECTIVES Machine learning (ML) for medical imaging is emerging for several organs and image modalities. Our objectives were to provide clinicians with an overview of this field by answering the following questions: (1) How is ML applied in liver computed tomography (CT) imaging? (2) How well do ML systems perform in liver CT imaging? (3) What are the clinical applications of ML in liver CT imaging? METHODS A systematic review was carried out according to the guidelines from the PRISMA-P statement. The search string focused on studies containing content relating to artificial intelligence, liver, and computed tomography. RESULTS One hundred ninety-one studies were included in the study. ML was applied to CT liver imaging by image analysis without clinicians' intervention in majority of studies while in newer studies the fusion of ML method with clinical intervention have been identified. Several were documented to perform very accurately on reliable but small data. Most models identified were deep learning-based, mainly using convolutional neural networks. Potentially many clinical applications of ML to CT liver imaging have been identified through our review including liver and its lesion segmentation and classification, segmentation of vascular structure inside the liver, fibrosis and cirrhosis staging, metastasis prediction, and evaluation of chemotherapy. CONCLUSION Several studies attempted to provide transparent result of the model. To make the model convenient for a clinical application, prospective clinical validation studies are in urgent call. Computer scientists and engineers should seek to cooperate with health professionals to ensure this. KEY POINTS • ML shows great potential for CT liver image tasks such as pixel-wise segmentation and classification of liver and liver lesions, fibrosis staging, metastasis prediction, and retrieval of relevant liver lesions from similar cases of other patients. • Despite presenting the result is not standardized, many studies have attempted to provide transparent results to interpret the machine learning method performance in the literature. • Prospective studies are in urgent call for clinical validation of ML method, preferably carried out by cooperation between clinicians and computer scientists.
Collapse
Affiliation(s)
- Keyur Radiya
- Department of Gastroenterological Surgery at University Hospital of North Norway (UNN), Tromso, Norway.
- Department of Clinical Medicine, UiT The Arctic University of Norway, Tromso, Norway.
| | - Henrik Lykke Joakimsen
- Institute of Clinical Medicine, UiT The Arctic University of Norway, Tromso, Norway
- Centre for Clinical Artificial Intelligence (SPKI), University Hospital of North Norway, Tromso, Norway
| | - Karl Øyvind Mikalsen
- Department of Clinical Medicine, UiT The Arctic University of Norway, Tromso, Norway
- Centre for Clinical Artificial Intelligence (SPKI), University Hospital of North Norway, Tromso, Norway
- UiT Machine Learning Group, Department of Physics and Technology, UiT the Arctic University of Norway, Tromso, Norway
| | - Eirik Kjus Aahlin
- Department of Gastroenterological Surgery at University Hospital of North Norway (UNN), Tromso, Norway
| | - Rolv-Ole Lindsetmo
- Department of Clinical Medicine, UiT The Arctic University of Norway, Tromso, Norway
- Head Clinic of Surgery, Oncology and Women Health, University Hospital of North Norway, Tromso, Norway
| | - Kim Erlend Mortensen
- Department of Gastroenterological Surgery at University Hospital of North Norway (UNN), Tromso, Norway
- Department of Clinical Medicine, UiT The Arctic University of Norway, Tromso, Norway
| |
Collapse
|
66
|
Li S, Ning Y, Ong MEH, Chakraborty B, Hong C, Xie F, Yuan H, Liu M, Buckland DM, Chen Y, Liu N. FedScore: A privacy-preserving framework for federated scoring system development. J Biomed Inform 2023; 146:104485. [PMID: 37660960 DOI: 10.1016/j.jbi.2023.104485] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2023] [Revised: 08/08/2023] [Accepted: 08/31/2023] [Indexed: 09/05/2023]
Abstract
OBJECTIVE We propose FedScore, a privacy-preserving federated learning framework for scoring system generation across multiple sites to facilitate cross-institutional collaborations. MATERIALS AND METHODS The FedScore framework includes five modules: federated variable ranking, federated variable transformation, federated score derivation, federated model selection and federated model evaluation. To illustrate usage and assess FedScore's performance, we built a hypothetical global scoring system for mortality prediction within 30 days after a visit to an emergency department using 10 simulated sites divided from a tertiary hospital in Singapore. We employed a pre-existing score generator to construct 10 local scoring systems independently at each site and we also developed a scoring system using centralized data for comparison. RESULTS We compared the acquired FedScore model's performance with that of other scoring models using the receiver operating characteristic (ROC) analysis. The FedScore model achieved an average area under the curve (AUC) value of 0.763 across all sites, with a standard deviation (SD) of 0.020. We also calculated the average AUC values and SDs for each local model, and the FedScore model showed promising accuracy and stability with a high average AUC value which was closest to the one of the pooled model and SD which was lower than that of most local models. CONCLUSION This study demonstrates that FedScore is a privacy-preserving scoring system generator with potentially good generalizability.
Collapse
Affiliation(s)
- Siqi Li
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore
| | - Yilin Ning
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore
| | - Marcus Eng Hock Ong
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore; Health Services Research Centre, Singapore Health Services, Singapore, Singapore; Department of Emergency Medicine, Singapore General Hospital, Singapore, Singapore
| | - Bibhas Chakraborty
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore; Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore; Department of Statistics and Data Science, National University of Singapore, Singapore, Singapore; Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, USA
| | - Chuan Hong
- Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, USA
| | - Feng Xie
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore; Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore
| | - Han Yuan
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore
| | - Mingxuan Liu
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore
| | - Daniel M Buckland
- Department of Emergency Medicine, Duke University School of Medicine, Durham, NC, USA
| | - Yong Chen
- Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania, Philadelphia, PA, USA
| | - Nan Liu
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore; Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore; Institute of Data Science, National University of Singapore, Singapore, Singapore.
| |
Collapse
|
67
|
Dong Z, Tao X, Du H, Wang J, Huang L, He C, Zhao Z, Mao X, Ai Y, Zhang B, Liu M, Xu H, Jiang Z, Sun Y, Li X, Liu Z, Chen J, Song Y, Liu G, Luo C, Li Y, Zeng X, Liu J, Zhu Y, Wu L, Yu H. Exploring the challenge of early gastric cancer diagnostic AI system face in multiple centers and its potential solutions. J Gastroenterol 2023; 58:978-989. [PMID: 37515597 DOI: 10.1007/s00535-023-02025-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/02/2023] [Accepted: 07/10/2023] [Indexed: 07/31/2023]
Abstract
BACKGROUND Artificial intelligence (AI) performed variously among test sets with different diversity due to sample selection bias, which can be stumbling block for AI applications. We previously tested AI named ENDOANGEL, diagnosing early gastric cancer (EGC) on single-center videos in man-machine competition. We aimed to re-test ENDOANGEL on multi-center videos to explore challenges applying AI in multiple centers, then upgrade ENDOANGEL and explore solutions to the challenge. METHODS ENDOANGEL was re-tested on multi-center videos retrospectively collected from 12 institutions and compared with performance in previously reported single-center videos. We then upgraded ENDOANGEL to ENDOANGEL-2022 with more training samples and novel algorithms and conducted competition between ENDOANGEL-2022 and endoscopists. ENDOANGEL-2022 was then tested on single-center videos and compared with performance in multi-center videos; the two AI systems were also compared with each other and endoscopists. RESULTS Forty-six EGCs and 54 non-cancers were included in multi-center video cohort. On diagnosing EGCs, compared with single-center videos, ENDOANGEL showed stable sensitivity (97.83% vs. 100.00%) while sharply decreased specificity (61.11% vs. 82.54%); ENDOANGEL-2022 showed similar tendency while achieving significantly higher specificity (79.63%, p < 0.01) making fewer mistakes on typical lesions than ENDOANGEL. On detecting gastric neoplasms, both AI showed stable sensitivity while sharply decreased specificity. Nevertheless, both AI outperformed endoscopists in the two competitions. CONCLUSIONS Great increase of false positives is a prominent challenge for applying EGC diagnostic AI in multiple centers due to high heterogeneity of negative cases. Optimizing AI by adding samples and using novel algorithms is promising to overcome this challenge.
Collapse
Affiliation(s)
- Zehua Dong
- Renmin Hospital of Wuhan University, Wuhan, China
- Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
- Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
| | - Xiao Tao
- Renmin Hospital of Wuhan University, Wuhan, China
- Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
- Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
| | - Hongliu Du
- Renmin Hospital of Wuhan University, Wuhan, China
- Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
- Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
| | - Junxiao Wang
- Renmin Hospital of Wuhan University, Wuhan, China
- Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
- Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
| | - Li Huang
- Renmin Hospital of Wuhan University, Wuhan, China
- Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
- Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
| | - Chiyi He
- Department of Gastroenterology, Yijishan Hospital of Wannan Medical College, Wuhu, 241001, Anhui, People's Republic of China
| | - Zhifeng Zhao
- Department of Digestive Endoscopy, The Fourth Hospital of China Medical University, Shenyang, 110032, Liaoning Province, People's Republic of China
| | - Xinli Mao
- Department of Gastroenterology, Taizhou Hospital of Zhejiang Province Affiliated to Wenzhou Medical University, Linhai, Zhejiang, China
| | - Yaowei Ai
- Department of Gastroenterology, The People's Hospital of China Three Gorges University, The First People's Hospital of Yichang, Yichang, China
| | - Beiping Zhang
- Department of Gastroenterology, The Second Affiliated Hospital of Guangzhou University of Chinese Medicine, Guangzhou, China
| | - Mei Liu
- Department of Gastroenterology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| | - Hong Xu
- Department of Endoscopy, The First Hospital of Jilin University, Changchun, China
| | - Zhenyu Jiang
- Department of Gastroenterology, The Second Affiliated Hospital of Baotou Medical College, Baotou, Inner Mongolia, China
| | - Yunwei Sun
- Department of Gastroenterology, Ruijin Hospital, Shanghai Jiaotong University, Gubei Branch, Shanghai, People's Republic of China
| | - Xiuling Li
- Department of Gastroenterology, School of Clinical Medicine, Henan Provincial People's Hospital, People's Hospital of Zhengzhou University, Henan University, Zhengzhou, Henan, China
| | - Zhihong Liu
- Department of Gastroenterology, Jilin City People's Hospital, Jilin, China
| | - Jinzhong Chen
- Endoscopy Center, School of Medicine, The First Affiliated Hospital of Xiamen University, Xiamen University, Xiamen, China
| | - Ying Song
- Department of Gastroenterology, Xi'an Gaoxin Hospital, Xi'an, 710032, Shaanxi Province, China
| | - Guowei Liu
- Yi Xin Clinic, Changzhou, Jiangsu, China
| | - Chaijie Luo
- Renmin Hospital of Wuhan University, Wuhan, China
- Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
- Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
| | - Yanxia Li
- Renmin Hospital of Wuhan University, Wuhan, China
- Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
- Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
| | - Xiaoquan Zeng
- Renmin Hospital of Wuhan University, Wuhan, China
- Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
- Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
| | - Jun Liu
- Renmin Hospital of Wuhan University, Wuhan, China
- Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
- Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
| | - Yijie Zhu
- Renmin Hospital of Wuhan University, Wuhan, China
- Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
- Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
| | - Lianlian Wu
- Renmin Hospital of Wuhan University, Wuhan, China.
- Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China.
- Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.
- Department of Gastroenterology, Renmin Hospital of Wuhan University, 99 Zhangzhidong Road, Wuhan, 430060, Hubei Province, China.
| | - Honggang Yu
- Renmin Hospital of Wuhan University, Wuhan, China.
- Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China.
- Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China.
- Department of Gastroenterology, Renmin Hospital of Wuhan University, 99 Zhangzhidong Road, Wuhan, 430060, Hubei Province, China.
| |
Collapse
|
68
|
McElfresh DC, Chen L, Oliva E, Joyce V, Rose S, Tamang S. A call for better validation of opioid overdose risk algorithms. J Am Med Inform Assoc 2023; 30:1741-1746. [PMID: 37428897 PMCID: PMC10531142 DOI: 10.1093/jamia/ocad110] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2023] [Revised: 05/11/2023] [Accepted: 07/01/2023] [Indexed: 07/12/2023] Open
Abstract
Clinical decision support (CDS) systems powered by predictive models have the potential to improve the accuracy and efficiency of clinical decision-making. However, without sufficient validation, these systems have the potential to mislead clinicians and harm patients. This is especially true for CDS systems used by opioid prescribers and dispensers, where a flawed prediction can directly harm patients. To prevent these harms, regulators and researchers have proposed guidance for validating predictive models and CDS systems. However, this guidance is not universally followed and is not required by law. We call on CDS developers, deployers, and users to hold these systems to higher standards of clinical and technical validation. We provide a case study on two CDS systems deployed on a national scale in the United States for predicting a patient's risk of adverse opioid-related events: the Stratification Tool for Opioid Risk Mitigation (STORM), used by the Veterans Health Administration, and NarxCare, a commercial system.
Collapse
Affiliation(s)
- Duncan C McElfresh
- Department of Health Policy, Stanford University, Stanford, California, USA
- Program Evaluation Resource Center, Office of Mental Health and Suicide Prevention, US Department of Veterans Affairs, Menlo Park, California, USA
| | - Lucia Chen
- Department of Health Policy, Stanford University, Stanford, California, USA
| | - Elizabeth Oliva
- Program Evaluation Resource Center, Office of Mental Health and Suicide Prevention, US Department of Veterans Affairs, Menlo Park, California, USA
| | - Vilija Joyce
- Program Evaluation Resource Center, Office of Mental Health and Suicide Prevention, US Department of Veterans Affairs, Menlo Park, California, USA
- Health Economics Resource Center, US Department of Veterans Affairs, Menlo Park, California, USA
| | - Sherri Rose
- Department of Health Policy, Stanford University, Stanford, California, USA
| | - Suzanne Tamang
- Program Evaluation Resource Center, Office of Mental Health and Suicide Prevention, US Department of Veterans Affairs, Menlo Park, California, USA
- Department of Medicine, Stanford University, Stanford, California, USA
| |
Collapse
|
69
|
Kostekci YE, Bakırarar B, Okulu E, Erdeve O, Atasay B, Arsan S. An Early Prediction Model for Estimating Bronchopulmonary Dysplasia in Preterm Infants. Neonatology 2023; 120:709-717. [PMID: 37725910 DOI: 10.1159/000533299] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/13/2023] [Accepted: 07/22/2023] [Indexed: 09/21/2023]
Abstract
INTRODUCTION Accurate assessment of the risk for bronchopulmonary dysplasia (BPD) is critical to determine the prognosis and identify infants who will benefit from preventive therapies. Clinical prediction models can support the identification of high-risk patients. In this study, we investigated the potential risk factors for BPD and compared machine learning models for predicting the outcome of BPD/death on days 1, 7, 14, and 28 in preterm infants. We also developed a local BPD estimator. METHODS This study involved 124 infants. We evaluated the composite outcome of BPD/death at a postmenstrual age of 36 weeks and identified risk factors that would improve BPD/death prediction. SPSS for Windows Version 11.5 and Weka 3.9 software were used for the data analysis. RESULTS To evaluate the combined effect of all variables, all risk factors were taken into consideration. Gestational age, birth weight, mode of respiratory support, intraventricular hemorrhage, necrotizing enterocolitis, surfactant requirement, and late-onset sepsis were risk factors on postnatal days 7, 14, and 28. In a comparison of four different time points (postnatal days 1, 7, 14, and 28), the day 7 model provided the best prediction. According to this model, when a patient was diagnosed with BPD/death, the accuracy rate was 89.5%. CONCLUSION The postnatal day 7 model was the best predictor of BPD or death. Future validation studies will help identify infants who may benefit from preventive therapies and develop individualized care.
Collapse
Affiliation(s)
- Yasemin Ezgi Kostekci
- Division of Neonatology, Department of Pediatrics, Ankara University Faculty of Medicine, Ankara, Turkey
| | - Batuhan Bakırarar
- Department of Biostatistics, Ankara University Faculty of Medicine, Ankara, Turkey
| | - Emel Okulu
- Division of Neonatology, Department of Pediatrics, Ankara University Faculty of Medicine, Ankara, Turkey
| | - Omer Erdeve
- Division of Neonatology, Department of Pediatrics, Ankara University Faculty of Medicine, Ankara, Turkey
| | - Begum Atasay
- Division of Neonatology, Department of Pediatrics, Ankara University Faculty of Medicine, Ankara, Turkey
| | - Saadet Arsan
- Division of Neonatology, Department of Pediatrics, Ankara University Faculty of Medicine, Ankara, Turkey
| |
Collapse
|
70
|
Liu M, Ning Y, Teixayavong S, Mertens M, Xu J, Ting DSW, Cheng LTE, Ong JCL, Teo ZL, Tan TF, RaviChandran N, Wang F, Celi LA, Ong MEH, Liu N. A translational perspective towards clinical AI fairness. NPJ Digit Med 2023; 6:172. [PMID: 37709945 PMCID: PMC10502051 DOI: 10.1038/s41746-023-00918-4] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2023] [Accepted: 09/04/2023] [Indexed: 09/16/2023] Open
Abstract
Artificial intelligence (AI) has demonstrated the ability to extract insights from data, but the fairness of such data-driven insights remains a concern in high-stakes fields. Despite extensive developments, issues of AI fairness in clinical contexts have not been adequately addressed. A fair model is normally expected to perform equally across subgroups defined by sensitive variables (e.g., age, gender/sex, race/ethnicity, socio-economic status, etc.). Various fairness measurements have been developed to detect differences between subgroups as evidence of bias, and bias mitigation methods are designed to reduce the differences detected. This perspective of fairness, however, is misaligned with some key considerations in clinical contexts. The set of sensitive variables used in healthcare applications must be carefully examined for relevance and justified by clear clinical motivations. In addition, clinical AI fairness should closely investigate the ethical implications of fairness measurements (e.g., potential conflicts between group- and individual-level fairness) to select suitable and objective metrics. Generally defining AI fairness as "equality" is not necessarily reasonable in clinical settings, as differences may have clinical justifications and do not indicate biases. Instead, "equity" would be an appropriate objective of clinical AI fairness. Moreover, clinical feedback is essential to developing fair and well-performing AI models, and efforts should be made to actively involve clinicians in the process. The adaptation of AI fairness towards healthcare is not self-evident due to misalignments between technical developments and clinical considerations. Multidisciplinary collaboration between AI researchers, clinicians, and ethicists is necessary to bridge the gap and translate AI fairness into real-life benefits.
Collapse
Affiliation(s)
- Mingxuan Liu
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore
| | - Yilin Ning
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore
| | | | - Mayli Mertens
- Centre for Ethics, Department of Philosophy, University of Antwerp, Antwerp, Belgium
- Antwerp Center on Responsible AI, University of Antwerp, Antwerp, Belgium
| | - Jie Xu
- Department of Health Outcomes and Biomedical Informatics, University of Florida, Gainesville, FL, USA
| | - Daniel Shu Wei Ting
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore
- SingHealth AI Office, Singapore Health Services, Singapore, Singapore
| | - Lionel Tim-Ee Cheng
- Department of Diagnostic Radiology, Singapore General Hospital, Singapore, Singapore
| | | | - Zhen Ling Teo
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore
| | - Ting Fang Tan
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore
| | | | - Fei Wang
- Department of Population Health Sciences, Weill Cornell Medicine, New York, NY, USA
| | - Leo Anthony Celi
- Laboratory for Computational Physiology, Massachusetts Institute of Technology, Cambridge, MA, USA
- Division of Pulmonary, Critical Care and Sleep Medicine, Beth Israel Deaconess Medical Center, Boston, MA, USA
- Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - Marcus Eng Hock Ong
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore
- Department of Emergency Medicine, Singapore General Hospital, Singapore, Singapore
| | - Nan Liu
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore.
- SingHealth AI Office, Singapore Health Services, Singapore, Singapore.
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore.
- Institute of Data Science, National University of Singapore, Singapore, Singapore.
| |
Collapse
|
71
|
Buslón N, Cortés A, Catuara-Solarz S, Cirillo D, Rementeria MJ. Raising awareness of sex and gender bias in artificial intelligence and health. Front Glob Womens Health 2023; 4:970312. [PMID: 37746321 PMCID: PMC10512182 DOI: 10.3389/fgwh.2023.970312] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2022] [Accepted: 08/18/2023] [Indexed: 09/26/2023] Open
Abstract
Historically, biomedical research has been led by and focused on men. The recent introduction of Artificial Intelligence (AI) in this area has further proven this practice to be discriminatory for other sexes and genders, more noticeably for women. To move towards a fair AI development, it is essential to include sex and gender diversity both in research practices and in the workplace. In this context, the Bioinfo4women (B4W) program of the Barcelona Supercomputing Center (i) promotes the participation of women scientists by improving their visibility, (ii) fosters international collaborations between institutions and programs and (iii) advances research on sex and gender bias in AI and health. In this article, we discuss methodology and results of a series of conferences, titled “Sex and Gender Bias in Artificial Intelligence and Health, organized by B4W and La Caixa Foundation from March to June 2021 in Barcelona, Spain. The series consisted of nine hybrid events, composed of keynote sessions and seminars open to the general audience, and two working groups with invited experts from different professional backgrounds (academic fields such as biology, engineering, and sociology, as well as NGOs, journalists, lawyers, policymakers, industry). Based on this awareness-raising action, we distilled key recommendations to facilitate the inclusion of sex and gender perspective into public policies, educational programs, industry, and biomedical research, among other sectors, and help overcome sex and gender biases in AI and health.
Collapse
Affiliation(s)
- Nataly Buslón
- Life Sciences Department, Barcelona Supercomputing Center, Barcelona, Spain
| | - Atia Cortés
- Life Sciences Department, Barcelona Supercomputing Center, Barcelona, Spain
| | | | - Davide Cirillo
- Life Sciences Department, Barcelona Supercomputing Center, Barcelona, Spain
| | | |
Collapse
|
72
|
Spencer KL, Absolom KL, Allsop MJ, Relton SD, Pearce J, Liao K, Naseer S, Salako O, Howdon D, Hewison J, Velikova G, Faivre-Finn C, Bekker HL, van der Veer SN. Fixing the Leaky Pipe: How to Improve the Uptake of Patient-Reported Outcomes-Based Prognostic and Predictive Models in Cancer Clinical Practice. JCO Clin Cancer Inform 2023; 7:e2300070. [PMID: 37976441 PMCID: PMC10681558 DOI: 10.1200/cci.23.00070] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2023] [Revised: 09/08/2023] [Accepted: 09/29/2023] [Indexed: 11/19/2023] Open
Abstract
PURPOSE This discussion paper outlines challenges and proposes solutions for successfully implementing prediction models that incorporate patient-reported outcomes (PROs) in cancer practice. METHODS We organized a full-day multidisciplinary meeting of people with expertise in cancer care delivery, PRO collection, PRO use in prediction modeling, computing, implementation, and decision science. The discussions presented here focused on identifying challenges to the development, implementation and use of prediction models incorporating PROs, and suggesting possible solutions. RESULTS Specific challenges and solutions were identified across three broad areas. (1) Understanding decision making and implementation: necessitating multidisciplinary collaboration in the early stages and throughout; early stakeholder engagement to define the decision problem and ensure acceptability of PROs in prediction; understanding patient/clinician interpretation of PRO predictions and uncertainty to optimize prediction impact; striving for model integration into existing electronic health records; and early regulatory alignment. (2) Recognizing the limitations to PRO collection and their impact on prediction: incorporating validated, clinically important PROs to maximize model generalizability and clinical engagement; and minimizing missing PRO data (resulting from both structural digital exclusion and time-varying factors) to avoid exacerbating existing inequalities. (3) Statistical and modeling challenges: incorporating statistical methods to address missing data; ensuring predictive modeling recognizes complex causal relationships; and considering temporal and geographic recalibration so that model predictions reflect the relevant population. CONCLUSION Developing and implementing PRO-based prediction models in cancer care requires extensive multidisciplinary working from the earliest stages, recognition of implementation challenges because of PRO collection and model presentation, and robust statistical methods to manage missing data, causality, and calibration. Prediction models incorporating PROs should be viewed as complex interventions, with their development and impact assessment carried out to reflect this.
Collapse
Affiliation(s)
- Katie L. Spencer
- Leeds Institute of Health Sciences, University of Leeds, Leeds, United Kingdom
- Leeds Cancer Centre, Leeds Teaching Hospitals NHS Trust, Leeds, United Kingdom
| | - Kate L. Absolom
- Leeds Institute of Health Sciences, University of Leeds, Leeds, United Kingdom
| | - Matthew J. Allsop
- Leeds Institute of Health Sciences, University of Leeds, Leeds, United Kingdom
| | - Samuel D. Relton
- Leeds Institute of Data Analytics, University of Leeds, Leeds, United Kingdom
| | - Jessica Pearce
- Leeds Cancer Centre, Leeds Teaching Hospitals NHS Trust, Leeds, United Kingdom
- Leeds Institute of Medical Research, University of Leeds, Leeds, United Kingdom
| | - Kuan Liao
- Division of Informatics, Imaging and Data Sciences, Faculty of Biology, Medicine and Health, Centre for Health Informatics, Manchester Academic Health Science Centre, The University of Manchester, Manchester, United Kingdom
| | - Sairah Naseer
- School of Medicine, University of Leeds, Leeds, United Kingdom
| | - Omolola Salako
- College of Medicine, University of Lagos, Lagos, Nigeria
| | - Daniel Howdon
- Leeds Institute of Health Sciences, University of Leeds, Leeds, United Kingdom
| | - Jenny Hewison
- Leeds Institute of Health Sciences, University of Leeds, Leeds, United Kingdom
| | - Galina Velikova
- Leeds Cancer Centre, Leeds Teaching Hospitals NHS Trust, Leeds, United Kingdom
- Leeds Institute of Medical Research, University of Leeds, Leeds, United Kingdom
| | - Corinne Faivre-Finn
- Institute of Cancer Sciences, University of Manchester, Manchester, United Kingdom
| | - Hilary L. Bekker
- Leeds Institute of Health Sciences, University of Leeds, Leeds, United Kingdom
| | - Sabine N. van der Veer
- Division of Informatics, Imaging and Data Sciences, Faculty of Biology, Medicine and Health, Centre for Health Informatics, Manchester Academic Health Science Centre, The University of Manchester, Manchester, United Kingdom
| |
Collapse
|
73
|
Liu J, Capurro D, Nguyen A, Verspoor K. Attention-based multimodal fusion with contrast for robust clinical prediction in the face of missing modalities. J Biomed Inform 2023; 145:104466. [PMID: 37549722 DOI: 10.1016/j.jbi.2023.104466] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2023] [Revised: 06/09/2023] [Accepted: 08/01/2023] [Indexed: 08/09/2023]
Abstract
OBJECTIVE With the increasing amount and growing variety of healthcare data, multimodal machine learning supporting integrated modeling of structured and unstructured data is an increasingly important tool for clinical machine learning tasks. However, it is non-trivial to manage the differences in dimensionality, volume, and temporal characteristics of data modalities in the context of a shared target task. Furthermore, patients can have substantial variations in the availability of data, while existing multimodal modeling methods typically assume data completeness and lack a mechanism to handle missing modalities. METHODS We propose a Transformer-based fusion model with modality-specific tokens that summarize the corresponding modalities to achieve effective cross-modal interaction accommodating missing modalities in the clinical context. The model is further refined by inter-modal, inter-sample contrastive learning to improve the representations for better predictive performance. We denote the model as Attention-based cRoss-MOdal fUsion with contRast (ARMOUR). We evaluate ARMOUR using two input modalities (structured measurements and unstructured text), six clinical prediction tasks, and two evaluation regimes, either including or excluding samples with missing modalities. RESULTS Our model shows improved performances over unimodal or multimodal baselines in both evaluation regimes, including or excluding patients with missing modalities in the input. The contrastive learning improves the representation power and is shown to be essential for better results. The simple setup of modality-specific tokens enables ARMOUR to handle patients with missing modalities and allows comparison with existing unimodal benchmark results. CONCLUSION We propose a multimodal model for robust clinical prediction to achieve improved performance while accommodating patients with missing modalities. This work could inspire future research to study the effective incorporation of multiple, more complex modalities of clinical data into a single model.
Collapse
Affiliation(s)
- Jinghui Liu
- Australian e-Health Research Centre, CSIRO, Queensland, Australia; School of Computing and Information Systems, The University of Melbourne, Victoria, Australia
| | - Daniel Capurro
- School of Computing and Information Systems, The University of Melbourne, Victoria, Australia; Centre for Digital Transformation of Health, The University of Melbourne, Victoria, Australia
| | - Anthony Nguyen
- Australian e-Health Research Centre, CSIRO, Queensland, Australia
| | - Karin Verspoor
- School of Computing and Information Systems, The University of Melbourne, Victoria, Australia; School of Computing Technologies, RMIT University, Victoria, Australia.
| |
Collapse
|
74
|
Trischitta V, Mastroianno M, Scarale MG, Prehn C, Salvemini L, Fontana A, Adamski J, Schena FP, Cosmo SD, Copetti M, Menzaghi C. Circulating metabolites improve the prediction of renal impairment in patients with type 2 diabetes. BMJ Open Diabetes Res Care 2023; 11:e003422. [PMID: 37734903 PMCID: PMC10514631 DOI: 10.1136/bmjdrc-2023-003422] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Accepted: 08/29/2023] [Indexed: 09/23/2023] Open
Abstract
INTRODUCTION Low glomerular filtration rate (GFR) is a leading cause of reduced lifespan in type 2 diabetes. Unravelling biomarkers capable to identify high-risk patients can help tackle this burden. We investigated the association between 188 serum metabolites and kidney function in type 2 diabetes and then whether the associated metabolites improve two established clinical models for predicting GFR decline in these patients. RESEARCH DESIGN AND METHODS Two cohorts comprising 849 individuals with type 2 diabetes (discovery and validation samples) and a follow-up study of 575 patients with estimated GFR (eGFR) decline were analyzed. RESULTS Ten metabolites were independently associated with low eGFR in the discovery sample, with nine of them being confirmed also in the validation sample (ORs range 1.3-2.4 per 1SD, p values range 1.9×10-2-2.5×10-9). Of these, five metabolites were also associated with eGFR decline (ie, tiglylcarnitine, decadienylcarnitine, total dimethylarginine, decenoylcarnitine and kynurenine) (β range -0.11 to -0.19, p values range 4.8×10-2 to 3.0×10-3). Indeed, tiglylcarnitine and kynurenine, which captured all the information of the other three markers, improved discrimination and reclassification (all p<0.01) of two clinical prediction models of GFR decline in people with diabetes. CONCLUSIONS Further studies are needed to validate our findings in larger cohorts of different clinical, environmental and genetic background.
Collapse
Affiliation(s)
- Vincenzo Trischitta
- Research Unit of Diabetes and Endocrine Diseases, Fondazione IRCCS Casa Sollievo della Sofferenza, San Giovanni Rotondo, Italy
- Experimental Medicine, University of Rome La Sapienza, Rome, Italy
| | - Mario Mastroianno
- Scientific Direction, Fondazione IRCCS "Casa Sollievo della Sofferenza", San Giovanni Rotondo, Italy
| | - Maria Giovanna Scarale
- Research Unit of Diabetes and Endocrine Diseases, Fondazione IRCCS Casa Sollievo della Sofferenza, San Giovanni Rotondo, Italy
| | - Cornelia Prehn
- Institute of Experimental Genetics, Helmholtz Zentrum München, Neuherberg, Germany
| | - Lucia Salvemini
- Research Unit of Diabetes and Endocrine Diseases, Fondazione IRCCS Casa Sollievo della Sofferenza, San Giovanni Rotondo, Italy
| | - Andrea Fontana
- Unit of Biostatistics, Fondazione IRCCS Casa Sollievo della Sofferenza, San Giovanni Rotondo, Italy
| | - Jerzy Adamski
- Institute of Experimental Genetics, Helmholtz Zentrum München, Neuherberg, Germany
- Department of Biochemistry, National University Singapore Yong Loo Lin School of Medicine, Singapore
| | | | - Salvatore De Cosmo
- Unit of Internal Medicine, IRCCS Casa Sollievo della Sofferenza San Giovanni Rotondo, Foggia, Italy
| | - Massimiliano Copetti
- Unit of Biostatistics, Fondazione IRCCS Casa Sollievo della Sofferenza, San Giovanni Rotondo, Italy
| | - Claudia Menzaghi
- Research Unit of Diabetes and Endocrine Diseases, Istituti di Ricovero e Cura a Carattere Scientifico Ospedale Casa Sollievo della Sofferenza, San Giovanni Rotondo, Italy
| |
Collapse
|
75
|
Yuan T, Edelmann D, Fan Z, Alwers E, Kather JN, Brenner H, Hoffmeister M. Machine learning in the identification of prognostic DNA methylation biomarkers among patients with cancer: A systematic review of epigenome-wide studies. Artif Intell Med 2023; 143:102589. [PMID: 37673571 DOI: 10.1016/j.artmed.2023.102589] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2022] [Revised: 04/19/2023] [Accepted: 04/30/2023] [Indexed: 09/08/2023]
Abstract
BACKGROUND DNA methylation biomarkers have great potential in improving prognostic classification systems for patients with cancer. Machine learning (ML)-based analytic techniques might help overcome the challenges of analyzing high-dimensional data in relatively small sample sizes. This systematic review summarizes the current use of ML-based methods in epigenome-wide studies for the identification of DNA methylation signatures associated with cancer prognosis. METHODS We searched three electronic databases including PubMed, EMBASE, and Web of Science for articles published until 2 January 2023. ML-based methods and workflows used to identify DNA methylation signatures associated with cancer prognosis were extracted and summarized. Two authors independently assessed the methodological quality of included studies by a seven-item checklist adapted from 'A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies (PROBAST)' and from the 'Reporting Recommendations for Tumor Marker Prognostic Studies (REMARK). Different ML methods and workflows used in included studies were summarized and visualized by a sunburst chart, a bubble chart, and Sankey diagrams, respectively. RESULTS Eighty-three studies were included in this review. Three major types of ML-based workflows were identified. 1) unsupervised clustering, 2) supervised feature selection, and 3) deep learning-based feature transformation. For the three workflows, the most frequently used ML techniques were consensus clustering, least absolute shrinkage and selection operator (LASSO), and autoencoder, respectively. The systematic review revealed that the performance of these approaches has not been adequately evaluated yet and that methodological and reporting flaws were common in the identified studies using ML techniques. CONCLUSIONS There is great heterogeneity in ML-based methodological strategies used by epigenome-wide studies to identify DNA methylation markers associated with cancer prognosis. In theory, most existing workflows could not handle the high multi-collinearity and potentially non-linearity interactions in epigenome-wide DNA methylation data. Benchmarking studies are needed to compare the relative performance of various approaches for specific cancer types. Adherence to relevant methodological and reporting guidelines are urgently needed.
Collapse
Affiliation(s)
- Tanwei Yuan
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany; Medical Faculty Heidelberg, Heidelberg University, Heidelberg, Germany
| | - Dominic Edelmann
- Division of Biostatistics, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Ziwen Fan
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Elizabeth Alwers
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Jakob Nikolas Kather
- Else Kroener Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany; Medical Oncology, National Center of Tumour Diseases (NCT), University Hospital Heidelberg, Heidelberg, Germany
| | - Hermann Brenner
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany; Division of Preventive Oncology, German Cancer Research Center (DKFZ) and National Center for Tumor Diseases (NCT), Heidelberg, Germany; German Cancer Consortium (DKTK), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Michael Hoffmeister
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany.
| |
Collapse
|
76
|
Wilson S, Tolley C, Mc Ardle R, Beswick E, Slight SP. Key Considerations When Developing and Implementing Digital Technology for Early Detection of Dementia-Causing Diseases Among Health Care Professionals: Qualitative Study. J Med Internet Res 2023; 25:e46711. [PMID: 37606986 PMCID: PMC10481214 DOI: 10.2196/46711] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2023] [Revised: 05/30/2023] [Accepted: 06/15/2023] [Indexed: 08/23/2023] Open
Abstract
BACKGROUND The World Health Organization (WHO) promotes using digital technologies to accelerate global attainment of health and well-being. This has led to a growth in research exploring the use of digital technology to aid early detection and preventative interventions for dementia-causing diseases such as Alzheimer disease. The opinions and perspectives of health care professionals must be incorporated into the development and implementation of technology to promote its successful adoption in clinical practice. OBJECTIVE This study aimed to explore health care professionals' perspectives on the key considerations of developing and implementing digital technologies for the early detection of dementia-causing diseases in the National Health Service (NHS). METHODS Health care professionals with patient-facing roles in primary or secondary care settings in the NHS were recruited through various web-based NHS clinical networks. Participants were interviewed to explore their experiences of the current dementia diagnostic practices, views on early detection and use of digital technology to aid these practices, and the challenges of implementing such interventions in health care. An inductive thematic analysis approach was applied to identify central concepts and themes in the interviews, allowing the data to determine our themes. A list of central concepts and themes was applied systematically to the whole data set using NVivo (version 1.6.1; QSR International). Using the constant comparison technique, the researchers moved backward and forward between these data and evolving explanations until a fit was made. RESULTS Eighteen semistructured interviews were conducted, with 11 primary and 7 secondary care health care professionals. We identified 3 main categories of considerations relevant to health care service users, health care professionals, and the digital health technology itself. Health care professionals recognized the potential of using digital technology to collect real-time data and the possible benefits of detecting dementia-causing diseases earlier if an effective intervention were available. However, some were concerned about postdetection management, questioning the point of an early detection of dementia-causing diseases if an effective intervention cannot be provided and feared this would only lead to increased anxiety in patients. Health care professionals also expressed mixed opinions on who should be screened for early detection. Some suggested it should be available to everyone to mitigate the chance of excluding those who are not in touch with their health care or are digitally excluded. Others were concerned about the resources that would be required to make the technology available to everyone. CONCLUSIONS This study highlights the need to design digital health technology in a way that is accessible to all and does not add burden to health care professionals. Further work is needed to ensure inclusive strategies are used in digital research to promote health equity.
Collapse
Affiliation(s)
- Sarah Wilson
- School of Pharmacy, Newcastle University, Newcastle upon Tyne, United Kingdom
| | - Clare Tolley
- School of Pharmacy, Newcastle University, Newcastle upon Tyne, United Kingdom
| | - Riona Mc Ardle
- Translational and Clinical Research Institute, Newcastle University, Newcastle upon Tyne, United Kingdom
| | - Emily Beswick
- School of Pharmacy, Newcastle University, Newcastle upon Tyne, United Kingdom
| | - Sarah P Slight
- School of Pharmacy, Newcastle University, Newcastle upon Tyne, United Kingdom
| |
Collapse
|
77
|
Hurvitz N, Ilan Y. The Constrained-Disorder Principle Assists in Overcoming Significant Challenges in Digital Health: Moving from "Nice to Have" to Mandatory Systems. Clin Pract 2023; 13:994-1014. [PMID: 37623270 PMCID: PMC10453547 DOI: 10.3390/clinpract13040089] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2023] [Revised: 08/16/2023] [Accepted: 08/18/2023] [Indexed: 08/26/2023] Open
Abstract
The success of artificial intelligence depends on whether it can penetrate the boundaries of evidence-based medicine, the lack of policies, and the resistance of medical professionals to its use. The failure of digital health to meet expectations requires rethinking some of the challenges faced. We discuss some of the most significant challenges faced by patients, physicians, payers, pharmaceutical companies, and health systems in the digital world. The goal of healthcare systems is to improve outcomes. Assisting in diagnosing, collecting data, and simplifying processes is a "nice to have" tool, but it is not essential. Many of these systems have yet to be shown to improve outcomes. Current outcome-based expectations and economic constraints make "nice to have," "assists," and "ease processes" insufficient. Complex biological systems are defined by their inherent disorder, bounded by dynamic boundaries, as described by the constrained disorder principle (CDP). It provides a platform for correcting systems' malfunctions by regulating their degree of variability. A CDP-based second-generation artificial intelligence system provides solutions to some challenges digital health faces. Therapeutic interventions are held to improve outcomes with these systems. In addition to improving clinically meaningful endpoints, CDP-based second-generation algorithms ensure patient and physician engagement and reduce the health system's costs.
Collapse
Affiliation(s)
| | - Yaron Ilan
- Hadassah Medical Center, Department of Medicine, Faculty of Medicine, Hebrew University, POB 1200, Jerusalem IL91120, Israel;
| |
Collapse
|
78
|
van der Vegt AH, Scott IA, Dermawan K, Schnetler RJ, Kalke VR, Lane PJ. Implementation frameworks for end-to-end clinical AI: derivation of the SALIENT framework. J Am Med Inform Assoc 2023; 30:1503-1515. [PMID: 37208863 PMCID: PMC10436156 DOI: 10.1093/jamia/ocad088] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Revised: 04/17/2023] [Accepted: 05/09/2023] [Indexed: 05/21/2023] Open
Abstract
OBJECTIVE To derive a comprehensive implementation framework for clinical AI models within hospitals informed by existing AI frameworks and integrated with reporting standards for clinical AI research. MATERIALS AND METHODS (1) Derive a provisional implementation framework based on the taxonomy of Stead et al and integrated with current reporting standards for AI research: TRIPOD, DECIDE-AI, CONSORT-AI. (2) Undertake a scoping review of published clinical AI implementation frameworks and identify key themes and stages. (3) Perform a gap analysis and refine the framework by incorporating missing items. RESULTS The provisional AI implementation framework, called SALIENT, was mapped to 5 stages common to both the taxonomy and the reporting standards. A scoping review retrieved 20 studies and 247 themes, stages, and subelements were identified. A gap analysis identified 5 new cross-stage themes and 16 new tasks. The final framework comprised 5 stages, 7 elements, and 4 components, including the AI system, data pipeline, human-computer interface, and clinical workflow. DISCUSSION This pragmatic framework resolves gaps in existing stage- and theme-based clinical AI implementation guidance by comprehensively addressing the what (components), when (stages), and how (tasks) of AI implementation, as well as the who (organization) and why (policy domains). By integrating research reporting standards into SALIENT, the framework is grounded in rigorous evaluation methodologies. The framework requires validation as being applicable to real-world studies of deployed AI models. CONCLUSIONS A novel end-to-end framework has been developed for implementing AI within hospital clinical practice that builds on previous AI implementation frameworks and research reporting standards.
Collapse
Affiliation(s)
- Anton H van der Vegt
- Centre for Health Services Research, The University of Queensland, Brisbane, Australia
| | - Ian A Scott
- Department of Internal Medicine and Clinical Epidemiology, Princess Alexandra Hospital, Brisbane, Australia
| | - Krishna Dermawan
- Centre for Information Resilience, The University of Queensland, St Lucia, Australia
| | - Rudolf J Schnetler
- School of Information Technology and Electrical Engineering, The University of Queensland, St Lucia, Australia
| | - Vikrant R Kalke
- Patient Safety and Quality, Clinical Excellence Queensland, Queensland Health, Brisbane, Australia
| | - Paul J Lane
- Safety Quality & Innovation, The Prince Charles Hospital, Queensland Health, Brisbane, Australia
| |
Collapse
|
79
|
Salih M, Austin C, Warty RR, Tiktin C, Rolnik DL, Momeni M, Rezatofighi H, Reddy S, Smith V, Vollenhoven B, Horta F. Embryo selection through artificial intelligence versus embryologists: a systematic review. Hum Reprod Open 2023; 2023:hoad031. [PMID: 37588797 PMCID: PMC10426717 DOI: 10.1093/hropen/hoad031] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2023] [Revised: 07/17/2023] [Indexed: 08/18/2023] Open
Abstract
STUDY QUESTION What is the present performance of artificial intelligence (AI) decision support during embryo selection compared to the standard embryo selection by embryologists? SUMMARY ANSWER AI consistently outperformed the clinical teams in all the studies focused on embryo morphology and clinical outcome prediction during embryo selection assessment. WHAT IS KNOWN ALREADY The ART success rate is ∼30%, with a worrying trend of increasing female age correlating with considerably worse results. As such, there have been ongoing efforts to address this low success rate through the development of new technologies. With the advent of AI, there is potential for machine learning to be applied in such a manner that areas limited by human subjectivity, such as embryo selection, can be enhanced through increased objectivity. Given the potential of AI to improve IVF success rates, it remains crucial to review the performance between AI and embryologists during embryo selection. STUDY DESIGN SIZE DURATION The search was done across PubMed, EMBASE, Ovid Medline, and IEEE Xplore from 1 June 2005 up to and including 7 January 2022. Included articles were also restricted to those written in English. Search terms utilized across all databases for the study were: ('Artificial intelligence' OR 'Machine Learning' OR 'Deep learning' OR 'Neural network') AND ('IVF' OR 'in vitro fertili*' OR 'assisted reproductive techn*' OR 'embryo'), where the character '*' refers the search engine to include any auto completion of the search term. PARTICIPANTS/MATERIALS SETTING METHODS A literature search was conducted for literature relating to AI applications to IVF. Primary outcomes of interest were accuracy, sensitivity, and specificity of the embryo morphology grade assessments and the likelihood of clinical outcomes, such as clinical pregnancy after IVF treatments. Risk of bias was assessed using the Modified Down and Black Checklist. MAIN RESULTS AND THE ROLE OF CHANCE Twenty articles were included in this review. There was no specific embryo assessment day across the studies-Day 1 until Day 5/6 of embryo development was investigated. The types of input for training AI algorithms were images and time-lapse (10/20), clinical information (6/20), and both images and clinical information (4/20). Each AI model demonstrated promise when compared to an embryologist's visual assessment. On average, the models predicted the likelihood of successful clinical pregnancy with greater accuracy than clinical embryologists, signifying greater reliability when compared to human prediction. The AI models performed at a median accuracy of 75.5% (range 59-94%) on predicting embryo morphology grade. The correct prediction (Ground Truth) was defined through the use of embryo images according to post embryologists' assessment following local respective guidelines. Using blind test datasets, the embryologists' accuracy prediction was 65.4% (range 47-75%) with the same ground truth provided by the original local respective assessment. Similarly, AI models had a median accuracy of 77.8% (range 68-90%) in predicting clinical pregnancy through the use of patient clinical treatment information compared to 64% (range 58-76%) when performed by embryologists. When both images/time-lapse and clinical information inputs were combined, the median accuracy by the AI models was higher at 81.5% (range 67-98%), while clinical embryologists had a median accuracy of 51% (range 43-59%). LIMITATIONS REASONS FOR CAUTION The findings of this review are based on studies that have not been prospectively evaluated in a clinical setting. Additionally, a fair comparison of all the studies were deemed unfeasible owing to the heterogeneity of the studies, development of the AI models, database employed and the study design and quality. WIDER IMPLICATIONS OF THE FINDINGS AI provides considerable promise to the IVF field and embryo selection. However, there needs to be a shift in developers' perception of the clinical outcome from successful implantation towards ongoing pregnancy or live birth. Additionally, existing models focus on locally generated databases and many lack external validation. STUDY FUNDING/COMPETING INTERESTS This study was funded by Monash Data Future Institute. All authors have no conflicts of interest to declare. REGISTRATION NUMBER CRD42021256333.
Collapse
Affiliation(s)
- M Salih
- Department of Obstetrics and Gynaecology, Monash University, Clayton, Victoria, Australia
- Department of Data Science and Artificial Intelligence, Faculty of Information Technology, Monash University, Clayton, Victoria, Australia
| | - C Austin
- Department of Obstetrics and Gynaecology, Monash University, Clayton, Victoria, Australia
- Department of Data Science and Artificial Intelligence, Faculty of Information Technology, Monash University, Clayton, Victoria, Australia
| | - R R Warty
- Department of Obstetrics and Gynaecology, Monash University, Clayton, Victoria, Australia
| | - C Tiktin
- School of Engineering, RMIT University, Melbourne, Victoria, Australia
| | - D L Rolnik
- Department of Obstetrics and Gynaecology, Monash University, Clayton, Victoria, Australia
- Women’s and Newborn Program, Monash Health, Melbourne, Victoria, Australia
| | - M Momeni
- Department of Obstetrics and Gynaecology, Monash University, Clayton, Victoria, Australia
| | - H Rezatofighi
- Department of Data Science and Artificial Intelligence, Faculty of Information Technology, Monash University, Clayton, Victoria, Australia
- Monash Data Future Institute, Monash University, Clayton, Victoria, Australia
| | - S Reddy
- School of Medicine, Deakin University, Geelong, Victoria, Australia
| | - V Smith
- Department of Obstetrics and Gynaecology, Monash University, Clayton, Victoria, Australia
| | - B Vollenhoven
- Department of Obstetrics and Gynaecology, Monash University, Clayton, Victoria, Australia
- Women’s and Newborn Program, Monash Health, Melbourne, Victoria, Australia
- Monash IVF, Melbourne, Victoria, Australia
| | - F Horta
- Department of Obstetrics and Gynaecology, Monash University, Clayton, Victoria, Australia
- Monash Data Future Institute, Monash University, Clayton, Victoria, Australia
- City Fertility, Melbourne, Victoria, Australia
| |
Collapse
|
80
|
Chalkidis G, McPherson JP, Beck A, Newman MG, Guo JW, Sloss EA, Staes CJ. External Validation of a Machine Learning Model to Predict 6-Month Mortality for Patients With Advanced Solid Tumors. JAMA Netw Open 2023; 6:e2327193. [PMID: 37535359 PMCID: PMC10401299 DOI: 10.1001/jamanetworkopen.2023.27193] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/05/2023] [Accepted: 06/23/2023] [Indexed: 08/04/2023] Open
Abstract
This prognostic study performed external validation of a machine learning model to predict 6-month mortality among patients with advanced solid tumors.
Collapse
Affiliation(s)
| | - Jordan P McPherson
- Huntsman Cancer Institute, Salt Lake City, Utah
- College of Pharmacy, University of Utah, Salt Lake City
| | - Anna Beck
- Huntsman Cancer Institute, Salt Lake City, Utah
| | | | - Jia-Wen Guo
- College of Nursing, University of Utah, Salt Lake City
- Department of Biomedical Informatics, University of Utah, Salt Lake City
| | | | - Catherine J Staes
- College of Nursing, University of Utah, Salt Lake City
- Department of Biomedical Informatics, University of Utah, Salt Lake City
| |
Collapse
|
81
|
van der Meijden S, Arbous M, Geerts B. Possibilities and challenges for artificial intelligence and machine learning in perioperative care. BJA Educ 2023; 23:288-294. [PMID: 37465235 PMCID: PMC10350557 DOI: 10.1016/j.bjae.2023.04.003] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/21/2023] [Indexed: 07/20/2023] Open
Affiliation(s)
- S.L. van der Meijden
- Healthplus.ai-R&D B.V., Amsterdam, The Netherlands
- Intensive Care Unit, Leiden University Medical Centre, Leiden, The Netherlands
| | - M.S. Arbous
- Intensive Care Unit, Leiden University Medical Centre, Leiden, The Netherlands
| | - B.F. Geerts
- Healthplus.ai-R&D B.V., Amsterdam, The Netherlands
| |
Collapse
|
82
|
Hassan N, Slight R, Morgan G, Bates DW, Gallier S, Sapey E, Slight S. Road map for clinicians to develop and evaluate AI predictive models to inform clinical decision-making. BMJ Health Care Inform 2023; 30:e100784. [PMID: 37558245 PMCID: PMC10414079 DOI: 10.1136/bmjhci-2023-100784] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2023] [Accepted: 07/24/2023] [Indexed: 08/11/2023] Open
Abstract
BACKGROUND Predictive models have been used in clinical care for decades. They can determine the risk of a patient developing a particular condition or complication and inform the shared decision-making process. Developing artificial intelligence (AI) predictive models for use in clinical practice is challenging; even if they have good predictive performance, this does not guarantee that they will be used or enhance decision-making. We describe nine stages of developing and evaluating a predictive AI model, recognising the challenges that clinicians might face at each stage and providing practical tips to help manage them. FINDINGS The nine stages included clarifying the clinical question or outcome(s) of interest (output), identifying appropriate predictors (features selection), choosing relevant datasets, developing the AI predictive model, validating and testing the developed model, presenting and interpreting the model prediction(s), licensing and maintaining the AI predictive model and evaluating the impact of the AI predictive model. The introduction of an AI prediction model into clinical practice usually consists of multiple interacting components, including the accuracy of the model predictions, physician and patient understanding and use of these probabilities, expected effectiveness of subsequent actions or interventions and adherence to these. Much of the difference in whether benefits are realised relates to whether the predictions are given to clinicians in a timely way that enables them to take an appropriate action. CONCLUSION The downstream effects on processes and outcomes of AI prediction models vary widely, and it is essential to evaluate the use in clinical practice using an appropriate study design.
Collapse
Affiliation(s)
- Nehal Hassan
- School of Pharmacy, Newcastle University School of Pharmacy, Newcastle Upon Tyne, UK
- Faculty of Medical Sciences, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
| | - Robert Slight
- Faculty of Medical Sciences, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
- Freeman Hospital, Newcastle Upon Tyne Hospitals NHS Foundation Trust, Newcastle Upon Tyne, UK
| | - Graham Morgan
- School of Computing, Newcastle University, Newcastle upon Tyne, UK
| | - David W Bates
- Department of General Internal Medicine, Harvard Medical School, Boston, Massachusetts, USA
| | - Suzy Gallier
- PIONEER Health Data Research Hub, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- Department of Health Informatics, PIONEER Health Data Research Hub, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | - Elizabeth Sapey
- PIONEER Health Data Research Hub, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- Department of Health Informatics, PIONEER Health Data Research Hub, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | - Sarah Slight
- School of Pharmacy, Newcastle University School of Pharmacy, Newcastle Upon Tyne, UK
- Faculty of Medical Sciences, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
| |
Collapse
|
83
|
Mertens E, Barrenechea-Pulache A, Sagastume D, Vasquez MS, Vandevijvere S, Peñalvo JL. Understanding the contribution of lifestyle in breast cancer risk prediction: a systematic review of models applicable to Europe. BMC Cancer 2023; 23:687. [PMID: 37480028 PMCID: PMC10360320 DOI: 10.1186/s12885-023-11174-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2023] [Accepted: 07/12/2023] [Indexed: 07/23/2023] Open
Abstract
BACKGROUND Breast cancer (BC) is a significant health concern among European women, with the highest prevalence rates among all cancers. Existing BC prediction models account for major risks such as hereditary, hormonal and reproductive factors, but research suggests that adherence to a healthy lifestyle can reduce the risk of developing BC to some extent. Understanding the influence and predictive role of lifestyle variables in current risk prediction models could help identify actionable, modifiable, targets among high-risk population groups. PURPOSE To systematically review population-based BC risk prediction models applicable to European populations and identify lifestyle predictors and their corresponding parameter values for a better understanding of their relative contribution to the prediction of incident BC. METHODS A systematic review was conducted in PubMed, Embase and Web of Science from January 2000 to August 2021. Risk prediction models were included if (i) developed and/or validated in adult cancer-free women in Europe, (ii) based on easily ascertained information, and (iii) reported models' final predictors. To investigate further the comparability of lifestyle predictors across models, estimates were standardised into risk ratios and visualised using forest plots. RESULTS From a total of 49 studies, 33 models were developed and 22 different existing models, mostly from Gail (22 studies) and Tyrer-Cuzick and co-workers (12 studies) were validated or modified for European populations. Family history of BC was the most frequently included predictor (31 models), while body mass index (BMI) and alcohol consumption (26 and 21 models, respectively) were the lifestyle predictors most often included, followed by smoking and physical activity (7 and 6 models respectively). Overall, for lifestyle predictors, their modest predictive contribution was greater for riskier lifestyle levels, though highly variable model estimates across different models. CONCLUSIONS Given the increasing BC incidence rates in Europe, risk models utilising readily available risk factors could greatly aid in widening the population coverage of screening efforts, while the addition of lifestyle factors could help improving model performance and serve as intervention targets of prevention programmes.
Collapse
Affiliation(s)
- Elly Mertens
- Unit of Non-Communicable Diseases, Department of Public Health, Institute of Tropical Medicine, Nationalestraat 155, 2000, Antwerp, Belgium.
| | - Antonio Barrenechea-Pulache
- Unit of Non-Communicable Diseases, Department of Public Health, Institute of Tropical Medicine, Nationalestraat 155, 2000, Antwerp, Belgium
| | - Diana Sagastume
- Unit of Non-Communicable Diseases, Department of Public Health, Institute of Tropical Medicine, Nationalestraat 155, 2000, Antwerp, Belgium
| | - Maria Salve Vasquez
- Health Information, Scientific Institute of Public Health (Sciensano), Brussels, Belgium
| | - Stefanie Vandevijvere
- Health Information, Scientific Institute of Public Health (Sciensano), Brussels, Belgium
| | - José L Peñalvo
- Unit of Non-Communicable Diseases, Department of Public Health, Institute of Tropical Medicine, Nationalestraat 155, 2000, Antwerp, Belgium
- Global Health Institute, University of Antwerp, Antwerp, Belgium
| |
Collapse
|
84
|
Banda JM, Shah NH, Periyakoil VS. Characterizing subgroup performance of probabilistic phenotype algorithms within older adults: a case study for dementia, mild cognitive impairment, and Alzheimer's and Parkinson's diseases. JAMIA Open 2023; 6:ooad043. [PMID: 37397506 PMCID: PMC10307941 DOI: 10.1093/jamiaopen/ooad043] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2023] [Revised: 06/06/2023] [Accepted: 06/22/2023] [Indexed: 07/04/2023] Open
Abstract
Objective Biases within probabilistic electronic phenotyping algorithms are largely unexplored. In this work, we characterize differences in subgroup performance of phenotyping algorithms for Alzheimer's disease and related dementias (ADRD) in older adults. Materials and methods We created an experimental framework to characterize the performance of probabilistic phenotyping algorithms under different racial distributions allowing us to identify which algorithms may have differential performance, by how much, and under what conditions. We relied on rule-based phenotype definitions as reference to evaluate probabilistic phenotype algorithms created using the Automated PHenotype Routine for Observational Definition, Identification, Training and Evaluation framework. Results We demonstrate that some algorithms have performance variations anywhere from 3% to 30% for different populations, even when not using race as an input variable. We show that while performance differences in subgroups are not present for all phenotypes, they do affect some phenotypes and groups more disproportionately than others. Discussion Our analysis establishes the need for a robust evaluation framework for subgroup differences. The underlying patient populations for the algorithms showing subgroup performance differences have great variance between model features when compared with the phenotypes with little to no differences. Conclusion We have created a framework to identify systematic differences in the performance of probabilistic phenotyping algorithms specifically in the context of ADRD as a use case. Differences in subgroup performance of probabilistic phenotyping algorithms are not widespread nor do they occur consistently. This highlights the great need for careful ongoing monitoring to evaluate, measure, and try to mitigate such differences.
Collapse
Affiliation(s)
- Juan M Banda
- Corresponding Author: Juan M. Banda, PhD, Department of Computer Science, College of Arts and Sciences, Georgia State University, 25 Park Place, Suite 752, Atlanta, GA 30303, USA;
| | - Nigam H Shah
- Stanford Center for Biomedical Informatics Research, Stanford University School of Medicine, Stanford, California, USA
| | - Vyjeyanthi S Periyakoil
- Stanford Department of Medicine, Palo Alto, California, USA
- VA Palo Alto Health Care System, Palo Alto, California, USA
| |
Collapse
|
85
|
Lee YT, Fujiwara N, Yang JD, Hoshida Y. Risk stratification and early detection biomarkers for precision HCC screening. Hepatology 2023; 78:319-362. [PMID: 36082510 PMCID: PMC9995677 DOI: 10.1002/hep.32779] [Citation(s) in RCA: 19] [Impact Index Per Article: 19.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/18/2022] [Revised: 08/25/2022] [Accepted: 08/28/2022] [Indexed: 12/08/2022]
Abstract
Hepatocellular carcinoma (HCC) mortality remains high primarily due to late diagnosis as a consequence of failed early detection. Professional societies recommend semi-annual HCC screening in at-risk patients with chronic liver disease to increase the likelihood of curative treatment receipt and improve survival. However, recent dynamic shift of HCC etiologies from viral to metabolic liver diseases has significantly increased the potential target population for the screening, whereas annual incidence rate has become substantially lower. Thus, with the contemporary HCC etiologies, the traditional screening approach might not be practical and cost-effective. HCC screening consists of (i) definition of rational at-risk population, and subsequent (ii) repeated application of early detection tests to the population at regular intervals. The suboptimal performance of the currently available HCC screening tests highlights an urgent need for new modalities and strategies to improve early HCC detection. In this review, we overview recent developments of clinical, molecular, and imaging-based tools to address the current challenge, and discuss conceptual framework and approaches of their clinical translation and implementation. These encouraging progresses are expected to transform the current "one-size-fits-all" HCC screening into individualized precision approaches to early HCC detection and ultimately improve the poor HCC prognosis in the foreseeable future.
Collapse
Affiliation(s)
- Yi-Te Lee
- California NanoSystems Institute, Crump Institute for Molecular Imaging, Department of Molecular and Medical Pharmacology, University of California, Los Angeles, Los Angeles, California
| | - Naoto Fujiwara
- Liver Tumor Translational Research Program, Simmons Comprehensive Cancer Center, Division of Digestive and Liver Diseases, Department of Internal Medicine, University of Texas Southwestern Medical Center, Dallas, Texas
| | - Ju Dong Yang
- Karsh Division of Gastroenterology and Hepatology, Cedars-Sinai Medical Center, Los Angeles, California; Comprehensive Transplant Center, Cedars-Sinai Medical Center, Los Angeles, Los Angeles, California; Samuel Oschin Comprehensive Cancer Institute, Cedars-Sinai Medical Center, Los Angeles, California
| | - Yujin Hoshida
- Liver Tumor Translational Research Program, Simmons Comprehensive Cancer Center, Division of Digestive and Liver Diseases, Department of Internal Medicine, University of Texas Southwestern Medical Center, Dallas, Texas
| |
Collapse
|
86
|
Eskofier BM, Klucken J. Predictive Models for Health Deterioration: Understanding Disease Pathways for Personalized Medicine. Annu Rev Biomed Eng 2023; 25:131-156. [PMID: 36854259 DOI: 10.1146/annurev-bioeng-110220-030247] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/02/2023]
Abstract
Artificial intelligence (AI) and machine learning (ML) methods are currently widely employed in medicine and healthcare. A PubMed search returns more than 100,000 articles on these topics published between 2018 and 2022 alone. Notwithstanding several recent reviews in various subfields of AI and ML in medicine, we have yet to see a comprehensive review around the methods' use in longitudinal analysis and prediction of an individual patient's health status within a personalized disease pathway. This review seeks to fill that gap. After an overview of the AI and ML methods employed in this field and of specific medical applications of models of this type, the review discusses the strengths and limitations of current studies and looks ahead to future strands of research in this field. We aim to enable interested readers to gain a detailed impression of the research currently available and accordingly plan future work around predictive models for deterioration in health status.
Collapse
Affiliation(s)
- Bjoern M Eskofier
- Machine Learning and Data Analytics Lab, Department of Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany;
| | - Jochen Klucken
- Digital Medicine Group, Luxembourg Centre for Systems Biomedicine, Université du Luxembourg, Belvaux, Luxembourg
- Digital Medicine Group, Department of Precision Health, Luxembourg Institute of Health, Strassen, Luxembourg
- Centre Hospitalier de Luxembourg, Luxembourg City, Luxembourg
| |
Collapse
|
87
|
Calumby RT, Duarte AA, Angelo MF, Santos E, Sarder P, dos-Santos WL, Oliveira LR. Toward Real-World Computational Nephropathology. Clin J Am Soc Nephrol 2023; 18:809-812. [PMID: 37027795 PMCID: PMC10278791 DOI: 10.2215/cjn.0000000000000168] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2022] [Accepted: 03/30/2023] [Indexed: 04/09/2023]
Affiliation(s)
- Rodrigo T. Calumby
- Advanced Data Analysis and Management Lab, Department of Exact Sciences, University of Feira de Santana, Feira de Santana, Brazil
| | - Angelo A. Duarte
- High Performance Computing Laboratory, Department of Technology, University of Feira de Santana, Feira de Santana, Brazil
| | - Michele F. Angelo
- Departament of Exact Sciences, University of Feira de Santana, Feira de Santana, Brazil
| | - Emanuele Santos
- Center of Sciences, Department of Computing, Federal University of Ceara, Fortaleza, Brazil
| | - Pinaki Sarder
- Intelligent Critical Care Center, Department of Medicine – Nephrology, University of Florida, Gainesville, Florida
| | - Washington L.C. dos-Santos
- Fundação Oswaldo Cruz, Gonçalo Moniz Institute, Structural and Molecular Pathology Laboratory, Salvador, Brazil
| | - Luciano R. Oliveira
- Intelligent Vision Research Lab, Computer Science Department, Institute of Computing, Federal University of Bahia, Salvador, Brazil
| |
Collapse
|
88
|
Hosein S, Drebin HM, Kurtansky NR, Bagge RO, Coit DG, Bartlett EK, Marchetti MA. Are the MIA and MSKCC nomograms useful in selecting patients with melanoma for sentinel lymph node biopsy? J Surg Oncol 2023; 127:1167-1173. [PMID: 36905337 PMCID: PMC10147582 DOI: 10.1002/jso.27231] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2023] [Accepted: 02/26/2023] [Indexed: 03/12/2023]
Abstract
BACKGROUND AND METHODS The Melanoma Institute of Australia (MIA) and Memorial Sloan Kettering Cancer Center (MSKCC) nomograms were developed to help guide sentinel lymph node biopsy (SLNB) decisions. Although statistically validated, whether these prediction models provide clinical benefit at National Comprehensive Cancer Network guideline-endorsed thresholds is unknown. We conducted a net benefit analysis to quantify the clinical utility of these nomograms at risk thresholds of 5%-10% compared to the alternative strategy of biopsying all patients. External validation data for MIA and MSKCC nomograms were extracted from respective published studies. RESULTS The MIA nomogram provided added net benefit at a risk threshold of 9% but net harm at 5%-8% and 10%. The MSKCC nomogram provided added net benefit at risk thresholds of 5% and 9%-10% but net harm at 6%-8%. When present, the magnitude of net benefit was small (1-3 net avoidable biopsies per 100 patients). CONCLUSION Neither model consistently provided added net benefit compared to performing SLNB for all patients. DISCUSSION Based on published data, use of the MIA or MSKCC nomograms as decision-making tools for SLNB at risk thresholds of 5%-10% does not clearly provide clinical benefit to patients.
Collapse
Affiliation(s)
- Sharif Hosein
- Dermatology Service, Department of Medicine, Memorial Sloan Kettering Cancer Center, New York, NY, United States
| | - Harrison M. Drebin
- Gastric and Mixed Tumor Service, Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, United States
| | - Nicholas R. Kurtansky
- Dermatology Service, Department of Medicine, Memorial Sloan Kettering Cancer Center, New York, NY, United States
| | - Roger Olofsson Bagge
- Department of Surgery, Sahlgrenska University Hospital, Sweden
- Sahlgrenska Center for Cancer Research, Department of Surgery, Institute of Clinical Sciences, Sahlgrenska Academy, University of Gothenburg
- Wallenberg Centre for Molecular and Translational Medicine, University of Gothenburg, Sweden
| | - Daniel G. Coit
- Gastric and Mixed Tumor Service, Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, United States
| | - Edmund K. Bartlett
- Gastric and Mixed Tumor Service, Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, United States
| | - Michael A. Marchetti
- Dermatology Service, Department of Medicine, Memorial Sloan Kettering Cancer Center, New York, NY, United States
| |
Collapse
|
89
|
Gottlieb M, Kline JA, Schneider AJ, Coates WC. ChatGPT and conversational artificial intelligence: Friend, foe, or future of research? Am J Emerg Med 2023; 70:81-83. [PMID: 37229893 DOI: 10.1016/j.ajem.2023.05.018] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2023] [Revised: 05/01/2023] [Accepted: 05/12/2023] [Indexed: 05/27/2023] Open
Abstract
Artificial intelligence (AI) and machine learning are increasingly utilized across healthcare. More recently, there has been a rise in the use AI within research, particularly through novel conversational AI platforms, such as ChatGPT. In this Controversies paper, we discuss the advantages, limitations, and future directions for ChatGPT and other forms of conversational AI in research and scholarly dissemination.
Collapse
Affiliation(s)
- Michael Gottlieb
- Department of Emergency Medicine, Rush University Medical Center, Chicago, IL, United States of America.
| | - Jeffrey A Kline
- Department of Emergency Medicine, Wayne State University School of Medicine, Detroit, MI, United States of America.
| | | | - Wendy C Coates
- Department of Emergency Medicine, University of California, David Geffen School of Medicine, Los Angeles, CA, United States of America
| |
Collapse
|
90
|
Masoumian Hosseini M, Masoumian Hosseini ST, Qayumi K, Ahmady S, Koohestani HR. The Aspects of Running Artificial Intelligence in Emergency Care; a Scoping Review. ARCHIVES OF ACADEMIC EMERGENCY MEDICINE 2023; 11:e38. [PMID: 37215232 PMCID: PMC10197918 DOI: 10.22037/aaem.v11i1.1974] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 05/24/2023]
Abstract
Introduction Artificial Inteligence (AI) application in emergency medicine is subject to ethical and legal inconsistencies. The purposes of this study were to map the extent of AI applications in emergency medicine, to identify ethical issues related to the use of AI, and to propose an ethical framework for its use. Methods A comprehensive literature collection was compiled through electronic databases/internet search engines (PubMed, Web of Science Platform, MEDLINE, Scopus, Google Scholar/Academia, and ERIC) and reference lists. We considered studies published between 1 January 2014 and 6 October 2022. Articles that did not self-classify as studies of an AI intervention, those that were not relevant to Emergency Departments (EDs), and articles that did not report outcomes or evaluations were excluded. Descriptive and thematic analyses of data extracted from the included articles were conducted. Results A total of 137 out of the 2175 citations in the original database were eligible for full-text evaluation. Of these articles, 47 were included in the scoping review and considered for theme extraction. This review covers seven main areas of AI techniques in emergency medicine: Machine Learning (ML) Algorithms (10.64%), prehospital emergency management (12.76%), triage, patient acuity and disposition of patients (19.15%), disease and condition prediction (23.40%), emergency department management (17.03%), the future impact of AI on Emergency Medical Services (EMS) (8.51%), and ethical issues (8.51%). Conclusion There has been a rapid increase in AI research in emergency medicine in recent years. Several studies have demonstrated the potential of AI in diverse contexts, particularly when improving patient outcomes through predictive modelling. According to the synthesis of studies in our review, AI-based decision-making lacks transparency. This feature makes AI decision-making opaque.
Collapse
Affiliation(s)
| | | | - Karim Qayumi
- Centre of Excellence for Simulation Education and Innovation, Department of Surgery, University of British Columbia, Vancouver, BC, Canada
| | - Soleiman Ahmady
- Department of Medical Education, Virtual School of Medical Education & Management, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Hamid Reza Koohestani
- Department of Nursing, Social Determinants of Health Research Center, Saveh University of Medical Sciences, Saveh, Iran
| |
Collapse
|
91
|
de Hond AAH, Shah VB, Kant IMJ, Van Calster B, Steyerberg EW, Hernandez-Boussard T. Perspectives on validation of clinical predictive algorithms. NPJ Digit Med 2023; 6:86. [PMID: 37149704 PMCID: PMC10163568 DOI: 10.1038/s41746-023-00832-9] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2023] [Accepted: 04/28/2023] [Indexed: 05/08/2023] Open
Affiliation(s)
- Anne A H de Hond
- Clinical AI Implementation and Research Lab, Leiden University Medical Centre, Leiden, the Netherlands.
- Department of Medicine (Biomedical Informatics), Stanford University, Stanford, CA, USA.
- Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, the Netherlands.
| | - Vaibhavi B Shah
- Department of Medicine (Biomedical Informatics), Stanford University, Stanford, CA, USA
| | - Ilse M J Kant
- Department of Digital Health, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Ben Van Calster
- Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, the Netherlands
- Department of Development & Regeneration, KU Leuven, Leuven, Belgium
| | - Ewout W Steyerberg
- Clinical AI Implementation and Research Lab, Leiden University Medical Centre, Leiden, the Netherlands
- Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, the Netherlands
| | - Tina Hernandez-Boussard
- Department of Medicine (Biomedical Informatics), Stanford University, Stanford, CA, USA
- Department of Biomedical Data Science, Stanford University, Stanford, CA, USA
- Department of Epidemiology & Population Health (by courtesy), Stanford University, Stanford, CA, USA
| |
Collapse
|
92
|
Liu LJ, Takeuchi T, Chen J, Neyra JA. Artificial Intelligence in Continuous Kidney Replacement Therapy. Clin J Am Soc Nephrol 2023; 18:671-674. [PMID: 36735382 PMCID: PMC10278853 DOI: 10.2215/cjn.0000000000000099] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2023] [Accepted: 01/20/2023] [Indexed: 02/04/2023]
Affiliation(s)
- Lucas J. Liu
- Department of Computer Science, University of Kentucky, Lexington, Kentucky
- Division of Biomedical Informatics, Department of Internal Medicine, University of Kentucky, Lexington, Kentucky
| | - Tomonori Takeuchi
- Department of Health Policy and Informatics, Tokyo Medical and Dental University, Tokyo, Japan
| | - Jin Chen
- Department of Computer Science, University of Kentucky, Lexington, Kentucky
- Division of Biomedical Informatics, Department of Internal Medicine, University of Kentucky, Lexington, Kentucky
| | - Javier A. Neyra
- Division of Nephrology, Bone and Mineral Metabolism, Department of Internal Medicine, University of Kentucky, Lexington, Kentucky
- Division of Nephrology, Department of Internal Medicine, University of Alabama at Birmingham, Birmingham, Alabama
| |
Collapse
|
93
|
Mo D, Zheng Q, Xiao B, Li L. Predicting thalassemia using deep neural network based on red blood cell indices. Clin Chim Acta 2023; 543:117329. [PMID: 37019327 DOI: 10.1016/j.cca.2023.117329] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2023] [Revised: 03/11/2023] [Accepted: 03/30/2023] [Indexed: 04/05/2023]
Abstract
BACKGROUND AND OBJECTIVE The traditional statistical screening method for thalassemia based on red blood cell (RBC) indices is being replaced by machine learning. Here, we developed deep neural networks (DNNs) that outperformed the traditional method for predicting thalassemia. METHOD Using a dataset of 8693 records comprising genetic tests and other 11 features we constructed 11 DNN models and 4 traditional statistical models and then compared their performances and analysed feature importance for interpreting DNN models. RESULTS The area under the receiver operating characteristic curve, accuracy, Youden's index, F1 score, sensitivity, specificity, positive predictive value and negative predictive value, were 0.960, 0.897, 0.794, 0.897, 0.883, 0.911, 0.914, and 0.882, respectively, for our best model, and compared with the traditional statistical model based on the mean corpuscular volume, these values were increased by 10.22%, 10.09%, 26.55%, 8.92%, 4.13%, 16.90%, 13.86% and 6.07%, respectively, and by 15.38%, 11.70%, 31.70%, 9.89%, 3.05%, 22.13%, 17.11% and 5.94%, respectively, for the mean cellular haemoglobin model. The DNN model performance will reduce without age, RBC distribution width (RDW), sex, or both WBC and PLT. CONCLUSIONS Our DNN model outperformed the current screening model. In 8 features, RDW and age were the most useful, followed by sex and the combination of WBC and PLT, the remaining nearly useless.
Collapse
Affiliation(s)
- Donghua Mo
- The First School of Clinical Medicine, Southern Medical University, Guangzhou, China; Clinical Laboratory Medicine Department, The Second Affiliated Hospital of Guangzhou Medical University, Guangzhou, China
| | - Qian Zheng
- Department of Cardiovascular, The First Affiliated Hospital of Sun Yat-sen University, Guangzhou, China
| | - Bin Xiao
- Department of Laboratory Medicine, The Sixth Affiliated Hospital of Guangzhou Medical University, Qingyuan People's Hospital, 511518 Qingyuan, China.
| | - Linhai Li
- The First School of Clinical Medicine, Southern Medical University, Guangzhou, China; Department of Laboratory Medicine, The Sixth Affiliated Hospital of Guangzhou Medical University, Qingyuan People's Hospital, 511518 Qingyuan, China.
| |
Collapse
|
94
|
Zaniletti I, Larson DR, Lewallen DG, Berry DJ, Maradit Kremers H. How to Develop and Validate Prediction Models for Orthopedic Outcomes. J Arthroplasty 2023; 38:627-633. [PMID: 36572235 PMCID: PMC10023373 DOI: 10.1016/j.arth.2022.12.032] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/12/2022] [Revised: 12/15/2022] [Accepted: 12/17/2022] [Indexed: 12/25/2022] Open
Abstract
Prediction models are common in medicine for predicting outcomes such as mortality, complications, or response to treatment. Despite the growing interest in these models in arthroplasty (and orthopaedics in general), few have been adopted in clinical practice. If robustly built and validated, prediction models can be excellent tools to support surgical decision making. In this paper, we provide an overview of the statistical concepts surrounding prediction models and outline practical steps for prediction model development and validation in arthroplasty research. Please visit the followinghttps://www.youtube.com/watch?v=9Yrit23Rkicfor a video that explains the highlights of the paper in practical terms.
Collapse
Affiliation(s)
| | - Dirk R. Larson
- Department of Quantitative Health Sciences, Mayo Clinic, Rochester, MN
| | | | - Daniel J. Berry
- Department of Orthopedic Surgery, Mayo Clinic, Rochester, MN
| | - Hilal Maradit Kremers
- Department of Quantitative Health Sciences, Mayo Clinic, Rochester, MN
- Department of Orthopedic Surgery, Mayo Clinic, Rochester, MN
| |
Collapse
|
95
|
Mintz I, Chowers M, Obolski U. Prediction of ciprofloxacin resistance in hospitalized patients using machine learning. COMMUNICATIONS MEDICINE 2023; 3:43. [PMID: 36977789 PMCID: PMC10050086 DOI: 10.1038/s43856-023-00275-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2022] [Accepted: 03/14/2023] [Indexed: 03/30/2023] Open
Abstract
BACKGROUND Ciprofloxacin is a widely used antibiotic that has lost efficiency due to extensive resistance. We developed machine learning (ML) models that predict the probability of ciprofloxacin resistance in hospitalized patients. METHODS Data were collected from electronic records of hospitalized patients with positive bacterial cultures, during 2016-2019. Susceptibility results to ciprofloxacin (n = 10,053 cultures) were obtained for Escherichia coli, Klebsiella pneumoniae, Morganella morganii, Pseudomonas aeruginosa, Proteus mirabilis and Staphylococcus aureus. An ensemble model, combining several base models, was developed to predict ciprofloxacin resistant cultures, either with (gnostic) or without (agnostic) information on the infecting bacterial species. RESULTS The ensemble models' predictions are well-calibrated, and yield ROC-AUCs (area under the receiver operating characteristic curve) of 0.737 (95%CI 0.715-0.758) and 0.837 (95%CI 0.821-0.854) on independent test-sets for the agnostic and gnostic datasets, respectively. Shapley additive explanations analysis identifies that influential variables are related to resistance of previous infections, where patients arrived from (hospital, nursing home, etc.), and recent resistance frequencies in the hospital. A decision curve analysis reveals that implementing our models can be beneficial in a wide range of cost-benefits considerations of ciprofloxacin administration. CONCLUSIONS This study develops ML models to predict ciprofloxacin resistance in hospitalized patients. The models achieve high predictive ability, are well calibrated, have substantial net-benefit across a wide range of conditions, and rely on predictors consistent with the literature. This is a further step on the way to inclusion of ML decision support systems into clinical practice.
Collapse
Affiliation(s)
- Igor Mintz
- School of Public Health, Tel Aviv University, Tel Aviv, Israel
- Porter School of the Environment and Earth Sciences, Tel Aviv University, Tel Aviv, Israel
| | - Michal Chowers
- Meir Medical Center, Kfar Saba, Israel
- Sackler School of Medicine, Tel Aviv University, Tel Aviv, Israel
| | - Uri Obolski
- School of Public Health, Tel Aviv University, Tel Aviv, Israel.
- Porter School of the Environment and Earth Sciences, Tel Aviv University, Tel Aviv, Israel.
| |
Collapse
|
96
|
Chen Y, Clayton EW, Novak LL, Anders S, Malin B. Human-Centered Design to Address Biases in Artificial Intelligence. J Med Internet Res 2023; 25:e43251. [PMID: 36961506 PMCID: PMC10132017 DOI: 10.2196/43251] [Citation(s) in RCA: 18] [Impact Index Per Article: 18.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2022] [Revised: 01/30/2023] [Accepted: 02/28/2023] [Indexed: 03/25/2023] Open
Abstract
The potential of artificial intelligence (AI) to reduce health care disparities and inequities is recognized, but it can also exacerbate these issues if not implemented in an equitable manner. This perspective identifies potential biases in each stage of the AI life cycle, including data collection, annotation, machine learning model development, evaluation, deployment, operationalization, monitoring, and feedback integration. To mitigate these biases, we suggest involving a diverse group of stakeholders, using human-centered AI principles. Human-centered AI can help ensure that AI systems are designed and used in a way that benefits patients and society, which can reduce health disparities and inequities. By recognizing and addressing biases at each stage of the AI life cycle, AI can achieve its potential in health care.
Collapse
Affiliation(s)
- You Chen
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, United States
- Department of Computer Science, Vanderbilt University, Nashville, TN, United States
| | - Ellen Wright Clayton
- Law School, Vanderbilt University, Nashville, TN, United States
- Center for Biomedical Ethics and Society, Vanderbilt University Medical Center, Nashville, TN, United States
- Department of Pediatrics, Vanderbilt University Medical Center, Nashville, TN, United States
| | - Laurie Lovett Novak
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, United States
| | - Shilo Anders
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, United States
- Department of Computer Science, Vanderbilt University, Nashville, TN, United States
- Department of Anesthesiology, Vanderbilt University Medical Center, Nashville, TN, United States
| | - Bradley Malin
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, United States
- Department of Computer Science, Vanderbilt University, Nashville, TN, United States
- Center for Biomedical Ethics and Society, Vanderbilt University Medical Center, Nashville, TN, United States
- Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN, United States
| |
Collapse
|
97
|
Walter W, Pohlkamp C, Meggendorfer M, Nadarajah N, Kern W, Haferlach C, Haferlach T. Artificial intelligence in hematological diagnostics: Game changer or gadget? Blood Rev 2023; 58:101019. [PMID: 36241586 DOI: 10.1016/j.blre.2022.101019] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2022] [Revised: 09/21/2022] [Accepted: 10/03/2022] [Indexed: 11/30/2022]
Abstract
The future of clinical diagnosis and treatment of hematologic diseases will inevitably involve the integration of artificial intelligence (AI)-based systems into routine practice to support the hematologists' decision making. Several studies have shown that AI-based models can already be used to automatically differentiate cells, reliably detect malignant cell populations, support chromosome banding analysis, and interpret clinical variants, contributing to early disease detection and prognosis. However, even the best tool can become useless if it is misapplied or the results are misinterpreted. Therefore, in order to comprehensively judge and correctly apply newly developed AI-based systems, the hematologist must have a basic understanding of the general concepts of machine learning. In this review, we provide the hematologist with a comprehensive overview of various machine learning techniques, their current implementations and approaches in different diagnostic subfields (e.g., cytogenetics, molecular genetics), and the limitations and unresolved challenges of the systems.
Collapse
Affiliation(s)
- Wencke Walter
- MLL Munich Leukemia Laboratory, Max-Lebsche-Platz 31, 81377 München, Germany.
| | - Christian Pohlkamp
- MLL Munich Leukemia Laboratory, Max-Lebsche-Platz 31, 81377 München, Germany.
| | - Manja Meggendorfer
- MLL Munich Leukemia Laboratory, Max-Lebsche-Platz 31, 81377 München, Germany.
| | - Niroshan Nadarajah
- MLL Munich Leukemia Laboratory, Max-Lebsche-Platz 31, 81377 München, Germany.
| | - Wolfgang Kern
- MLL Munich Leukemia Laboratory, Max-Lebsche-Platz 31, 81377 München, Germany.
| | - Claudia Haferlach
- MLL Munich Leukemia Laboratory, Max-Lebsche-Platz 31, 81377 München, Germany.
| | - Torsten Haferlach
- MLL Munich Leukemia Laboratory, Max-Lebsche-Platz 31, 81377 München, Germany.
| |
Collapse
|
98
|
Henry JA. Culture intelligent workflow, structure, and steps. Front Artif Intell 2023; 6:985469. [PMID: 36925615 PMCID: PMC10011165 DOI: 10.3389/frai.2023.985469] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2022] [Accepted: 02/06/2023] [Indexed: 03/08/2023] Open
Abstract
Introduction Technologies abstract intelligence and provide predictor and precision insight in workflows that manage disorders, similar to cardiology and hematological disease. Positive perceptions of Artificial Intelligence (AI) that support Machine Learning (ML) and Deep Learning (DL) manage transformations with a safe system that improves wellbeing. In sections, workflow introduces an eXamination (X = AI) as an end-to-end structure to culture workstreams in a step-by-step design to manage populace health in a governed system. Method To better healthcare outcomes, communities and personnel benefit from an explanation and an interpretive that elucidates workflow for citizens or practitioners to comprehend personalized platforms. Therefore, the author undertook structure and practice reviews and appraised perspectives that impact the management of AI in public health and medicine. Results Figures for the management of AI workflow illustrate and inform on the model, structure, culture, assurance, process steps, values, and governance required for abstract insights in public health and medicine. The papers' end-to-end structure with explanans in a work culture interprets the step-by-step designs that manage the success of AI. Personalized care graphics offer an explanandum in the management of biological analytic value. Discussion Healthcare leadership collaboratives plan population health with an upstream, workplace and workstream format. Secure workflow and safety wellbeing system requirements prove that genomics and AI improve medicine. Therefore, the paper discusses group understanding of current practice, ethics, policy, and legality. Conclusion "Culture, intelligent workflow, structure, and steps" improve wellbeing with personalized care and align a percept for national opportunities, regional control, and local needs. Personalized practice cultures support analytic systems to describe, predict, precision, and prescript medicine in population health management eXaminations.
Collapse
Affiliation(s)
- James Andrew Henry
- Institute of Biomedical Sciences, London, United Kingdom
- Society for Advanced Blood Management, Mount Royal, NJ, United States
- British Blood Transfusion Society, Birmingham, United Kingdom
| |
Collapse
|
99
|
Mavragani A, Ozoude MM, Williams KS, Sadiq-Onilenla RA, Ojo SA, Wasarme LB, Walsh S, Edomwande M. The Need to Prioritize Model-Updating Processes in Clinical Artificial Intelligence (AI) Models: Protocol for a Scoping Review. JMIR Res Protoc 2023; 12:e37685. [PMID: 36795464 PMCID: PMC9982723 DOI: 10.2196/37685] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2022] [Revised: 11/10/2022] [Accepted: 11/28/2022] [Indexed: 11/29/2022] Open
Abstract
BACKGROUND With an increase in the number of artificial intelligence (AI) and machine learning (ML) algorithms available for clinical settings, appropriate model updating and implementation of updates are imperative to ensure applicability, reproducibility, and patient safety. OBJECTIVE The objective of this scoping review was to evaluate and assess the model-updating practices of AI and ML clinical models that are used in direct patient-provider clinical decision-making. METHODS We used the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist and the PRISMA-P protocol guidance in addition to a modified CHARMS (Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies) checklist to conduct this scoping review. A comprehensive medical literature search of databases, including Embase, MEDLINE, PsycINFO, Cochrane, Scopus, and Web of Science, was conducted to identify AI and ML algorithms that would impact clinical decision-making at the level of direct patient care. Our primary end point is the rate at which model updating is recommended by published algorithms; we will also conduct an assessment of study quality and risk of bias in all publications reviewed. In addition, we will evaluate the rate at which published algorithms include ethnic and gender demographic distribution information in their training data as a secondary end point. RESULTS Our initial literature search yielded approximately 13,693 articles, with approximately 7810 articles to consider for full reviews among our team of 7 reviewers. We plan to complete the review process and disseminate the results by spring of 2023. CONCLUSIONS Although AI and ML applications in health care have the potential to improve patient care by reducing errors between measurement and model output, currently there exists more hype than hope because of the lack of proper external validation of these models. We expect to find that the AI and ML model-updating methods are proxies for model applicability and generalizability on implementation. Our findings will add to the field by determining the degree to which published models meet the criteria for clinical validity, real-life implementation, and best practices to optimize model development, and in so doing, reduce the overpromise and underachievement of the contemporary model development process. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID) PRR1-10.2196/37685.
Collapse
Affiliation(s)
| | | | | | | | - Soji Akin Ojo
- Pharmaceutical Product Development (PPD), Thermo Fisher Scientific, Wilmington, NC, United States
| | | | - Samantha Walsh
- Levy Library, Icahn School of Medicine at Mount Sinai, New York, NY, United States
| | | |
Collapse
|
100
|
de Hond AAH, Kant IMJ, Fornasa M, Cinà G, Elbers PWG, Thoral PJ, Sesmu Arbous M, Steyerberg EW. Predicting Readmission or Death After Discharge From the ICU: External Validation and Retraining of a Machine Learning Model. Crit Care Med 2023; 51:291-300. [PMID: 36524820 PMCID: PMC9848213 DOI: 10.1097/ccm.0000000000005758] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022]
Abstract
OBJECTIVES Many machine learning (ML) models have been developed for application in the ICU, but few models have been subjected to external validation. The performance of these models in new settings therefore remains unknown. The objective of this study was to assess the performance of an existing decision support tool based on a ML model predicting readmission or death within 7 days after ICU discharge before, during, and after retraining and recalibration. DESIGN A gradient boosted ML model was developed and validated on electronic health record data from 2004 to 2021. We performed an independent validation of this model on electronic health record data from 2011 to 2019 from a different tertiary care center. SETTING Two ICUs in tertiary care centers in The Netherlands. PATIENTS Adult patients who were admitted to the ICU and stayed for longer than 12 hours. INTERVENTIONS None. MEASUREMENTS AND MAIN RESULTS We assessed discrimination by area under the receiver operating characteristic curve (AUC) and calibration (slope and intercept). We retrained and recalibrated the original model and assessed performance via a temporal validation design. The final retrained model was cross-validated on all data from the new site. Readmission or death within 7 days after ICU discharge occurred in 577 of 10,052 ICU admissions (5.7%) at the new site. External validation revealed moderate discrimination with an AUC of 0.72 (95% CI 0.67-0.76). Retrained models showed improved discrimination with AUC 0.79 (95% CI 0.75-0.82) for the final validation model. Calibration was poor initially and good after recalibration via isotonic regression. CONCLUSIONS In this era of expanding availability of ML models, external validation and retraining are key steps to consider before applying ML models to new settings. Clinicians and decision-makers should take this into account when considering applying new ML models to their local settings.
Collapse
Affiliation(s)
- Anne A H de Hond
- Department of Information Technology and Digital Innovation, Leiden University Medical Centre, Leiden, The Netherlands
- Department of Biomedical Informatics, Stanford Medicine, Stanford, CA
- Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, The Netherlands
| | - Ilse M J Kant
- Department of Information Technology and Digital Innovation, Leiden University Medical Centre, Leiden, The Netherlands
- Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, The Netherlands
| | | | - Giovanni Cinà
- Pacmed, Stadhouderskade 55, Amsterdam, The Netherlands
- Institute of Logic, Language and Computation, University of Amsterdam, Amsterdam, The Netherlands
| | - Paul W G Elbers
- Department of Intensive Care Medicine, Laboratory for Critical Care Computational Intelligence, Amsterdam UMC, Amsterdam, The Netherlands
| | - Patrick J Thoral
- Department of Intensive Care Medicine, Laboratory for Critical Care Computational Intelligence, Amsterdam UMC, Amsterdam, The Netherlands
| | - M Sesmu Arbous
- Department of Intensive Care Medicine, Leiden University Medical Centre, Leiden, The Netherlands
| | - Ewout W Steyerberg
- Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, The Netherlands
| |
Collapse
|