1
|
Gonzalez R, Saha A, Campbell CJ, Nejat P, Lokker C, Norgan AP. Seeing the random forest through the decision trees. Supporting learning health systems from histopathology with machine learning models: Challenges and opportunities. J Pathol Inform 2024; 15:100347. [PMID: 38162950 PMCID: PMC10755052 DOI: 10.1016/j.jpi.2023.100347] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2023] [Revised: 10/06/2023] [Accepted: 11/01/2023] [Indexed: 01/03/2024] Open
Abstract
This paper discusses some overlooked challenges faced when working with machine learning models for histopathology and presents a novel opportunity to support "Learning Health Systems" with them. Initially, the authors elaborate on these challenges after separating them according to their mitigation strategies: those that need innovative approaches, time, or future technological capabilities and those that require a conceptual reappraisal from a critical perspective. Then, a novel opportunity to support "Learning Health Systems" by integrating hidden information extracted by ML models from digitalized histopathology slides with other healthcare big data is presented.
Collapse
Affiliation(s)
- Ricardo Gonzalez
- DeGroote School of Business, McMaster University, Hamilton, Ontario, Canada
- Division of Computational Pathology and Artificial Intelligence, Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, United States
| | - Ashirbani Saha
- Department of Oncology, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
- Escarpment Cancer Research Institute, McMaster University and Hamilton Health Sciences, Hamilton, Ontario, Canada
| | - Clinton J.V. Campbell
- William Osler Health System, Brampton, Ontario, Canada
- Department of Pathology and Molecular Medicine, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
| | - Peyman Nejat
- Department of Artificial Intelligence and Informatics, Mayo Clinic, Rochester, MN, United States
| | - Cynthia Lokker
- Health Information Research Unit, Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada
| | - Andrew P. Norgan
- Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, United States
| |
Collapse
|
2
|
Saha A, Bosma JS, Twilt JJ, van Ginneken B, Bjartell A, Padhani AR, Bonekamp D, Villeirs G, Salomon G, Giannarini G, Kalpathy-Cramer J, Barentsz J, Maier-Hein KH, Rusu M, Rouvière O, van den Bergh R, Panebianco V, Kasivisvanathan V, Obuchowski NA, Yakar D, Elschot M, Veltman J, Fütterer JJ, de Rooij M, Huisman H. Artificial intelligence and radiologists in prostate cancer detection on MRI (PI-CAI): an international, paired, non-inferiority, confirmatory study. Lancet Oncol 2024; 25:879-887. [PMID: 38876123 DOI: 10.1016/s1470-2045(24)00220-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2024] [Revised: 04/16/2024] [Accepted: 04/18/2024] [Indexed: 06/16/2024]
Abstract
BACKGROUND Artificial intelligence (AI) systems can potentially aid the diagnostic pathway of prostate cancer by alleviating the increasing workload, preventing overdiagnosis, and reducing the dependence on experienced radiologists. We aimed to investigate the performance of AI systems at detecting clinically significant prostate cancer on MRI in comparison with radiologists using the Prostate Imaging-Reporting and Data System version 2.1 (PI-RADS 2.1) and the standard of care in multidisciplinary routine practice at scale. METHODS In this international, paired, non-inferiority, confirmatory study, we trained and externally validated an AI system (developed within an international consortium) for detecting Gleason grade group 2 or greater cancers using a retrospective cohort of 10 207 MRI examinations from 9129 patients. Of these examinations, 9207 cases from three centres (11 sites) based in the Netherlands were used for training and tuning, and 1000 cases from four centres (12 sites) based in the Netherlands and Norway were used for testing. In parallel, we facilitated a multireader, multicase observer study with 62 radiologists (45 centres in 20 countries; median 7 [IQR 5-10] years of experience in reading prostate MRI) using PI-RADS (2.1) on 400 paired MRI examinations from the testing cohort. Primary endpoints were the sensitivity, specificity, and the area under the receiver operating characteristic curve (AUROC) of the AI system in comparison with that of all readers using PI-RADS (2.1) and in comparison with that of the historical radiology readings made during multidisciplinary routine practice (ie, the standard of care with the aid of patient history and peer consultation). Histopathology and at least 3 years (median 5 [IQR 4-6] years) of follow-up were used to establish the reference standard. The statistical analysis plan was prespecified with a primary hypothesis of non-inferiority (considering a margin of 0·05) and a secondary hypothesis of superiority towards the AI system, if non-inferiority was confirmed. This study was registered at ClinicalTrials.gov, NCT05489341. FINDINGS Of the 10 207 examinations included from Jan 1, 2012, through Dec 31, 2021, 2440 cases had histologically confirmed Gleason grade group 2 or greater prostate cancer. In the subset of 400 testing cases in which the AI system was compared with the radiologists participating in the reader study, the AI system showed a statistically superior and non-inferior AUROC of 0·91 (95% CI 0·87-0·94; p<0·0001), in comparison to the pool of 62 radiologists with an AUROC of 0·86 (0·83-0·89), with a lower boundary of the two-sided 95% Wald CI for the difference in AUROC of 0·02. At the mean PI-RADS 3 or greater operating point of all readers, the AI system detected 6·8% more cases with Gleason grade group 2 or greater cancers at the same specificity (57·7%, 95% CI 51·6-63·3), or 50·4% fewer false-positive results and 20·0% fewer cases with Gleason grade group 1 cancers at the same sensitivity (89·4%, 95% CI 85·3-92·9). In all 1000 testing cases where the AI system was compared with the radiology readings made during multidisciplinary practice, non-inferiority was not confirmed, as the AI system showed lower specificity (68·9% [95% CI 65·3-72·4] vs 69·0% [65·5-72·5]) at the same sensitivity (96·1%, 94·0-98·2) as the PI-RADS 3 or greater operating point. The lower boundary of the two-sided 95% Wald CI for the difference in specificity (-0·04) was greater than the non-inferiority margin (-0·05) and a p value below the significance threshold was reached (p<0·001). INTERPRETATION An AI system was superior to radiologists using PI-RADS (2.1), on average, at detecting clinically significant prostate cancer and comparable to the standard of care. Such a system shows the potential to be a supportive tool within a primary diagnostic setting, with several associated benefits for patients and radiologists. Prospective validation is needed to test clinical applicability of this system. FUNDING Health~Holland and EU Horizon 2020.
Collapse
Affiliation(s)
- Anindo Saha
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, Netherlands; Minimally Invasive Image-Guided Intervention Center, Radboud University Medical Center, Nijmegen, Netherlands.
| | - Joeran S Bosma
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, Netherlands
| | - Jasper J Twilt
- Minimally Invasive Image-Guided Intervention Center, Radboud University Medical Center, Nijmegen, Netherlands
| | - Bram van Ginneken
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, Netherlands
| | - Anders Bjartell
- Department of Urology, Skåne University Hospital, Malmö, Sweden; Division of Translational Cancer Research, Lund University Cancer Centre, Lund, Sweden
| | - Anwar R Padhani
- Paul Strickland Scanner Centre, Mount Vernon Cancer Centre, London, UK
| | - David Bonekamp
- Division of Radiology, Deutsches Krebsforschungszentrum Heidelberg, Heidelberg, Germany
| | - Geert Villeirs
- Department of Diagnostic Sciences, Ghent University Hospital, Ghent, Belgium
| | - Georg Salomon
- Martini Clinic, Prostate Cancer Center, University Medical Centre Hamburg-Eppendorf, Hamburg, Germany
| | - Gianluca Giannarini
- Urology Unit, Santa Maria della Misericordia University Hospital, Udine, Italy
| | - Jayashree Kalpathy-Cramer
- Division of Artificial Medical Intelligence in Ophthalmology, University of Colorado, Aurora, CO, USA
| | - Jelle Barentsz
- Department of Medical Imaging, Andros Clinics, Arnhem, Netherlands
| | - Klaus H Maier-Hein
- Division of Medical Image Computing, Deutsches Krebsforschungszentrum Heidelberg, Heidelberg, Germany; Pattern Analysis and Learning Group, Department of Radiation Oncology, Heidelberg University Hospital, Heidelberg, Germany
| | - Mirabela Rusu
- Departments of Radiology, Urology and Biomedical Data Science, Stanford University, Stanford, CA, USA
| | - Olivier Rouvière
- Department of Urinary and Vascular Imaging, Hôpital Edouard Herriot, Hospices Civils de Lyon, Lyon, France; Faculté de Médecine Lyon-Est, Université de Lyon, Lyon, France
| | | | - Valeria Panebianco
- Department of Radiological Sciences, Oncology and Pathology, Sapienza University of Rome, Rome, Italy
| | - Veeru Kasivisvanathan
- Division of Surgery and Interventional Sciences, University College London and University College London Hospital, London, UK
| | - Nancy A Obuchowski
- Department of Quantitative Health Sciences and Department of Diagnostic Radiology, Cleveland Clinic Foundation, Cleveland OH, USA
| | - Derya Yakar
- Department of Radiology, University Medical Center Groningen, Netherlands; Department of Radiology, Netherlands Cancer Institute, Amsterdam, Netherlands
| | - Mattijs Elschot
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, Tronheim, Norway; Department of Radiology and Nuclear Medicine, St Olavs Hospital, Trondheim University Hospital, Trondheim, Norway
| | - Jeroen Veltman
- Department of Radiology, Ziekenhuisgroep Twente, Hengelo, Netherlands; Department of Multi-Modality Medical Imaging, Technical Medical Centre, University of Twente, Enschede, Netherlands
| | - Jurgen J Fütterer
- Minimally Invasive Image-Guided Intervention Center, Radboud University Medical Center, Nijmegen, Netherlands
| | - Maarten de Rooij
- Department of Medical Imaging, Radboud University Medical Center, Nijmegen, Netherlands
| | - Henkjan Huisman
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, Netherlands; Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, Tronheim, Norway
| |
Collapse
|
3
|
Takahashi K, Usuzaki T, Inamori R. Vision Transformer-based Deep Learning Models Accelerate Further Research for Predicting Neurosurgical Intervention. Radiol Artif Intell 2024; 6:e240117. [PMID: 38864744 DOI: 10.1148/ryai.240117] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/13/2024]
Affiliation(s)
- Kengo Takahashi
- Tohoku University Graduate School of Medicine, 2-1 Seiryomachi, Aoba-ku, Sendai, Miyagi 980-8573, Japan
| | - Takuma Usuzaki
- Department of Diagnostic Radiology, Tohoku University Hospital, Miyagi, Japan
| | - Ryusei Inamori
- Tohoku University Graduate School of Medicine, 2-1 Seiryomachi, Aoba-ku, Sendai, Miyagi 980-8573, Japan
| |
Collapse
|
4
|
Kim S, Kang U, Gu J, Kim J, Park J, Hwang GW, Park S, Jang HJ, Seong TY, Lee S. Artificial Multimodal Neuron with Associative Learning Capabilities: Acquisition, Extinction, and Spontaneous Recovery. ACS APPLIED MATERIALS & INTERFACES 2024. [PMID: 38950119 DOI: 10.1021/acsami.4c02343] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/03/2024]
Abstract
Associative multimodal artificial intelligence (AMAI) has gained significant attention across various fields, yet its implementation poses challenges due to the burden on computing and memory resources. To address these challenges, researchers have paid increasing attention to neuromorphic devices based on novel materials and structures, which can implement classical conditioning behaviors with simplified circuitry. Herein, we introduce an artificial multimodal neuron device that shows not only the acquisition behavior but also the extinction and the spontaneous recovery behaviors for the first time. Being composed of an ovonic threshold switch (OTS)-based neuron device, a conductive bridge memristor (CBM)-based synapse device, and a few passive electrical elements, such observed behaviors of this neuron device are explained in terms of the electroforming and the diffusion of metallic ions in the CBM. We believe that the proposed associative learning neuron device will shed light on the way of developing large-scale AMAI systems by providing inspiration to devise an associative learning network with improved energy efficiency.
Collapse
Affiliation(s)
- Sangheon Kim
- Center for Neuromorphic Engineering, Korea Institute of Science and Technology, Seoul 02792, Rep. of Korea
- Department of Materials Science and Engineering, Korea University, Seoul 02841, Rep. of Korea
| | - Unhyeon Kang
- Center for Neuromorphic Engineering, Korea Institute of Science and Technology, Seoul 02792, Rep. of Korea
- Materials Science & Engineering, Seoul National University, Seoul 08826, Rep. of Korea
| | - Jiyoung Gu
- Center for Neuromorphic Engineering, Korea Institute of Science and Technology, Seoul 02792, Rep. of Korea
- Department of Materials Science & Engineering, Seoul National University of Science and Technology, Seoul 01811, Rep. of Korea
| | - Jaewook Kim
- Center for Neuromorphic Engineering, Korea Institute of Science and Technology, Seoul 02792, Rep. of Korea
| | - Jongkil Park
- Center for Neuromorphic Engineering, Korea Institute of Science and Technology, Seoul 02792, Rep. of Korea
| | - Gyu Weon Hwang
- Center for Neuromorphic Engineering, Korea Institute of Science and Technology, Seoul 02792, Rep. of Korea
| | - Seongsik Park
- Center for Neuromorphic Engineering, Korea Institute of Science and Technology, Seoul 02792, Rep. of Korea
| | - Hyun Jae Jang
- Center for Neuromorphic Engineering, Korea Institute of Science and Technology, Seoul 02792, Rep. of Korea
| | - Tae-Yeon Seong
- Department of Materials Science and Engineering, Korea University, Seoul 02841, Rep. of Korea
| | - Suyoun Lee
- Center for Neuromorphic Engineering, Korea Institute of Science and Technology, Seoul 02792, Rep. of Korea
- Division of Nano & Information Technology, Korea University of Science and Technology, Daejon 34316, Rep. of Korea
| |
Collapse
|
5
|
Sonmez SC, Sevgi M, Antaki F, Huemer J, Keane PA. Generative artificial intelligence in ophthalmology: current innovations, future applications and challenges. Br J Ophthalmol 2024:bjo-2024-325458. [PMID: 38925907 DOI: 10.1136/bjo-2024-325458] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2024] [Accepted: 06/03/2024] [Indexed: 06/28/2024]
Abstract
The rapid advancements in generative artificial intelligence are set to significantly influence the medical sector, particularly ophthalmology. Generative adversarial networks and diffusion models enable the creation of synthetic images, aiding the development of deep learning models tailored for specific imaging tasks. Additionally, the advent of multimodal foundational models, capable of generating images, text and videos, presents a broad spectrum of applications within ophthalmology. These range from enhancing diagnostic accuracy to improving patient education and training healthcare professionals. Despite the promising potential, this area of technology is still in its infancy, and there are several challenges to be addressed, including data bias, safety concerns and the practical implementation of these technologies in clinical settings.
Collapse
Affiliation(s)
| | - Mertcan Sevgi
- Institute of Ophthalmology, University College London, London, UK
- Moorfields Eye Hospital, NIHR Moorfields Biomedical Research Centre, London, UK
| | - Fares Antaki
- Institute of Ophthalmology, University College London, London, UK
- Moorfields Eye Hospital, NIHR Moorfields Biomedical Research Centre, London, UK
- The CHUM School of Artificial Intelligence in Healthcare, Montreal, Quebec, Canada
| | - Josef Huemer
- Moorfields Eye Hospital, NIHR Moorfields Biomedical Research Centre, London, UK
- Department of Ophthalmology and Optometry, Kepler University Hospital, Linz, Austria
| | - Pearse A Keane
- Institute of Ophthalmology, University College London, London, UK
- Moorfields Eye Hospital, NIHR Moorfields Biomedical Research Centre, London, UK
| |
Collapse
|
6
|
Hong N, Whittier DE, Glüer CC, Leslie WD. The potential role for artificial intelligence in fracture risk prediction. Lancet Diabetes Endocrinol 2024:S2213-8587(24)00153-0. [PMID: 38942044 DOI: 10.1016/s2213-8587(24)00153-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/06/2024] [Revised: 05/16/2024] [Accepted: 05/16/2024] [Indexed: 06/30/2024]
Abstract
Osteoporotic fractures are a major health challenge in older adults. Despite the availability of safe and effective therapies for osteoporosis, these therapies are underused in individuals at high risk for fracture, calling for better case-finding and fracture risk assessment strategies. Artificial intelligence (AI) and machine learning (ML) hold promise for enhancing identification of individuals at high risk for fracture by distilling useful features from high-dimensional data derived from medical records, imaging, and wearable devices. AI-ML could enable automated opportunistic screening for vertebral fractures and osteoporosis, home-based monitoring and intervention targeting lifestyle factors, and integration of multimodal features to leverage fracture prediction, ultimately aiding improved fracture risk assessment and individualised treatment. Optimism must be balanced with consideration for the explainability of AI-ML models, biases (including information inequity in numerically under-represented populations), model limitations, and net clinical benefit and workload impact. Clinical integration of AI-ML algorithms has the potential to transform osteoporosis management, offering a more personalised approach to reduce the burden of osteoporotic fractures.
Collapse
Affiliation(s)
- Namki Hong
- Department of Internal Medicine, Endocrine Research Institute, Severance Hospital, Yonsei University College of Medicine, Seoul, Korea; Institute for Innovation in Digital Healthcare, Yonsei University Health System, Seoul, Korea.
| | - Danielle E Whittier
- McCaig Institute for Bone and Joint Health and Alberta Children's Hospital Research Institute, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Claus-C Glüer
- Section Biomedical Imaging, Department of Radiology and Neuroradiology, University Medical Center Schleswig-Holstein, Kiel University, Kiel, Germany
| | - William D Leslie
- Department of Internal Medicine, University of Manitoba, Winnipeg, MB, Canada
| |
Collapse
|
7
|
Ahmad F, Muhmood T. Clinical translation of nanomedicine with integrated digital medicine and machine learning interventions. Colloids Surf B Biointerfaces 2024; 241:114041. [PMID: 38897022 DOI: 10.1016/j.colsurfb.2024.114041] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2024] [Revised: 06/11/2024] [Accepted: 06/13/2024] [Indexed: 06/21/2024]
Abstract
Nanomaterials based therapeutics transform the ways of disease prevention, diagnosis and treatment with increasing sophistications in nanotechnology at a breakneck pace, but very few could reach to the clinic due to inconsistencies in preclinical studies followed by regulatory hinderances. To tackle this, integrating the nanomedicine discovery with digital medicine provide technologies as tools of specific biological activity measurement. Hence, overcome the redundancies in nanomedicine discovery by the on-site data acquisition and analytics through integrating intelligent sensors and artificial intelligence (AI) or machine learning (ML). Integrated AI/ML wearable sensors directly gather clinically relevant biochemical information from the subject's body and process data for physicians to make right clinical decision(s) in a time and cost-effective way. This review summarizes insights and recommend the infusion of actionable big data computation enabled sensors in burgeoning field of nanomedicine at academia, research institutes, and pharmaceutical industries, with a potential of clinical translation. Furthermore, many blind spots are present in modern clinically relevant computation, one of which could prevent ML-guided low-cost new nanomedicine development from being successfully translated into the clinic was also discussed.
Collapse
Affiliation(s)
- Farooq Ahmad
- State Key Laboratory of Chemistry and Utilization of Carbon Based Energy Resources, College of Chemistry, Xinjiang University, Urumqi 830017, China.
| | - Tahir Muhmood
- International Iberian Nanotechnology Laboratory (INL), Avenida Mestre José Veiga, Braga 4715-330, Portugal.
| |
Collapse
|
8
|
Johnson Z, Saikia MJ. Digital Twins for Healthcare Using Wearables. Bioengineering (Basel) 2024; 11:606. [PMID: 38927842 PMCID: PMC11200512 DOI: 10.3390/bioengineering11060606] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2024] [Revised: 05/31/2024] [Accepted: 06/07/2024] [Indexed: 06/28/2024] Open
Abstract
Digital twins are a relatively new form of digital modeling that has been gaining popularity in recent years. This is in large part due to their ability to update in real time to their physical counterparts and connect across multiple devices. As a result, much interest has been directed towards using digital twins in the healthcare industry. Recent advancements in smart wearable technologies have allowed for the utilization of human digital twins in healthcare. Human digital twins can be generated using biometric data from the patient gathered from wearables. These data can then be used to enhance patient care through a variety of means, such as simulated clinical trials, disease prediction, and monitoring treatment progression remotely. This revolutionary method of patient care is still in its infancy, and as such, there is limited research on using wearables to generate human digital twins for healthcare applications. This paper reviews the literature pertaining to human digital twins, including methods, applications, and challenges. The paper also presents a conceptual method for creating human body digital twins using wearable sensors.
Collapse
Affiliation(s)
| | - Manob Jyoti Saikia
- Department of Electrical Engineering, University of North Florida, Jacksonville, FL 32224, USA
| |
Collapse
|
9
|
Wang Z, Gregg B, Du L. Regulatory barriers to US-China collaboration for generative AI development in genomic research. CELL GENOMICS 2024; 4:100564. [PMID: 38795704 DOI: 10.1016/j.xgen.2024.100564] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/29/2023] [Revised: 03/14/2024] [Accepted: 04/24/2024] [Indexed: 05/28/2024]
Abstract
Here, we examine the challenges posed by laws in the United States and China for generative-AI-assisted genomic research collaboration. We recommend renewing the Agreement on Cooperation in Science and Technology to promote responsible principles for sharing human genomic data and to enhance transparency in research.
Collapse
Affiliation(s)
- Zhangyu Wang
- PhD student in International Law, Faculty of Law, University of Macau, Avenida da Universidade, Taipa, Macau, China
| | - Benjamin Gregg
- School of Law, The University of Texas at Austin, 727 E. Dean Keeton Street, Austin, TX 78705, USA
| | - Li Du
- Faculty of Law, University of Macau, Avenida da Universidade, Taipa, Macau, China.
| |
Collapse
|
10
|
Curion F, Theis FJ. Machine learning integrative approaches to advance computational immunology. Genome Med 2024; 16:80. [PMID: 38862979 PMCID: PMC11165829 DOI: 10.1186/s13073-024-01350-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2023] [Accepted: 05/23/2024] [Indexed: 06/13/2024] Open
Abstract
The study of immunology, traditionally reliant on proteomics to evaluate individual immune cells, has been revolutionized by single-cell RNA sequencing. Computational immunologists play a crucial role in analysing these datasets, moving beyond traditional protein marker identification to encompass a more detailed view of cellular phenotypes and their functional roles. Recent technological advancements allow the simultaneous measurements of multiple cellular components-transcriptome, proteome, chromatin, epigenetic modifications and metabolites-within single cells, including in spatial contexts within tissues. This has led to the generation of complex multiscale datasets that can include multimodal measurements from the same cells or a mix of paired and unpaired modalities. Modern machine learning (ML) techniques allow for the integration of multiple "omics" data without the need for extensive independent modelling of each modality. This review focuses on recent advancements in ML integrative approaches applied to immunological studies. We highlight the importance of these methods in creating a unified representation of multiscale data collections, particularly for single-cell and spatial profiling technologies. Finally, we discuss the challenges of these holistic approaches and how they will be instrumental in the development of a common coordinate framework for multiscale studies, thereby accelerating research and enabling discoveries in the computational immunology field.
Collapse
Affiliation(s)
- Fabiola Curion
- Institute of Computational Biology, Helmholtz Center Munich, Munich, Germany
- Department of Mathematics, School of Computation, Information and Technology, Technical University of Munich, Munich, Germany
| | - Fabian J Theis
- Institute of Computational Biology, Helmholtz Center Munich, Munich, Germany.
- Department of Mathematics, School of Computation, Information and Technology, Technical University of Munich, Munich, Germany.
- School of Life Sciences Weihenstephan, Technical University of Munich, Munich, Germany.
| |
Collapse
|
11
|
Sezgin E, McKay I. Behavioral health and generative AI: a perspective on future of therapies and patient care. NPJ MENTAL HEALTH RESEARCH 2024; 3:25. [PMID: 38849499 PMCID: PMC11161641 DOI: 10.1038/s44184-024-00067-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/27/2023] [Accepted: 04/06/2024] [Indexed: 06/09/2024]
Affiliation(s)
- Emre Sezgin
- The Abigail Wexner Research Institute at Nationwide Children's Hospital, Columbus, OH, USA.
- The Ohio State University College of Medicine, Columbus, OH, USA.
| | - Ian McKay
- The Ohio State University College of Medicine, Columbus, OH, USA
- Department of Psychiatry and Behavioral Health, Nationwide Children's Hospital, Columbus, OH, USA
| |
Collapse
|
12
|
Kikuchi T, Hanaoka S, Nakao T, Takenaga T, Nomura Y, Mori H, Yoshikawa T. Synthesis of Hybrid Data Consisting of Chest Radiographs and Tabular Clinical Records Using Dual Generative Models for COVID-19 Positive Cases. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024; 37:1217-1227. [PMID: 38351224 PMCID: PMC11169290 DOI: 10.1007/s10278-024-01015-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/26/2023] [Revised: 12/21/2023] [Accepted: 12/22/2023] [Indexed: 06/13/2024]
Abstract
To generate synthetic medical data incorporating image-tabular hybrid data by merging an image encoding/decoding model with a table-compatible generative model and assess their utility. We used 1342 cases from the Stony Brook University Covid-19-positive cases, comprising chest X-ray radiographs (CXRs) and tabular clinical data as a private dataset (pDS). We generated a synthetic dataset (sDS) through the following steps: (I) dimensionally reducing CXRs in the pDS using a pretrained encoder of the auto-encoding generative adversarial networks (αGAN) and integrating them with the correspondent tabular clinical data; (II) training the conditional tabular GAN (CTGAN) on this combined data to generate synthetic records, encompassing encoded image features and clinical data; and (III) reconstructing synthetic images from these encoded image features in the sDS using a pretrained decoder of the αGAN. The utility of sDS was assessed by the performance of the prediction models for patient outcomes (deceased or discharged). For the pDS test set, the area under the receiver operating characteristic (AUC) curve was calculated to compare the performance of prediction models trained separately with pDS, sDS, or a combination of both. We created an sDS comprising CXRs with a resolution of 256 × 256 pixels and tabular data containing 13 variables. The AUC for the outcome was 0.83 when the model was trained with the pDS, 0.74 with the sDS, and 0.87 when combining pDS and sDS for training. Our method is effective for generating synthetic records consisting of both images and tabular clinical data.
Collapse
Affiliation(s)
- Tomohiro Kikuchi
- Department of Computational Diagnostic Radiology and Preventive Medicine, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8655, Japan.
- Department of Radiology, School of Medicine, Jichi Medical University, 3311-1 Yakushiji, Shimotsuke, Tochigi, 329-0498, Japan.
| | - Shouhei Hanaoka
- Department of Radiology, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo, Japan
| | - Takahiro Nakao
- Department of Computational Diagnostic Radiology and Preventive Medicine, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8655, Japan
| | - Tomomi Takenaga
- Department of Radiology, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo, Japan
| | - Yukihiro Nomura
- Department of Computational Diagnostic Radiology and Preventive Medicine, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8655, Japan
- Center for Frontier Medical Engineering, Chiba University, 1-33 Yayoi-cho, Inage-ku, Chiba, 263-8522, Japan
| | - Harushi Mori
- Department of Radiology, School of Medicine, Jichi Medical University, 3311-1 Yakushiji, Shimotsuke, Tochigi, 329-0498, Japan
| | - Takeharu Yoshikawa
- Department of Computational Diagnostic Radiology and Preventive Medicine, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8655, Japan
| |
Collapse
|
13
|
Levenson RM, Singh Y, Rieck B, Hathaway QA, Farrelly C, Rozenblit J, Prasanna P, Erickson B, Choudhary A, Carlsson G, Sarkar D. Advancing Precision Medicine: Algebraic Topology and Differential Geometry in Radiology and Computational Pathology. J Transl Med 2024; 104:102060. [PMID: 38626875 DOI: 10.1016/j.labinv.2024.102060] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2023] [Revised: 04/08/2024] [Accepted: 04/10/2024] [Indexed: 05/19/2024] Open
Abstract
Precision medicine aims to provide personalized care based on individual patient characteristics, rather than guideline-directed therapies for groups of diseases or patient demographics. Images-both radiology- and pathology-derived-are a major source of information on presence, type, and status of disease. Exploring the mathematical relationship of pixels in medical imaging ("radiomics") and cellular-scale structures in digital pathology slides ("pathomics") offers powerful tools for extracting both qualitative and, increasingly, quantitative data. These analytical approaches, however, may be significantly enhanced by applying additional methods arising from fields of mathematics such as differential geometry and algebraic topology that remain underexplored in this context. Geometry's strength lies in its ability to provide precise local measurements, such as curvature, that can be crucial for identifying abnormalities at multiple spatial levels. These measurements can augment the quantitative features extracted in conventional radiomics, leading to more nuanced diagnostics. By contrast, topology serves as a robust shape descriptor, capturing essential features such as connected components and holes. The field of topological data analysis was initially founded to explore the shape of data, with functional network connectivity in the brain being a prominent example. Increasingly, its tools are now being used to explore organizational patterns of physical structures in medical images and digitized pathology slides. By leveraging tools from both differential geometry and algebraic topology, researchers and clinicians may be able to obtain a more comprehensive, multi-layered understanding of medical images and contribute to precision medicine's armamentarium.
Collapse
Affiliation(s)
- Richard M Levenson
- Department of Pathology and Laboratory Medicine, University of California Davis, Davis, California.
| | - Yashbir Singh
- Department of Radiology, Mayo Clinic, Rochester, Minnesota.
| | - Bastian Rieck
- Helmholtz Munich and Technical University of Munich, Munich, Germany
| | - Quincy A Hathaway
- Department of Medical Education, West Virginia University, Morgantown, West Virginia
| | | | | | - Prateek Prasanna
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, New York
| | | | | | - Gunnar Carlsson
- Department of Mathematics, Stanford University, Stanford, California
| | - Deepa Sarkar
- Institute of Genomic Health, Ichan school of Medicine, Mount Sinai, New York
| |
Collapse
|
14
|
Ganjizadeh A, Zawada SJ, Langer SG, Erickson BJ. Visualizing Clinical Data Retrieval and Curation in Multimodal Healthcare AI Research: A Technical Note on RIL-workflow. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024; 37:1239-1247. [PMID: 38366291 PMCID: PMC11169146 DOI: 10.1007/s10278-024-00977-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/19/2023] [Revised: 12/02/2023] [Accepted: 12/04/2023] [Indexed: 02/18/2024]
Abstract
Curating and integrating data from sources are bottlenecks to procuring robust training datasets for artificial intelligence (AI) models in healthcare. While numerous applications can process discrete types of clinical data, it is still time-consuming to integrate heterogenous data types. Therefore, there exists a need for more efficient retrieval and storage of curated patient data from dissimilar sources, such as biobanks, health records, and sensors. We describe a customizable, modular data retrieval application (RIL-workflow), which integrates clinical notes, images, and prescription data, and show its feasibility applied to research at our institution. It uses the workflow automation platform Camunda (Camunda Services GmbH, Berlin, Germany) to collect internal data from Fast Healthcare Interoperability Resources (FHIR) and Digital Imaging and Communications in Medicine (DICOM) sources. Using the web-based graphical user interface (GUI), the workflow runs tasks to completion according to visual representation, retrieving and storing results for patients meeting study inclusion criteria while segregating errors for human review. We showcase RIL-workflow with its library of ready-to-use modules, enabling researchers to specify human input or automation at fixed steps. We validated our workflow by demonstrating its capability to aggregate, curate, and handle errors related to data from multiple sources to generate a multimodal database for clinical AI research. Further, we solicited user feedback to highlight the pros and cons associated with RIL-workflow. The source code is available at github.com/magnooj/RIL-workflow.
Collapse
Affiliation(s)
- Ali Ganjizadeh
- Mayo Clinic Artificial Intelligence Laboratory, 200 1st Street SW, Rochester, MN, 55902, USA
- Mayo Clinic Department of Radiology, 200 1st Street SW, Rochester, MN, 55902, USA
| | - Stephanie J Zawada
- Mayo Clinic Artificial Intelligence Laboratory, 200 1st Street SW, Rochester, MN, 55902, USA
- Mayo Clinic College of Medicine and Science, 5777 E. Mayo Boulevard, Scottsdale, AZ, 85054, USA
| | - Steve G Langer
- Mayo Clinic Artificial Intelligence Laboratory, 200 1st Street SW, Rochester, MN, 55902, USA
- Mayo Clinic Department of Radiology, 200 1st Street SW, Rochester, MN, 55902, USA
| | - Bradley J Erickson
- Mayo Clinic Artificial Intelligence Laboratory, 200 1st Street SW, Rochester, MN, 55902, USA.
- Mayo Clinic Department of Radiology, 200 1st Street SW, Rochester, MN, 55902, USA.
- Mayo Clinic College of Medicine and Science, 5777 E. Mayo Boulevard, Scottsdale, AZ, 85054, USA.
| |
Collapse
|
15
|
Xie W, Yu J, Huang L, For LS, Zheng Z, Chen X, Wang Y, Liu Z, Peng C, Wong KC. DeepSeq2Drug: An expandable ensemble end-to-end anti-viral drug repurposing benchmark framework by multi-modal embeddings and transfer learning. Comput Biol Med 2024; 175:108487. [PMID: 38653064 DOI: 10.1016/j.compbiomed.2024.108487] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2024] [Revised: 03/26/2024] [Accepted: 04/15/2024] [Indexed: 04/25/2024]
Abstract
Drug repurposing is promising in multiple scenarios, such as emerging viral outbreak controls and cost reductions of drug discovery. Traditional graph-based drug repurposing methods are limited to fast, large-scale virtual screens, as they constrain the counts for drugs and targets and fail to predict novel viruses or drugs. Moreover, though deep learning has been proposed for drug repurposing, only a few methods have been used, including a group of pre-trained deep learning models for embedding generation and transfer learning. Hence, we propose DeepSeq2Drug to tackle the shortcomings of previous methods. We leverage multi-modal embeddings and an ensemble strategy to complement the numbers of drugs and viruses and to guarantee the novel prediction. This framework (including the expanded version) involves four modal types: six NLP models, four CV models, four graph models, and two sequence models. In detail, we first make a pipeline and calculate the predictive performance of each pair of viral and drug embeddings. Then, we select the best embedding pairs and apply an ensemble strategy to conduct anti-viral drug repurposing. To validate the effect of the proposed ensemble model, a monkeypox virus (MPV) case study is conducted to reflect the potential predictive capability. This framework could be a benchmark method for further pre-trained deep learning optimization and anti-viral drug repurposing tasks. We also build software further to make the proposed model easier to reuse. The code and software are freely available at http://deepseq2drug.cs.cityu.edu.hk.
Collapse
Affiliation(s)
- Weidun Xie
- Department of Computer Science, City University of Hong Kong, Kowloon Tong, Hong Kong SAR, China
| | - Jixiang Yu
- Department of Computer Science, City University of Hong Kong, Kowloon Tong, Hong Kong SAR, China
| | - Lei Huang
- Department of Computer Science, City University of Hong Kong, Kowloon Tong, Hong Kong SAR, China
| | - Lek Shyuen For
- Department of Computer Science, City University of Hong Kong, Kowloon Tong, Hong Kong SAR, China
| | - Zetian Zheng
- Department of Computer Science, City University of Hong Kong, Kowloon Tong, Hong Kong SAR, China
| | - Xingjian Chen
- Cutaneous Biology Research Center, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Yuchen Wang
- Department of Computer Science, City University of Hong Kong, Kowloon Tong, Hong Kong SAR, China
| | - Zhichao Liu
- Sir William Dunn School of Pathology, University of Oxford, UK
| | - Chengbin Peng
- College of Information Science and Engineering, Ningbo University, Ningbo, China
| | - Ka-Chun Wong
- Department of Computer Science, City University of Hong Kong, Kowloon Tong, Hong Kong SAR, China; Shenzhen Research Institute, City University of Hong Kong, Shenzhen, China; Hong Kong Institute for Data Science, City University of Hong Kong, Kowloon Tong, Hong Kong SAR, China.
| |
Collapse
|
16
|
Bozzo A, Tsui JMG, Bhatnagar S, Forsberg J. Deep Learning and Multimodal Artificial Intelligence in Orthopaedic Surgery. J Am Acad Orthop Surg 2024; 32:e523-e532. [PMID: 38652882 PMCID: PMC11075751 DOI: 10.5435/jaaos-d-23-00831] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/13/2023] [Revised: 02/13/2024] [Accepted: 03/01/2024] [Indexed: 04/25/2024] Open
Abstract
This review article focuses on the applications of deep learning with neural networks and multimodal neural networks in the orthopaedic domain. By providing practical examples of how artificial intelligence (AI) is being applied successfully in orthopaedic surgery, particularly in the realm of imaging data sets and the integration of clinical data, this study aims to provide orthopaedic surgeons with the necessary tools to not only evaluate existing literature but also to consider AI's potential in their own clinical or research pursuits. We first review standard deep neural networks which can analyze numerical clinical variables, then describe convolutional neural networks which can analyze image data, and then introduce multimodal AI models which analyze various types of different data. Then, we contrast these deep learning techniques with related but more limited techniques such as radiomics, describe how to interpret deep learning studies, and how to initiate such studies at your institution. Ultimately, by empowering orthopaedic surgeons with the knowledge and know-how of deep learning, this review aspires to facilitate the translation of research into clinical practice, thereby enhancing the efficacy and precision of real-world orthopaedic care for patients.
Collapse
Affiliation(s)
- Anthony Bozzo
- From the Division of Orthopaedic Surgery, McGill University, Canada (Bozzo), the Division of Radiation Oncology, McGill University, Canada (Tsui), the Department of Epidemiology and Biostatistics, Department of Diagnostic Radiology, McGill University, Canada (Bhatnagar), and the Memorial Sloan Kettering Cancer Center (Forsberg)
| | - James M. G. Tsui
- From the Division of Orthopaedic Surgery, McGill University, Canada (Bozzo), the Division of Radiation Oncology, McGill University, Canada (Tsui), the Department of Epidemiology and Biostatistics, Department of Diagnostic Radiology, McGill University, Canada (Bhatnagar), and the Memorial Sloan Kettering Cancer Center (Forsberg)
| | - Sahir Bhatnagar
- From the Division of Orthopaedic Surgery, McGill University, Canada (Bozzo), the Division of Radiation Oncology, McGill University, Canada (Tsui), the Department of Epidemiology and Biostatistics, Department of Diagnostic Radiology, McGill University, Canada (Bhatnagar), and the Memorial Sloan Kettering Cancer Center (Forsberg)
| | - Jonathan Forsberg
- From the Division of Orthopaedic Surgery, McGill University, Canada (Bozzo), the Division of Radiation Oncology, McGill University, Canada (Tsui), the Department of Epidemiology and Biostatistics, Department of Diagnostic Radiology, McGill University, Canada (Bhatnagar), and the Memorial Sloan Kettering Cancer Center (Forsberg)
| |
Collapse
|
17
|
Nordblom N, Büttner M, Schwendicke F. Artificial Intelligence in Orthodontics: Critical Review. J Dent Res 2024; 103:577-584. [PMID: 38682436 PMCID: PMC11118788 DOI: 10.1177/00220345241235606] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/01/2024] Open
Abstract
With increasing digitalization in orthodontics, certain orthodontic manufacturing processes such as the fabrication of indirect bonding trays, aligner production, or wire bending can be automated. However, orthodontic treatment planning and evaluation remains a specialist's task and responsibility. As the prediction of growth in orthodontic patients and response to orthodontic treatment is inherently complex and individual, orthodontists make use of features gathered from longitudinal, multimodal, and standardized orthodontic data sets. Currently, these data sets are used by the orthodontist to make informed, rule-based treatment decisions. In research, artificial intelligence (AI) has been successfully applied to assist orthodontists with the extraction of relevant data from such data sets. Here, AI has been applied for the analysis of clinical imagery, such as automated landmark detection in lateral cephalograms but also for evaluation of intraoral scans or photographic data. Furthermore, AI is applied to help orthodontists with decision support for treatment decisions such as the need for orthognathic surgery or for orthodontic tooth extractions. One major challenge in current AI research in orthodontics is the limited generalizability, as most studies use unicentric data with high risks of bias. Moreover, comparing AI across different studies and tasks is virtually impossible as both outcomes and outcome metrics vary widely, and underlying data sets are not standardized. Notably, only few AI applications in orthodontics have reached full clinical maturity and regulatory approval, and researchers in the field are tasked with tackling real-world evaluation and implementation of AI into the orthodontic workflow.
Collapse
Affiliation(s)
- N.F. Nordblom
- Department of Oral Diagnostics, Digital Health and Health Services Research, Charité–Universitätsmedizin Berlin, Berlin, Germany
| | - M. Büttner
- Department of Oral Diagnostics, Digital Health and Health Services Research, Charité–Universitätsmedizin Berlin, Berlin, Germany
| | - F. Schwendicke
- Department of Conservative Dentistry and Periodontology, University Hospital, Ludwig-Maximilians University of Munich, Munich, Germany
| |
Collapse
|
18
|
Bharel M, Auerbach J, Nguyen V, DeSalvo KB. Transforming Public Health Practice With Generative Artificial Intelligence. Health Aff (Millwood) 2024; 43:776-782. [PMID: 38830160 DOI: 10.1377/hlthaff.2024.00050] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/05/2024]
Abstract
Public health practice appears poised to undergo a transformative shift as a result of the latest advancements in artificial intelligence (AI). These changes will usher in a new era of public health, charged with responding to deficiencies identified during the COVID-19 pandemic and managing investments required to meet the health needs of the twenty-first century. In this Commentary, we explore how AI is being used in public health, and we describe the advanced capabilities of generative AI models capable of producing synthetic content such as images, videos, audio, text, and other digital content. Viewing the use of AI from the perspective of health departments in the United States, we examine how this new technology can support core public health functions with a focus on near-term opportunities to improve communication, optimize organizational performance, and generate novel insights to drive decision making. Finally, we review the challenges and risks associated with these technologies, offering suggestions for health officials to harness the new tools to accomplish public health goals.
Collapse
|
19
|
Yu KH, Healey E, Leong TY, Kohane IS, Manrai AK. Medical Artificial Intelligence and Human Values. N Engl J Med 2024; 390:1895-1904. [PMID: 38810186 DOI: 10.1056/nejmra2214183] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 05/31/2024]
Affiliation(s)
- Kun-Hsing Yu
- From the Department of Biomedical Informatics, Harvard Medical School (K.-H.Y., E.H., I.S.K., A.K.M.), the Department of Pathology, Brigham and Women's Hospital (K.-H.Y.), and the Harvard-MIT Division of Health Sciences and Technology (E.H.) - all in Boston; and the School of Computing, National University of Singapore, Singapore (T.-Y.L.)
| | - Elizabeth Healey
- From the Department of Biomedical Informatics, Harvard Medical School (K.-H.Y., E.H., I.S.K., A.K.M.), the Department of Pathology, Brigham and Women's Hospital (K.-H.Y.), and the Harvard-MIT Division of Health Sciences and Technology (E.H.) - all in Boston; and the School of Computing, National University of Singapore, Singapore (T.-Y.L.)
| | - Tze-Yun Leong
- From the Department of Biomedical Informatics, Harvard Medical School (K.-H.Y., E.H., I.S.K., A.K.M.), the Department of Pathology, Brigham and Women's Hospital (K.-H.Y.), and the Harvard-MIT Division of Health Sciences and Technology (E.H.) - all in Boston; and the School of Computing, National University of Singapore, Singapore (T.-Y.L.)
| | - Isaac S Kohane
- From the Department of Biomedical Informatics, Harvard Medical School (K.-H.Y., E.H., I.S.K., A.K.M.), the Department of Pathology, Brigham and Women's Hospital (K.-H.Y.), and the Harvard-MIT Division of Health Sciences and Technology (E.H.) - all in Boston; and the School of Computing, National University of Singapore, Singapore (T.-Y.L.)
| | - Arjun K Manrai
- From the Department of Biomedical Informatics, Harvard Medical School (K.-H.Y., E.H., I.S.K., A.K.M.), the Department of Pathology, Brigham and Women's Hospital (K.-H.Y.), and the Harvard-MIT Division of Health Sciences and Technology (E.H.) - all in Boston; and the School of Computing, National University of Singapore, Singapore (T.-Y.L.)
| |
Collapse
|
20
|
Stevens J, Tezel O, Bonnefil V, Hapstack M, Atreya MR. Biological basis of critical illness subclasses: from the bedside to the bench and back again. Crit Care 2024; 28:186. [PMID: 38812006 PMCID: PMC11137966 DOI: 10.1186/s13054-024-04959-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2024] [Accepted: 05/17/2024] [Indexed: 05/31/2024] Open
Abstract
Critical illness syndromes including sepsis, acute respiratory distress syndrome, and acute kidney injury (AKI) are associated with high in-hospital mortality and long-term adverse health outcomes among survivors. Despite advancements in care, clinical and biological heterogeneity among patients continues to hamper identification of efficacious therapies. Precision medicine offers hope by identifying patient subclasses based on clinical, laboratory, biomarker and 'omic' data and potentially facilitating better alignment of interventions. Within the previous two decades, numerous studies have made strides in identifying gene-expression based endotypes and clinico-biomarker based phenotypes among critically ill patients associated with differential outcomes and responses to treatment. In this state-of-the-art review, we summarize the biological similarities and differences across the various subclassification schemes among critically ill patients. In addition, we highlight current translational gaps, the need for advanced scientific tools, human-relevant disease models, to gain a comprehensive understanding of the molecular mechanisms underlying critical illness subclasses.
Collapse
Affiliation(s)
- Joseph Stevens
- Division of Immunobiology, Graduate Program, College of Medicine, University of Cincinnati, Cincinnati, OH, 45267, USA
| | - Oğuzhan Tezel
- Division of Critical Care Medicine, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, 45229, USA
| | - Valentina Bonnefil
- Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH, 45627, USA
| | - Matthew Hapstack
- Division of Critical Care Medicine, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, 45229, USA
| | - Mihir R Atreya
- Division of Critical Care Medicine, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, 45229, USA.
- Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH, 45627, USA.
| |
Collapse
|
21
|
Lei B, Li Y, Fu W, Yang P, Chen S, Wang T, Xiao X, Niu T, Fu Y, Wang S, Han H, Qin J. Alzheimer's disease diagnosis from multi-modal data via feature inductive learning and dual multilevel graph neural network. Med Image Anal 2024; 97:103213. [PMID: 38850625 DOI: 10.1016/j.media.2024.103213] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2023] [Revised: 05/13/2024] [Accepted: 05/17/2024] [Indexed: 06/10/2024]
Abstract
Multi-modal data can provide complementary information of Alzheimer's disease (AD) and its development from different perspectives. Such information is closely related to the diagnosis, prevention, and treatment of AD, and hence it is necessary and critical to study AD through multi-modal data. Existing learning methods, however, usually ignore the influence of feature heterogeneity and directly fuse features in the last stages. Furthermore, most of these methods only focus on local fusion features or global fusion features, neglecting the complementariness of features at different levels and thus not sufficiently leveraging information embedded in multi-modal data. To overcome these shortcomings, we propose a novel framework for AD diagnosis that fuses gene, imaging, protein, and clinical data. Our framework learns feature representations under the same feature space for different modalities through a feature induction learning (FIL) module, thereby alleviating the impact of feature heterogeneity. Furthermore, in our framework, local and global salient multi-modal feature interaction information at different levels is extracted through a novel dual multilevel graph neural network (DMGNN). We extensively validate the proposed method on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset and experimental results demonstrate our method consistently outperforms other state-of-the-art multi-modal fusion methods. The code is publicly available on the GitHub website. (https://github.com/xiankantingqianxue/MIA-code.git).
Collapse
Affiliation(s)
- Baiying Lei
- National-Regional Key Technology Engineering Lab. for Medical Ultrasound, Guangdong Key Lab. for Biomedical Measurements and Ultrasound Imaging, Marshall Lab. of Biomedical Engineering, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, 518060, China
| | - Yafeng Li
- National-Regional Key Technology Engineering Lab. for Medical Ultrasound, Guangdong Key Lab. for Biomedical Measurements and Ultrasound Imaging, Marshall Lab. of Biomedical Engineering, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, 518060, China
| | - Wanyi Fu
- Department of Electronic Engineering, Tsinghua University, Beijing Key Laboratory of Magnetic Resonance Imaging Devices and Technology, China
| | - Peng Yang
- National-Regional Key Technology Engineering Lab. for Medical Ultrasound, Guangdong Key Lab. for Biomedical Measurements and Ultrasound Imaging, Marshall Lab. of Biomedical Engineering, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, 518060, China
| | - Shaobin Chen
- National-Regional Key Technology Engineering Lab. for Medical Ultrasound, Guangdong Key Lab. for Biomedical Measurements and Ultrasound Imaging, Marshall Lab. of Biomedical Engineering, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, 518060, China
| | - Tianfu Wang
- National-Regional Key Technology Engineering Lab. for Medical Ultrasound, Guangdong Key Lab. for Biomedical Measurements and Ultrasound Imaging, Marshall Lab. of Biomedical Engineering, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, 518060, China
| | - Xiaohua Xiao
- The First Affiliated Hospital of Shenzhen University, Shenzhen University Medical School, Shenzhen University, Shenzhen Second People's Hospital, Shenzhen, 530031, China
| | - Tianye Niu
- Shenzhen Bay Laboratory, Shenzhen, 518067, China
| | - Yu Fu
- Department of Neurology, Peking University Third Hospital, No. 49, North Garden Rd., Haidian District, Beijing, 100191, China.
| | - Shuqiang Wang
- Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055, China.
| | - Hongbin Han
- Institute of Medical Technology, Peking University Health Science Center, Department of Radiology, Peking University Third Hospital, Beijing Key Laboratory of Magnetic Resonance Imaging Devices and Technology, Beijing, 100191, China; The second hospital of Dalian Medical University,Research and developing center of medical technology, Dalian, 116027, China.
| | - Jing Qin
- Center for Smart Health, School of Nursing, The Hong Kong Polytechnic University, Hong Kong, China
| |
Collapse
|
22
|
Kong SW, Lee IH, Collen LV, Manrai AK, Snapper SB, Mandl KD. Discordance between a deep learning model and clinical-grade variant pathogenicity classification in a rare disease cohort. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2024:2024.05.22.24307756. [PMID: 38826236 PMCID: PMC11142383 DOI: 10.1101/2024.05.22.24307756] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/04/2024]
Abstract
Genetic testing has become an essential component in the diagnosis and management of a wide range of clinical conditions, from cancer to developmental disorders, especially in rare Mendelian diseases. Efforts to identify rare phenotype-associated variants have predominantly focused on protein-truncating variants, while the interpretation of missense variants presents a considerable challenge. Deep learning algorithms excel in various applications across biomedical tasks1,2, yet accurately distinguishing between pathogenic and benign genetic variants remains an elusive goal3-5. Specifically, even the most sophisticated models encounter difficulties in accurately assessing the pathogenicity of missense variants of uncertain significance (VUS). Our investigation of AlphaMissense (AM)5, the latest iteration of deep learning methods for predicting the potential functional impact of missense variants and assessing gene essentiality, reveals important limitations in its ability to identify pathogenic missense variants within a rare disease cohort. Indeed, AM struggles to accurately assess the pathogenicity of variants in intrinsically disordered regions (IDRs), leading to unreliable gene-level essentiality scores for certain genes containing IDRs. This limitation highlights the challenges in applying AM faces in the context of clinical genetics6.
Collapse
Affiliation(s)
- Sek Won Kong
- Computational Health Informatics Program, Boston Children’s Hospital, Boston, MA 02215
- Department of Pediatrics, Harvard Medical School, Boston, MA 02115
| | - In-Hee Lee
- Computational Health Informatics Program, Boston Children’s Hospital, Boston, MA 02215
| | - Lauren V. Collen
- Department of Pediatrics, Harvard Medical School, Boston, MA 02115
- Division of Gastroenterology, Hepatology, and Nutrition, Boston Children’s Hospital, Boston, MA 02215
| | - Arjun K. Manrai
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA 02115
| | - Scott B. Snapper
- Department of Pediatrics, Harvard Medical School, Boston, MA 02115
- Division of Gastroenterology, Hepatology, and Nutrition, Boston Children’s Hospital, Boston, MA 02215
| | - Kenneth D. Mandl
- Computational Health Informatics Program, Boston Children’s Hospital, Boston, MA 02215
- Department of Pediatrics, Harvard Medical School, Boston, MA 02115
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA 02115
| |
Collapse
|
23
|
Nakhod V, Krivenko A, Butkova T, Malsagova K, Kaysheva A. Advances in Molecular and Genetic Technologies and the Problems Related to Their Application in Personalized Medicine. J Pers Med 2024; 14:555. [PMID: 38929775 PMCID: PMC11204801 DOI: 10.3390/jpm14060555] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2024] [Revised: 05/03/2024] [Accepted: 05/10/2024] [Indexed: 06/28/2024] Open
Abstract
Advances in the global personalized medicine market are directly related to innovations and developments in molecular and genetic technologies. This review focuses on the key trends in the development of these technologies in the healthcare sector. The existing global developments having an impact on the evolution of the personalized medicine market are reviewed. Efficient measures to support the development of molecular and genetic technologies are proposed.
Collapse
Affiliation(s)
- Valeriya Nakhod
- Institute of Biomedical Chemistry, 10 bld. 8, Pogodinskaya str., 119121 Moscow, Russia
| | | | | | | | | |
Collapse
|
24
|
Zilker S, Weinzierl S, Kraus M, Zschech P, Matzner M. A machine learning framework for interpretable predictions in patient pathways: The case of predicting ICU admission for patients with symptoms of sepsis. Health Care Manag Sci 2024:10.1007/s10729-024-09673-8. [PMID: 38771522 DOI: 10.1007/s10729-024-09673-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2023] [Accepted: 04/13/2024] [Indexed: 05/22/2024]
Abstract
Proactive analysis of patient pathways helps healthcare providers anticipate treatment-related risks, identify outcomes, and allocate resources. Machine learning (ML) can leverage a patient's complete health history to make informed decisions about future events. However, previous work has mostly relied on so-called black-box models, which are unintelligible to humans, making it difficult for clinicians to apply such models. Our work introduces PatWay-Net, an ML framework designed for interpretable predictions of admission to the intensive care unit (ICU) for patients with symptoms of sepsis. We propose a novel type of recurrent neural network and combine it with multi-layer perceptrons to process the patient pathways and produce predictive yet interpretable results. We demonstrate its utility through a comprehensive dashboard that visualizes patient health trajectories, predictive outcomes, and associated risks. Our evaluation includes both predictive performance - where PatWay-Net outperforms standard models such as decision trees, random forests, and gradient-boosted decision trees - and clinical utility, validated through structured interviews with clinicians. By providing improved predictive accuracy along with interpretable and actionable insights, PatWay-Net serves as a valuable tool for healthcare decision support in the critical case of patients with symptoms of sepsis.
Collapse
Affiliation(s)
- Sandra Zilker
- Technische Hochschule Nürnberg Georg Simon Ohm, Professorship for Business Analytics, Hohfederstraße 40, 90489, Nuremberg, Germany.
- Friedrich-Alexander-Universität Erlangen-Nürnberg, Chair of Digital Industrial Service Systems, Fürther Straße 248, 90429, Nuremberg, Germany.
| | - Sven Weinzierl
- Friedrich-Alexander-Universität Erlangen-Nürnberg, Chair of Digital Industrial Service Systems, Fürther Straße 248, 90429, Nuremberg, Germany
| | - Mathias Kraus
- University of Regensburg, Chair for Explainable AI in Business Value Creation, Bajuwarenstraße 4, 93053, Regensburg, Germany
| | - Patrick Zschech
- Leipzig University, Professorship for Intelligent Information Systems and Processes, Grimmaische Straße 12, 04109, Leipzig, Germany
| | - Martin Matzner
- Friedrich-Alexander-Universität Erlangen-Nürnberg, Chair of Digital Industrial Service Systems, Fürther Straße 248, 90429, Nuremberg, Germany
| |
Collapse
|
25
|
Demuth S, Paris J, Faddeenkov I, De Sèze J, Gourraud PA. Clinical applications of deep learning in neuroinflammatory diseases: A scoping review. Rev Neurol (Paris) 2024:S0035-3787(24)00522-8. [PMID: 38772806 DOI: 10.1016/j.neurol.2024.04.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2024] [Revised: 03/26/2024] [Accepted: 04/09/2024] [Indexed: 05/23/2024]
Abstract
BACKGROUND Deep learning (DL) is an artificial intelligence technology that has aroused much excitement for predictive medicine due to its ability to process raw data modalities such as images, text, and time series of signals. OBJECTIVES Here, we intend to give the clinical reader elements to understand this technology, taking neuroinflammatory diseases as an illustrative use case of clinical translation efforts. We reviewed the scope of this rapidly evolving field to get quantitative insights about which clinical applications concentrate the efforts and which data modalities are most commonly used. METHODS We queried the PubMed database for articles reporting DL algorithms for clinical applications in neuroinflammatory diseases and the radiology.healthairegister.com website for commercial algorithms. RESULTS The review included 148 articles published between 2018 and 2024 and five commercial algorithms. The clinical applications could be grouped as computer-aided diagnosis, individual prognosis, functional assessment, the segmentation of radiological structures, and the optimization of data acquisition. Our review highlighted important discrepancies in efforts. The segmentation of radiological structures and computer-aided diagnosis currently concentrate most efforts with an overrepresentation of imaging. Various model architectures have addressed different applications, relatively low volume of data, and diverse data modalities. We report the high-level technical characteristics of the algorithms and synthesize narratively the clinical applications. Predictive performances and some common a priori on this topic are finally discussed. CONCLUSION The currently reported efforts position DL as an information processing technology, enhancing existing modalities of paraclinical investigations and bringing perspectives to make innovative ones actionable for healthcare.
Collapse
Affiliation(s)
- S Demuth
- Inserm U1064, CR2TI - Center for Research in Transplantation and Translational Immunology, Nantes University, 44000 Nantes, France; Inserm U1119 : biopathologie de la myéline, neuroprotection et stratégies thérapeutiques, University of Strasbourg, 1, rue Eugène-Boeckel - CS 60026, 67084 Strasbourg, France.
| | - J Paris
- Inserm U1064, CR2TI - Center for Research in Transplantation and Translational Immunology, Nantes University, 44000 Nantes, France
| | - I Faddeenkov
- Inserm U1064, CR2TI - Center for Research in Transplantation and Translational Immunology, Nantes University, 44000 Nantes, France
| | - J De Sèze
- Inserm U1119 : biopathologie de la myéline, neuroprotection et stratégies thérapeutiques, University of Strasbourg, 1, rue Eugène-Boeckel - CS 60026, 67084 Strasbourg, France; Department of Neurology, University Hospital of Strasbourg, 1, avenue Molière, 67200 Strasbourg, France; Inserm CIC 1434 Clinical Investigation Center, University Hospital of Strasbourg, 1, avenue Molière, 67200 Strasbourg, France
| | - P-A Gourraud
- Inserm U1064, CR2TI - Center for Research in Transplantation and Translational Immunology, Nantes University, 44000 Nantes, France; "Data clinic", Department of Public Health, University Hospital of Nantes, Nantes, France
| |
Collapse
|
26
|
Liu X, Shen H, Zhang L, Huang W, Zhang S, Zhang B. Immunotherapy for recurrent or metastatic nasopharyngeal carcinoma. NPJ Precis Oncol 2024; 8:101. [PMID: 38755255 PMCID: PMC11099100 DOI: 10.1038/s41698-024-00601-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/01/2024] [Accepted: 05/03/2024] [Indexed: 05/18/2024] Open
Abstract
Immunotherapy, particularly immune checkpoint inhibitors (ICIs), such as anti-programmed death 1/programmed death-ligand 1 (PD-1/PD-L1) therapy, has emerged as a pivotal treatment modality for solid tumors, including recurrent or metastatic nasopharyngeal carcinoma (R/M-NPC). Despite the advancements in the utilization of ICIs, there is still room for further improving patient outcomes. Another promising approach to immunotherapy for R/M-NPC involves adoptive cell therapy (ACT), which aims to stimulate systemic anti-tumor immunity. However, individual agent therapies targeting dendritic cells (DCs) appear to still be in the clinical trial phase. This current review underscores the potential of immunotherapy as a valuable adjunct to the treatment paradigm for R/M-NPC patients. Further research is warranted to enhance the efficacy of immunotherapy through the implementation of strategies such as combination therapies and overcoming immune suppression. Additionally, the development of a biomarker-based scoring system is essential for identifying suitable candidates for precision immunotherapy.
Collapse
Affiliation(s)
- Xin Liu
- The First Affiliated Hospital of Jinan University, Guangzhou, Guangdong, China
- Graduate College, Jinan University, Guangzhou, Guangdong, China
| | - Hui Shen
- The First Affiliated Hospital of Jinan University, Guangzhou, Guangdong, China
- Graduate College, Jinan University, Guangzhou, Guangdong, China
| | - Lu Zhang
- The First Affiliated Hospital of Jinan University, Guangzhou, Guangdong, China
| | - Wenhui Huang
- The First Affiliated Hospital of Jinan University, Guangzhou, Guangdong, China
| | - Shuixing Zhang
- The First Affiliated Hospital of Jinan University, Guangzhou, Guangdong, China.
| | - Bin Zhang
- The First Affiliated Hospital of Jinan University, Guangzhou, Guangdong, China.
| |
Collapse
|
27
|
Coles CE, Earl H, Anderson BO, Barrios CH, Bienz M, Bliss JM, Cameron DA, Cardoso F, Cui W, Francis PA, Jagsi R, Knaul FM, McIntosh SA, Phillips KA, Radbruch L, Thompson MK, André F, Abraham JE, Bhattacharya IS, Franzoi MA, Drewett L, Fulton A, Kazmi F, Inbah Rajah D, Mutebi M, Ng D, Ng S, Olopade OI, Rosa WE, Rubasingham J, Spence D, Stobart H, Vargas Enciso V, Vaz-Luis I, Villarreal-Garza C. The Lancet Breast Cancer Commission. Lancet 2024; 403:1895-1950. [PMID: 38636533 DOI: 10.1016/s0140-6736(24)00747-5] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/24/2023] [Revised: 12/18/2023] [Accepted: 04/09/2024] [Indexed: 04/20/2024]
Affiliation(s)
| | - Helena Earl
- Department of Oncology, University of Cambridge, Cambridge, UK
| | - Benjamin O Anderson
- Global Breast Cancer Initiative, World Health Organisation and Departments of Surgery and Global Health Medicine, University of Washington, Seattle, WA, USA
| | - Carlos H Barrios
- Oncology Research Center, Hospital São Lucas, Porto Alegre, Brazil
| | - Maya Bienz
- Mount Vernon Cancer Centre, East and North Hertfordshire NHS Trust, London, UK; Cambridge University Hospitals NHS Foundation Trust, Cambridge, UK
| | | | - David A Cameron
- Institute of Genetics and Cancer and Usher Institute, University of Edinburgh, Edinburgh, UK
| | - Fatima Cardoso
- Breast Unit, Champalimaud Clinical Center/Champalimaud Foundation, Lisbon, Portugal
| | - Wanda Cui
- Department of Medical Oncology, Peter MacCallum Cancer Centre, Melbourne, VIC, Australia; Sir Peter MacCallum Department of Oncology, The University of Melbourne, Melbourne, VIC, Australia
| | - Prudence A Francis
- Department of Medical Oncology, Peter MacCallum Cancer Centre, Melbourne, VIC, Australia; Sir Peter MacCallum Department of Oncology, The University of Melbourne, Melbourne, VIC, Australia
| | - Reshma Jagsi
- Emory University School of Medicine, Atlanta, GA, USA
| | - Felicia Marie Knaul
- Institute for Advanced Study of the Americas, University of Miami, Miami, FL, USA; Sylvester Comprehensive Cancer Center, University of Miami, Miami, FL, USA; Tómatelo a Pecho, Mexico City, Mexico
| | - Stuart A McIntosh
- School of Medicine, Dentistry and Biomedical Sciences, Queen's University Belfast, Belfast, UK
| | - Kelly-Anne Phillips
- Department of Medical Oncology, Peter MacCallum Cancer Centre, Melbourne, VIC, Australia; Sir Peter MacCallum Department of Oncology, The University of Melbourne, Melbourne, VIC, Australia
| | - Lukas Radbruch
- Department of Palliative Medicine, University Hospital Bonn, Bonn, Germany
| | | | | | - Jean E Abraham
- Department of Oncology, University of Cambridge, Cambridge, UK
| | | | | | - Lynsey Drewett
- Royal Devon University Healthcare NHS Foundation Trust, Exeter, UK
| | | | - Farasat Kazmi
- Norfolk and Norwich University Hospitals NHS Foundation Trust, Norwich, UK
| | | | | | - Dianna Ng
- Memorial Sloan Kettering Cancer Center, New York City, NY, USA
| | - Szeyi Ng
- The Institute of Cancer Research, London, UK
| | | | - William E Rosa
- Memorial Sloan Kettering Cancer Center, New York City, NY, USA
| | | | | | | | | | | | - Cynthia Villarreal-Garza
- Breast Cancer Center, Hospital Zambrano Hellion TecSalud, Tecnologico de Monterrey, Monterrey, Mexico
| |
Collapse
|
28
|
Han R, Acosta JN, Shakeri Z, Ioannidis JPA, Topol EJ, Rajpurkar P. Randomised controlled trials evaluating artificial intelligence in clinical practice: a scoping review. Lancet Digit Health 2024; 6:e367-e373. [PMID: 38670745 PMCID: PMC11068159 DOI: 10.1016/s2589-7500(24)00047-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2023] [Revised: 03/01/2024] [Accepted: 03/04/2024] [Indexed: 04/28/2024]
Abstract
This scoping review of randomised controlled trials on artificial intelligence (AI) in clinical practice reveals an expanding interest in AI across clinical specialties and locations. The USA and China are leading in the number of trials, with a focus on deep learning systems for medical imaging, particularly in gastroenterology and radiology. A majority of trials (70 [81%] of 86) report positive primary endpoints, primarily related to diagnostic yield or performance; however, the predominance of single-centre trials, little demographic reporting, and varying reports of operational efficiency raise concerns about the generalisability and practicality of these results. Despite the promising outcomes, considering the likelihood of publication bias and the need for more comprehensive research including multicentre trials, diverse outcome measures, and improved reporting standards is crucial. Future AI trials should prioritise patient-relevant outcomes to fully understand AI's true effects and limitations in health care.
Collapse
Affiliation(s)
- Ryan Han
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA; Department of Computer Science, Stanford University, Stanford, CA, USA; University of California Los Angeles-Caltech Medical Scientist Training Program, Los Angeles, CA, USA
| | - Julián N Acosta
- Department of Neurology, Yale School of Medicine, New Haven, CT, USA; Rad AI, San Francisco, CA, USA
| | - Zahra Shakeri
- Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
| | - John P A Ioannidis
- Stanford Prevention Research Center, Department of Medicine, Stanford University, Stanford, CA, USA; Meta-Research Innovation Center at Stanford, Stanford University, Stanford, CA, USA
| | - Eric J Topol
- Scripps Research Translational Institute, Scripps Research, La Jolla, CA, USA.
| | - Pranav Rajpurkar
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
29
|
Usuzaki T, Takahashi K, Inamori R, Morishita Y, Shizukuishi T, Takagi H, Ishikuro M, Obara T, Takase K. Identifying key factors for predicting O6-Methylguanine-DNA methyltransferase status in adult patients with diffuse glioma: a multimodal analysis of demographics, radiomics, and MRI by variable Vision Transformer. Neuroradiology 2024; 66:761-773. [PMID: 38472373 PMCID: PMC11031474 DOI: 10.1007/s00234-024-03329-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2023] [Accepted: 03/04/2024] [Indexed: 03/14/2024]
Abstract
PURPOSE This study aimed to perform multimodal analysis by vision transformer (vViT) in predicting O6-methylguanine-DNA methyl transferase (MGMT) promoter status among adult patients with diffuse glioma using demographics (sex and age), radiomic features, and MRI. METHODS The training and test datasets contained 122 patients with 1,570 images and 30 patients with 484 images, respectively. The radiomic features were extracted from enhancing tumors (ET), necrotic tumor cores (NCR), and the peritumoral edematous/infiltrated tissues (ED) using contrast-enhanced T1-weighted images (CE-T1WI) and T2-weighted images (T2WI). The vViT had 9 sectors; 1 demographic sector, 6 radiomic sectors (CE-T1WI ET, CE-T1WI NCR, CE-T1WI ED, T2WI ET, T2WI NCR, and T2WI ED), 2 image sectors (CE-T1WI, and T2WI). Accuracy and area under the curve of receiver-operating characteristics (AUC-ROC) were calculated for the test dataset. The performance of vViT was compared with AlexNet, GoogleNet, VGG16, and ResNet by McNemar and Delong test. Permutation importance (PI) analysis with the Mann-Whitney U test was performed. RESULTS The accuracy was 0.833 (95% confidence interval [95%CI]: 0.714-0.877) and the area under the curve of receiver-operating characteristics was 0.840 (0.650-0.995) in the patient-based analysis. The vViT had higher accuracy than VGG16 and ResNet, and had higher AUC-ROC than GoogleNet (p<0.05). The ED radiomic features extracted from the T2-weighted image demonstrated the highest importance (PI=0.239, 95%CI: 0.237-0.240) among all other sectors (p<0.0001). CONCLUSION The vViT is a competent deep learning model in predicting MGMT status. The ED radiomic features of the T2-weighted image demonstrated the most dominant contribution.
Collapse
Affiliation(s)
- Takuma Usuzaki
- Department of Diagnostic Radiology, Tohoku University Hospital, 1-1 Seiryo-machi, Aoba-ku, Sendai, Miyagi, Miyagi, 980-8574, Japan.
| | - Kengo Takahashi
- Tohoku University Graduate School of Medicine, 2-1 Seiryo-machi, Aoba-ku, Sendai, Miyagi, Miyagi, 980-8573, Japan
| | - Ryusei Inamori
- Tohoku University Graduate School of Medicine, 2-1 Seiryo-machi, Aoba-ku, Sendai, Miyagi, Miyagi, 980-8573, Japan
| | - Yohei Morishita
- Department of Diagnostic Radiology, Tohoku University Hospital, 1-1 Seiryo-machi, Aoba-ku, Sendai, Miyagi, Miyagi, 980-8574, Japan
| | - Takashi Shizukuishi
- Department of Diagnostic Radiology, Tohoku University Hospital, 1-1 Seiryo-machi, Aoba-ku, Sendai, Miyagi, Miyagi, 980-8574, Japan
| | - Hidenobu Takagi
- Department of Diagnostic Radiology, Tohoku University Hospital, 1-1 Seiryo-machi, Aoba-ku, Sendai, Miyagi, Miyagi, 980-8574, Japan
- Department of Advanced MRI Collaborative Research, Graduate School of Medicine, Tohoku University, 2-1 Seiryo-machi, Aoba-ku, Sendai, Miyagi, Miyagi, 980-8573, Japan
| | - Mami Ishikuro
- Tohoku University Graduate School of Medicine, Division of Molecular Epidemiology, 2-1 Seiryo-machi, Aoba-ku, Sendai, Miyagi, Miyagi, 980-8573, Japan
| | - Taku Obara
- Tohoku University Graduate School of Medicine, Division of Molecular Epidemiology, 2-1 Seiryo-machi, Aoba-ku, Sendai, Miyagi, Miyagi, 980-8573, Japan
- Tohoku University Graduate School of Medicine, Division of Molecular Epidemiology, Department of Preventive Medicine and Epidemiology, Tohoku Medical Megabank Organization, 2-1 Seiryo-machi, Aoba-ku, Sendai, Miyagi, Miyagi, 980-8573, Japan
- Tohoku University Hospital, Department of Pharmaceutical Sciences, 1-1 Seiryo-machi, Aoba-ku, Sendai, Miyagi, Miyagi, 980-8574, Japan
| | - Kei Takase
- Department of Diagnostic Radiology, Tohoku University Hospital, 1-1 Seiryo-machi, Aoba-ku, Sendai, Miyagi, Miyagi, 980-8574, Japan
| |
Collapse
|
30
|
Thakur GK, Thakur A, Kulkarni S, Khan N, Khan S. Deep Learning Approaches for Medical Image Analysis and Diagnosis. Cureus 2024; 16:e59507. [PMID: 38826977 PMCID: PMC11144045 DOI: 10.7759/cureus.59507] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2024] [Accepted: 05/01/2024] [Indexed: 06/04/2024] Open
Abstract
In addition to enhancing diagnostic accuracy, deep learning techniques offer the potential to streamline workflows, reduce interpretation time, and ultimately improve patient outcomes. The scalability and adaptability of deep learning algorithms enable their deployment across diverse clinical settings, ranging from radiology departments to point-of-care facilities. Furthermore, ongoing research efforts focus on addressing the challenges of data heterogeneity, model interpretability, and regulatory compliance, paving the way for seamless integration of deep learning solutions into routine clinical practice. As the field continues to evolve, collaborations between clinicians, data scientists, and industry stakeholders will be paramount in harnessing the full potential of deep learning for advancing medical image analysis and diagnosis. Furthermore, the integration of deep learning algorithms with other technologies, including natural language processing and computer vision, may foster multimodal medical data analysis and clinical decision support systems to improve patient care. The future of deep learning in medical image analysis and diagnosis is promising. With each success and advancement, this technology is getting closer to being leveraged for medical purposes. Beyond medical image analysis, patient care pathways like multimodal imaging, imaging genomics, and intelligent operating rooms or intensive care units can benefit from deep learning models.
Collapse
Affiliation(s)
- Gopal Kumar Thakur
- Department of Data Sciences, Harrisburg University of Science and Technology, Harrisburg, USA
| | - Abhishek Thakur
- Department of Data Sciences, Harrisburg University of Science and Technology, Harrisburg, USA
| | - Shridhar Kulkarni
- Department of Data Sciences, Harrisburg University of Science and Technology, Harrisburg, USA
| | - Naseebia Khan
- Department of Data Sciences, Harrisburg University of Science and Technology, Harrisburg, USA
| | - Shahnawaz Khan
- Department of Computer Application, Bundelkhand University, Jhansi, IND
| |
Collapse
|
31
|
Voigtlaender S, Pawelczyk J, Geiger M, Vaios EJ, Karschnia P, Cudkowicz M, Dietrich J, Haraldsen IRJH, Feigin V, Owolabi M, White TL, Świeboda P, Farahany N, Natarajan V, Winter SF. Artificial intelligence in neurology: opportunities, challenges, and policy implications. J Neurol 2024; 271:2258-2273. [PMID: 38367046 DOI: 10.1007/s00415-024-12220-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2023] [Revised: 01/20/2024] [Accepted: 01/22/2024] [Indexed: 02/19/2024]
Abstract
Neurological conditions are the leading cause of disability and mortality combined, demanding innovative, scalable, and sustainable solutions. Brain health has become a global priority with adoption of the World Health Organization's Intersectoral Global Action Plan in 2022. Simultaneously, rapid advancements in artificial intelligence (AI) are revolutionizing neurological research and practice. This scoping review of 66 original articles explores the value of AI in neurology and brain health, systematizing the landscape for emergent clinical opportunities and future trends across the care trajectory: prevention, risk stratification, early detection, diagnosis, management, and rehabilitation. AI's potential to advance personalized precision neurology and global brain health directives hinges on resolving core challenges across four pillars-models, data, feasibility/equity, and regulation/innovation-through concerted pursuit of targeted recommendations. Paramount actions include swift, ethical, equity-focused integration of novel technologies into clinical workflows, mitigating data-related issues, counteracting digital inequity gaps, and establishing robust governance frameworks balancing safety and innovation.
Collapse
Affiliation(s)
- Sebastian Voigtlaender
- Systems Neuroscience Division, Max-Planck-Institute for Biological Cybernetics, Tübingen, Germany
- Virtual Diagnostics Team, QuantCo Inc., Cambridge, MA, USA
| | - Johannes Pawelczyk
- Faculty of Medicine, Ruprecht-Karls-University, Heidelberg, Germany
- Graduate Center of Medicine and Health, Technical University Munich, Munich, Germany
| | - Mario Geiger
- Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA
- NVIDIA, Zurich, Switzerland
| | - Eugene J Vaios
- Department of Radiation Oncology, Duke University Medical Center, Durham, NC, USA
| | - Philipp Karschnia
- Department of Neurosurgery, Ludwig-Maximilians-University and University Hospital Munich, Munich, Germany
- Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
| | - Merit Cudkowicz
- Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
| | - Jorg Dietrich
- Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
| | - Ira R J Hebold Haraldsen
- Department of Neurology, Division of Clinical Neuroscience, Oslo University Hospital, Oslo, Norway
| | - Valery Feigin
- National Institute for Stroke and Applied Neurosciences, Auckland University of Technology, Auckland, New Zealand
| | - Mayowa Owolabi
- Center for Genomics and Precision Medicine, College of Medicine, University of Ibadan, Ibadan, Nigeria
- Neurology Unit, Department of Medicine, University of Ibadan, Ibadan, Nigeria
- Blossom Specialist Medical Center, Ibadan, Nigeria
- Lebanese American University of Beirut, Beirut, Lebanon
| | - Tara L White
- Department of Behavioral and Social Sciences, Brown University, Providence, RI, USA
| | | | | | | | - Sebastian F Winter
- Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA.
| |
Collapse
|
32
|
Blindenbach J, Kang J, Hong S, Karam C, Lehner T, Gürsoy G. Ultra-secure storage and analysis of genetic data for the advancement of precision medicine. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.04.16.589793. [PMID: 38695012 PMCID: PMC11061874 DOI: 10.1101/2024.04.16.589793] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 05/04/2024]
Abstract
Cloud computing provides the opportunity to store the ever-growing genotype-phenotype data sets needed to achieve the full potential of precision medicine. However, due to the sensitive nature of this data and the patchwork of data privacy laws across states and countries, additional security protections are proving necessary to ensure data privacy and security. Here we present SQUiD, a secure queryable database for storing and analyzing genotype-phenotype data. With SQUiD, genotype-phenotype data can be stored in a low-security, low-cost public cloud in the encrypted form, which researchers can securely query without the public cloud ever being able to decrypt the data. We demonstrate the usability of SQUiD by replicating various commonly used calculations such as polygenic risk scores, cohort creation for GWAS, MAF filtering, and patient similarity analysis both on synthetic and UK Biobank data. Our work represents a new and scalable platform enabling the realization of precision medicine without security and privacy concerns.
Collapse
Affiliation(s)
- Jacob Blindenbach
- Department of Computer Science, Columbia University
- Department of Biomedical Informatics, Columbia University
- New York Genome Center
- These authors contributed equally
| | - Jiayi Kang
- COSIC, KU Leuven
- These authors contributed equally
| | - Seungwan Hong
- Department of Biomedical Informatics, Columbia University
- New York Genome Center
- These authors contributed equally
| | | | | | - Gamze Gürsoy
- Department of Computer Science, Columbia University
- Department of Biomedical Informatics, Columbia University
- New York Genome Center
| |
Collapse
|
33
|
Kalyani RR, Allende-Vigo MZ, Antinori-Lent KJ, Close KL, Das SR, Deroze P, Edelman SV, El Sayed NA, Kerr D, Neumiller JJ, Norton A. Prioritizing Patient Experiences in the Management of Diabetes and Its Complications: An Endocrine Society Position Statement. J Clin Endocrinol Metab 2024; 109:1155-1178. [PMID: 38381587 DOI: 10.1210/clinem/dgad745] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/11/2023] [Indexed: 02/23/2024]
Abstract
Diabetes can be an arduous journey both for people with diabetes (PWD) and their caregivers. While the journey of every person with diabetes is unique, common themes emerge in managing this disease. To date, the experiences of PWD have not been fully considered to successfully implement the recommended standards of diabetes care in practice. It is critical for health-care providers (HCPs) to recognize perspectives of PWD to achieve optimal health outcomes. Further, existing tools are available to facilitate patient-centered care but are often underused. This statement summarizes findings from multistakeholder expert roundtable discussions hosted by the Endocrine Society that aimed to identify existing gaps in the management of diabetes and its complications and to identify tools needed to empower HCPs and PWD to address their many challenges. The roundtables included delegates from professional societies, governmental organizations, patient advocacy organizations, and social enterprises committed to making life better for PWD. Each section begins with a clinical scenario that serves as a framework to achieve desired health outcomes and includes a discussion of resources for HCPs to deliver patient-centered care in clinical practice. As diabetes management evolves, achieving this goal will also require the development of new tools to help guide HCPs in supporting PWD, as well as concrete strategies for the efficient uptake of these tools in clinical practice to minimize provider burden. Importantly, coordination among various stakeholders including PWD, HCPs, caregivers, policymakers, and payers is critical at all stages of the patient journey.
Collapse
Affiliation(s)
- Rita R Kalyani
- Division of Endocrinology, Diabetes, & Metabolism, Johns Hopkins University School of Medicine, Baltimore, MD 21287, USA
| | | | | | | | - Sandeep R Das
- Division of Cardiology, UT Southwestern Medical Center, Dallas, TX 75390, USA
| | - Phyllisa Deroze
- dQ&A, The Diabetes Research Company, San Francisco, CA 94117, USA
| | - Steven V Edelman
- Division of Endocrinology, Diabetes & Metabolism at the University of California at San Diego, San Diego, CA 92103, USA
| | - Nuha A El Sayed
- American Diabetes Association, Harvard Medical School, Boston, MA 02215, USA
| | - David Kerr
- Director of Digital Health, Diabetes Technology Society, Santa Barbara, CA 94010, USA
| | - Joshua J Neumiller
- Department of Pharmacotherapy, College of Pharmacy and Pharmaceutical Sciences, Washington State University, Spokane, WA 99202, USA
| | - Anna Norton
- DiabetesSisters, #180, 1112 W Boughton Road, Bolingbrook, IL 60440, USA
| |
Collapse
|
34
|
Panebianco V, Pecoraro M, Novelli S, Catalano C. Bridging the gap between human beings and digital twins in radiology. Eur Radiol 2024:10.1007/s00330-024-10766-9. [PMID: 38625614 DOI: 10.1007/s00330-024-10766-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2024] [Revised: 03/12/2024] [Accepted: 03/15/2024] [Indexed: 04/17/2024]
Affiliation(s)
- Valeria Panebianco
- Department of Radiological Sciences, Oncology and Pathology, Sapienza University of Rome, Rome, Italy.
| | - Martina Pecoraro
- Department of Radiological Sciences, Oncology and Pathology, Sapienza University of Rome, Rome, Italy
| | - Simone Novelli
- Department of Mechanical and Aerospace Engineering, Sapienza University of Rome, Rome, Italy
- Liver Failure Group, Institute for Liver and Digestive Health, UCL Medical School, Royal Free Hospital, London, UK
| | - Carlo Catalano
- Department of Radiological Sciences, Oncology and Pathology, Sapienza University of Rome, Rome, Italy
| |
Collapse
|
35
|
Hirosawa T, Harada Y, Tokumasu K, Ito T, Suzuki T, Shimizu T. Evaluating ChatGPT-4's Diagnostic Accuracy: Impact of Visual Data Integration. JMIR Med Inform 2024; 12:e55627. [PMID: 38592758 DOI: 10.2196/55627] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2023] [Revised: 02/14/2024] [Accepted: 03/13/2024] [Indexed: 04/10/2024] Open
Abstract
BACKGROUND In the evolving field of health care, multimodal generative artificial intelligence (AI) systems, such as ChatGPT-4 with vision (ChatGPT-4V), represent a significant advancement, as they integrate visual data with text data. This integration has the potential to revolutionize clinical diagnostics by offering more comprehensive analysis capabilities. However, the impact on diagnostic accuracy of using image data to augment ChatGPT-4 remains unclear. OBJECTIVE This study aims to assess the impact of adding image data on ChatGPT-4's diagnostic accuracy and provide insights into how image data integration can enhance the accuracy of multimodal AI in medical diagnostics. Specifically, this study endeavored to compare the diagnostic accuracy between ChatGPT-4V, which processed both text and image data, and its counterpart, ChatGPT-4, which only uses text data. METHODS We identified a total of 557 case reports published in the American Journal of Case Reports from January 2022 to March 2023. After excluding cases that were nondiagnostic, pediatric, and lacking image data, we included 363 case descriptions with their final diagnoses and associated images. We compared the diagnostic accuracy of ChatGPT-4V and ChatGPT-4 without vision based on their ability to include the final diagnoses within differential diagnosis lists. Two independent physicians evaluated their accuracy, with a third resolving any discrepancies, ensuring a rigorous and objective analysis. RESULTS The integration of image data into ChatGPT-4V did not significantly enhance diagnostic accuracy, showing that final diagnoses were included in the top 10 differential diagnosis lists at a rate of 85.1% (n=309), comparable to the rate of 87.9% (n=319) for the text-only version (P=.33). Notably, ChatGPT-4V's performance in correctly identifying the top diagnosis was inferior, at 44.4% (n=161), compared with 55.9% (n=203) for the text-only version (P=.002, χ2 test). Additionally, ChatGPT-4's self-reports showed that image data accounted for 30% of the weight in developing the differential diagnosis lists in more than half of cases. CONCLUSIONS Our findings reveal that currently, ChatGPT-4V predominantly relies on textual data, limiting its ability to fully use the diagnostic potential of visual information. This study underscores the need for further development of multimodal generative AI systems to effectively integrate and use clinical image data. Enhancing the diagnostic performance of such AI systems through improved multimodal data integration could significantly benefit patient care by providing more accurate and comprehensive diagnostic insights. Future research should focus on overcoming these limitations, paving the way for the practical application of advanced AI in medicine.
Collapse
Affiliation(s)
- Takanobu Hirosawa
- Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, Shimotsuga, Japan
| | - Yukinori Harada
- Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, Shimotsuga, Japan
| | - Kazuki Tokumasu
- Department of General Medicine, Okayama University Graduate School of Medicine, Dentistry and Pharmaceutical Sciences, Okayama, Japan
| | | | - Tomoharu Suzuki
- Department of Hospital Medicine, Urasoe General Hospital, Okinawa, Japan
| | - Taro Shimizu
- Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, Shimotsuga, Japan
| |
Collapse
|
36
|
Lo Gullo R, Brunekreef J, Marcus E, Han LK, Eskreis-Winkler S, Thakur SB, Mann R, Groot Lipman K, Teuwen J, Pinker K. AI Applications to Breast MRI: Today and Tomorrow. J Magn Reson Imaging 2024. [PMID: 38581127 DOI: 10.1002/jmri.29358] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2023] [Revised: 03/07/2024] [Accepted: 03/09/2024] [Indexed: 04/08/2024] Open
Abstract
In breast imaging, there is an unrelenting increase in the demand for breast imaging services, partly explained by continuous expanding imaging indications in breast diagnosis and treatment. As the human workforce providing these services is not growing at the same rate, the implementation of artificial intelligence (AI) in breast imaging has gained significant momentum to maximize workflow efficiency and increase productivity while concurrently improving diagnostic accuracy and patient outcomes. Thus far, the implementation of AI in breast imaging is at the most advanced stage with mammography and digital breast tomosynthesis techniques, followed by ultrasound, whereas the implementation of AI in breast magnetic resonance imaging (MRI) is not moving along as rapidly due to the complexity of MRI examinations and fewer available dataset. Nevertheless, there is persisting interest in AI-enhanced breast MRI applications, even as the use of and indications of breast MRI continue to expand. This review presents an overview of the basic concepts of AI imaging analysis and subsequently reviews the use cases for AI-enhanced MRI interpretation, that is, breast MRI triaging and lesion detection, lesion classification, prediction of treatment response, risk assessment, and image quality. Finally, it provides an outlook on the barriers and facilitators for the adoption of AI in breast MRI. LEVEL OF EVIDENCE: 5 TECHNICAL EFFICACY: Stage 6.
Collapse
Affiliation(s)
- Roberto Lo Gullo
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York City, New York, USA
| | - Joren Brunekreef
- AI for Oncology, Netherlands Cancer Institute, Amsterdam, the Netherlands
| | - Eric Marcus
- AI for Oncology, Netherlands Cancer Institute, Amsterdam, the Netherlands
| | - Lynn K Han
- Weill Cornell Medical College, New York-Presbyterian Hospital, New York City, New York, USA
| | - Sarah Eskreis-Winkler
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York City, New York, USA
| | - Sunitha B Thakur
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York City, New York, USA
- Department of Medical Physics, Memorial Sloan Kettering Cancer Center, New York City, New York, USA
| | - Ritse Mann
- AI for Oncology, Netherlands Cancer Institute, Amsterdam, the Netherlands
- Department of Medical Imaging, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Kevin Groot Lipman
- AI for Oncology, Netherlands Cancer Institute, Amsterdam, the Netherlands
- Department of Medical Imaging, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Jonas Teuwen
- AI for Oncology, Netherlands Cancer Institute, Amsterdam, the Netherlands
- Department of Medical Imaging, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Katja Pinker
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York City, New York, USA
| |
Collapse
|
37
|
Subbiah V, Horgan D, Subbiah IM. A Vision for Democratizing Next-Generation Oncology Clinical Trials. Cancer Discov 2024; 14:579-584. [PMID: 38571427 DOI: 10.1158/2159-8290.cd-24-0017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/05/2024]
Abstract
SUMMARY Revolutionary advancements in oncology have transformed lives, but the clinical trials ecosystem encounters challenges, including restricted access to innovative therapies and a lack of diversity in participant representation. A vision emerges for democratized, globally accessible oncology trials, necessitating collaboration among researchers, clinicians, patients, and policymakers to shift from converting complex, exclusive trials into a dynamic, inclusive force against cancer.
Collapse
Affiliation(s)
- Vivek Subbiah
- Sarah Cannon Research Institute, Nashville, Tennessee
| | - Denis Horgan
- European Alliance for Personalized Medicine, Brussels, Belgium
| | - Ishwaria M Subbiah
- Sarah Cannon Research Institute, Nashville, Tennessee
- The US Oncology Network, The Woodlands, Texas
| |
Collapse
|
38
|
Muse ED, Topol EJ. Transforming the cardiometabolic disease landscape: Multimodal AI-powered approaches in prevention and management. Cell Metab 2024; 36:670-683. [PMID: 38428435 PMCID: PMC10990799 DOI: 10.1016/j.cmet.2024.02.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/26/2023] [Revised: 01/25/2024] [Accepted: 02/06/2024] [Indexed: 03/03/2024]
Abstract
The rise of artificial intelligence (AI) has revolutionized various scientific fields, particularly in medicine, where it has enabled the modeling of complex relationships from massive datasets. Initially, AI algorithms focused on improved interpretation of diagnostic studies such as chest X-rays and electrocardiograms in addition to predicting patient outcomes and future disease onset. However, AI has evolved with the introduction of transformer models, allowing analysis of the diverse, multimodal data sources existing in medicine today. Multimodal AI holds great promise in more accurate disease risk assessment and stratification as well as optimizing the key driving factors in cardiometabolic disease: blood pressure, sleep, stress, glucose control, weight, nutrition, and physical activity. In this article we outline the current state of medical AI in cardiometabolic disease, highlighting the potential of multimodal AI to augment personalized prevention and treatment strategies in cardiometabolic disease.
Collapse
Affiliation(s)
- Evan D Muse
- Scripps Research Translational Institute, Scripps Research, La Jolla, CA 92037, USA; Division of Cardiovascular Diseases, Scripps Clinic, La Jolla, CA 92037, USA
| | - Eric J Topol
- Scripps Research Translational Institute, Scripps Research, La Jolla, CA 92037, USA; Division of Cardiovascular Diseases, Scripps Clinic, La Jolla, CA 92037, USA.
| |
Collapse
|
39
|
Wimbarti S, Kairupan BHR, Tallei TE. Critical review of self-diagnosis of mental health conditions using artificial intelligence. Int J Ment Health Nurs 2024; 33:344-358. [PMID: 38345132 DOI: 10.1111/inm.13303] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/20/2023] [Revised: 01/26/2024] [Accepted: 01/30/2024] [Indexed: 03/10/2024]
Abstract
The advent of artificial intelligence (AI) has revolutionised various aspects of our lives, including mental health nursing. AI-driven tools and applications have provided a convenient and accessible means for individuals to assess their mental well-being within the confines of their homes. Nonetheless, the widespread trend of self-diagnosing mental health conditions through AI poses considerable risks. This review article examines the perils associated with relying on AI for self-diagnosis in mental health, highlighting the constraints and possible adverse outcomes that can arise from such practices. It delves into the ethical, psychological, and social implications, underscoring the vital role of mental health professionals, including psychologists, psychiatrists, and nursing specialists, in providing professional assistance and guidance. This article aims to highlight the importance of seeking professional assistance and guidance in addressing mental health concerns, especially in the era of AI-driven self-diagnosis.
Collapse
Affiliation(s)
- Supra Wimbarti
- Faculty of Psychology, Universitas Gadjah Mada, Yogyakarta, Indonesia
| | - B H Ralph Kairupan
- Department of Psychiatry, Faculty of Medicine, Sam Ratulangi University, Manado, North Sulawesi, Indonesia
| | - Trina Ekawati Tallei
- Department of Biology, Faculty of Mathematics and Natural Sciences, Sam Ratulangi University, Manado, North Sulawesi, Indonesia
- Department of Biology, Faculty of Medicine, Sam Ratulangi University, Manado, North Sulawesi, Indonesia
| |
Collapse
|
40
|
Naik K, Goyal RK, Foschini L, Chak CW, Thielscher C, Zhu H, Lu J, Lehár J, Pacanoswki MA, Terranova N, Mehta N, Korsbo N, Fakhouri T, Liu Q, Gobburu J. Current Status and Future Directions: The Application of Artificial Intelligence/Machine Learning for Precision Medicine. Clin Pharmacol Ther 2024; 115:673-686. [PMID: 38103204 DOI: 10.1002/cpt.3152] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2023] [Accepted: 11/28/2023] [Indexed: 12/18/2023]
Abstract
Technological innovations, such as artificial intelligence (AI) and machine learning (ML), have the potential to expedite the goal of precision medicine, especially when combined with increased capacity for voluminous data from multiple sources and expanded therapeutic modalities; however, they also present several challenges. In this communication, we first discuss the goals of precision medicine, and contextualize the use of AI in precision medicine by showcasing innovative applications (e.g., prediction of tumor growth and overall survival, biomarker identification using biomedical images, and identification of patient population for clinical practice) which were presented during the February 2023 virtual public workshop entitled "Application of Artificial Intelligence and Machine Learning for Precision Medicine," hosted by the US Food and Drug Administration (FDA) and University of Maryland Center of Excellence in Regulatory Science and Innovation (M-CERSI). Next, we put forward challenges brought about by the multidisciplinary nature of AI, particularly highlighting the need for AI to be trustworthy. To address such challenges, we subsequently note practical approaches, viz., differential privacy, synthetic data generation, and federated learning. The proposed strategies - some of which are highlighted presentations from the workshop - are for the protection of personal information and intellectual property. In addition, methods such as the risk-based management approach and the need for an agile regulatory ecosystem are discussed. Finally, we lay out a call for action that includes sharing of data and algorithms, development of regulatory guidance documents, and pooling of expertise from a broad-spectrum of stakeholders to enhance the application of AI in precision medicine.
Collapse
Affiliation(s)
- Kunal Naik
- Office of Clinical Pharmacology, Office of Translational Sciences, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, Maryland, USA
| | - Rahul K Goyal
- Center for Translational Medicine, University of Maryland School of Pharmacy, Baltimore, Maryland, USA
| | | | | | | | - Hao Zhu
- Office of Clinical Pharmacology, Office of Translational Sciences, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, Maryland, USA
| | - James Lu
- Modeling & Simulation/Clinical Pharmacology, Genentech Inc., South San Francisco, California, USA
| | | | - Michael A Pacanoswki
- Office of Clinical Pharmacology, Office of Translational Sciences, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, Maryland, USA
| | - Nadia Terranova
- Quantitative Pharmacology, Ares Trading S.A. (an affiliate of Merck KGaA, Darmstadt, Germany), Lausanne, Switzerland
| | - Neha Mehta
- Office of Clinical Pharmacology, Office of Translational Sciences, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, Maryland, USA
| | | | - Tala Fakhouri
- Office of Medical Policy, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, Maryland, USA
| | - Qi Liu
- Office of Clinical Pharmacology, Office of Translational Sciences, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, Maryland, USA
| | - Jogarao Gobburu
- Center for Translational Medicine, University of Maryland School of Pharmacy, Baltimore, Maryland, USA
| |
Collapse
|
41
|
Li Z, Brittan M, Mills NL. A Multimodal Omics Framework to Empower Target Discovery for Cardiovascular Regeneration. Cardiovasc Drugs Ther 2024; 38:223-236. [PMID: 37421484 PMCID: PMC10959818 DOI: 10.1007/s10557-023-07484-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 06/19/2023] [Indexed: 07/10/2023]
Abstract
Ischaemic heart disease is a global healthcare challenge with high morbidity and mortality. Early revascularisation in acute myocardial infarction has improved survival; however, limited regenerative capacity and microvascular dysfunction often lead to impaired function and the development of heart failure. New mechanistic insights are required to identify robust targets for the development of novel strategies to promote regeneration. Single-cell RNA sequencing (scRNA-seq) has enabled profiling and analysis of the transcriptomes of individual cells at high resolution. Applications of scRNA-seq have generated single-cell atlases for multiple species, revealed distinct cellular compositions for different regions of the heart, and defined multiple mechanisms involved in myocardial injury-induced regeneration. In this review, we summarise findings from studies of healthy and injured hearts in multiple species and spanning different developmental stages. Based on this transformative technology, we propose a multi-species, multi-omics, meta-analysis framework to drive the discovery of new targets to promote cardiovascular regeneration.
Collapse
Affiliation(s)
- Ziwen Li
- BHF Centre for Cardiovascular Science, The Queen's Medical Research Institute, University of Edinburgh, Edinburgh, UK.
| | - Mairi Brittan
- BHF Centre for Cardiovascular Science, The Queen's Medical Research Institute, University of Edinburgh, Edinburgh, UK
| | - Nicholas L Mills
- BHF Centre for Cardiovascular Science, The Queen's Medical Research Institute, University of Edinburgh, Edinburgh, UK
| |
Collapse
|
42
|
Harun R, Lu J, Kassir N, Zhang W. Machine Learning-Based Quantification of Patient Factors Impacting Remission in Patients With Ulcerative Colitis: Insights from Etrolizumab Phase III Clinical Trials. Clin Pharmacol Ther 2024; 115:815-824. [PMID: 37828747 DOI: 10.1002/cpt.3076] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2023] [Accepted: 10/04/2023] [Indexed: 10/14/2023]
Abstract
Etrolizumab, an investigational anti-β7 integrin monoclonal antibody, has undergone evaluation for safety and efficacy in phase III clinical trials on patients with moderate to severe ulcerative colitis (UC). Etrolizumab was terminated because mixed efficacy results were shown in the induction and maintenance phase in patients with UC. In this post hoc analysis, we characterized the impact of explanatory variables on the probability of remission using XGBoost machine learning (ML) models alongside with the SHapley Additive exPlanations framework for explainability. We used patient-level data encompassing demographics, physiology, disease history, clinical questionnaires, histology, serum biomarkers, and etrolizumab drug exposure to develop ML models aimed at predicting remission. Baseline covariates and early etrolizumab exposure at week 4 in the induction phase were utilized to develop an induction ML model, whereas covariates from the end of the induction phase and early etrolizumab exposure at week 4 in the maintenance phase were used to develop a maintenance ML model. Both the induction and maintenance ML models exhibited good predictive performance, achieving an area under the receiver operating characteristic curve (AUROC) of 0.74 ± 0.03 and 0.75 ± 0.06 (mean ± SD), respectively. Compared with placebo, the highest tertile of etrolizumab exposure contributed to 15.0% (95% confidence interval (CI): 9.7-19.9) and 17.0% (95% CI: 8.1-26.4) increases in remission probability in the induction and maintenance phases, respectively. Additionally, the key covariates that predicted remission were CRP, MAdCAM-1, and stool frequency for the induction phase and white blood cells, fecal calprotectin and age for the maintenance phase. These findings hold significant implications for establishing stratification factors in the design of future clinical trials.
Collapse
Affiliation(s)
- Rashed Harun
- Clinical Pharmacology, Genentech, Inc., South San Francisco, California, USA
- PTC Genomics, Bioinformatics & Biospecimens, Genentech, Inc., South San Francisco, California, USA
| | - James Lu
- Clinical Pharmacology, Genentech, Inc., South San Francisco, California, USA
| | - Nastya Kassir
- Clinical Pharmacology, Genentech, Inc., South San Francisco, California, USA
| | - Wenhui Zhang
- Clinical Pharmacology, Genentech, Inc., South San Francisco, California, USA
| |
Collapse
|
43
|
Cornillet M, Villard C, Rorsman F, Molinaro A, Nilsson E, Kechagias S, von Seth E, Bergquist A. The Swedish initiative for the st udy of Primary sclerosing cholangitis (SUPRIM). EClinicalMedicine 2024; 70:102526. [PMID: 38500838 PMCID: PMC10945116 DOI: 10.1016/j.eclinm.2024.102526] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/04/2023] [Revised: 02/19/2024] [Accepted: 02/20/2024] [Indexed: 03/20/2024] Open
Abstract
Background Despite more than 50 years of research and parallel improvements in hepatology and oncology, there is still today neither a treatment to prevent disease progression in primary sclerosing cholangitis (PSC), nor reliable early diagnostic tools for the associated hepatobiliary cancers. Importantly, the limited understanding of the underlying biological mechanisms in PSC and its natural history not only affects the identification of new drug targets but implies a lack of surrogate markers that hampers the design of clinical trials and the evaluation of drug efficacy. The lack of easy access to large representative well-characterised prospective resources is an important contributing factor to the current situation. Methods We here present the SUPRIM cohort, a national multicentre prospective longitudinal study of unselected PSC patients capturing the representative diversity of PSC phenotypes. We describe the 10-year effort of inclusion and follow-up, an intermediate analysis report including original results, and the associated research resource. All included patients gave written informed consent (recruitment: November 2011-April 2016). Findings Out of 512 included patients, 452 patients completed the five-year follow-up without endpoint outcomes. Liver transplantation was performed in 54 patients (10%) and hepatobiliary malignancy was diagnosed in 15 patients (3%). We draw a comprehensive landscape of the multidimensional clinical and biological heterogeneity of PSC illustrating the diversity of PSC phenotypes. Performances of available predictive scores are compared and perspectives on the continuation of the SUPRIM cohort are provided. Interpretation We envision the SUPRIM cohort as an open-access collaborative resource to accelerate the generation of new knowledge and independent validations of promising ones with the aim to uncover reliable diagnostics, prognostic tools, surrogate markers, and new treatment targets by 2040. Funding This work was supported by the Swedish Cancer Society, Stockholm County Council, and the Cancer Research Funds of Radiumhemmet.
Collapse
Affiliation(s)
- Martin Cornillet
- Department of Medicine Huddinge, Karolinska Institutet, Stockholm, Sweden
| | - Christina Villard
- Department of Medicine Huddinge, Karolinska Institutet, Stockholm, Sweden
- Department of Transplantation Surgery, Karolinska University Hospital, Stockholm, Sweden
| | - Fredrik Rorsman
- Department of Gastroenterology and Hepatology, Akademiska University Hospital, Uppsala, Sweden
| | - Antonio Molinaro
- Department of Hepatology, Sahlgrenska University Hospital, Göteborg, Sweden
| | - Emma Nilsson
- Gastroenterology Clinic, Skåne University Hospital, Sweden
| | - Stergios Kechagias
- Department of Health, Medicine and Caring Sciences, Linköping University, Linköping, Sweden
| | - Erik von Seth
- Department of Medicine Huddinge, Karolinska Institutet, Stockholm, Sweden
- Department of Upper Abdominal Diseases, Karolinska University Hospital, Stockholm, Sweden
| | - Annika Bergquist
- Department of Medicine Huddinge, Karolinska Institutet, Stockholm, Sweden
- Department of Upper Abdominal Diseases, Karolinska University Hospital, Stockholm, Sweden
| |
Collapse
|
44
|
Terranova N, Renard D, Shahin MH, Menon S, Cao Y, Hop CECA, Hayes S, Madrasi K, Stodtmann S, Tensfeldt T, Vaddady P, Ellinwood N, Lu J. Artificial Intelligence for Quantitative Modeling in Drug Discovery and Development: An Innovation and Quality Consortium Perspective on Use Cases and Best Practices. Clin Pharmacol Ther 2024; 115:658-672. [PMID: 37716910 DOI: 10.1002/cpt.3053] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2023] [Accepted: 09/11/2023] [Indexed: 09/18/2023]
Abstract
Recent breakthroughs in artificial intelligence (AI) and machine learning (ML) have ushered in a new era of possibilities across various scientific domains. One area where these advancements hold significant promise is model-informed drug discovery and development (MID3). To foster a wider adoption and acceptance of these advanced algorithms, the Innovation and Quality (IQ) Consortium initiated the AI/ML working group in 2021 with the aim of promoting their acceptance among the broader scientific community as well as by regulatory agencies. By drawing insights from workshops organized by the working group and attended by key stakeholders across the biopharma industry, academia, and regulatory agencies, this white paper provides a perspective from the IQ Consortium. The range of applications covered in this white paper encompass the following thematic topics: (i) AI/ML-enabled Analytics for Pharmacometrics and Quantitative Systems Pharmacology (QSP) Workflows; (ii) Explainable Artificial Intelligence and its Applications in Disease Progression Modeling; (iii) Natural Language Processing (NLP) in Quantitative Pharmacology Modeling; and (iv) AI/ML Utilization in Drug Discovery. Additionally, the paper offers a set of best practices to ensure an effective and responsible use of AI, including considering the context of use, explainability and generalizability of models, and having human-in-the-loop. We believe that embracing the transformative power of AI in quantitative modeling while adopting a set of good practices can unlock new opportunities for innovation, increase efficiency, and ultimately bring benefits to patients.
Collapse
Affiliation(s)
- Nadia Terranova
- Quantitative Pharmacology, Merck KGaA, Lausanne, Switzerland
| | - Didier Renard
- Full Development Pharmacometrics, Novartis Pharma AG, Basel, Switzerland
| | | | - Sujatha Menon
- Clinical Pharmacology, Pfizer Inc., Groton, Connecticut, USA
| | - Youfang Cao
- Clinical Pharmacology and Translational Medicine, Eisai Inc., Nutley, New Jersey, USA
| | | | - Sean Hayes
- Quantitative Pharmacology & Pharmacometrics, Merck & Co. Inc., Rahway, New Jersey, USA
| | - Kumpal Madrasi
- Modeling & Simulation, Sanofi, Bridgewater, New Jersey, USA
| | - Sven Stodtmann
- Pharmacometrics, AbbVie Deutschland GmbH & Co. KG, Ludwigshafen, Germany
| | | | - Pavan Vaddady
- Quantitative Clinical Pharmacology, Daiichi Sankyo, Inc., Basking Ridge, New Jersey, USA
| | | | - James Lu
- Clinical Pharmacology, Genentech Inc., South San Francisco, California, USA
| |
Collapse
|
45
|
Samaranayake L. IDJ Pioneers Efforts to Reframe Dental Health Care Through Artificial Intelligence (AI). Int Dent J 2024; 74:177-178. [PMID: 38548452 PMCID: PMC10988283 DOI: 10.1016/j.identj.2024.03.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/02/2024] Open
|
46
|
Ohta T, Hananoe A, Fukushima-Nomura A, Ashizaki K, Sekita A, Seita J, Kawakami E, Sakurada K, Amagai M, Koseki H, Kawasaki H. Best practices for multimodal clinical data management and integration: An atopic dermatitis research case. Allergol Int 2024; 73:255-263. [PMID: 38102028 DOI: 10.1016/j.alit.2023.11.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2023] [Revised: 10/06/2023] [Accepted: 11/03/2023] [Indexed: 12/17/2023] Open
Abstract
BACKGROUND In clinical research on multifactorial diseases such as atopic dermatitis, data-driven medical research has become more widely used as means to clarify diverse pathological conditions and to realize precision medicine. However, modern clinical data, characterized as large-scale, multimodal, and multi-center, causes difficulties in data integration and management, which limits productivity in clinical data science. METHODS We designed a generic data management flow to collect, cleanse, and integrate data to handle different types of data generated at multiple institutions by 10 types of clinical studies. We developed MeDIA (Medical Data Integration Assistant), a software to browse the data in an integrated manner and extract subsets for analysis. RESULTS MeDIA integrates and visualizes data and information on research participants obtained from multiple studies. It then provides a sophisticated interface that supports data management and helps data scientists retrieve the data sets they need. Furthermore, the system promotes the use of unified terms such as identifiers or sampling dates to reduce the cost of pre-processing by data analysts. We also propose best practices in clinical data management flow, which we learned from the development and implementation of MeDIA. CONCLUSIONS The MeDIA system solves the problem of multimodal clinical data integration, from complex text data such as medical records to big data such as omics data from a large number of patients. The system and the proposed best practices can be applied not only to allergic diseases but also to other diseases to promote data-driven medical research.
Collapse
Affiliation(s)
- Tazro Ohta
- Medical Data Mathematical Reasoning Team, Advanced Data Science Project, RIKEN Information R&D and Strategy Headquarters, RIKEN, Kanagawa, Japan; Institute for Advanced Academic Research, Chiba University, Chiba, Japan; Department of Artificial Intelligence Medicine, Graduate School of Medicine, Chiba University, Chiba, Japan
| | - Ayaka Hananoe
- Medical Data Mathematical Reasoning Team, Advanced Data Science Project, RIKEN Information R&D and Strategy Headquarters, RIKEN, Kanagawa, Japan; Laboratory for Developmental Genetics, RIKEN Center for Integrative Medical Sciences, RIKEN, Kanagawa, Japan; Department of Dermatology, Keio University School of Medicine, Tokyo, Japan
| | | | - Koichi Ashizaki
- Laboratory for Developmental Genetics, RIKEN Center for Integrative Medical Sciences, RIKEN, Kanagawa, Japan; Department of Dermatology, Keio University School of Medicine, Tokyo, Japan; Advanced Data Science Project, RIKEN Information R&D and Strategy Headquarters, RIKEN, Kanagawa, Japan
| | - Aiko Sekita
- Laboratory for Developmental Genetics, RIKEN Center for Integrative Medical Sciences, RIKEN, Kanagawa, Japan
| | - Jun Seita
- Laboratory for Integrative Genomics, RIKEN Center for Integrative Medical Sciences, RIKEN, Kanagawa, Japan; Medical Data Deep Learning Team, Advanced Data Science Project, RIKEN Information R&D and Strategy Headquarters, RIKEN, Kanagawa, Japan; Medical Data Sharing Unit, Infrastructure Research and Development Division, RIKEN Information R&D and Strategy Headquarters, RIKEN, Saitama, Japan
| | - Eiryo Kawakami
- Medical Data Mathematical Reasoning Team, Advanced Data Science Project, RIKEN Information R&D and Strategy Headquarters, RIKEN, Kanagawa, Japan; Institute for Advanced Academic Research, Chiba University, Chiba, Japan; Department of Artificial Intelligence Medicine, Graduate School of Medicine, Chiba University, Chiba, Japan
| | - Kazuhiro Sakurada
- Advanced Data Science Project, RIKEN Information R&D and Strategy Headquarters, RIKEN, Kanagawa, Japan; Department of Extended Intelligence for Medicine, The Ishii-Ishibashi Laboratory, Keio University School of Medicine, Tokyo, Japan
| | - Masayuki Amagai
- Department of Dermatology, Keio University School of Medicine, Tokyo, Japan; Laboratory for Skin Homeostasis, RIKEN Center for Integrative Medical Sciences, RIKEN, Kanagawa, Japan
| | - Haruhiko Koseki
- Laboratory for Developmental Genetics, RIKEN Center for Integrative Medical Sciences, RIKEN, Kanagawa, Japan
| | - Hiroshi Kawasaki
- Laboratory for Developmental Genetics, RIKEN Center for Integrative Medical Sciences, RIKEN, Kanagawa, Japan; Department of Dermatology, Keio University School of Medicine, Tokyo, Japan; Laboratory for Skin Homeostasis, RIKEN Center for Integrative Medical Sciences, RIKEN, Kanagawa, Japan.
| |
Collapse
|
47
|
Chen X, Huang L. Computational model for drug research. Brief Bioinform 2024; 25:bbae158. [PMID: 38581423 PMCID: PMC10998638 DOI: 10.1093/bib/bbae158] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2024] [Accepted: 03/22/2024] [Indexed: 04/08/2024] Open
Abstract
This special issue focuses on computational model for drug research regarding drug bioactivity prediction, drug-related interaction prediction, modelling for immunotherapy and modelling for treatment of a specific disease, as conveyed by the following six research and four review articles. Notably, these 10 papers described a wide variety of in-depth drug research from the computational perspective and may represent a snapshot of the wide research landscape.
Collapse
Affiliation(s)
- Xing Chen
- School of Science, Jiangnan University, Wuxi, 214122, China
| | - Li Huang
- The Future Laboratory, Tsinghua University, Beijing, 100084, China
| |
Collapse
|
48
|
Zhou Y, Cosentino J, Yun T, Biradar MI, Shreibati J, Lai D, Schwantes-An TH, Luben R, McCaw Z, Engmann J, Providencia R, Schmidt AF, Munroe P, Yang H, Carroll A, Khawaja AP, McLean CY, Behsaz B, Hormozdiari F. Utilizing multimodal AI to improve genetic analyses of cardiovascular traits. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2024:2024.03.19.24304547. [PMID: 38562791 PMCID: PMC10984061 DOI: 10.1101/2024.03.19.24304547] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/04/2024]
Abstract
Electronic health records, biobanks, and wearable biosensors contain multiple high-dimensional clinical data (HDCD) modalities (e.g., ECG, Photoplethysmography (PPG), and MRI) for each individual. Access to multimodal HDCD provides a unique opportunity for genetic studies of complex traits because different modalities relevant to a single physiological system (e.g., circulatory system) encode complementary and overlapping information. We propose a novel multimodal deep learning method, M-REGLE, for discovering genetic associations from a joint representation of multiple complementary HDCD modalities. We showcase the effectiveness of this model by applying it to several cardiovascular modalities. M-REGLE jointly learns a lower representation (i.e., latent factors) of multimodal HDCD using a convolutional variational autoencoder, performs genome wide association studies (GWAS) on each latent factor, then combines the results to study the genetics of the underlying system. To validate the advantages of M-REGLE and multimodal learning, we apply it to common cardiovascular modalities (PPG and ECG), and compare its results to unimodal learning methods in which representations are learned from each data modality separately, but the downstream genetic analyses are performed on the combined unimodal representations. M-REGLE identifies 19.3% more loci on the 12-lead ECG dataset, 13.0% more loci on the ECG lead I + PPG dataset, and its genetic risk score significantly outperforms the unimodal risk score at predicting cardiac phenotypes, such as atrial fibrillation (Afib), in multiple biobanks.
Collapse
Affiliation(s)
| | | | | | - Mahantesh I Biradar
- NIHR Biomedical Research Centre at Moorfields Eye Hospital & UCL Institute of Ophthalmology, London EC1V 9EL, UK
- MRC Epidemiology Unit, University of Cambridge, Cambridge CB2 0SL, UK
| | | | - Dongbing Lai
- Department of Medical and Molecular Genetics, Indiana University School of Medicine, Indianapolis, IN 46202, USA
| | - Tae-Hwi Schwantes-An
- Department of Medical and Molecular Genetics, Indiana University School of Medicine, Indianapolis, IN 46202, USA
| | - Robert Luben
- NIHR Biomedical Research Centre at Moorfields Eye Hospital & UCL Institute of Ophthalmology, London EC1V 9EL, UK
- MRC Epidemiology Unit, University of Cambridge, Cambridge CB2 0SL, UK
| | - Zachary McCaw
- Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Jorgen Engmann
- Center for Translational Genomics, Population Science and Experimental Medicine, Institute of Cardiovascular Science, University College London, UK
| | - Rui Providencia
- Institute of Health Informatics Research, University College London, London, UK
- Electrophysiology Department, Barts Heart Centre, St. Bartholomew's Hospital, London, UK
| | - Amand Floriaan Schmidt
- Department of Cardiology; Amsterdam University Medical Centres, Amsterdam, The Netherlands
- Institute of Cardiovascular Science; University College London, London, UK
- Division of Heart and Lungs, University Medical Center Utrecht, Utrecht, Netherlands
| | - Patricia Munroe
- William Harvey Research Institute, Barts and the London Faculty of Medicine and Dentistry, Queen Mary University of London, London, UK
| | - Howard Yang
- Google Research, San Francisco CA, 94105 USA
| | | | - Anthony P Khawaja
- NIHR Biomedical Research Centre at Moorfields Eye Hospital & UCL Institute of Ophthalmology, London EC1V 9EL, UK
- MRC Epidemiology Unit, University of Cambridge, Cambridge CB2 0SL, UK
| | | | | | | |
Collapse
|
49
|
Nakao Y, Nishihara T, Sasaki R, Fukushima M, Miuma S, Miyaaki H, Akazawa Y, Nakao K. Investigation of deep learning model for predicting immune checkpoint inhibitor treatment efficacy on contrast-enhanced computed tomography images of hepatocellular carcinoma. Sci Rep 2024; 14:6576. [PMID: 38503827 PMCID: PMC10951210 DOI: 10.1038/s41598-024-57078-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2023] [Accepted: 03/14/2024] [Indexed: 03/21/2024] Open
Abstract
Although the use of immune checkpoint inhibitors (ICIs)-targeted agents for unresectable hepatocellular carcinoma (HCC) is promising, individual response variability exists. Therefore, we developed an artificial intelligence (AI)-based model to predict treatment efficacy using pre-ICIs contrast-enhanced computed tomography (CT) imaging characteristics. We evaluated the efficacy of atezolizumab and bevacizumab in 43 patients at the Nagasaki University Hospital from 2020 to 2022 using the modified Response Evaluation Criteria in Solid Tumors. A total of 197 Progressive Disease (PD), 271 Partial Response (PR), and 342 Stable Disease (SD) contrast CT images of HCC were used for training. We used ResNet-18 as the Convolutional Neural Network (CNN) model and YOLOv5, YOLOv7, YOLOv8 as the You Only Look Once (YOLO) model with precision-recall curves and class activation maps (CAMs) for diagnostic performance evaluation and model interpretation, respectively. The 3D t-distributed Stochastic Neighbor Embedding was used for image feature analysis. The YOLOv7 model demonstrated Precision 53.7%, Recall 100%, F1 score 69.8%, mAP@0.5 99.5% for PD, providing accurate and clinically versatile predictions by identifying decisive points. The ResNet-18 model had Precision 100% and Recall 100% for PD. However, the CAMs sites did not align with the tumors, suggesting the CNN model is not predicting that a given CT slice is PD, PR, or SD, but that it accurately predicts Individual Patient's CT slices. Preparing substantial training data for tumor drug effect prediction models is challenging compared to general tumor diagnosis models; hence, large-scale validation using an efficient YOLO model is warranted.
Collapse
Affiliation(s)
- Yasuhiko Nakao
- Department of Gastroenterology and Hepatology, Nagasaki University Graduate School of Biomedical Sciences, 1-7-1 Sakamoto, Nagasaki City, Nagasaki, Japan.
| | - Takahito Nishihara
- Department of Gastroenterology and Hepatology, Nagasaki University Graduate School of Biomedical Sciences, 1-7-1 Sakamoto, Nagasaki City, Nagasaki, Japan
- Department of Gastroenterology and Hepatology, Isahaya General Hospital, 24-1 Eishohigashimachi, Isahaya, Nagasaki, Japan
| | - Ryu Sasaki
- Department of Gastroenterology and Hepatology, Nagasaki University Graduate School of Biomedical Sciences, 1-7-1 Sakamoto, Nagasaki City, Nagasaki, Japan
| | - Masanori Fukushima
- Department of Gastroenterology and Hepatology, Nagasaki University Graduate School of Biomedical Sciences, 1-7-1 Sakamoto, Nagasaki City, Nagasaki, Japan
| | - Satoshi Miuma
- Department of Gastroenterology and Hepatology, Nagasaki University Graduate School of Biomedical Sciences, 1-7-1 Sakamoto, Nagasaki City, Nagasaki, Japan
| | - Hisamitsu Miyaaki
- Department of Gastroenterology and Hepatology, Nagasaki University Graduate School of Biomedical Sciences, 1-7-1 Sakamoto, Nagasaki City, Nagasaki, Japan
| | - Yuko Akazawa
- Department of Gastroenterology and Hepatology, Nagasaki University Graduate School of Biomedical Sciences, 1-7-1 Sakamoto, Nagasaki City, Nagasaki, Japan
- Department of Histology and Cell Biology, Nagasaki University Graduate School of Biomedical Sciences, 1-12-4 Sakamoto, Nagasaki City, Nagasaki, Japan
| | - Kazuhiko Nakao
- Department of Gastroenterology and Hepatology, Nagasaki University Graduate School of Biomedical Sciences, 1-7-1 Sakamoto, Nagasaki City, Nagasaki, Japan
| |
Collapse
|
50
|
Bhatia A, Hanna J, Stuart T, Kasper KA, Clausen DM, Gutruf P. Wireless Battery-free and Fully Implantable Organ Interfaces. Chem Rev 2024; 124:2205-2280. [PMID: 38382030 DOI: 10.1021/acs.chemrev.3c00425] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/23/2024]
Abstract
Advances in soft materials, miniaturized electronics, sensors, stimulators, radios, and battery-free power supplies are resulting in a new generation of fully implantable organ interfaces that leverage volumetric reduction and soft mechanics by eliminating electrochemical power storage. This device class offers the ability to provide high-fidelity readouts of physiological processes, enables stimulation, and allows control over organs to realize new therapeutic and diagnostic paradigms. Driven by seamless integration with connected infrastructure, these devices enable personalized digital medicine. Key to advances are carefully designed material, electrophysical, electrochemical, and electromagnetic systems that form implantables with mechanical properties closely matched to the target organ to deliver functionality that supports high-fidelity sensors and stimulators. The elimination of electrochemical power supplies enables control over device operation, anywhere from acute, to lifetimes matching the target subject with physical dimensions that supports imperceptible operation. This review provides a comprehensive overview of the basic building blocks of battery-free organ interfaces and related topics such as implantation, delivery, sterilization, and user acceptance. State of the art examples categorized by organ system and an outlook of interconnection and advanced strategies for computation leveraging the consistent power influx to elevate functionality of this device class over current battery-powered strategies is highlighted.
Collapse
Affiliation(s)
- Aman Bhatia
- Department of Biomedical Engineering, The University of Arizona, Tucson, Arizona 85721, United States
| | - Jessica Hanna
- Department of Biomedical Engineering, The University of Arizona, Tucson, Arizona 85721, United States
| | - Tucker Stuart
- Department of Biomedical Engineering, The University of Arizona, Tucson, Arizona 85721, United States
| | - Kevin Albert Kasper
- Department of Biomedical Engineering, The University of Arizona, Tucson, Arizona 85721, United States
| | - David Marshall Clausen
- Department of Biomedical Engineering, The University of Arizona, Tucson, Arizona 85721, United States
| | - Philipp Gutruf
- Department of Biomedical Engineering, The University of Arizona, Tucson, Arizona 85721, United States
- Department of Electrical and Computer Engineering, The University of Arizona, Tucson, Arizona 85721, United States
- Bio5 Institute, The University of Arizona, Tucson, Arizona 85721, United States
- Neuroscience Graduate Interdisciplinary Program (GIDP), The University of Arizona, Tucson, Arizona 85721, United States
| |
Collapse
|