1
|
Kim Y, Park H, Yoon HJ, Suh J, Kang SH, Lim YH, Jang DH, Park JH, Shin ES, Bae JW, Lee JH, Oh JH, Kang DY, Kweon J, Jo MW, Park DW, Kim YH, Ahn JM. Fully automated quantitative coronary angiography versus optical coherence tomography guidance for coronary stent implantation (FLASH): Study protocol for a randomized controlled noninferiority trial. Am Heart J 2024; 275:86-95. [PMID: 38723880 DOI: 10.1016/j.ahj.2024.05.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/18/2024] [Revised: 05/05/2024] [Accepted: 05/05/2024] [Indexed: 07/01/2024]
Abstract
BACKGROUND Artificial intelligence-based quantitative coronary angiography (AI-QCA) has been developed to provide a more objective and reproducible data about the severity of coronary artery stenosis and the dimensions of the vessel for intervention in real-time, overcoming the limitations of significant inter- and intraobserver variability, and time-consuming nature of on-site QCA, without requiring extra time and effort. Compared with the subjective nature of visually estimated conventional CAG guidance, AI-QCA guidance provides a more practical and standardized angiography-based approach. Although the advantage of intravascular imaging-guided PCI is increasingly recognized, their broader adoption is limited by clinical and economic barriers in many catheterization laboratories. METHODS The FLASH (fully automated quantitative coronary angiography versus optical coherence tomography guidance for coronary stent implantation) trial is a randomized, investigator-initiated, multicenter, open-label, noninferiority trial comparing the AI-QCA-assisted PCI strategy with optical coherence tomography-guided PCI strategy in patients with significant coronary artery disease. All operators will utilize a novel, standardized AI-QCA software and PCI protocol in the AI-QCA-assisted group. A total of 400 patients will be randomized to either group at a 1:1 ratio. The primary endpoint is the minimal stent area (mm2), determined by the final OCT run after completion of PCI. Clinical follow-up and cost-effectiveness evaluations are planned at 1 month and 6 months for all patients enrolled in the study. RESULTS Enrollment of a total of 400 patients from the 13 participating centers in South Korea will be completed in February 2024. Follow-up of the last enrolled patients will be completed in August 2024, and primary results will be available by late 2024. CONCLUSION The FLASH is the first clinical trial to evaluate the feasibility of AI-QCA-assisted PCI, and will provide the clinical evidence on AI-QCA assistance in the field of coronary intervention. CLINICAL TRIAL REGISTRATION URL: https://www. CLINICALTRIALS gov. Unique identifier: NCT05388357.
Collapse
Affiliation(s)
- Yongcheol Kim
- Division of Cardiology, Department of Internal Medicine, Yonsei University College of Medicine and Cardiovascular Center, Yongin Severance Hospital, Yongin, Korea
| | - Hanbit Park
- Department of Medicine, Division of Cardiology, Gangneung Asan Hospital, University of Ulsan College of Medicine, Gangneung, Korea
| | - Hyuck-Jun Yoon
- Department of Internal Medicine and Cardiovascular Research Institute, Keimyung University Dongsan Hospital, Daegu, Korea
| | - Jon Suh
- Department of Cardiology, Soon Chun Hyang University Hospital Bucheon, Bucheon, Korea
| | - Si-Hyuck Kang
- Department of Internal Medicine, Cardiovascular Center, Seoul National University Bundang Hospital, Seongnam, Republic of Korea
| | - Young-Hyo Lim
- Department of Internal Medicine, Division of Cardiology, Hanyang University College of Medicine, Seoul, Korea
| | - Duck Hyun Jang
- Department of Internal Medicine, Division of Cardiology, Sejong General Hospital, Bucheon, Korea
| | - Jae Hyoung Park
- Department of Cardiology, Cardiovascular Center, Korea University Anam Hospital, Korea University College of Medicine, Seoul, Korea
| | - Eun-Seok Shin
- Department of Cardiology, Ulsan University Hospital, University of Ulsan College of Medicine, Ulsan, Korea
| | - Jang-Whan Bae
- Department of Internal Medicine, Division of Cardiology, Chungbuk National University College of Medicine, Cheongju, Korea
| | - Jang Hoon Lee
- Department of Internal Medicine, Kyungpook National University Hospital, School of Medicine, Kyungpook National University, Daegu, Korea
| | - Jun-Hyok Oh
- Department of Cardiology and Medical Research Institute, Pusan, Pusan National University, National University Hospital, Busan, Korea
| | - Do-Yoon Kang
- Division of Cardiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Korea
| | - Jihoon Kweon
- Department of Biomedical Engineering, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Korea
| | - Min-Woo Jo
- Department of Preventive Medicine, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Duk-Woo Park
- Division of Cardiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Korea
| | - Young-Hak Kim
- Division of Cardiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Korea
| | - Jung-Min Ahn
- Division of Cardiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Korea
| |
Collapse
|
2
|
Holste G, Lin M, Zhou R, Wang F, Liu L, Yan Q, Van Tassel SH, Kovacs K, Chew EY, Lu Z, Wang Z, Peng Y. Harnessing the power of longitudinal medical imaging for eye disease prognosis using Transformer-based sequence modeling. NPJ Digit Med 2024; 7:216. [PMID: 39152209 PMCID: PMC11329720 DOI: 10.1038/s41746-024-01207-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2024] [Accepted: 07/29/2024] [Indexed: 08/19/2024] Open
Abstract
Deep learning has enabled breakthroughs in automated diagnosis from medical imaging, with many successful applications in ophthalmology. However, standard medical image classification approaches only assess disease presence at the time of acquisition, neglecting the common clinical setting of longitudinal imaging. For slow, progressive eye diseases like age-related macular degeneration (AMD) and primary open-angle glaucoma (POAG), patients undergo repeated imaging over time to track disease progression and forecasting the future risk of developing a disease is critical to properly plan treatment. Our proposed Longitudinal Transformer for Survival Analysis (LTSA) enables dynamic disease prognosis from longitudinal medical imaging, modeling the time to disease from sequences of fundus photography images captured over long, irregular time periods. Using longitudinal imaging data from the Age-Related Eye Disease Study (AREDS) and Ocular Hypertension Treatment Study (OHTS), LTSA significantly outperformed a single-image baseline in 19/20 head-to-head comparisons on late AMD prognosis and 18/20 comparisons on POAG prognosis. A temporal attention analysis also suggested that, while the most recent image is typically the most influential, prior imaging still provides additional prognostic value.
Collapse
Affiliation(s)
- Gregory Holste
- Department of Population Health Sciences, Weill Cornell Medicine, New York, NY, USA
- Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX, USA
| | - Mingquan Lin
- Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX, USA
- Department of Surgery, University of Minnesota, Minneapolis, MN, USA
| | - Ruiwen Zhou
- Center for Biostatistics and Data Science, Washington University School of Medicine, St. Louis, MO, USA
| | - Fei Wang
- Department of Population Health Sciences, Weill Cornell Medicine, New York, NY, USA
| | - Lei Liu
- Center for Biostatistics and Data Science, Washington University School of Medicine, St. Louis, MO, USA
| | - Qi Yan
- Department of Obstetrics & Gynecology, Columbia University Irving Medical Center, New York, NY, USA
| | - Sarah H Van Tassel
- Israel Englander Department of Ophthalmology, Weill Cornell Medicine, New York, NY, USA
| | - Kyle Kovacs
- Israel Englander Department of Ophthalmology, Weill Cornell Medicine, New York, NY, USA
| | - Emily Y Chew
- Division of Epidemiology and Clinical Applications, National Eye Institute, National Institutes of Health (NIH), Bethesda, MD, USA
| | - Zhiyong Lu
- National Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), Bethesda, MD, USA
| | - Zhangyang Wang
- Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX, USA.
| | - Yifan Peng
- Department of Population Health Sciences, Weill Cornell Medicine, New York, NY, USA.
| |
Collapse
|
3
|
Iglesias G, Talavera E, Troya J, Díaz-Álvarez A, García-Remesal M. Artificial intelligence model for tumoral clinical decision support systems. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 253:108228. [PMID: 38810378 DOI: 10.1016/j.cmpb.2024.108228] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/02/2023] [Revised: 04/21/2024] [Accepted: 05/13/2024] [Indexed: 05/31/2024]
Abstract
BACKGROUND AND OBJECTIVE Comparative diagnostic in brain tumor evaluation makes possible to use the available information of a medical center to compare similar cases when a new patient is evaluated. By leveraging Artificial Intelligence models, the proposed system is able of retrieving the most similar cases of brain tumors for a given query. The primary objective is to enhance the diagnostic process by generating more accurate representations of medical images, with a particular focus on patient-specific normal features and pathologies. A key distinction from previous models lies in its ability to produce enriched image descriptors solely from binary information, eliminating the need for costly and difficult to obtain tumor segmentation. METHODS The proposed model uses Artificial Intelligence to detect patient features to recommend the most similar cases from a database. The system not only suggests similar cases but also balances the representation of healthy and abnormal features in its design. This not only encourages the generalization of its use but also aids clinicians in their decision-making processes. This generalization makes possible for future research in different medical diagnosis areas with almost not any change in the system. RESULTS We conducted a comparative analysis of our approach in relation to similar studies. The proposed architecture obtains a Dice coefficient of 0.474 in both tumoral and healthy regions of the patients, which outperforms previous literature. Our proposed model excels at extracting and combining anatomical and pathological features from brain Magnetic Resonances (MRs), achieving state-of-the-art results while relying on less expensive label information. This substantially reduces the overall cost of the training process. Our findings highlight the significant potential for improving the efficiency and accuracy of comparative diagnostics and the treatment of tumoral pathologies. CONCLUSIONS This paper provides substantial grounds for further exploration of the broader applicability and optimization of the proposed architecture to enhance clinical decision-making. The novel approach presented in this work marks a significant advancement in the field of medical diagnosis, particularly in the context of Artificial Intelligence-assisted image retrieval, and promises to reduce costs and improve the quality of patient care using Artificial Intelligence as a support tool instead of a black box system.
Collapse
Affiliation(s)
- Guillermo Iglesias
- Departamento de Sistemas Informáticos, Escuela Técnica Superior de Ingeniería de Sistemas Informáticos, Universidad Politécnica de Madrid, Spain.
| | - Edgar Talavera
- Departamento de Sistemas Informáticos, Escuela Técnica Superior de Ingeniería de Sistemas Informáticos, Universidad Politécnica de Madrid, Spain.
| | - Jesús Troya
- Infanta Leonor University Hospital. Madrid., Spain
| | - Alberto Díaz-Álvarez
- Departamento de Sistemas Informáticos, Escuela Técnica Superior de Ingeniería de Sistemas Informáticos, Universidad Politécnica de Madrid, Spain.
| | - Miguel García-Remesal
- Biomedical Informatics Group, Departamento de Inteligencia Artificial, Escuela Técnica Superior de Ingenieros Informáticos, Universidad Politécnica de Madrid, Spain.
| |
Collapse
|
4
|
Auñón J, Hurtado-Ramírez D, Porras-Díaz L, Irigoyen-Peña B, Rahmian S, Al-Khazraji Y, Soler-Garrido J, Kotsev A. Evaluation and utilisation of privacy enhancing technologies-A data spaces perspective. Data Brief 2024; 55:110560. [PMID: 38948408 PMCID: PMC11214200 DOI: 10.1016/j.dib.2024.110560] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2024] [Revised: 05/14/2024] [Accepted: 05/21/2024] [Indexed: 07/02/2024] Open
Abstract
Data sharing has facilitated the digitisation of society. We can access our bank accounts or make an appointment with our doctor anytime and anywhere. To achieve this, we have to share certain information, whether personal, professional, etc. This may seem like a minor cost for an individual user, but actually the data economy as the backbone of a digital transformation that is reshaping all aspects of human life. However, one of the major concerns arises regarding what happens to such individual data; once shared, control over it is often lost. For that reason, users and companies are reluctant to share their data. The European Union, through its European Strategy for Data, is establishing a policy and legal framework for establishing a single market for data in Europe by improving the trust and fairness of the data economy. Data spaces are a commitment to sharing data in a reliable and secure way, but this endeavour should, of course, not be at the expense of privacy rights. In recent years, Privacy-Enhancing Technologies (PETs) have emerged to achieve data sharing and privacy preservation that can address the requirements of data spaces around sensitive citizen and business data. In this work, we review existing PETs and assess their relevance, technological maturity, and applicability in the context of common European data spaces. Finally, we illustrate the benefits of secure data sharing via Federated Learning in a healthcare use case, where the preservation of privacy is a primer requirement and is therefore to be guaranteed.
Collapse
Affiliation(s)
- J.M. Auñón
- GMV, Department of Artificial Intelligence and Big Data, Isaac Newton 11, Tres Cantos, Madrid 28760, Spain
| | - D. Hurtado-Ramírez
- GMV, Department of Artificial Intelligence and Big Data, Isaac Newton 11, Tres Cantos, Madrid 28760, Spain
| | - L. Porras-Díaz
- GMV, Department of Artificial Intelligence and Big Data, Isaac Newton 11, Tres Cantos, Madrid 28760, Spain
| | - B. Irigoyen-Peña
- GMV, Department of Artificial Intelligence and Big Data, Isaac Newton 11, Tres Cantos, Madrid 28760, Spain
| | - S. Rahmian
- GMV, Department of Artificial Intelligence and Big Data, Isaac Newton 11, Tres Cantos, Madrid 28760, Spain
| | - Y. Al-Khazraji
- GMV, Department of Artificial Intelligence and Big Data, Isaac Newton 11, Tres Cantos, Madrid 28760, Spain
| | - J. Soler-Garrido
- European Commission, Joint Research Centre (JRC), Algorithmic Transparency Unit, Inca Garcilaso 3, Seville 41092, Spain
| | - A. Kotsev
- European Commission, Joint Research Centre (JRC), Digital Economy Unit, Via Enrico Fermi 2749, Ispra 21027, Italy
| |
Collapse
|
5
|
Mamidi IS, Dunham ME, Adkins LK, McWhorter AJ, Fang Z, Banh BT. Laryngeal Cancer Screening During Flexible Video Laryngoscopy Using Large Computer Vision Models. Ann Otol Rhinol Laryngol 2024; 133:720-728. [PMID: 38755974 DOI: 10.1177/00034894241253376] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/18/2024]
Abstract
OBJECTIVE Develop an artificial intelligence assisted computer vision model to screen for laryngeal cancer during flexible laryngoscopy. METHODS Using laryngeal images and flexible laryngoscopy video recordings, we developed computer vision models to classify video frames for usability and cancer screening. A separate model segments any identified lesions on the frames. We used these computer vision models to construct a video stream annotation system. This system classifies findings from flexible laryngoscopy as "potentially malignant" or "probably benign" and segments any detected lesions. Additionally, the model provides a confidence level for each classification. RESULTS The overall accuracy of the flexible laryngoscopy cancer screening model was 92%. For cancer screening, it achieved a sensitivity of 97.7% and a specificity of 76.9%. The segmentation model attained an average precision at a 0.50 intersection-over-union of 0.595. The confidence level for positive screening results can assist clinicians in counseling patients regarding the findings. CONCLUSION Our model is highly sensitive and adequately specific for laryngeal cancer screening. Segmentation helps endoscopists identify and describe potential lesions. Further optimization is required to enable the model's deployment in clinical settings for real-time annotation during flexible laryngoscopy.
Collapse
Affiliation(s)
- Ishwarya S Mamidi
- Department of Otolaryngology-Head and Neck Surgery, Louisiana State University Health Sciences Center, New Orleans, LA, USA
| | - Michael E Dunham
- Department of Otolaryngology-Head and Neck Surgery, Louisiana State University Health Sciences Center, New Orleans, LA, USA
| | - Lacey K Adkins
- Department of Otolaryngology-Head and Neck Surgery, Louisiana State University Health Sciences Center, New Orleans, LA, USA
| | - Andrew J McWhorter
- Department of Otolaryngology-Head and Neck Surgery, Louisiana State University Health Sciences Center, New Orleans, LA, USA
| | - Zhide Fang
- Biostatistics Program, School of Public Health, Louisiana State University Health Sciences Center, New Orleans LA, USA
| | - Britney T Banh
- Our Lady of the Lake Voice Center, Our Lady of the Lake Regional Medical Center, Baton Rouge, LA, USA
| |
Collapse
|
6
|
Pareek A, Ro DH, Karlsson J, Martin RK. Machine learning/artificial intelligence in sports medicine: state of the art and future directions. J ISAKOS 2024; 9:635-644. [PMID: 38336099 DOI: 10.1016/j.jisako.2024.01.013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2022] [Revised: 12/30/2023] [Accepted: 01/25/2024] [Indexed: 02/12/2024]
Abstract
Machine learning (ML) is changing the way health care is practiced and recent applications of these novel statistical techniques have started to impact orthopaedic sports medicine. Machine learning enables the analysis of large volumes of data to establish complex relationships between "input" and "output" variables. These relationships may be more complex than could be established through traditional statistical analysis and can lead to the ability to predict the "output" with high levels of accuracy. Supervised learning is the most common ML approach for healthcare data and recent studies have developed algorithms to predict patient-specific outcome after surgical procedures such as hip arthroscopy and anterior cruciate ligament reconstruction. Deep learning is a higher-level ML approach that facilitates the processing and interpretation of complex datasets through artificial neural networks that are inspired by the way the human brain processes information. In orthopaedic sports medicine, deep learning has primarily been used for automatic image (computer vision) and text (natural language processing) interpretation. While applications in orthopaedic sports medicine have been increasing exponentially, one significant barrier to widespread adoption of ML remains clinician unfamiliarity with the associated methods and concepts. The goal of this review is to introduce these concepts, review current machine learning models in orthopaedic sport medicine, and discuss future opportunities for innovation within the specialty.
Collapse
Affiliation(s)
- Ayoosh Pareek
- Sports Medicine and Shoulder Service, Hospital for Special Surgery, New York, 10021, USA; Department of Orthopaedics, Institute of Clinical Sciences, Sahlgrenska Academy, Gothenburg University, Gothenburg, 43180, Sweden.
| | - Du Hyun Ro
- Department of Orthopedic Surgery, Seoul National University Hospital, Seoul, 03080, South Korea; CONNECTEVE Co., Ltd, Seoul, 03080, South Korea
| | - Jón Karlsson
- Department of Orthopaedics, Institute of Clinical Sciences, Sahlgrenska Academy, Gothenburg University, Gothenburg, 43180, Sweden
| | - R Kyle Martin
- Department of Orthopedic Surgery, University of Minnesota, Minneapolis, MN, 55454, USA; Department of Orthopedic Surgery, CentraCare, Saint Cloud, MN, 56303, USA; Oslo Sports Trauma Research Center, Norwegian School of Sport Sciences, Oslo, 0806, Norway
| |
Collapse
|
7
|
Chen W, Yi Z, Lim LJR, Lim RQR, Zhang A, Qian Z, Huang J, He J, Liu B. Deep learning and remote photoplethysmography powered advancements in contactless physiological measurement. Front Bioeng Biotechnol 2024; 12:1420100. [PMID: 39104628 PMCID: PMC11298756 DOI: 10.3389/fbioe.2024.1420100] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2024] [Accepted: 06/27/2024] [Indexed: 08/07/2024] Open
Abstract
In recent decades, there has been ongoing development in the application of computer vision (CV) in the medical field. As conventional contact-based physiological measurement techniques often restrict a patient's mobility in the clinical environment, the ability to achieve continuous, comfortable and convenient monitoring is thus a topic of interest to researchers. One type of CV application is remote imaging photoplethysmography (rPPG), which can predict vital signs using a video or image. While contactless physiological measurement techniques have an excellent application prospect, the lack of uniformity or standardization of contactless vital monitoring methods limits their application in remote healthcare/telehealth settings. Several methods have been developed to improve this limitation and solve the heterogeneity of video signals caused by movement, lighting, and equipment. The fundamental algorithms include traditional algorithms with optimization and developing deep learning (DL) algorithms. This article aims to provide an in-depth review of current Artificial Intelligence (AI) methods using CV and DL in contactless physiological measurement and a comprehensive summary of the latest development of contactless measurement techniques for skin perfusion, respiratory rate, blood oxygen saturation, heart rate, heart rate variability, and blood pressure.
Collapse
Affiliation(s)
- Wei Chen
- Department of Hand Surgery, Beijing Jishuitan Hospital, Capital Medical University, Beijing, China
| | - Zhe Yi
- Department of Hand Surgery, Beijing Jishuitan Hospital, Capital Medical University, Beijing, China
| | - Lincoln Jian Rong Lim
- Department of Medical Imaging, Western Health, Footscray Hospital, Footscray, VIC, Australia
- Department of Surgery, The University of Melbourne, Melbourne, VIC, Australia
| | - Rebecca Qian Ru Lim
- Department of Hand & Reconstructive Microsurgery, Singapore General Hospital, Singapore, Singapore
| | - Aijie Zhang
- Department of Hand Surgery, Beijing Jishuitan Hospital, Capital Medical University, Beijing, China
| | - Zhen Qian
- Institute of Intelligent Diagnostics, Beijing United-Imaging Research Institute of Intelligent Imaging, Beijing, China
| | - Jiaxing Huang
- Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| | - Jia He
- Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| | - Bo Liu
- Department of Hand Surgery, Beijing Jishuitan Hospital, Capital Medical University, Beijing, China
- Beijing Research Institute of Traumatology and Orthopaedics, Beijing, China
| |
Collapse
|
8
|
Jeon K, Park WY, Kahn CE, Nagy P, You SC, Yoon SH. Advancing Medical Imaging Research Through Standardization: The Path to Rapid Development, Rigorous Validation, and Robust Reproducibility. Invest Radiol 2024:00004424-990000000-00232. [PMID: 38985896 DOI: 10.1097/rli.0000000000001106] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/12/2024]
Abstract
ABSTRACT Artificial intelligence (AI) has made significant advances in radiology. Nonetheless, challenges in AI development, validation, and reproducibility persist, primarily due to the lack of high-quality, large-scale, standardized data across the world. Addressing these challenges requires comprehensive standardization of medical imaging data and seamless integration with structured medical data.Developed by the Observational Health Data Sciences and Informatics community, the OMOP Common Data Model enables large-scale international collaborations with structured medical data. It ensures syntactic and semantic interoperability, while supporting the privacy-protected distribution of research across borders. The recently proposed Medical Imaging Common Data Model is designed to encompass all DICOM-formatted medical imaging data and integrate imaging-derived features with clinical data, ensuring their provenance.The harmonization of medical imaging data and its seamless integration with structured clinical data at a global scale will pave the way for advanced AI research in radiology. This standardization will enable federated learning, ensuring privacy-preserving collaboration across institutions and promoting equitable AI through the inclusion of diverse patient populations. Moreover, it will facilitate the development of foundation models trained on large-scale, multimodal datasets, serving as powerful starting points for specialized AI applications. Objective and transparent algorithm validation on a standardized data infrastructure will enhance reproducibility and interoperability of AI systems, driving innovation and reliability in clinical applications.
Collapse
Affiliation(s)
- Kyulee Jeon
- From the Department of Biomedical Systems Informatics, Yonsei University, Seoul, South Korea (K.J., S.C.Y.); Institution for Innovation in Digital Healthcare, Yonsei University, Seoul, South Korea (K.J., S.C.Y.); Biomedical Informatics and Data Science, Johns Hopkins University, Baltimore, MD (W.Y.P., P.N.); Department of Radiology, University of Pennsylvania, Philadelphia, PA (C.E.K.); and Department of Radiology, Seoul National University College of Medicine, Seoul National University Hospital, Seoul, South Korea (S.H.Y.)
| | | | | | | | | | | |
Collapse
|
9
|
Holste G, Oikonomou EK, Mortazavi BJ, Wang Z, Khera R. Efficient deep learning-based automated diagnosis from echocardiography with contrastive self-supervised learning. COMMUNICATIONS MEDICINE 2024; 4:133. [PMID: 38971887 PMCID: PMC11227494 DOI: 10.1038/s43856-024-00538-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2023] [Accepted: 05/31/2024] [Indexed: 07/08/2024] Open
Abstract
BACKGROUND Advances in self-supervised learning (SSL) have enabled state-of-the-art automated medical image diagnosis from small, labeled datasets. This label efficiency is often desirable, given the difficulty of obtaining expert labels for medical image recognition tasks. However, most efforts toward SSL in medical imaging are not adapted to video-based modalities, such as echocardiography. METHODS We developed a self-supervised contrastive learning approach, EchoCLR, for echocardiogram videos with the goal of learning strong representations for efficient fine-tuning on downstream cardiac disease diagnosis. EchoCLR pretraining involves (i) contrastive learning, where the model is trained to identify distinct videos of the same patient, and (ii) frame reordering, where the model is trained to predict the correct of video frames after being randomly shuffled. RESULTS When fine-tuned on small portions of labeled data, EchoCLR pretraining significantly improves classification performance for left ventricular hypertrophy (LVH) and aortic stenosis (AS) over other transfer learning and SSL approaches across internal and external test sets. When fine-tuning on 10% of available training data (519 studies), an EchoCLR-pretrained model achieves 0.72 AUROC (95% CI: [0.69, 0.75]) on LVH classification, compared to 0.61 AUROC (95% CI: [0.57, 0.64]) with a standard transfer learning approach. Similarly, using 1% of available training data (53 studies), EchoCLR pretraining achieves 0.82 AUROC (95% CI: [0.79, 0.84]) on severe AS classification, compared to 0.61 AUROC (95% CI: [0.58, 0.65]) with transfer learning. CONCLUSIONS EchoCLR is unique in its ability to learn representations of echocardiogram videos and demonstrates that SSL can enable label-efficient disease classification from small amounts of labeled data.
Collapse
Affiliation(s)
- Gregory Holste
- Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX, USA
| | - Evangelos K Oikonomou
- Section of Cardiovascular Medicine, Department of Internal Medicine, Yale School of Medicine, New Haven, CT, USA
| | - Bobak J Mortazavi
- Department of Computer Science & Engineering, Texas A&M University, College Station, TX, USA
- Center for Outcomes Research and Evaluation, Yale New Haven Hospital, New Haven, CT, USA
| | - Zhangyang Wang
- Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX, USA
| | - Rohan Khera
- Section of Cardiovascular Medicine, Department of Internal Medicine, Yale School of Medicine, New Haven, CT, USA.
- Center for Outcomes Research and Evaluation, Yale New Haven Hospital, New Haven, CT, USA.
- Section of Biomedical Informatics and Data Science, Yale School of Medicine, New Haven, CT, USA.
- Section of Health Informatics, Department of Biostatistics, Yale School of Public Health, New Haven, CT, USA.
| |
Collapse
|
10
|
Xu S, Wang X, Sun Q, Dong K. MWIRGas-YOLO: Gas Leakage Detection Based on Mid-Wave Infrared Imaging. SENSORS (BASEL, SWITZERLAND) 2024; 24:4345. [PMID: 39001124 PMCID: PMC11244429 DOI: 10.3390/s24134345] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/03/2024] [Revised: 07/02/2024] [Accepted: 07/02/2024] [Indexed: 07/16/2024]
Abstract
The integration of visual algorithms with infrared imaging technology has become an effective tool for industrial gas leak detection. However, existing research has mostly focused on simple scenarios where a gas plume is clearly visible, with limited studies on detecting gas in complex scenes where target contours are blurred and contrast is low. This paper uses a cooled mid-wave infrared (MWIR) system to provide high sensitivity and fast response imaging and proposes the MWIRGas-YOLO network for detecting gas leaks in mid-wave infrared imaging. This network effectively detects low-contrast gas leakage and segments the gas plume within the scene. In MWIRGas-YOLO, it utilizes the global attention mechanism (GAM) to fully focus on gas plume targets during feature fusion, adds a small target detection layer to enhance information on small-sized targets, and employs transfer learning of similar features from visible light smoke to provide the model with prior knowledge of infrared gas features. Using a cooled mid-wave infrared imager to collect gas leak images, the experimental results show that the proposed algorithm significantly improves the performance over the original model. The segment mean average precision reached 96.1% (mAP50) and 47.6% (mAP50:95), respectively, outperforming the other mainstream algorithms. This can provide an effective reference for research on infrared imaging for gas leak detection.
Collapse
Affiliation(s)
| | - Xia Wang
- Key Laboratory of Photoelectronic Imaging Technology and System, Ministry of Education of China, Beijing Institute of Technology, Beijing 100081, China; (S.X.); (Q.S.); (K.D.)
| | | | | |
Collapse
|
11
|
Szymaszek P, Tyszka-Czochara M, Ortyl J. Application of Photoactive Compounds in Cancer Theranostics: Review on Recent Trends from Photoactive Chemistry to Artificial Intelligence. Molecules 2024; 29:3164. [PMID: 38999115 PMCID: PMC11243723 DOI: 10.3390/molecules29133164] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2024] [Revised: 06/14/2024] [Accepted: 06/25/2024] [Indexed: 07/14/2024] Open
Abstract
According to the World Health Organization (WHO) and the International Agency for Research on Cancer (IARC), the number of cancer cases and deaths worldwide is predicted to nearly double by 2030, reaching 21.7 million cases and 13 million fatalities. The increase in cancer mortality is due to limitations in the diagnosis and treatment options that are currently available. The close relationship between diagnostics and medicine has made it possible for cancer patients to receive precise diagnoses and individualized care. This article discusses newly developed compounds with potential for photodynamic therapy and diagnostic applications, as well as those already in use. In addition, it discusses the use of artificial intelligence in the analysis of diagnostic images obtained using, among other things, theranostic agents.
Collapse
Affiliation(s)
- Patryk Szymaszek
- Department of Biotechnology and Physical Chemistry, Faculty of Chemical Engineering and Technology, Cracow University of Technology, Warszawska 24, 31-155 Kraków, Poland
| | | | - Joanna Ortyl
- Department of Biotechnology and Physical Chemistry, Faculty of Chemical Engineering and Technology, Cracow University of Technology, Warszawska 24, 31-155 Kraków, Poland
- Photo HiTech Ltd., Bobrzyńskiego 14, 30-348 Kraków, Poland
- Photo4Chem Ltd., Juliusza Lea 114/416A-B, 31-133 Cracow, Poland
| |
Collapse
|
12
|
Park C, Kang JW, Lee DE, Son W, Lee SM, Park C, Kim M. W-DRAG: A joint framework of WGAN with data random augmentation optimized for generative networks for bone marrow edema detection in dual energy CT. Comput Med Imaging Graph 2024; 115:102387. [PMID: 38703602 DOI: 10.1016/j.compmedimag.2024.102387] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2023] [Revised: 04/17/2024] [Accepted: 04/17/2024] [Indexed: 05/06/2024]
Abstract
Dual-energy computed tomography (CT) is an excellent substitute for identifying bone marrow edema in magnetic resonance imaging. However, it is rarely used in practice owing to its low contrast. To overcome this problem, we constructed a framework based on deep learning techniques to screen for diseases using axial bone images and to identify the local positions of bone lesions. To address the limited availability of labeled samples, we developed a new generative adversarial network (GAN) that extends expressions beyond conventional augmentation (CA) methods based on geometric transformations. We theoretically and experimentally determined that combining the concepts of data augmentation optimized for GAN training (DAG) and Wasserstein GAN yields a considerably stable generation of synthetic images and effectively aligns their distribution with that of real images, thereby achieving a high degree of similarity. The classification model was trained using real and synthetic samples. Consequently, the GAN technique used in the diagnostic test had an improved F1 score of approximately 7.8% compared with CA. The final F1 score was 80.24%, and the recall and precision were 84.3% and 88.7%, respectively. The results obtained using the augmented samples outperformed those obtained using pure real samples without augmentation. In addition, we adopted explainable AI techniques that leverage a class activation map (CAM) and principal component analysis to facilitate visual analysis of the network's results. The framework was designed to suggest an attention map and scattering plot to visually explain the disease predictions of the network.
Collapse
Affiliation(s)
- Chunsu Park
- Department of Information Convergence Engineering, Pusan National University, Yangsan, Republic of Korea
| | - Jeong-Woon Kang
- Department of Information Convergence Engineering, Pusan National University, Yangsan, Republic of Korea
| | - Doen-Eon Lee
- Department of Information Convergence Engineering, Pusan National University, Yangsan, Republic of Korea
| | - Wookon Son
- Department of Radiology, Research Institute for Convergence of Biomedical Science and Technology, Pusan National University Yangsan Hospital, Yangsan, Republic of Korea
| | - Sang-Min Lee
- Department of Orthopedics, Research Institute for Convergence of Biomedical Science and Technology, Pusan National University Yangsan Hospital, Yangsan, Republic of Korea
| | - Chankue Park
- Department of Radiology, Research Institute for Convergence of Biomedical Science and Technology, Pusan National University Yangsan Hospital, Yangsan, Republic of Korea.
| | - MinWoo Kim
- Department of Biomedical Convergence Engineering, Pusan National University, Yangsan, Republic of Korea; Center for Artificial Intelligence Research, Pusan National University, Busan, Republic of Korea.
| |
Collapse
|
13
|
Daneshvar N, Pandita D, Erickson S, Snyder Sulmasy L, DeCamp M. Artificial Intelligence in the Provision of Health Care: An American College of Physicians Policy Position Paper. Ann Intern Med 2024; 177:964-967. [PMID: 38830215 DOI: 10.7326/m24-0146] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 06/05/2024] Open
Abstract
Internal medicine physicians are increasingly interacting with systems that implement artificial intelligence (AI) and machine learning (ML) technologies. Some physicians and health care systems are even developing their own AI models, both within and outside of electronic health record (EHR) systems. These technologies have various applications throughout the provision of health care, such as clinical documentation, diagnostic image processing, and clinical decision support. With the growing availability of vast amounts of patient data and unprecedented levels of clinician burnout, the proliferation of these technologies is cautiously welcomed by some physicians. Others think it presents challenges to the patient-physician relationship and the professional integrity of physicians. These dispositions are understandable, given the "black box" nature of some AI models, for which specifications and development methods can be closely guarded or proprietary, along with the relative lagging or absence of appropriate regulatory scrutiny and validation. This American College of Physicians (ACP) position paper describes the College's foundational positions and recommendations regarding the use of AI- and ML-enabled tools and systems in the provision of health care. Many of the College's positions and recommendations, such as those related to patient-centeredness, privacy, and transparency, are founded on principles in the ACP Ethics Manual. They are also derived from considerations for the clinical safety and effectiveness of the tools as well as their potential consequences regarding health disparities. The College calls for more research on the clinical and ethical implications of these technologies and their effects on patient health and well-being.
Collapse
Affiliation(s)
| | - Deepti Pandita
- University of California Irvine Health, Laguna Niguel, California (D.P.)
| | - Shari Erickson
- American College of Physicians, Washington, DC (N.D., S.E.)
| | | | - Matthew DeCamp
- University of Colorado Anschutz Medical Campus, Aurora, Colorado (M.D.)
| |
Collapse
|
14
|
Aden D, Zaheer S, Khan S. Possible benefits, challenges, pitfalls, and future perspective of using ChatGPT in pathology. REVISTA ESPANOLA DE PATOLOGIA : PUBLICACION OFICIAL DE LA SOCIEDAD ESPANOLA DE ANATOMIA PATOLOGICA Y DE LA SOCIEDAD ESPANOLA DE CITOLOGIA 2024; 57:198-210. [PMID: 38971620 DOI: 10.1016/j.patol.2024.04.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/29/2024] [Revised: 02/22/2024] [Accepted: 04/16/2024] [Indexed: 07/08/2024]
Abstract
The much-hyped artificial intelligence (AI) model called ChatGPT developed by Open AI can have great benefits for physicians, especially pathologists, by saving time so that they can use their time for more significant work. Generative AI is a special class of AI model, which uses patterns and structures learned from existing data and can create new data. Utilizing ChatGPT in Pathology offers a multitude of benefits, encompassing the summarization of patient records and its promising prospects in Digital Pathology, as well as its valuable contributions to education and research in this field. However, certain roadblocks need to be dealt like integrating ChatGPT with image analysis which will act as a revolution in the field of pathology by increasing diagnostic accuracy and precision. The challenges with the use of ChatGPT encompass biases from its training data, the need for ample input data, potential risks related to bias and transparency, and the potential adverse outcomes arising from inaccurate content generation. Generation of meaningful insights from the textual information which will be efficient in processing different types of image data, such as medical images, and pathology slides. Due consideration should be given to ethical and legal issues including bias.
Collapse
Affiliation(s)
- Durre Aden
- Department of Pathology, Hamdard Institute of Medical Sciences and Research, Jamia Hamdard, New Delhi, India
| | - Sufian Zaheer
- Department of Pathology, Vardhman Mahavir Medical College and Safdarjung Hospital, New Delhi, India.
| | - Sabina Khan
- Department of Pathology, Hamdard Institute of Medical Sciences and Research, Jamia Hamdard, New Delhi, India
| |
Collapse
|
15
|
Fan L, Shen Y, Lou D, Gu N. Progress in the Computer-Aided Analysis in Multiple Aspects of Nanocatalysis Research. Adv Healthc Mater 2024:e2401576. [PMID: 38936401 DOI: 10.1002/adhm.202401576] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2024] [Revised: 06/08/2024] [Indexed: 06/29/2024]
Abstract
Making the utmost of the differences and advantages of multiple disciplines, interdisciplinary integration breaks the science boundaries and accelerates the progress in mutual quests. As an organic connection of material science, enzymology, and biomedicine, nanozyme-related research is further supported by computer technology, which injects in new vitality, and contributes to in-depth understanding, unprecedented insights, and broadened application possibilities. Utilizing computer-aided first-principles method, high-speed and high-throughput mathematic, physic, and chemic models are introduced to perform atomic-level kinetic analysis for nanocatalytic reaction process, and theoretically illustrate the underlying nanozymetic mechanism and structure-function relationship. On this basis, nanozymes with desirable properties can be designed and demand-oriented synthesized without repeated trial-and-error experiments. Besides that, computational analysis and device also play an indispensable role in nanozyme-based detecting methods to realize automatic readouts with improved accuracy and reproducibility. Here, this work focuses on the crossing of nanocatalysis research and computational technology, to inspire the research in computer-aided analysis in nanozyme field to a greater extent.
Collapse
Affiliation(s)
- Lin Fan
- Medical School of Nanjing University, Nanjing, 210093, P. R. China
- School of Integrated Circuit Science and Engineering (Industry-Education Integration School), Nanjing University of Posts and Telecommunications, Nanjing, 210023, P. R. China
| | - Yilei Shen
- School of Integrated Circuit Science and Engineering (Industry-Education Integration School), Nanjing University of Posts and Telecommunications, Nanjing, 210023, P. R. China
| | - Doudou Lou
- Nanjing Institute for Food and Drug Control, Nanjing, 211198, P. R. China
| | - Ning Gu
- Medical School of Nanjing University, Nanjing, 210093, P. R. China
| |
Collapse
|
16
|
Friedrich MU, Roenn AJ, Palmisano C, Alty J, Paschen S, Deuschl G, Ip CW, Volkmann J, Muthuraman M, Peach R, Reich MM. Validation and application of computer vision algorithms for video-based tremor analysis. NPJ Digit Med 2024; 7:165. [PMID: 38906946 PMCID: PMC11192937 DOI: 10.1038/s41746-024-01153-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2023] [Accepted: 05/29/2024] [Indexed: 06/23/2024] Open
Abstract
Tremor is one of the most common neurological symptoms. Its clinical and neurobiological complexity necessitates novel approaches for granular phenotyping. Instrumented neurophysiological analyses have proven useful, but are highly resource-intensive and lack broad accessibility. In contrast, bedside scores are simple to administer, but lack the granularity to capture subtle but relevant tremor features. We utilise the open-source computer vision pose tracking algorithm Mediapipe to track hands in clinical video recordings and use the resulting time series to compute canonical tremor features. This approach is compared to marker-based 3D motion capture, wrist-worn accelerometry, clinical scoring and a second, specifically trained tremor-specific algorithm in two independent clinical cohorts. These cohorts consisted of 66 patients diagnosed with essential tremor, assessed in different task conditions and states of deep brain stimulation therapy. We find that Mediapipe-derived tremor metrics exhibit high convergent clinical validity to scores (Spearman's ρ = 0.55-0.86, p≤ .01) as well as an accuracy of up to 2.60 mm (95% CI [-3.13, 8.23]) and ≤0.21 Hz (95% CI [-0.05, 0.46]) for tremor amplitude and frequency measurements, matching gold-standard equipment. Mediapipe, but not the disease-specific algorithm, was capable of analysing videos involving complex configurational changes of the hands. Moreover, it enabled the extraction of tremor features with diagnostic and prognostic relevance, a dimension which conventional tremor scores were unable to provide. Collectively, this demonstrates that current computer vision algorithms can be transformed into an accurate and highly accessible tool for video-based tremor analysis, yielding comparable results to gold standard tremor recordings.
Collapse
Affiliation(s)
- Maximilian U Friedrich
- Center for Brain Circuit Therapeutics, Brigham and Women's Hospital, Boston, MA, USA.
- Harvard Medical School, Boston, MA, USA.
- Department of Neurology, University Hospital Wurzburg, Wuerzburg, Germany.
| | - Anna-Julia Roenn
- Department of Neurology, University Hospital Wurzburg, Wuerzburg, Germany
| | - Chiara Palmisano
- Department of Neurology, University Hospital Wurzburg, Wuerzburg, Germany
| | - Jane Alty
- Wicking Dementia Research and Education Centre, College of Health and Medicine, University of Tasmania, Hobart, Tasmania, Australia
| | | | | | - Chi Wang Ip
- Department of Neurology, University Hospital Wurzburg, Wuerzburg, Germany
| | - Jens Volkmann
- Department of Neurology, University Hospital Wurzburg, Wuerzburg, Germany
| | | | - Robert Peach
- Department of Neurology, University Hospital Wurzburg, Wuerzburg, Germany
- Department of Brain Sciences, Imperial College, London, UK
| | - Martin M Reich
- Department of Neurology, University Hospital Wurzburg, Wuerzburg, Germany.
| |
Collapse
|
17
|
Dubois C, Eigen D, Simon F, Couloigner V, Gormish M, Chalumeau M, Schmoll L, Cohen JF. Development and validation of a smartphone-based deep-learning-enabled system to detect middle-ear conditions in otoscopic images. NPJ Digit Med 2024; 7:162. [PMID: 38902477 PMCID: PMC11189910 DOI: 10.1038/s41746-024-01159-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2023] [Accepted: 06/10/2024] [Indexed: 06/22/2024] Open
Abstract
Middle-ear conditions are common causes of primary care visits, hearing impairment, and inappropriate antibiotic use. Deep learning (DL) may assist clinicians in interpreting otoscopic images. This study included patients over 5 years old from an ambulatory ENT practice in Strasbourg, France, between 2013 and 2020. Digital otoscopic images were obtained using a smartphone-attached otoscope (Smart Scope, Karl Storz, Germany) and labeled by a senior ENT specialist across 11 diagnostic classes (reference standard). An Inception-v2 DL model was trained using 41,664 otoscopic images, and its diagnostic accuracy was evaluated by calculating class-specific estimates of sensitivity and specificity. The model was then incorporated into a smartphone app called i-Nside. The DL model was evaluated on a validation set of 3,962 images and a held-out test set comprising 326 images. On the validation set, all class-specific estimates of sensitivity and specificity exceeded 98%. On the test set, the DL model achieved a sensitivity of 99.0% (95% confidence interval: 94.5-100) and a specificity of 95.2% (91.5-97.6) for the binary classification of normal vs. abnormal images; wax plugs were detected with a sensitivity of 100% (94.6-100) and specificity of 97.7% (95.0-99.1); other class-specific estimates of sensitivity and specificity ranged from 33.3% to 92.3% and 96.0% to 100%, respectively. We present an end-to-end DL-enabled system able to achieve expert-level diagnostic accuracy for identifying normal tympanic aspects and wax plugs within digital otoscopic images. However, the system's performance varied for other middle-ear conditions. Further prospective validation is necessary before wider clinical deployment.
Collapse
Affiliation(s)
| | | | - François Simon
- Department of Pediatric Otolaryngology, Necker-Enfants malades Hospital, APHP, Université Paris Cité, Paris, France
| | - Vincent Couloigner
- Department of Pediatric Otolaryngology, Necker-Enfants malades Hospital, APHP, Université Paris Cité, Paris, France
| | | | - Martin Chalumeau
- Inserm UMR1153 (CRESS), Université Paris Cité, Paris, France
- Department of General Pediatrics and Pediatric Infectious Diseases, Necker-Enfants malades Hospital, APHP, Université Paris Cité, Paris, France
| | | | - Jérémie F Cohen
- Inserm UMR1153 (CRESS), Université Paris Cité, Paris, France.
- Department of General Pediatrics and Pediatric Infectious Diseases, Necker-Enfants malades Hospital, APHP, Université Paris Cité, Paris, France.
| |
Collapse
|
18
|
Peach R, Friedrich M, Fronemann L, Muthuraman M, Schreglmann SR, Zeller D, Schrader C, Krauss JK, Schnitzler A, Wittstock M, Helmers AK, Paschen S, Kühn A, Skogseid IM, Eisner W, Mueller J, Matthies C, Reich M, Volkmann J, Ip CW. Head movement dynamics in dystonia: a multi-centre retrospective study using visual perceptive deep learning. NPJ Digit Med 2024; 7:160. [PMID: 38890413 PMCID: PMC11189529 DOI: 10.1038/s41746-024-01140-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2023] [Accepted: 05/22/2024] [Indexed: 06/20/2024] Open
Abstract
Dystonia is a neurological movement disorder characterised by abnormal involuntary movements and postures, particularly affecting the head and neck. However, current clinical assessment methods for dystonia rely on simplified rating scales which lack the ability to capture the intricate spatiotemporal features of dystonic phenomena, hindering clinical management and limiting understanding of the underlying neurobiology. To address this, we developed a visual perceptive deep learning framework that utilizes standard clinical videos to comprehensively evaluate and quantify disease states and the impact of therapeutic interventions, specifically deep brain stimulation. This framework overcomes the limitations of traditional rating scales and offers an efficient and accurate method that is rater-independent for evaluating and monitoring dystonia patients. To evaluate the framework, we leveraged semi-standardized clinical video data collected in three retrospective, longitudinal cohort studies across seven academic centres. We extracted static head angle excursions for clinical validation and derived kinematic variables reflecting naturalistic head dynamics to predict dystonia severity, subtype, and neuromodulation effects. The framework was also applied to a fully independent cohort of generalised dystonia patients for comparison between dystonia sub-types. Computer vision-derived measurements of head angle excursions showed a strong correlation with clinically assigned scores. Across comparisons, we identified consistent kinematic features from full video assessments encoding information critical to disease severity, subtype, and effects of neural circuit interventions, independent of static head angle deviations used in scoring. Our visual perceptive machine learning framework reveals kinematic pathosignatures of dystonia, potentially augmenting clinical management, facilitating scientific translation, and informing personalized precision neurology approaches.
Collapse
Affiliation(s)
- Robert Peach
- Department of Neurology, University Hospital Würzburg, Würzburg, 97080, Germany.
- Department of Brain Sciences, Imperial College London, London, UK.
| | - Maximilian Friedrich
- Department of Neurology, University Hospital Würzburg, Würzburg, 97080, Germany
- Center for Brain Circuit Therapeutics, Brigham & Women's Hospital, Boston, USA
- Harvard Medical School, Boston, USA
| | - Lara Fronemann
- Department of Neurology, University Hospital Würzburg, Würzburg, 97080, Germany
| | | | | | - Daniel Zeller
- Department of Neurology, University Hospital Würzburg, Würzburg, 97080, Germany
| | - Christoph Schrader
- Department of Neurology and Clinical Neurophysiology, Hannover Medical School, Hannover, Germany
| | - Joachim K Krauss
- Department of Neurosurgery, Hannover Medical School, Hannover, Germany
| | - Alfons Schnitzler
- Institute of Clinical Neuroscience and Medical Psychology, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
| | | | - Ann-Kristin Helmers
- Department of Neurology, UKSH, Kiel Campus Christian-Albrechts-University, Kiel, Germany
| | - Steffen Paschen
- Department of Neurology, Christian-Albrechts-University, Kiel, Germany
| | - Andrea Kühn
- Department of Neurology, Movement Disorder and Neuromodulation Unit, Charité - Universitätsmedizin, Berlin, Germany
| | - Inger Marie Skogseid
- Movement Disorders Unit, Department of Neurology, Oslo University Hospital, Rikshospitalet, Oslo, Norway
| | - Wilhelm Eisner
- Department of Neurology, Innsbruck Medical University, 6020, Innsbruck, Austria
| | - Joerg Mueller
- Klinik für Neurologie mit Stroke Unit, Vivantes Klinikum Spandau, Berlin, Germany
| | - Cordula Matthies
- Department of Neurology, University Hospital Würzburg, Würzburg, 97080, Germany
| | - Martin Reich
- Department of Neurology, University Hospital Würzburg, Würzburg, 97080, Germany
| | - Jens Volkmann
- Department of Neurology, University Hospital Würzburg, Würzburg, 97080, Germany
| | - Chi Wang Ip
- Department of Neurology, University Hospital Würzburg, Würzburg, 97080, Germany.
| |
Collapse
|
19
|
Liu J, Zhang Y, Wang K, Yavuz MC, Chen X, Yuan Y, Li H, Yang Y, Yuille A, Tang Y, Zhou Z. Universal and extensible language-vision models for organ segmentation and tumor detection from abdominal computed tomography. Med Image Anal 2024; 97:103226. [PMID: 38852215 DOI: 10.1016/j.media.2024.103226] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2023] [Revised: 03/30/2024] [Accepted: 05/27/2024] [Indexed: 06/11/2024]
Abstract
The advancement of artificial intelligence (AI) for organ segmentation and tumor detection is propelled by the growing availability of computed tomography (CT) datasets with detailed, per-voxel annotations. However, these AI models often struggle with flexibility for partially annotated datasets and extensibility for new classes due to limitations in the one-hot encoding, architectural design, and learning scheme. To overcome these limitations, we propose a universal, extensible framework enabling a single model, termed Universal Model, to deal with multiple public datasets and adapt to new classes (e.g., organs/tumors). Firstly, we introduce a novel language-driven parameter generator that leverages language embeddings from large language models, enriching semantic encoding compared with one-hot encoding. Secondly, the conventional output layers are replaced with lightweight, class-specific heads, allowing Universal Model to simultaneously segment 25 organs and six types of tumors and ease the addition of new classes. We train our Universal Model on 3410 CT volumes assembled from 14 publicly available datasets and then test it on 6173 CT volumes from four external datasets. Universal Model achieves first place on six CT tasks in the Medical Segmentation Decathlon (MSD) public leaderboard and leading performance on the Beyond The Cranial Vault (BTCV) dataset. In summary, Universal Model exhibits remarkable computational efficiency (6× faster than other dataset-specific models), demonstrates strong generalization across different hospitals, transfers well to numerous downstream tasks, and more importantly, facilitates the extensibility to new classes while alleviating the catastrophic forgetting of previously learned classes. Codes, models, and datasets are available at https://github.com/ljwztc/CLIP-Driven-Universal-Model.
Collapse
Affiliation(s)
- Jie Liu
- City University of Hong Kong, Hong Kong
| | - Yixiao Zhang
- Johns Hopkins University, United States of America
| | - Kang Wang
- University of California, San Francisco, United States of America
| | - Mehmet Can Yavuz
- University of California, San Francisco, United States of America
| | - Xiaoxi Chen
- University of Illinois Urbana-Champaign, United States of America
| | | | | | - Yang Yang
- University of California, San Francisco, United States of America
| | - Alan Yuille
- Johns Hopkins University, United States of America
| | | | - Zongwei Zhou
- Johns Hopkins University, United States of America.
| |
Collapse
|
20
|
Cheng CT, Lin HH, Hsu CP, Chen HW, Huang JF, Hsieh CH, Fu CY, Chung IF, Liao CH. Deep Learning for Automated Detection and Localization of Traumatic Abdominal Solid Organ Injuries on CT Scans. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024; 37:1113-1123. [PMID: 38366294 PMCID: PMC11169164 DOI: 10.1007/s10278-024-01038-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/25/2023] [Revised: 01/31/2024] [Accepted: 02/01/2024] [Indexed: 02/18/2024]
Abstract
Computed tomography (CT) is the most commonly used diagnostic modality for blunt abdominal trauma (BAT), significantly influencing management approaches. Deep learning models (DLMs) have shown great promise in enhancing various aspects of clinical practice. There is limited literature available on the use of DLMs specifically for trauma image evaluation. In this study, we developed a DLM aimed at detecting solid organ injuries to assist medical professionals in rapidly identifying life-threatening injuries. The study enrolled patients from a single trauma center who received abdominal CT scans between 2008 and 2017. Patients with spleen, liver, or kidney injury were categorized as the solid organ injury group, while others were considered negative cases. Only images acquired from the trauma center were enrolled. A subset of images acquired in the last year was designated as the test set, and the remaining images were utilized to train and validate the detection models. The performance of each model was assessed using metrics such as the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity, positive predictive value, and negative predictive value based on the best Youden index operating point. The study developed the models using 1302 (87%) scans for training and tested them on 194 (13%) scans. The spleen injury model demonstrated an accuracy of 0.938 and a specificity of 0.952. The accuracy and specificity of the liver injury model were reported as 0.820 and 0.847, respectively. The kidney injury model showed an accuracy of 0.959 and a specificity of 0.989. We developed a DLM that can automate the detection of solid organ injuries by abdominal CT scans with acceptable diagnostic accuracy. It cannot replace the role of clinicians, but we can expect it to be a potential tool to accelerate the process of therapeutic decisions for trauma care.
Collapse
Affiliation(s)
- Chi-Tung Cheng
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan, Taiwan
| | - Hou-Hsien Lin
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan, Taiwan
| | - Chih-Po Hsu
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan, Taiwan
| | - Huan-Wu Chen
- Department of Medical Imaging & Intervention, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan, Taiwan
| | - Jen-Fu Huang
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan, Taiwan
| | - Chi-Hsun Hsieh
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan, Taiwan
| | - Chih-Yuan Fu
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan, Taiwan
| | - I-Fang Chung
- Institute of Biomedical Informatics, National Yang Ming Chiao Tung University, Taipei, Taiwan
| | - Chien-Hung Liao
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan, Taiwan.
| |
Collapse
|
21
|
Liu J, Li J, Duan Y, Zhou Y, Fan X, Li S, Chang S. MA-MIL: Sampling point-level abnormal ECG location method via weakly supervised learning. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 250:108164. [PMID: 38718709 DOI: 10.1016/j.cmpb.2024.108164] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/11/2023] [Revised: 03/21/2024] [Accepted: 04/04/2024] [Indexed: 05/15/2024]
Abstract
BACKGROUND AND OBJECTIVE Current automatic electrocardiogram (ECG) diagnostic systems could provide classification outcomes but often lack explanations for these results. This limitation hampers their application in clinical diagnoses. Previous supervised learning could not highlight abnormal segmentation output accurately enough for clinical application without manual labeling of large ECG datasets. METHOD In this study, we present a multi-instance learning framework called MA-MIL, which has designed a multi-layer and multi-instance structure that is aggregated step by step at different scales. We evaluated our method using the public MIT-BIH dataset and our private dataset. RESULTS The results show that our model performed well in both ECG classification output and heartbeat level, sub-heartbeat level abnormal segment detection, with accuracy and F1 scores of 0.987 and 0.986 for ECG classification and 0.968 and 0.949 for heartbeat level abnormal detection, respectively. Compared to visualization methods, the IoU values of MA-MIL improved by at least 17 % and at most 31 % across all categories. CONCLUSIONS MA-MIL could accurately locate the abnormal ECG segment, offering more trustworthy results for clinical application.
Collapse
Affiliation(s)
- Jin Liu
- Division of Biomedical Engineering, China Medical University, China
| | - Jiadong Li
- Division of Biomedical Engineering, China Medical University, China
| | - Yuxin Duan
- Division of Biomedical Engineering, China Medical University, China
| | - Yang Zhou
- Division of Biomedical Engineering, China Medical University, China
| | - Xiaoxue Fan
- Division of Biomedical Engineering, China Medical University, China
| | - Shuo Li
- School of Life Sciences, China Medical University, Shenyang, China
| | - Shijie Chang
- Division of Biomedical Engineering, China Medical University, China.
| |
Collapse
|
22
|
Nolin-Lapalme A, Theriault-Lauzier P, Corbin D, Tastet O, Sharma A, Hussin JG, Kadoury S, Jiang R, Krahn AD, Gallo R, Avram R. Maximising Large Language Model Utility in Cardiovascular Care: A Practical Guide. Can J Cardiol 2024:S0828-282X(24)00415-X. [PMID: 38825181 DOI: 10.1016/j.cjca.2024.05.024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2024] [Revised: 05/03/2024] [Accepted: 05/16/2024] [Indexed: 06/04/2024] Open
Abstract
Large language models (LLMs) have emerged as powerful tools in artificial intelligence, demonstrating remarkable capabilities in natural language processing and generation. In this article, we explore the potential applications of LLMs in enhancing cardiovascular care and research. We discuss how LLMs can be used to simplify complex medical information, improve patient-physician communication, and automate tasks such as summarising medical articles and extracting key information. In addition, we highlight the role of LLMs in categorising and analysing unstructured data, such as medical notes and test results, which could revolutionise data handling and interpretation in cardiovascular research. However, we also emphasise the limitations and challenges associated with LLMs, including potential biases, reasoning opacity, and the need for rigourous validation in medical contexts. This review provides a practical guide for cardiovascular professionals to understand and harness the power of LLMs while navigating their limitations. We conclude by discussing the future directions and implications of LLMs in transforming cardiovascular care and research.
Collapse
Affiliation(s)
- Alexis Nolin-Lapalme
- Department of Medicine, Montréal Heart Institute, Université de Montréal, Montréal, Canada; Mila-Québec AI Institute, Montréal, Canada; Heartwise (heartwise.ai), Montréal Heart Institute, Montréal, Canada
| | | | - Denis Corbin
- Department of Medicine, Montréal Heart Institute, Université de Montréal, Montréal, Canada; Heartwise (heartwise.ai), Montréal Heart Institute, Montréal, Canada
| | - Olivier Tastet
- Heartwise (heartwise.ai), Montréal Heart Institute, Montréal, Canada
| | - Abhinav Sharma
- DREAM-CV Lab, Department of Cardiology, McGill University, Montréal, Canada
| | - Julie G Hussin
- Department of Medicine, Montréal Heart Institute, Université de Montréal, Montréal, Canada
| | | | - River Jiang
- Centre for Cardiovascular Innovation, Division of Cardiology, University of British Columbia, Vancouver, Canada
| | - Andrew D Krahn
- Centre for Cardiovascular Innovation, Division of Cardiology, University of British Columbia, Vancouver, Canada
| | - Richard Gallo
- Department of Medicine, Montréal Heart Institute, Université de Montréal, Montréal, Canada
| | - Robert Avram
- Department of Medicine, Montréal Heart Institute, Université de Montréal, Montréal, Canada; Polytechnique Montréal, Montréal, Canada.
| |
Collapse
|
23
|
Brahmi Z, Mahyoob M, Al-Sarem M, Algaraady J, Bousselmi K, Alblwi A. Exploring the Role of Machine Learning in Diagnosing and Treating Speech Disorders: A Systematic Literature Review. Psychol Res Behav Manag 2024; 17:2205-2232. [PMID: 38835654 PMCID: PMC11149643 DOI: 10.2147/prbm.s460283] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2024] [Accepted: 05/07/2024] [Indexed: 06/06/2024] Open
Abstract
Purpose Speech disorders profoundly impact the overall quality of life by impeding social operations and hindering effective communication. This study addresses the gap in systematic reviews concerning machine learning-based assistive technology for individuals with speech disorders. The overarching purpose is to offer a comprehensive overview of the field through a Systematic Literature Review (SLR) and provide valuable insights into the landscape of ML-based solutions and related studies. Methods The research employs a systematic approach, utilizing a Systematic Literature Review (SLR) methodology. The study extensively examines the existing literature on machine learning-based assistive technology for speech disorders. Specific attention is given to ML techniques, characteristics of exploited datasets in the training phase, speaker languages, feature extraction techniques, and the features employed by ML algorithms. Originality This study contributes to the existing literature by systematically exploring the machine learning landscape in assistive technology for speech disorders. The originality lies in the focused investigation of ML-speech recognition for impaired speech disorder users over ten years (2014-2023). The emphasis on systematic research questions related to ML techniques, dataset characteristics, languages, feature extraction techniques, and feature sets adds a unique and comprehensive perspective to the current discourse. Findings The systematic literature review identifies significant trends and critical studies published between 2014 and 2023. In the analysis of the 65 papers from prestigious journals, support vector machines and neural networks (CNN, DNN) were the most utilized ML technique (20%, 16.92%), with the most studied disease being Dysarthria (35/65, 54% studies). Furthermore, an upsurge in using neural network-based architectures, mainly CNN and DNN, was observed after 2018. Almost half of the included studies were published between 2021 and 2022).
Collapse
Affiliation(s)
- Zaki Brahmi
- Department of Computer Science, Taibah University, Madina, Kingdom of Saudi Arabia
| | - Mohammad Mahyoob
- Department of Languages and Translation, Taibah University, Madina, Kingdom of Saudi Arabia
| | - Mohammed Al-Sarem
- Department of Computer Science, Taibah University, Madina, Kingdom of Saudi Arabia
| | | | - Khadija Bousselmi
- Department of Computer Science, LISTIC, University of Savoie Mont Blanc, Chambéry, France
| | - Abdulaziz Alblwi
- Department of Computer Science, Taibah University, Madina, Kingdom of Saudi Arabia
| |
Collapse
|
24
|
Karkehabadi H, Khoshbin E, Ghasemi N, Mahavi A, Mohammad-Rahimi H, Sadr S. Deep learning for determining the difficulty of endodontic treatment: a pilot study. BMC Oral Health 2024; 24:574. [PMID: 38760686 PMCID: PMC11102254 DOI: 10.1186/s12903-024-04235-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2023] [Accepted: 04/08/2024] [Indexed: 05/19/2024] Open
Abstract
BACKGROUND To develop and validate a deep learning model for automated assessment of endodontic case difficulty from periapical radiographs. METHODS A dataset of 1,386 periapical radiographs was compiled from two clinical sites. Two dentists and two endodontists annotated the radiographs for difficulty using the "simple assessment" criteria from the American Association of Endodontists' case difficulty assessment form in the Endocase application. A classification task labeled cases as "easy" or "hard", while regression predicted overall difficulty scores. Convolutional neural networks (i.e. VGG16, ResNet18, ResNet50, ResNext50, and Inception v2) were used, with a baseline model trained via transfer learning from ImageNet weights. Other models was pre-trained using self-supervised contrastive learning (i.e. BYOL, SimCLR, MoCo, and DINO) on 20,295 unlabeled dental radiographs to learn representation without manual labels. Both models were evaluated using 10-fold cross-validation, with performance compared to seven human examiners (three general dentists and four endodontists) on a hold-out test set. RESULTS The baseline VGG16 model attained 87.62% accuracy in classifying difficulty. Self-supervised pretraining did not improve performance. Regression predicted scores with ± 3.21 score error. All models outperformed human raters, with poor inter-examiner reliability. CONCLUSION This pilot study demonstrated the feasibility of automated endodontic difficulty assessment via deep learning models.
Collapse
Affiliation(s)
- Hamed Karkehabadi
- Department of Endodontics, Dental School, Hamadan University of Medical Sciences, Hamadan, Iran
- Department of Endodontics, Dental Research Center, Hamadan University of Medical Sciences, Hamadan, Iran
| | - Elham Khoshbin
- Department of Endodontics, Dental School, Hamadan University of Medical Sciences, Hamadan, Iran
| | - Nikoo Ghasemi
- Faculty of Dentistry, Zanjan University of Medical Sciences, Zanjan, Iran
| | - Amal Mahavi
- Department of Endodontics, Dental School, Hamadan University of Medical Sciences, Hamadan, Iran
| | - Hossein Mohammad-Rahimi
- Topic Group Dental Diagnostics and Digital Dentistry, ITU/WHO Focus Group AI on Health, Berlin, Federal Republic of Germany
| | - Soroush Sadr
- Department of Endodontics, Dental School, Hamadan University of Medical Sciences, Hamadan, Iran.
- Dental School, Hamadan University of Medical Sciences, Shahid Fahmideh Street, PO Box 6517838677, Hamadan, Iran.
| |
Collapse
|
25
|
Gosavi AA, Nandgude TD, Mishra RK, Puri DB. Exploring the Potential of Artificial Intelligence as a Facilitating Tool for Formulation Development in Fluidized Bed Processor: a Comprehensive Review. AAPS PharmSciTech 2024; 25:111. [PMID: 38740666 DOI: 10.1208/s12249-024-02816-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2024] [Accepted: 04/23/2024] [Indexed: 05/16/2024] Open
Abstract
This in-depth study looks into how artificial intelligence (AI) could be used to make formulation development easier in fluidized bed processes (FBP). FBP is complex and involves numerous variables, making optimization challenging. Various AI techniques have addressed this challenge, including machine learning, neural networks, genetic algorithms, and fuzzy logic. By integrating AI with experimental design, process modeling, and optimization strategies, intelligent systems for FBP can be developed. The advantages of AI in this context include improved process understanding, reduced time and cost, enhanced product quality, and robust formulation optimization. However, data availability, model interpretability, and regulatory compliance challenges must be addressed. Case studies demonstrate successful applications of AI in decision-making, process outcome prediction, and scale-up. AI can improve efficiency, quality, and cost-effectiveness in significant ways. Still, it is important to think carefully about data quality, how easy it is to understand, and how to follow the rules. Future research should focus on fully harnessing the potential of AI to advance formulation development in FBP.
Collapse
Affiliation(s)
- Aachal A Gosavi
- Department of Pharmaceutics, Dr. D. Y. Patil Institute of Pharmaceutical Sciences and Research, Pimpri, Pune, India
| | - Tanaji D Nandgude
- Department of Pharmaceutics, JSPM University's School of Pharmaceutical Sciences, Wagholi, Pune, India
| | - Rakesh K Mishra
- Department of Pharmaceutics, Dr. D. Y. Patil Institute of Pharmaceutical Sciences and Research, Pimpri, Pune, India.
| | - Dhiraj B Puri
- Department of Mechanical Engineering, Birla Institute of Technology and Science-Pilani, K K Birla Goa Campus, Zuarinagar, Sancoale, Goa, India
| |
Collapse
|
26
|
Song AH, Williams M, Williamson DFK, Chow SSL, Jaume G, Gao G, Zhang A, Chen B, Baras AS, Serafin R, Colling R, Downes MR, Farré X, Humphrey P, Verrill C, True LD, Parwani AV, Liu JTC, Mahmood F. Analysis of 3D pathology samples using weakly supervised AI. Cell 2024; 187:2502-2520.e17. [PMID: 38729110 PMCID: PMC11168832 DOI: 10.1016/j.cell.2024.03.035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Revised: 01/15/2024] [Accepted: 03/25/2024] [Indexed: 05/12/2024]
Abstract
Human tissue, which is inherently three-dimensional (3D), is traditionally examined through standard-of-care histopathology as limited two-dimensional (2D) cross-sections that can insufficiently represent the tissue due to sampling bias. To holistically characterize histomorphology, 3D imaging modalities have been developed, but clinical translation is hampered by complex manual evaluation and lack of computational platforms to distill clinical insights from large, high-resolution datasets. We present TriPath, a deep-learning platform for processing tissue volumes and efficiently predicting clinical outcomes based on 3D morphological features. Recurrence risk-stratification models were trained on prostate cancer specimens imaged with open-top light-sheet microscopy or microcomputed tomography. By comprehensively capturing 3D morphologies, 3D volume-based prognostication achieves superior performance to traditional 2D slice-based approaches, including clinical/histopathological baselines from six certified genitourinary pathologists. Incorporating greater tissue volume improves prognostic performance and mitigates risk prediction variability from sampling bias, further emphasizing the value of capturing larger extents of heterogeneous morphology.
Collapse
Affiliation(s)
- Andrew H Song
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Mane Williams
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA; Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Drew F K Williamson
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Sarah S L Chow
- Department of Mechanical Engineering, Bioengineering, and Laboratory Medicine & Pathology, University of Washington, Seattle, WA, USA
| | - Guillaume Jaume
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Gan Gao
- Department of Mechanical Engineering, Bioengineering, and Laboratory Medicine & Pathology, University of Washington, Seattle, WA, USA
| | - Andrew Zhang
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA; Harvard-MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Bowen Chen
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Alexander S Baras
- Department of Pathology, Johns Hopkins University School of Medicine, Baltimore, MD, USA; Department of Biomedical Engineering, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Robert Serafin
- Department of Mechanical Engineering, Bioengineering, and Laboratory Medicine & Pathology, University of Washington, Seattle, WA, USA
| | - Richard Colling
- Nuffield Department of Surgical Sciences, University of Oxford, UK; Department of Cellular Pathology, Oxford University Hospitals NHS Foundations Trust, John Radcliffe Hospital, Oxford, UK
| | - Michelle R Downes
- Sunnybrook Health Sciences Centre, University of Toronto, Toronto, ON, Canada
| | - Xavier Farré
- Public Health Agency of Catalonia, Lleida, Spain
| | - Peter Humphrey
- Department of Pathology, Yale School of Medicine, New Haven, CT, USA
| | - Clare Verrill
- Nuffield Department of Surgical Sciences, University of Oxford, UK; Department of Cellular Pathology, Oxford University Hospitals NHS Foundations Trust, John Radcliffe Hospital, Oxford, UK; NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Lawrence D True
- Department of Laboratory Medicine & Pathology, University of Washington School of Medicine, Seattle, WA, USA
| | - Anil V Parwani
- Department of Pathology, The Ohio State University, Columbus, OH, USA
| | - Jonathan T C Liu
- Department of Mechanical Engineering, Bioengineering, and Laboratory Medicine & Pathology, University of Washington, Seattle, WA, USA.
| | - Faisal Mahmood
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA.
| |
Collapse
|
27
|
Barmpas K, Panagakis Y, Zoumpourlis G, Adamos DA, Laskaris N, Zafeiriou S. A causal perspective on brainwave modeling for brain-computer interfaces. J Neural Eng 2024; 21:036001. [PMID: 38621380 DOI: 10.1088/1741-2552/ad3eb5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2023] [Accepted: 04/15/2024] [Indexed: 04/17/2024]
Abstract
Objective. Machine learning (ML) models have opened up enormous opportunities in the field of brain-computer Interfaces (BCIs). Despite their great success, they usually face severe limitations when they are employed in real-life applications outside a controlled laboratory setting.Approach. Mixing causal reasoning, identifying causal relationships between variables of interest, with brainwave modeling can change one's viewpoint on some of these major challenges which can be found in various stages in the ML pipeline, ranging from data collection and data pre-processing to training methods and techniques.Main results. In this work, we employ causal reasoning and present a framework aiming to breakdown and analyze important challenges of brainwave modeling for BCIs.Significance. Furthermore, we present how general ML practices as well as brainwave-specific techniques can be utilized and solve some of these identified challenges. And finally, we discuss appropriate evaluation schemes in order to measure these techniques' performance and efficiently compare them with other methods that will be developed in the future.
Collapse
Affiliation(s)
- Konstantinos Barmpas
- Department of Computing, Imperial College London, London SW7 2RH, United Kingdom
- Cogitat Ltd, London, United Kingdom
| | - Yannis Panagakis
- Department of Informatics and Telecommunications, National and Kapodistrian University of Athens, Athens 15784, Greece
- Archimedes Research Unit, Research Center Athena, Athens 15125, Greece
- Cogitat Ltd, London, United Kingdom
| | | | - Dimitrios A Adamos
- Department of Computing, Imperial College London, London SW7 2RH, United Kingdom
- Cogitat Ltd, London, United Kingdom
| | - Nikolaos Laskaris
- School of Informatics, Aristotle University of Thessaloniki, Thessaloniki 54124, Greece
- Cogitat Ltd, London, United Kingdom
| | - Stefanos Zafeiriou
- Department of Computing, Imperial College London, London SW7 2RH, United Kingdom
- Cogitat Ltd, London, United Kingdom
| |
Collapse
|
28
|
Wolcott ZC, English SW. Artificial intelligence to enhance prehospital stroke diagnosis and triage: a perspective. Front Neurol 2024; 15:1389056. [PMID: 38756217 PMCID: PMC11096539 DOI: 10.3389/fneur.2024.1389056] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2024] [Accepted: 04/22/2024] [Indexed: 05/18/2024] Open
Abstract
As health systems organize to deliver the highest quality stroke care to their patients, there is increasing emphasis being placed on prehospital stroke recognition, accurate diagnosis, and efficient triage to improve outcomes after stroke. Emergency medical services (EMS) personnel currently rely heavily on dispatch accuracy, stroke screening tools, bypass protocols and prehospital notification to care for patients with suspected stroke, but novel tools including mobile stroke units and telemedicine-enabled ambulances are already changing the landscape of prehospital stroke care. Herein, the authors provide our perspective on the current state of prehospital stroke diagnosis and triage including several of these emerging trends. Then, we provide commentary to highlight potential artificial intelligence (AI) applications to improve stroke detection, improve accurate and timely dispatch, enhance EMS training and performance, and develop novel stroke diagnostic tools for prehospital use.
Collapse
|
29
|
Kaji ES, Grove AF, Taunton MJ. Present and Future Optimization of Orthopaedic Care Through Machine Learning Algorithms. J Arthroplasty 2024; 39:1171-1172. [PMID: 38642965 DOI: 10.1016/j.arth.2024.03.043] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 04/22/2024] Open
|
30
|
Farabi Maleki S, Yousefi M, Afshar S, Pedrammehr S, Lim CP, Jafarizadeh A, Asadi H. Artificial Intelligence for Multiple Sclerosis Management Using Retinal Images: Pearl, Peaks, and Pitfalls. Semin Ophthalmol 2024; 39:271-288. [PMID: 38088176 DOI: 10.1080/08820538.2023.2293030] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/27/2023] [Accepted: 11/23/2023] [Indexed: 03/28/2024]
Abstract
Multiple sclerosis (MS) is a complex autoimmune disease characterized by inflammatory processes, demyelination, neurodegeneration, and axonal damage within the central nervous system (CNS). Retinal imaging, particularly Optical coherence tomography (OCT), has emerged as a crucial tool for investigating MS-related retinal injury. The integration of artificial intelligence(AI) has shown promise in enhancing OCT analysis for MS. Researchers are actively utilizing AI algorithms to accurately detect and classify MS-related abnormalities, leading to improved efficiency in diagnosis, monitoring, and personalized treatment planning. The prognostic value of AI in predicting MS disease progression has garnered substantial attention. Machine learning (ML) and deep learning (DL) algorithms can analyze longitudinal OCT data to forecast the course of the disease, providing critical information for personalized treatment planning and improved patient outcomes. Early detection of high-risk patients allows for targeted interventions to mitigate disability progression effectively. As such, AI-driven approaches yielded remarkable abilities in classifying distinct MS subtypes based on retinal features, aiding in disease characterization and guiding tailored therapeutic strategies. Additionally, these algorithms have enhanced the accuracy and efficiency of OCT image segmentation, streamlined diagnostic processes, and reduced human error. This study reviews the current research studies on the integration of AI,including ML and DL algorithms, with OCT in the context of MS. It examines the advancements, challenges, potential prospects, and ethical concerns of AI-powered techniques in enhancing MS diagnosis, monitoring disease progression, revolutionizing patient care, the development of patient screening tools, and supported clinical decision-making based on OCT images.
Collapse
Affiliation(s)
| | - Milad Yousefi
- Faculty of Mathematics, Statistics and Computer Sciences, University of Tabriz, Tabriz, Iran
| | - Sayeh Afshar
- Nikookari Eye Center, Tabriz University of Medical Sciences, Tabriz, Iran
| | | | - Chee Peng Lim
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Burwood, Australia
| | - Ali Jafarizadeh
- Nikookari Eye Center, Tabriz University of Medical Sciences, Tabriz, Iran
| | - Houshyar Asadi
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Burwood, Australia
| |
Collapse
|
31
|
Yang CN, Chen WL, Yeh HH, Chu HS, Wu JH, Hsieh YT. Convolutional Neural Network-Based Prediction of Axial Length Using Color Fundus Photography. Transl Vis Sci Technol 2024; 13:23. [PMID: 38809531 DOI: 10.1167/tvst.13.5.23] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/30/2024] Open
Abstract
Purpose To develop convolutional neural network (CNN)-based models for predicting the axial length (AL) using color fundus photography (CFP) and explore associated clinical and structural characteristics. Methods This study enrolled 1105 fundus images from 467 participants with ALs ranging from 19.91 to 32.59 mm, obtained at National Taiwan University Hospital between 2020 and 2021. The AL measurements obtained from a scanning laser interferometer served as the gold standard. The accuracy of prediction was compared among CNN-based models with different inputs, including CFP, age, and/or sex. Heatmaps were interpreted by integrated gradients. Results Using age, sex, and CFP as input, the mean ± standard deviation absolute error (MAE) for AL prediction by the model was 0.771 ± 0.128 mm, outperforming models that used age and sex alone (1.263 ± 0.115 mm; P < 0.001) and CFP alone (0.831 ± 0.216 mm; P = 0.016) by 39.0% and 7.31%, respectively. The removal of relatively poor-quality CFPs resulted in a slight MAE reduction to 0.759 ± 0.120 mm without statistical significance (P = 0.24). The inclusion of age and CFP improved prediction accuracy by 5.59% (P = 0.043), while adding sex had no significant improvement (P = 0.41). The optic disc and temporal peripapillary area were highlighted as the focused areas on the heatmaps. Conclusions Deep learning-based prediction of AL using CFP was fairly accurate and enhanced by age inclusion. The optic disc and temporal peripapillary area may contain crucial structural information for AL prediction in CFP. Translational Relevance This study might aid AL assessments and the understanding of the morphologic characteristics of the fundus related to AL.
Collapse
Affiliation(s)
- Che-Ning Yang
- School of Medicine, National Taiwan University, Taipei, Taiwan
| | - Wei-Li Chen
- School of Medicine, National Taiwan University, Taipei, Taiwan
- Department of Ophthalmology, National Taiwan University Hospital, Taipei, Taiwan
| | - Hsu-Hang Yeh
- Department of Ophthalmology, National Taiwan University Hospital, Taipei, Taiwan
| | - Hsiao-Sang Chu
- Department of Ophthalmology, National Taiwan University Hospital, Taipei, Taiwan
- Graduate Institute of Clinical Medicine, College of Medicine, National Taiwan University, Taipei, Taiwan
| | - Jo-Hsuan Wu
- Shiley Eye Institute and Viterbi Family Department of Ophthalmology, University of California San Diego, La Jolla, CA, USA
| | - Yi-Ting Hsieh
- School of Medicine, National Taiwan University, Taipei, Taiwan
- Department of Ophthalmology, National Taiwan University Hospital, Taipei, Taiwan
| |
Collapse
|
32
|
Ayhan MS, Neubauer J, Uzel MM, Gelisken F, Berens P. Interpretable detection of epiretinal membrane from optical coherence tomography with deep neural networks. Sci Rep 2024; 14:8484. [PMID: 38605115 PMCID: PMC11009346 DOI: 10.1038/s41598-024-57798-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2022] [Accepted: 03/21/2024] [Indexed: 04/13/2024] Open
Abstract
This study aimed to automatically detect epiretinal membranes (ERM) in various OCT-scans of the central and paracentral macula region and classify them by size using deep-neural-networks (DNNs). To this end, 11,061 OCT-images were included and graded according to the presence of an ERM and its size (small 100-1000 µm, large > 1000 µm). The data set was divided into training, validation and test sets (75%, 10%, 15% of the data, respectively). An ensemble of DNNs was trained and saliency maps were generated using Guided-Backprob. OCT-scans were also transformed into a one-dimensional-value using t-SNE analysis. The DNNs' receiver-operating-characteristics on the test set showed a high performance for no-ERM, small-ERM and large-ERM cases (AUC: 0.99, 0.92, 0.99, respectively; 3-way accuracy: 89%), with small-ERMs being the most difficult ones to detect. t-SNE analysis sorted cases by size and, in particular, revealed increased classification uncertainty at the transitions between groups. Saliency maps reliably highlighted ERM, regardless of the presence of other OCT features (i.e. retinal-thickening, intraretinal pseudo-cysts, epiretinal-proliferation) and entities such as ERM-retinoschisis, macular-pseudohole and lamellar-macular-hole. This study showed therefore that DNNs can reliably detect and grade ERMs according to their size not only in the fovea but also in the paracentral region. This is also achieved in cases of hard-to-detect, small-ERMs. In addition, the generated saliency maps can be used to highlight small-ERMs that might otherwise be missed. The proposed model could be used for screening-programs or decision-support-systems in the future.
Collapse
Affiliation(s)
- Murat Seçkin Ayhan
- Institute for Ophthalmic Research, University of Tübingen, Elfriede Aulhorn Str. 7, 72076, Tübingen, Germany
| | - Jonas Neubauer
- University Eye Clinic, University of Tübingen, Tübingen, Germany
| | - Mehmet Murat Uzel
- University Eye Clinic, University of Tübingen, Tübingen, Germany
- Department of Ophthalmology, Balıkesir University School of Medicine, Balıkesir, Turkey
| | - Faik Gelisken
- University Eye Clinic, University of Tübingen, Tübingen, Germany.
| | - Philipp Berens
- Institute for Ophthalmic Research, University of Tübingen, Elfriede Aulhorn Str. 7, 72076, Tübingen, Germany.
- Tübingen AI Center, Tübingen, Germany.
| |
Collapse
|
33
|
Walter C, Weissert C, Gizewski E, Burckhardt I, Mannsperger H, Hänselmann S, Busch W, Zimmermann S, Nolte O. Performance evaluation of machine-assisted interpretation of Gram stains from positive blood cultures. J Clin Microbiol 2024; 62:e0087623. [PMID: 38506525 PMCID: PMC11005413 DOI: 10.1128/jcm.00876-23] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2023] [Accepted: 02/24/2024] [Indexed: 03/21/2024] Open
Abstract
Manual microscopy of Gram stains from positive blood cultures (PBCs) is crucial for diagnosing bloodstream infections but remains labor intensive, time consuming, and subjective. This study aimed to evaluate a scan and analysis system that combines fully automated digital microscopy with deep convolutional neural networks (CNNs) to assist the interpretation of Gram stains from PBCs for routine laboratory use. The CNN was trained to classify images of Gram stains based on staining and morphology into seven different classes: background/false-positive, Gram-positive cocci in clusters (GPCCL), Gram-positive cocci in pairs (GPCP), Gram-positive cocci in chains (GPCC), rod-shaped bacilli (RSB), yeasts, and polymicrobial specimens. A total of 1,555 Gram-stained slides of PBCs were scanned, pre-classified, and reviewed by medical professionals. The results of assisted Gram stain interpretation were compared to those of manual microscopy and cultural species identification by matrix-assisted laser desorption/ionization time-of-flight mass spectrometry (MALDI-TOF MS). The comparison of assisted Gram stain interpretation and manual microscopy yielded positive/negative percent agreement values of 95.8%/98.0% (GPCCL), 87.6%/99.3% (GPCP/GPCC), 97.4%/97.8% (RSB), 83.3%/99.3% (yeasts), and 87.0%/98.5% (negative/false positive). The assisted Gram stain interpretation, when compared to MALDI-TOF MS species identification, also yielded similar results. During the analytical performance study, assisted interpretation showed excellent reproducibility and repeatability. Any microorganism in PBCs should be detectable at the determined limit of detection of 105 CFU/mL. Although the CNN-based interpretation of Gram stains from PBCs is not yet ready for clinical implementation, it has potential for future integration and advancement.
Collapse
Affiliation(s)
- Christian Walter
- Department of Infectious Diseases, Medical Microbiology and Hygiene, Medical Faculty Heidelberg, Heidelberg University, Heidelberg, Germany
- University Hospital Heidelberg, Heidelberg, Germany
| | - Christoph Weissert
- Division of Human Microbiology, Centre for Laboratory Medicine, St. Gall, Switzerland
| | - Eve Gizewski
- MetaSystems Hard & Software GmbH, Altlussheim, Germany
| | - Irene Burckhardt
- Department of Infectious Diseases, Medical Microbiology and Hygiene, Medical Faculty Heidelberg, Heidelberg University, Heidelberg, Germany
- University Hospital Heidelberg, Heidelberg, Germany
| | | | | | | | - Stefan Zimmermann
- Department of Infectious Diseases, Medical Microbiology and Hygiene, Medical Faculty Heidelberg, Heidelberg University, Heidelberg, Germany
- University Hospital Heidelberg, Heidelberg, Germany
| | - Oliver Nolte
- Division of Human Microbiology, Centre for Laboratory Medicine, St. Gall, Switzerland
| |
Collapse
|
34
|
Ahn S, Lee HS. Applicability of Spatial Technology in Cancer Research. Cancer Res Treat 2024; 56:343-356. [PMID: 38291743 PMCID: PMC11016655 DOI: 10.4143/crt.2023.1302] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2023] [Accepted: 01/29/2024] [Indexed: 02/01/2024] Open
Abstract
This review explores spatial mapping technologies in cancer research, highlighting their crucial role in understanding the complexities of the tumor microenvironment (TME). The TME, which is an intricate ecosystem of diverse cell types, has a significant impact on tumor dynamics and treatment outcomes. This review closely examines cutting-edge spatial mapping technologies, categorizing them into capture-, imaging-, and antibody-based approaches. Each technology was scrutinized for its advantages and disadvantages, factoring in aspects such as spatial profiling area, multiplexing capabilities, and resolution. Additionally, we draw attention to the nuanced choices researchers face, with capture-based methods lending themselves to hypothesis generation, and imaging/antibody-based methods that fit neatly into hypothesis testing. Looking ahead, we anticipate a scenario in which multi-omics data are seamlessly integrated, artificial intelligence enhances data analysis, and spatiotemporal profiling opens up new dimensions.
Collapse
Affiliation(s)
- Sangjeong Ahn
- Department of Pathology, Korea University Anam Hospital, Korea University College of Medicine, Seoul, Korea
- Artificial Intelligence Center, Korea University Anam Hospital, Korea University College of Medicine, Seoul, Korea
- Department of Medical Informatics, Korea University College of Medicine, Seoul, Korea
| | - Hye Seung Lee
- Department of Pathology, Seoul National University Hospital, Seoul National University College of Medicine, Seoul, Korea
- Cancer Research Institute, Seoul National University College of Medicine, Seoul, Korea
| |
Collapse
|
35
|
Roth CJ, Petersilge C, Clunie D, Towbin AJ, Cram D, Primo R, Li X, Berkowitz SJ, Barnosky V, Krupinski EA. HIMSS-SIIM Enterprise Imaging Community White Papers: Reflections and Future Directions. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024; 37:429-443. [PMID: 38336948 PMCID: PMC11031499 DOI: 10.1007/s10278-024-00992-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 01/23/2024] [Indexed: 02/12/2024]
Affiliation(s)
| | - Cheryl Petersilge
- Vidagos University of Pittsburgh School of Medicine, Pittsburgh, PA, USA
| | | | - Alexander J Towbin
- Department of Radiology, Cincinnati Children's Hospital, Cincinnati, OH, USA.
- Department of Radiology, University of Cincinnati College of Medicine, Cincinnati, OH, USA.
| | - Dawn Cram
- PaxeraHealth, 85 Wells Ave Suite 120, Newton, MA, 02459, USA
| | - Rik Primo
- Primo Medical Imaging Informatics Inc, Chicago, IL, USA
| | - Xin Li
- Department of Radiology, Hospital of the University of Pennsylvania, Philadelphia, PA, USA
| | - Seth J Berkowitz
- Department of Radiology, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, 02115, USA
| | - Victoria Barnosky
- Robert Morris University, Moon Township, Suazio, Philadelphia, PA, USA
| | | |
Collapse
|
36
|
Quistberg DA. Potential of artificial intelligence in injury prevention research and practice. Inj Prev 2024; 30:89-91. [PMID: 38307714 PMCID: PMC11003389 DOI: 10.1136/ip-2023-045203] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2023] [Accepted: 01/18/2024] [Indexed: 02/04/2024]
Abstract
There is increasing interest and use of artificial Intelligence algorithms and methods in biomedical research and practice, particularly as the technology has made significant advances in the past decade and become more accessible to more disciplines. This editorial briefly reviews this technology and its potential for injury prevention research and practice, proposing ways that it can be used to advance the discipline, as well as the potential pitfalls, concerns and biases that accompany it.
Collapse
Affiliation(s)
- D Alex Quistberg
- Urban Health Collaborative, Drexel University, Philadelphia, Pennsylvania, USA
- Environmental & Occupational Health, Drexel University, Philadelphia, Pennsylvania, USA
| |
Collapse
|
37
|
Wei ML, Tada M, So A, Torres R. Artificial intelligence and skin cancer. Front Med (Lausanne) 2024; 11:1331895. [PMID: 38566925 PMCID: PMC10985205 DOI: 10.3389/fmed.2024.1331895] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2023] [Accepted: 02/26/2024] [Indexed: 04/04/2024] Open
Abstract
Artificial intelligence is poised to rapidly reshape many fields, including that of skin cancer screening and diagnosis, both as a disruptive and assistive technology. Together with the collection and availability of large medical data sets, artificial intelligence will become a powerful tool that can be leveraged by physicians in their diagnoses and treatment plans for patients. This comprehensive review focuses on current progress toward AI applications for patients, primary care providers, dermatologists, and dermatopathologists, explores the diverse applications of image and molecular processing for skin cancer, and highlights AI's potential for patient self-screening and improving diagnostic accuracy for non-dermatologists. We additionally delve into the challenges and barriers to clinical implementation, paths forward for implementation and areas of active research.
Collapse
Affiliation(s)
- Maria L. Wei
- Department of Dermatology, University of California, San Francisco, San Francisco, CA, United States
- Dermatology Service, San Francisco VA Health Care System, San Francisco, CA, United States
| | - Mikio Tada
- Institute for Neurodegenerative Diseases, University of California, San Francisco, San Francisco, CA, United States
| | - Alexandra So
- School of Medicine, University of California, San Francisco, San Francisco, CA, United States
| | - Rodrigo Torres
- Dermatology Service, San Francisco VA Health Care System, San Francisco, CA, United States
| |
Collapse
|
38
|
Shahid MS, French AP, Valstar MF, Yakubov GE. Research in methodologies for modelling the oral cavity. Biomed Phys Eng Express 2024; 10:032001. [PMID: 38350128 DOI: 10.1088/2057-1976/ad28cc] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2023] [Accepted: 02/13/2024] [Indexed: 02/15/2024]
Abstract
The paper aims to explore the current state of understanding surrounding in silico oral modelling. This involves exploring methodologies, technologies and approaches pertaining to the modelling of the whole oral cavity; both internally and externally visible structures that may be relevant or appropriate to oral actions. Such a model could be referred to as a 'complete model' which includes consideration of a full set of facial features (i.e. not only mouth) as well as synergistic stimuli such as audio and facial thermal data. 3D modelling technologies capable of accurately and efficiently capturing a complete representation of the mouth for an individual have broad applications in the study of oral actions, due to their cost-effectiveness and time efficiency. This review delves into the field of clinical phonetics to classify oral actions pertaining to both speech and non-speech movements, identifying how the various vocal organs play a role in the articulatory and masticatory process. Vitaly, it provides a summation of 12 articulatory recording methods, forming a tool to be used by researchers in identifying which method of recording is appropriate for their work. After addressing the cost and resource-intensive limitations of existing methods, a new system of modelling is proposed that leverages external to internal correlation modelling techniques to create a more efficient models of the oral cavity. The vision is that the outcomes will be applicable to a broad spectrum of oral functions related to physiology, health and wellbeing, including speech, oral processing of foods as well as dental health. The applications may span from speech correction, designing foods for the aging population, whilst in the dental field we would be able to gain information about patient's oral actions that would become part of creating a personalised dental treatment plan.
Collapse
Affiliation(s)
| | - Andrew P French
- School of Computer Science, University of Nottingham, NG8 1BB, United Kingdom
- School of Biosciences, University of Nottingham, LE12 5RD, United Kingdom
| | - Michel F Valstar
- School of Computer Science, University of Nottingham, NG8 1BB, United Kingdom
| | - Gleb E Yakubov
- School of Biosciences, University of Nottingham, LE12 5RD, United Kingdom
| |
Collapse
|
39
|
Luo H, Yin W, Wang J, Zhang G, Liang W, Luo J, Yan C. Drug-drug interactions prediction based on deep learning and knowledge graph: A review. iScience 2024; 27:109148. [PMID: 38405609 PMCID: PMC10884936 DOI: 10.1016/j.isci.2024.109148] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/27/2024] Open
Abstract
Drug-drug interactions (DDIs) can produce unpredictable pharmacological effects and lead to adverse events that have the potential to cause irreversible damage to the organism. Traditional methods to detect DDIs through biological or pharmacological analysis are time-consuming and expensive, therefore, there is an urgent need to develop computational methods to effectively predict drug-drug interactions. Currently, deep learning and knowledge graph techniques which can effectively extract features of entities have been widely utilized to develop DDI prediction methods. In this research, we aim to systematically review DDI prediction researches applying deep learning and graph knowledge. The available biomedical data and public databases related to drugs are firstly summarized in this review. Then, we discuss the existing drug-drug interactions prediction methods which have utilized deep learning and knowledge graph techniques and group them into three main classes: deep learning-based methods, knowledge graph-based methods, and methods that combine deep learning with knowledge graph. We comprehensively analyze the commonly used drug related data and various DDI prediction methods, and compare these prediction methods on benchmark datasets. Finally, we briefly discuss the challenges related to drug-drug interactions prediction, including asymmetric DDIs prediction and high-order DDI prediction.
Collapse
Affiliation(s)
- Huimin Luo
- School of Computer and Information Engineering, Henan University, Kaifeng, China
- Henan Key Laboratory of Big Data Analysis and Processing, Henan University, Kaifeng, China
| | - Weijie Yin
- School of Computer and Information Engineering, Henan University, Kaifeng, China
| | - Jianlin Wang
- School of Computer and Information Engineering, Henan University, Kaifeng, China
- Academy for Advanced Interdisciplinary Studies, Zhengzhou, China
| | - Ge Zhang
- School of Computer and Information Engineering, Henan University, Kaifeng, China
- Henan Key Laboratory of Big Data Analysis and Processing, Henan University, Kaifeng, China
| | - Wenjuan Liang
- School of Computer and Information Engineering, Henan University, Kaifeng, China
| | - Junwei Luo
- College of Computer Science and Technology, Henan Polytechnic University, Jiaozuo, China
| | - Chaokun Yan
- School of Computer and Information Engineering, Henan University, Kaifeng, China
- Academy for Advanced Interdisciplinary Studies, Zhengzhou, China
| |
Collapse
|
40
|
Wang S, Han J, Huang J, Islam K, Shi Y, Zhou Y, Kim D, Zhou J, Lian Z, Liu Y, Huang J. Deep learning-based predictive classification of functional subpopulations of hematopoietic stem cells and multipotent progenitors. Stem Cell Res Ther 2024; 15:74. [PMID: 38475857 PMCID: PMC10935795 DOI: 10.1186/s13287-024-03682-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2023] [Accepted: 02/23/2024] [Indexed: 03/14/2024] Open
Abstract
BACKGROUND Hematopoietic stem cells (HSCs) and multipotent progenitors (MPPs) play a pivotal role in maintaining lifelong hematopoiesis. The distinction between stem cells and other progenitors, as well as the assessment of their functions, has long been a central focus in stem cell research. In recent years, deep learning has emerged as a powerful tool for cell image analysis and classification/prediction. METHODS In this study, we explored the feasibility of employing deep learning techniques to differentiate murine HSCs and MPPs based solely on their morphology, as observed through light microscopy (DIC) images. RESULTS After rigorous training and validation using extensive image datasets, we successfully developed a three-class classifier, referred to as the LSM model, capable of reliably distinguishing long-term HSCs, short-term HSCs, and MPPs. The LSM model extracts intrinsic morphological features unique to different cell types, irrespective of the methods used for cell identification and isolation, such as surface markers or intracellular GFP markers. Furthermore, employing the same deep learning framework, we created a two-class classifier that effectively discriminates between aged HSCs and young HSCs. This discovery is particularly significant as both cell types share identical surface markers yet serve distinct functions. This classifier holds the potential to offer a novel, rapid, and efficient means of assessing the functional states of HSCs, thus obviating the need for time-consuming transplantation experiments. CONCLUSION Our study represents the pioneering use of deep learning to differentiate HSCs and MPPs under steady-state conditions. This novel and robust deep learning-based platform will provide a basis for the future development of a new generation stem cell identification and separation system. It may also provide new insight into the molecular mechanisms underlying stem cell self-renewal.
Collapse
Affiliation(s)
- Shen Wang
- Department of Mechanical Engineering and Mechanics, Lehigh University, Bethlehem, PA, USA
| | - Jianzhong Han
- Coriell Institute for Medical Research, Camden, NJ, USA
| | - Jingru Huang
- Shanghai Key Laboratory of Medical Epigenetics, Laboratory of Cancer Epigenetics, Institutes of Biomedical Sciences, Medical College of Fudan University, Chinese Academy of Medical Sciences, Shanghai, People's Republic of China
| | - Khayrul Islam
- Department of Mechanical Engineering and Mechanics, Lehigh University, Bethlehem, PA, USA
| | - Yuheng Shi
- Shanghai Key Laboratory of Medical Epigenetics, Laboratory of Cancer Epigenetics, Institutes of Biomedical Sciences, Medical College of Fudan University, Chinese Academy of Medical Sciences, Shanghai, People's Republic of China
| | - Yuyuan Zhou
- Department of Bioengineering, Lehigh University, Bethlehem, PA, USA
| | - Dongwook Kim
- Coriell Institute for Medical Research, Camden, NJ, USA
| | - Jane Zhou
- Health and Human Biology, Brown University, Providence, RI, USA
| | - Zhaorui Lian
- Coriell Institute for Medical Research, Camden, NJ, USA
| | - Yaling Liu
- Department of Mechanical Engineering and Mechanics, Lehigh University, Bethlehem, PA, USA.
- Department of Bioengineering, Lehigh University, Bethlehem, PA, USA.
| | - Jian Huang
- Coriell Institute for Medical Research, Camden, NJ, USA.
- Cooper Medical School of Rowan University, Camden, NJ, USA.
- Center for Metabolic Disease Research, Temple University Lewis Katz School of Medicine, Philadelphia, PA, USA.
| |
Collapse
|
41
|
Luo X, Li P, Chen H, Zhou K, Piao S, Yang L, Hu B, Geng D. Automatic segmentation of hepatocellular carcinoma on dynamic contrast-enhanced MRI based on deep learning. Phys Med Biol 2024; 69:065008. [PMID: 38330492 DOI: 10.1088/1361-6560/ad2790] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2023] [Accepted: 02/08/2024] [Indexed: 02/10/2024]
Abstract
Objective. Precise hepatocellular carcinoma (HCC) detection is crucial for clinical management. While studies focus on computed tomography-based automatic algorithms, there is a rareness of research on automatic detection based on dynamic contrast enhanced (DCE) magnetic resonance imaging. This study is to develop an automatic detection and segmentation deep learning model for HCC using DCE.Approach: DCE images acquired from 2016 to 2021 were retrospectively collected. Then, 382 patients (301 male; 81 female) with 466 lesions pathologically confirmed were included and divided into an 80% training-validation set and a 20% independent test set. For external validation, 51 patients (42 male; 9 female) in another hospital from 2018 to 2021 were included. The U-net architecture was modified to accommodate multi-phasic DCE input. The model was trained with the training-validation set using five-fold cross-validation, and furtherly evaluated with the independent test set using comprehensive metrics for segmentation and detection performance. The proposed automatic segmentation model consisted of five main steps: phase registration, automatic liver region extraction using a pre-trained model, automatic HCC lesion segmentation using the multi-phasic deep learning model, ensemble of five-fold predictions, and post-processing using connected component analysis to enhance the performance to refine predictions and eliminate false positives.Main results. The proposed model achieved a mean dice similarity coefficient (DSC) of 0.81 ± 0.11, a sensitivity of 94.41 ± 15.50%, a precision of 94.19 ± 17.32%, and 0.14 ± 0.48 false positive lesions per patient in the independent test set. The model detected 88% (80/91) HCC lesions in the condition of DSC > 0.5, and the DSC per tumor was 0.80 ± 0.13. In the external set, the model detected 92% (58/62) lesions with 0.12 ± 0.33 false positives per patient, and the DSC per tumor was 0.75 ± 0.10.Significance.This study developed an automatic detection and segmentation deep learning model for HCC using DCE, which yielded promising post-processed results in accurately identifying and delineating HCC lesions.
Collapse
Affiliation(s)
- Xiao Luo
- Academy for Engineering and Technology, Fudan University, Shanghai, People's Republic of China
| | - Peiwen Li
- Department of Radiology, Huashan Hospital, Fudan University, Shanghai, People's Republic of China
| | - Hongyi Chen
- Academy for Engineering and Technology, Fudan University, Shanghai, People's Republic of China
| | - Kun Zhou
- Academy for Engineering and Technology, Fudan University, Shanghai, People's Republic of China
| | - Sirong Piao
- Department of Radiology, Huashan Hospital, Fudan University, Shanghai, People's Republic of China
- Department of Radiology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, People's Republic China
| | - Liqin Yang
- Department of Radiology, Huashan Hospital, Fudan University, Shanghai, People's Republic of China
- Shanghai Engineering Research Center of Intelligent Imaging for Critical Brain Diseases, Shanghai, People's Republic China
- Institute of Functional and Molecular Medical Imaging, Fudan University, Shanghai, People's Republic of China
| | - Bin Hu
- Department of Radiology, Huashan Hospital, Fudan University, Shanghai, People's Republic of China
| | - Daoying Geng
- Academy for Engineering and Technology, Fudan University, Shanghai, People's Republic of China
- Department of Radiology, Huashan Hospital, Fudan University, Shanghai, People's Republic of China
- Shanghai Engineering Research Center of Intelligent Imaging for Critical Brain Diseases, Shanghai, People's Republic China
- Institute of Functional and Molecular Medical Imaging, Fudan University, Shanghai, People's Republic of China
| |
Collapse
|
42
|
Wang P, Leong QY, Lau NY, Ng WY, Kwek SP, Tan L, Song SW, You K, Chong LM, Zhuang I, Ong YH, Foo N, Tadeo X, Kumar KS, Vijayakumar S, Sapanel Y, Raczkowska MN, Remus A, Blasiak A, Ho D. N-of-1 medicine. Singapore Med J 2024; 65:167-175. [PMID: 38527301 PMCID: PMC11060644 DOI: 10.4103/singaporemedj.smj-2023-243] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2023] [Accepted: 01/19/2024] [Indexed: 03/27/2024]
Abstract
ABSTRACT The fields of precision and personalised medicine have led to promising advances in tailoring treatment to individual patients. Examples include genome/molecular alteration-guided drug selection, single-patient gene therapy design and synergy-based drug combination development, and these approaches can yield substantially diverse recommendations. Therefore, it is important to define each domain and delineate their commonalities and differences in an effort to develop novel clinical trial designs, streamline workflow development, rethink regulatory considerations, create value in healthcare and economics assessments, and other factors. These and other segments are essential to recognise the diversity within these domains to accelerate their respective workflows towards practice-changing healthcare. To emphasise these points, this article elaborates on the concept of digital health and digital medicine-enabled N-of-1 medicine, which individualises combination regimen and dosing using a patient's own data. We will conclude with recommendations for consideration when developing novel workflows based on emerging digital-based platforms.
Collapse
Affiliation(s)
- Peter Wang
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore
| | - Qiao Ying Leong
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
| | - Ni Yin Lau
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
| | - Wei Ying Ng
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
| | - Siong Peng Kwek
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
| | - Lester Tan
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore
| | - Shang-Wei Song
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore
| | - Kui You
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore
| | - Li Ming Chong
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore
| | - Isaiah Zhuang
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
| | - Yoong Hun Ong
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
| | - Nigel Foo
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore
| | - Xavier Tadeo
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
| | - Kirthika Senthil Kumar
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
| | - Smrithi Vijayakumar
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
| | - Yoann Sapanel
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
- Singapore’s Health District @ Queenstown, Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| | - Marlena Natalia Raczkowska
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
| | - Alexandria Remus
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore
- Heat Resilience Performance Centre (HRPC), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| | - Agata Blasiak
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore
- Department of Pharmacology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| | - Dean Ho
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore
- Singapore’s Health District @ Queenstown, Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- Department of Pharmacology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The Bia-Echo Asia Centre for Reproductive Longevity and Equality, Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| |
Collapse
|
43
|
Wang ZC, Fan ZZ, Liu XY, Zhu MJ, Jiang SS, Tian S, Chen BH, Wu LM. Deep Learning for Discrimination of Hypertrophic Cardiomyopathy and Hypertensive Heart Disease on MRI Native T1 Maps. J Magn Reson Imaging 2024; 59:837-848. [PMID: 37431848 DOI: 10.1002/jmri.28904] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2023] [Revised: 06/05/2023] [Accepted: 06/06/2023] [Indexed: 07/12/2023] Open
Abstract
BACKGROUND Native T1 and radiomics were used for hypertrophic cardiomyopathy (HCM) and hypertensive heart disease (HHD) differentiation previously. The current problem is that global native T1 remains modest discrimination performance and radiomics requires feature extraction beforehand. Deep learning (DL) is a promising technique in differential diagnosis. However, its feasibility for discriminating HCM and HHD has not been investigated. PURPOSE To examine the feasibility of DL in differentiating HCM and HHD based on T1 images and compare its diagnostic performance with other methods. STUDY TYPE Retrospective. POPULATION 128 HCM patients (men, 75; age, 50 years ± 16) and 59 HHD patients (men, 40; age, 45 years ± 17). FIELD STRENGTH/SEQUENCE 3.0T; Balanced steady-state free precession, phase-sensitive inversion recovery (PSIR) and multislice native T1 mapping. ASSESSMENT Compare HCM and HHD patients baseline data. Myocardial T1 values were extracted from native T1 images. Radiomics was implemented through feature extraction and Extra Trees Classifier. The DL network is ResNet32. Different input including myocardial ring (DL-myo), myocardial ring bounding box (DL-box) and the surrounding tissue without myocardial ring (DL-nomyo) were tested. We evaluate diagnostic performance through AUC of ROC curve. STATISTICAL TESTS Accuracy, sensitivity, specificity, ROC, and AUC were calculated. Independent t test, Mann-Whitney U-test and Chi-square test were adopted for HCM and HHD comparison. P < 0.05 was considered statistically significant. RESULTS DL-myo, DL-box, and DL-nomyo models showed an AUC (95% confidential interval) of 0.830 (0.702-0.959), 0.766 (0.617-0.915), 0.795 (0.654-0.936) in the testing set. AUC of native T1 and radiomics were 0.545 (0.352-0.738) and 0.800 (0.655-0.944) in the testing set. DATA CONCLUSION The DL method based on T1 mapping seems capable of discriminating HCM and HHD. Considering diagnostic performance, the DL network outperformed the native T1 method. Compared with radiomics, DL won an advantage for its high specificity and automated working mode. LEVEL OF EVIDENCE 4 TECHNICAL EFFICACY STAGE: 2.
Collapse
Affiliation(s)
- Zi-Chen Wang
- Ottawa-Shanghai Joint School of Medicine, School of Medicine, Shanghai Jiao Tong University, Shanghai, China
| | - Zhang-Zhengyi Fan
- Ottawa-Shanghai Joint School of Medicine, School of Medicine, Shanghai Jiao Tong University, Shanghai, China
| | - Xi-Yuan Liu
- Ottawa-Shanghai Joint School of Medicine, School of Medicine, Shanghai Jiao Tong University, Shanghai, China
| | - Ming-Jie Zhu
- Ottawa-Shanghai Joint School of Medicine, School of Medicine, Shanghai Jiao Tong University, Shanghai, China
| | | | | | - Bing-Hua Chen
- Department of Radiology, Renji Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China
| | - Lian-Ming Wu
- Department of Radiology, Renji Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China
| |
Collapse
|
44
|
Benboujja F, Hartnick E, Zablah E, Hersh C, Callans K, Villamor P, Yager PH, Hartnick C. Overcoming language barriers in pediatric care: a multilingual, AI-driven curriculum for global healthcare education. Front Public Health 2024; 12:1337395. [PMID: 38454985 PMCID: PMC10917955 DOI: 10.3389/fpubh.2024.1337395] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2023] [Accepted: 01/22/2024] [Indexed: 03/09/2024] Open
Abstract
Background Online medical education often faces challenges related to communication and comprehension barriers, particularly when the instructional language differs from the healthcare providers' and caregivers' native languages. Our study addresses these challenges within pediatric healthcare by employing generative language models to produce a linguistically tailored, multilingual curriculum that covers the topics of team training, surgical procedures, perioperative care, patient journeys, and educational resources for healthcare providers and caregivers. Methods An interdisciplinary group formulated a video curriculum in English, addressing the nuanced challenges of pediatric healthcare. Subsequently, it was translated into Spanish, primarily emphasizing Latin American demographics, utilizing OpenAI's GPT-4. Videos were enriched with synthetic voice profiles of native speakers to uphold the consistency of the narrative. Results We created a collection of 45 multilingual video modules, each ranging from 3 to 8 min in length and covering essential topics such as teamwork, how to improve interpersonal communication, "How I Do It" surgical procedures, as well as focused topics in anesthesia, intensive care unit care, ward nursing, and transitions from hospital to home. Through AI-driven translation, this comprehensive collection ensures global accessibility and offers healthcare professionals and caregivers a linguistically inclusive resource for elevating standards of pediatric care worldwide. Conclusion This development of multilingual educational content marks a progressive step toward global standardization of pediatric care. By utilizing advanced language models for translation, we ensure that the curriculum is inclusive and accessible. This initiative aligns well with the World Health Organization's Digital Health Guidelines, advocating for digitally enabled healthcare education.
Collapse
Affiliation(s)
- Fouzi Benboujja
- Department of Otolaryngology, Mass Eye and Ear, Harvard Medical School, Boston, MA, United States
| | | | - Evelyn Zablah
- Department of Otolaryngology, Mass Eye and Ear, Harvard Medical School, Boston, MA, United States
| | - Cheryl Hersh
- Pediatric Airway, Voice and Swallowing Center, Massachusetts General Hospital for Children, Boston, MA, United States
| | - Kevin Callans
- Department of Otolaryngology, Mass Eye and Ear, Harvard Medical School, Boston, MA, United States
- Massachusetts General Hospital for Children, Boston, MA, United States
| | - Perla Villamor
- Hospital Serena del Mar, Cartagena, Colombia
- Hospital Infantil Napoleón Franco Pareja, Cartagena, Colombia
| | - Phoebe H. Yager
- Pediatric Intensive Care Unit, Massachusetts General Hospital, Harvard Medical School, Boston, MA, United States
| | - Christopher Hartnick
- Department of Otolaryngology, Mass Eye and Ear, Harvard Medical School, Boston, MA, United States
| |
Collapse
|
45
|
Wu X, Zhang D, Li G, Gao X, Metcalfe B, Chen L. Data augmentation for invasive brain-computer interfaces based on stereo-electroencephalography (SEEG). J Neural Eng 2024; 21:016026. [PMID: 38237174 DOI: 10.1088/1741-2552/ad200e] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2023] [Accepted: 01/18/2024] [Indexed: 02/23/2024]
Abstract
Objective.Deep learning is increasingly used for brain-computer interfaces (BCIs). However, the quantity of available data is sparse, especially for invasive BCIs. Data augmentation (DA) methods, such as generative models, can help to address this sparseness. However, all the existing studies on brain signals were based on convolutional neural networks and ignored the temporal dependence. This paper attempted to enhance generative models by capturing the temporal relationship from a time-series perspective.Approach. A conditional generative network (conditional transformer-based generative adversarial network (cTGAN)) based on the transformer model was proposed. The proposed method was tested using a stereo-electroencephalography (SEEG) dataset which was recorded from eight epileptic patients performing five different movements. Three other commonly used DA methods were also implemented: noise injection (NI), variational autoencoder (VAE), and conditional Wasserstein generative adversarial network with gradient penalty (cWGANGP). Using the proposed method, the artificial SEEG data was generated, and several metrics were used to compare the data quality, including visual inspection, cosine similarity (CS), Jensen-Shannon distance (JSD), and the effect on the performance of a deep learning-based classifier.Main results. Both the proposed cTGAN and the cWGANGP methods were able to generate realistic data, while NI and VAE outputted inferior samples when visualized as raw sequences and in a lower dimensional space. The cTGAN generated the best samples in terms of CS and JSD and outperformed cWGANGP significantly in enhancing the performance of a deep learning-based classifier (each of them yielding a significant improvement of 6% and 3.4%, respectively).Significance. This is the first time that DA methods have been applied to invasive BCIs based on SEEG. In addition, this study demonstrated the advantages of the model that preserves the temporal dependence from a time-series perspective.
Collapse
Affiliation(s)
- Xiaolong Wu
- The Centre for Autonomous Robotics (CENTAUR), Department of Electronic & Electrical Engineering, University of Bath, Bath, United Kingdom
| | - Dingguo Zhang
- The Centre for Autonomous Robotics (CENTAUR), Department of Electronic & Electrical Engineering, University of Bath, Bath, United Kingdom
| | - Guangye Li
- School of Mechanical Engineering, Shanghai Jiao Tong University, People's Republic of China
| | - Xin Gao
- The Centre for Autonomous Robotics (CENTAUR), Department of Electronic & Electrical Engineering, University of Bath, Bath, United Kingdom
| | - Benjamin Metcalfe
- The Centre for Autonomous Robotics (CENTAUR), Department of Electronic & Electrical Engineering, University of Bath, Bath, United Kingdom
| | - Liang Chen
- Liang Chen is with Huashan Hospital, Fudan University, People's Republic of China
| |
Collapse
|
46
|
Burguete-Lopez A, Makarenko M, Bonifazi M, Menezes de Oliveira BN, Getman F, Tian Y, Mazzone V, Li N, Giammona A, Liberale C, Fratalocchi A. Real-time simultaneous refractive index and thickness mapping of sub-cellular biology at the diffraction limit. Commun Biol 2024; 7:154. [PMID: 38321111 PMCID: PMC10847501 DOI: 10.1038/s42003-024-05839-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2023] [Accepted: 01/20/2024] [Indexed: 02/08/2024] Open
Abstract
Mapping the cellular refractive index (RI) is a central task for research involving the composition of microorganisms and the development of models providing automated medical screenings with accuracy beyond 95%. These models require significantly enhancing the state-of-the-art RI mapping capabilities to provide large amounts of accurate RI data at high throughput. Here, we present a machine-learning-based technique that obtains a biological specimen's real-time RI and thickness maps from a single image acquired with a conventional color camera. This technology leverages a suitably engineered nanostructured membrane that stretches a biological analyte over its surface and absorbs transmitted light, generating complex reflection spectra from each sample point. The technique does not need pre-existing sample knowledge. It achieves 10-4 RI sensitivity and sub-nanometer thickness resolution on diffraction-limited spatial areas. We illustrate practical application by performing sub-cellular segmentation of HCT-116 colorectal cancer cells, obtaining complete three-dimensional reconstruction of the cellular regions with a characteristic length of 30 μm. These results can facilitate the development of real-time label-free technologies for biomedical studies on microscopic multicellular dynamics.
Collapse
Affiliation(s)
- Arturo Burguete-Lopez
- PRIMALIGHT, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE), King Abdullah University of Science and Technology (KAUST), Thuwal, 23955-6900, Saudi Arabia
| | - Maksim Makarenko
- PRIMALIGHT, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE), King Abdullah University of Science and Technology (KAUST), Thuwal, 23955-6900, Saudi Arabia
| | - Marcella Bonifazi
- PRIMALIGHT, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE), King Abdullah University of Science and Technology (KAUST), Thuwal, 23955-6900, Saudi Arabia
- Physik-Institut, University of Zurich, Winterthurerstrasse 190, Zurich, 8057, Switzerland
| | - Barbara Nicoly Menezes de Oliveira
- PRIMALIGHT, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE), King Abdullah University of Science and Technology (KAUST), Thuwal, 23955-6900, Saudi Arabia
| | - Fedor Getman
- PRIMALIGHT, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE), King Abdullah University of Science and Technology (KAUST), Thuwal, 23955-6900, Saudi Arabia
| | - Yi Tian
- PRIMALIGHT, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE), King Abdullah University of Science and Technology (KAUST), Thuwal, 23955-6900, Saudi Arabia
| | - Valerio Mazzone
- PRIMALIGHT, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE), King Abdullah University of Science and Technology (KAUST), Thuwal, 23955-6900, Saudi Arabia
- Physik-Institut, University of Zurich, Winterthurerstrasse 190, Zurich, 8057, Switzerland
| | - Ning Li
- PRIMALIGHT, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE), King Abdullah University of Science and Technology (KAUST), Thuwal, 23955-6900, Saudi Arabia
| | - Alessandro Giammona
- Biological and Environmental Science and Engineering Division (BESE), King Abdullah University of Science and Technology (KAUST), Thuwal, 23955-6900, Saudi Arabia
- Institute of Molecular Bioimaging and Physiology (IBFM), National Research Council (CNR), Segrate, Italy
| | - Carlo Liberale
- Biological and Environmental Science and Engineering Division (BESE), King Abdullah University of Science and Technology (KAUST), Thuwal, 23955-6900, Saudi Arabia
| | - Andrea Fratalocchi
- PRIMALIGHT, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE), King Abdullah University of Science and Technology (KAUST), Thuwal, 23955-6900, Saudi Arabia.
| |
Collapse
|
47
|
Zheng F, Li M, Wang Y, Yu W, Wang R, Chen Z, Xiao N, Lu Y. Intensive vision-guided network for radiology report generation. Phys Med Biol 2024; 69:045008. [PMID: 38157546 DOI: 10.1088/1361-6560/ad1995] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2023] [Accepted: 12/29/2023] [Indexed: 01/03/2024]
Abstract
Objective.Automatic radiology report generation is booming due to its huge application potential for the healthcare industry. However, existing computer vision and natural language processing approaches to tackle this problem are limited in two aspects. First, when extracting image features, most of them neglect multi-view reasoning in vision and model single-view structure of medical images, such as space-view or channel-view. However, clinicians rely on multi-view imaging information for comprehensive judgment in daily clinical diagnosis. Second, when generating reports, they overlook context reasoning with multi-modal information and focus on pure textual optimization utilizing retrieval-based methods. We aim to address these two issues by proposing a model that better simulates clinicians perspectives and generates more accurate reports.Approach.Given the above limitation in feature extraction, we propose a globally-intensive attention (GIA) module in the medical image encoder to simulate and integrate multi-view vision perception. GIA aims to learn three types of vision perception: depth view, space view, and pixel view. On the other hand, to address the above problem in report generation, we explore how to involve multi-modal signals to generate precisely matched reports, i.e. how to integrate previously predicted words with region-aware visual content in next word prediction. Specifically, we design a visual knowledge-guided decoder (VKGD), which can adaptively consider how much the model needs to rely on visual information and previously predicted text to assist next word prediction. Hence, our final intensive vision-guided network framework includes a GIA-guided visual encoder and the VKGD.Main results.Experiments on two commonly-used datasets IU X-RAY and MIMIC-CXR demonstrate the superior ability of our method compared with other state-of-the-art approaches.Significance.Our model explores the potential of simulating clinicians perspectives and automatically generates more accurate reports, which promotes the exploration of medical automation and intelligence.
Collapse
Affiliation(s)
- Fudan Zheng
- Sun Yat-Sen University, No. 132 Waihuandong Road, Guangzhou Higher Education Mega Center, Guangzhou, 510006, People's Republic of China
| | - Mengfei Li
- Sun Yat-Sen University, No. 132 Waihuandong Road, Guangzhou Higher Education Mega Center, Guangzhou, 510006, People's Republic of China
| | - Ying Wang
- National SuperComputer Center in Guangzhou, No. 132 Waihuandong Road, Guangzhou Higher Education Mega Center, Guangzhou, 510006, People's Republic of China
| | - Weijiang Yu
- Huawei Technologies Co., Ltd, Huawei Industrial Park, Bantian, Longgang District, Shenzhen, 518129, People's Republic of China
| | - Ruixuan Wang
- Sun Yat-Sen University, No. 132 Waihuandong Road, Guangzhou Higher Education Mega Center, Guangzhou, 510006, People's Republic of China
| | - Zhiguang Chen
- Sun Yat-Sen University, No. 132 Waihuandong Road, Guangzhou Higher Education Mega Center, Guangzhou, 510006, People's Republic of China
- National SuperComputer Center in Guangzhou, No. 132 Waihuandong Road, Guangzhou Higher Education Mega Center, Guangzhou, 510006, People's Republic of China
| | - Nong Xiao
- Sun Yat-Sen University, No. 132 Waihuandong Road, Guangzhou Higher Education Mega Center, Guangzhou, 510006, People's Republic of China
- National SuperComputer Center in Guangzhou, No. 132 Waihuandong Road, Guangzhou Higher Education Mega Center, Guangzhou, 510006, People's Republic of China
| | - Yutong Lu
- Sun Yat-Sen University, No. 132 Waihuandong Road, Guangzhou Higher Education Mega Center, Guangzhou, 510006, People's Republic of China
- National SuperComputer Center in Guangzhou, No. 132 Waihuandong Road, Guangzhou Higher Education Mega Center, Guangzhou, 510006, People's Republic of China
| |
Collapse
|
48
|
Courtman M, Kim D, Wit H, Wang H, Sun L, Ifeachor E, Mullin S, Thurston M. Deep Learning Detection of Aneurysm Clips for Magnetic Resonance Imaging Safety. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024; 37:72-80. [PMID: 38343241 DOI: 10.1007/s10278-023-00932-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/05/2023] [Revised: 10/03/2023] [Accepted: 10/19/2023] [Indexed: 03/02/2024]
Abstract
Flagging the presence of metal devices before a head MRI scan is essential to allow appropriate safety checks. There is an unmet need for an automated system which can flag aneurysm clips prior to MRI appointments. We assess the accuracy with which a machine learning model can classify the presence or absence of an aneurysm clip on CT images. A total of 280 CT head scans were collected, 140 with aneurysm clips visible and 140 without. The data were used to retrain a pre-trained image classification neural network to classify CT localizer images. Models were developed using fivefold cross-validation and then tested on a holdout test set. A mean sensitivity of 100% and a mean accuracy of 82% were achieved. Predictions were explained using SHapley Additive exPlanations (SHAP), which highlighted that appropriate regions of interest were informing the models. Models were also trained from scratch to classify three-dimensional CT head scans. These did not exceed the sensitivity of the localizer models. This work illustrates an application of computer vision image classification to enhance current processes and improve patient safety.
Collapse
Affiliation(s)
- Megan Courtman
- Faculty of Science and Engineering, School of Engineering, Computing and Mathematics, University of Plymouth, Plymouth, PL4 8AA, UK.
| | - Daniel Kim
- Department of Radiology, Royal Cornwall Hospitals NHS Trust, Truro, TR1 3LJ, UK
| | - Huub Wit
- Department of Radiology, Torbay and South Devon NHS Trust, Torquay, TQ2 7AA, UK
| | - Hongrui Wang
- Department of Radiology, University Hospitals Plymouth NHS Trust, Plymouth, PL6 8DH, UK
| | - Lingfen Sun
- Faculty of Science and Engineering, School of Engineering, Computing and Mathematics, University of Plymouth, Plymouth, PL4 8AA, UK
| | - Emmanuel Ifeachor
- Faculty of Science and Engineering, School of Engineering, Computing and Mathematics, University of Plymouth, Plymouth, PL4 8AA, UK
| | - Stephen Mullin
- Plymouth Institute of Health and Care Research, University of Plymouth, Plymouth, PL4 8AA, UK
| | - Mark Thurston
- Department of Radiology, University Hospitals Plymouth NHS Trust, Plymouth, PL6 8DH, UK
| |
Collapse
|
49
|
Gerbasi A, Dagliati A, Albi G, Chiesa M, Andreini D, Baggiano A, Mushtaq S, Pontone G, Bellazzi R, Colombo G. CAD-RADS scoring of coronary CT angiography with Multi-Axis Vision Transformer: A clinically-inspired deep learning pipeline. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 244:107989. [PMID: 38141455 DOI: 10.1016/j.cmpb.2023.107989] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/26/2023] [Revised: 11/10/2023] [Accepted: 12/17/2023] [Indexed: 12/25/2023]
Abstract
BACKGROUND AND OBJECTIVE The standard non-invasive imaging technique used to assess the severity and extent of Coronary Artery Disease (CAD) is Coronary Computed Tomography Angiography (CCTA). However, manual grading of each patient's CCTA according to the CAD-Reporting and Data System (CAD-RADS) scoring is time-consuming and operator-dependent, especially in borderline cases. This work proposes a fully automated, and visually explainable, deep learning pipeline to be used as a decision support system for the CAD screening procedure. The pipeline performs two classification tasks: firstly, identifying patients who require further clinical investigations and secondly, classifying patients into subgroups based on the degree of stenosis, according to commonly used CAD-RADS thresholds. METHODS The pipeline pre-processes multiplanar projections of the coronary arteries, extracted from the original CCTAs, and classifies them using a fine-tuned Multi-Axis Vision Transformer architecture. With the aim of emulating the current clinical practice, the model is trained to assign a per-patient score by stacking the bi-dimensional longitudinal cross-sections of the three main coronary arteries along channel dimension. Furthermore, it generates visually interpretable maps to assess the reliability of the predictions. RESULTS When run on a database of 1873 three-channel images of 253 patients collected at the Monzino Cardiology Center in Milan, the pipeline obtained an AUC of 0.87 and 0.93 for the two classification tasks, respectively. CONCLUSION According to our knowledge, this is the first model trained to assign CAD-RADS scores learning solely from patient scores and not requiring finer imaging annotation steps that are not part of the clinical routine.
Collapse
Affiliation(s)
- Alessia Gerbasi
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Via Ferrata 5, Pavia, Italy.
| | - Arianna Dagliati
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Via Ferrata 5, Pavia, Italy
| | - Giuseppe Albi
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Via Ferrata 5, Pavia, Italy
| | | | - Daniele Andreini
- Centro Cardiologico Monzino IRCCS, Milan, Italy; Department of Biomedical and Clinical Sciences, University of Milan, Milan, Italy
| | - Andrea Baggiano
- Centro Cardiologico Monzino IRCCS, Milan, Italy; Department of Clinical Sciences and Community Health, University of Milan, Milan, Italy
| | | | - Gianluca Pontone
- Centro Cardiologico Monzino IRCCS, Milan, Italy; Department of Biomedical, Surgical and Dental Sciences, University of Milan, Milan, Italy
| | - Riccardo Bellazzi
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Via Ferrata 5, Pavia, Italy; IRCCS Istituti Clinici Scientifici Maugeri, Pavia, Pavia, Italy
| | | |
Collapse
|
50
|
Goodman ED, Patel KK, Zhang Y, Locke W, Kennedy CJ, Mehrotra R, Ren S, Guan M, Zohar O, Downing M, Chen HW, Clark JZ, Berrigan MT, Brat GA, Yeung-Levy S. Analyzing Surgical Technique in Diverse Open Surgical Videos With Multitask Machine Learning. JAMA Surg 2024; 159:185-192. [PMID: 38055227 PMCID: PMC10701669 DOI: 10.1001/jamasurg.2023.6262] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Accepted: 09/04/2023] [Indexed: 12/07/2023]
Abstract
Objective To overcome limitations of open surgery artificial intelligence (AI) models by curating the largest collection of annotated videos and to leverage this AI-ready data set to develop a generalizable multitask AI model capable of real-time understanding of clinically significant surgical behaviors in prospectively collected real-world surgical videos. Design, Setting, and Participants The study team programmatically queried open surgery procedures on YouTube and manually annotated selected videos to create the AI-ready data set used to train a multitask AI model for 2 proof-of-concept studies, one generating surgical signatures that define the patterns of a given procedure and the other identifying kinematics of hand motion that correlate with surgeon skill level and experience. The Annotated Videos of Open Surgery (AVOS) data set includes 1997 videos from 23 open-surgical procedure types uploaded to YouTube from 50 countries over the last 15 years. Prospectively recorded surgical videos were collected from a single tertiary care academic medical center. Deidentified videos were recorded of surgeons performing open surgical procedures and analyzed for correlation with surgical training. Exposures The multitask AI model was trained on the AI-ready video data set and then retrospectively applied to the prospectively collected video data set. Main Outcomes and Measures Analysis of open surgical videos in near real-time, performance on AI-ready and prospectively collected videos, and quantification of surgeon skill. Results Using the AI-ready data set, the study team developed a multitask AI model capable of real-time understanding of surgical behaviors-the building blocks of procedural flow and surgeon skill-across space and time. Through principal component analysis, a single compound skill feature was identified, composed of a linear combination of kinematic hand attributes. This feature was a significant discriminator between experienced surgeons and surgical trainees across 101 prospectively collected surgical videos of 14 operators. For each unit increase in the compound feature value, the odds of the operator being an experienced surgeon were 3.6 times higher (95% CI, 1.67-7.62; P = .001). Conclusions and Relevance In this observational study, the AVOS-trained model was applied to analyze prospectively collected open surgical videos and identify kinematic descriptors of surgical skill related to efficiency of hand motion. The ability to provide AI-deduced insights into surgical structure and skill is valuable in optimizing surgical skill acquisition and ultimately improving surgical care.
Collapse
Affiliation(s)
- Emmett D. Goodman
- Department of Computer Science, Stanford University, Stanford, California
- Department of Biomedical Data Science, Stanford University, Stanford, California
| | - Krishna K. Patel
- Department of Computer Science, Stanford University, Stanford, California
- Department of Biomedical Data Science, Stanford University, Stanford, California
| | - Yilun Zhang
- Department of Surgery, Beth Israel Deaconess Medical Center, Boston, Massachusetts
| | - William Locke
- Department of Computer Science, Stanford University, Stanford, California
- Department of Biomedical Data Science, Stanford University, Stanford, California
| | - Chris J. Kennedy
- Department of Surgery, Beth Israel Deaconess Medical Center, Boston, Massachusetts
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts
| | - Rohan Mehrotra
- Department of Computer Science, Stanford University, Stanford, California
- Department of Biomedical Data Science, Stanford University, Stanford, California
| | - Stephen Ren
- Department of Computer Science, Stanford University, Stanford, California
- Department of Biomedical Data Science, Stanford University, Stanford, California
| | - Melody Guan
- Department of Computer Science, Stanford University, Stanford, California
- Department of Biomedical Data Science, Stanford University, Stanford, California
| | - Orr Zohar
- Department of Biomedical Data Science, Stanford University, Stanford, California
- Department of Electrical Engineering, Stanford University, Stanford, California
| | - Maren Downing
- Department of Surgery, Beth Israel Deaconess Medical Center, Boston, Massachusetts
| | - Hao Wei Chen
- Department of Surgery, Beth Israel Deaconess Medical Center, Boston, Massachusetts
| | - Jevin Z. Clark
- Department of Surgery, Beth Israel Deaconess Medical Center, Boston, Massachusetts
| | - Margaret T. Berrigan
- Department of Surgery, Beth Israel Deaconess Medical Center, Boston, Massachusetts
| | - Gabriel A. Brat
- Department of Surgery, Beth Israel Deaconess Medical Center, Boston, Massachusetts
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts
| | - Serena Yeung-Levy
- Department of Computer Science, Stanford University, Stanford, California
- Department of Biomedical Data Science, Stanford University, Stanford, California
- Department of Electrical Engineering, Stanford University, Stanford, California
- Clinical Excellence Research Center, Stanford University School of Medicine, Stanford, California
| |
Collapse
|