1
|
Iglesias G, Talavera E, Troya J, Díaz-Álvarez A, García-Remesal M. Artificial intelligence model for tumoral clinical decision support systems. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 253:108228. [PMID: 38810378 DOI: 10.1016/j.cmpb.2024.108228] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/02/2023] [Revised: 04/21/2024] [Accepted: 05/13/2024] [Indexed: 05/31/2024]
Abstract
BACKGROUND AND OBJECTIVE Comparative diagnostic in brain tumor evaluation makes possible to use the available information of a medical center to compare similar cases when a new patient is evaluated. By leveraging Artificial Intelligence models, the proposed system is able of retrieving the most similar cases of brain tumors for a given query. The primary objective is to enhance the diagnostic process by generating more accurate representations of medical images, with a particular focus on patient-specific normal features and pathologies. A key distinction from previous models lies in its ability to produce enriched image descriptors solely from binary information, eliminating the need for costly and difficult to obtain tumor segmentation. METHODS The proposed model uses Artificial Intelligence to detect patient features to recommend the most similar cases from a database. The system not only suggests similar cases but also balances the representation of healthy and abnormal features in its design. This not only encourages the generalization of its use but also aids clinicians in their decision-making processes. This generalization makes possible for future research in different medical diagnosis areas with almost not any change in the system. RESULTS We conducted a comparative analysis of our approach in relation to similar studies. The proposed architecture obtains a Dice coefficient of 0.474 in both tumoral and healthy regions of the patients, which outperforms previous literature. Our proposed model excels at extracting and combining anatomical and pathological features from brain Magnetic Resonances (MRs), achieving state-of-the-art results while relying on less expensive label information. This substantially reduces the overall cost of the training process. Our findings highlight the significant potential for improving the efficiency and accuracy of comparative diagnostics and the treatment of tumoral pathologies. CONCLUSIONS This paper provides substantial grounds for further exploration of the broader applicability and optimization of the proposed architecture to enhance clinical decision-making. The novel approach presented in this work marks a significant advancement in the field of medical diagnosis, particularly in the context of Artificial Intelligence-assisted image retrieval, and promises to reduce costs and improve the quality of patient care using Artificial Intelligence as a support tool instead of a black box system.
Collapse
Affiliation(s)
- Guillermo Iglesias
- Departamento de Sistemas Informáticos, Escuela Técnica Superior de Ingeniería de Sistemas Informáticos, Universidad Politécnica de Madrid, Spain.
| | - Edgar Talavera
- Departamento de Sistemas Informáticos, Escuela Técnica Superior de Ingeniería de Sistemas Informáticos, Universidad Politécnica de Madrid, Spain.
| | - Jesús Troya
- Infanta Leonor University Hospital. Madrid., Spain
| | - Alberto Díaz-Álvarez
- Departamento de Sistemas Informáticos, Escuela Técnica Superior de Ingeniería de Sistemas Informáticos, Universidad Politécnica de Madrid, Spain.
| | - Miguel García-Remesal
- Biomedical Informatics Group, Departamento de Inteligencia Artificial, Escuela Técnica Superior de Ingenieros Informáticos, Universidad Politécnica de Madrid, Spain.
| |
Collapse
|
2
|
Park C, Kang JW, Lee DE, Son W, Lee SM, Park C, Kim M. W-DRAG: A joint framework of WGAN with data random augmentation optimized for generative networks for bone marrow edema detection in dual energy CT. Comput Med Imaging Graph 2024; 115:102387. [PMID: 38703602 DOI: 10.1016/j.compmedimag.2024.102387] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2023] [Revised: 04/17/2024] [Accepted: 04/17/2024] [Indexed: 05/06/2024]
Abstract
Dual-energy computed tomography (CT) is an excellent substitute for identifying bone marrow edema in magnetic resonance imaging. However, it is rarely used in practice owing to its low contrast. To overcome this problem, we constructed a framework based on deep learning techniques to screen for diseases using axial bone images and to identify the local positions of bone lesions. To address the limited availability of labeled samples, we developed a new generative adversarial network (GAN) that extends expressions beyond conventional augmentation (CA) methods based on geometric transformations. We theoretically and experimentally determined that combining the concepts of data augmentation optimized for GAN training (DAG) and Wasserstein GAN yields a considerably stable generation of synthetic images and effectively aligns their distribution with that of real images, thereby achieving a high degree of similarity. The classification model was trained using real and synthetic samples. Consequently, the GAN technique used in the diagnostic test had an improved F1 score of approximately 7.8% compared with CA. The final F1 score was 80.24%, and the recall and precision were 84.3% and 88.7%, respectively. The results obtained using the augmented samples outperformed those obtained using pure real samples without augmentation. In addition, we adopted explainable AI techniques that leverage a class activation map (CAM) and principal component analysis to facilitate visual analysis of the network's results. The framework was designed to suggest an attention map and scattering plot to visually explain the disease predictions of the network.
Collapse
Affiliation(s)
- Chunsu Park
- Department of Information Convergence Engineering, Pusan National University, Yangsan, Republic of Korea
| | - Jeong-Woon Kang
- Department of Information Convergence Engineering, Pusan National University, Yangsan, Republic of Korea
| | - Doen-Eon Lee
- Department of Information Convergence Engineering, Pusan National University, Yangsan, Republic of Korea
| | - Wookon Son
- Department of Radiology, Research Institute for Convergence of Biomedical Science and Technology, Pusan National University Yangsan Hospital, Yangsan, Republic of Korea
| | - Sang-Min Lee
- Department of Orthopedics, Research Institute for Convergence of Biomedical Science and Technology, Pusan National University Yangsan Hospital, Yangsan, Republic of Korea
| | - Chankue Park
- Department of Radiology, Research Institute for Convergence of Biomedical Science and Technology, Pusan National University Yangsan Hospital, Yangsan, Republic of Korea.
| | - MinWoo Kim
- Department of Biomedical Convergence Engineering, Pusan National University, Yangsan, Republic of Korea; Center for Artificial Intelligence Research, Pusan National University, Busan, Republic of Korea.
| |
Collapse
|
3
|
Friedrich MU, Roenn AJ, Palmisano C, Alty J, Paschen S, Deuschl G, Ip CW, Volkmann J, Muthuraman M, Peach R, Reich MM. Validation and application of computer vision algorithms for video-based tremor analysis. NPJ Digit Med 2024; 7:165. [PMID: 38906946 PMCID: PMC11192937 DOI: 10.1038/s41746-024-01153-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2023] [Accepted: 05/29/2024] [Indexed: 06/23/2024] Open
Abstract
Tremor is one of the most common neurological symptoms. Its clinical and neurobiological complexity necessitates novel approaches for granular phenotyping. Instrumented neurophysiological analyses have proven useful, but are highly resource-intensive and lack broad accessibility. In contrast, bedside scores are simple to administer, but lack the granularity to capture subtle but relevant tremor features. We utilise the open-source computer vision pose tracking algorithm Mediapipe to track hands in clinical video recordings and use the resulting time series to compute canonical tremor features. This approach is compared to marker-based 3D motion capture, wrist-worn accelerometry, clinical scoring and a second, specifically trained tremor-specific algorithm in two independent clinical cohorts. These cohorts consisted of 66 patients diagnosed with essential tremor, assessed in different task conditions and states of deep brain stimulation therapy. We find that Mediapipe-derived tremor metrics exhibit high convergent clinical validity to scores (Spearman's ρ = 0.55-0.86, p≤ .01) as well as an accuracy of up to 2.60 mm (95% CI [-3.13, 8.23]) and ≤0.21 Hz (95% CI [-0.05, 0.46]) for tremor amplitude and frequency measurements, matching gold-standard equipment. Mediapipe, but not the disease-specific algorithm, was capable of analysing videos involving complex configurational changes of the hands. Moreover, it enabled the extraction of tremor features with diagnostic and prognostic relevance, a dimension which conventional tremor scores were unable to provide. Collectively, this demonstrates that current computer vision algorithms can be transformed into an accurate and highly accessible tool for video-based tremor analysis, yielding comparable results to gold standard tremor recordings.
Collapse
Affiliation(s)
- Maximilian U Friedrich
- Center for Brain Circuit Therapeutics, Brigham and Women's Hospital, Boston, MA, USA.
- Harvard Medical School, Boston, MA, USA.
- Department of Neurology, University Hospital Wurzburg, Wuerzburg, Germany.
| | - Anna-Julia Roenn
- Department of Neurology, University Hospital Wurzburg, Wuerzburg, Germany
| | - Chiara Palmisano
- Department of Neurology, University Hospital Wurzburg, Wuerzburg, Germany
| | - Jane Alty
- Wicking Dementia Research and Education Centre, College of Health and Medicine, University of Tasmania, Hobart, Tasmania, Australia
| | | | | | - Chi Wang Ip
- Department of Neurology, University Hospital Wurzburg, Wuerzburg, Germany
| | - Jens Volkmann
- Department of Neurology, University Hospital Wurzburg, Wuerzburg, Germany
| | | | - Robert Peach
- Department of Neurology, University Hospital Wurzburg, Wuerzburg, Germany
- Department of Brain Sciences, Imperial College, London, UK
| | - Martin M Reich
- Department of Neurology, University Hospital Wurzburg, Wuerzburg, Germany.
| |
Collapse
|
4
|
Dubois C, Eigen D, Simon F, Couloigner V, Gormish M, Chalumeau M, Schmoll L, Cohen JF. Development and validation of a smartphone-based deep-learning-enabled system to detect middle-ear conditions in otoscopic images. NPJ Digit Med 2024; 7:162. [PMID: 38902477 PMCID: PMC11189910 DOI: 10.1038/s41746-024-01159-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2023] [Accepted: 06/10/2024] [Indexed: 06/22/2024] Open
Abstract
Middle-ear conditions are common causes of primary care visits, hearing impairment, and inappropriate antibiotic use. Deep learning (DL) may assist clinicians in interpreting otoscopic images. This study included patients over 5 years old from an ambulatory ENT practice in Strasbourg, France, between 2013 and 2020. Digital otoscopic images were obtained using a smartphone-attached otoscope (Smart Scope, Karl Storz, Germany) and labeled by a senior ENT specialist across 11 diagnostic classes (reference standard). An Inception-v2 DL model was trained using 41,664 otoscopic images, and its diagnostic accuracy was evaluated by calculating class-specific estimates of sensitivity and specificity. The model was then incorporated into a smartphone app called i-Nside. The DL model was evaluated on a validation set of 3,962 images and a held-out test set comprising 326 images. On the validation set, all class-specific estimates of sensitivity and specificity exceeded 98%. On the test set, the DL model achieved a sensitivity of 99.0% (95% confidence interval: 94.5-100) and a specificity of 95.2% (91.5-97.6) for the binary classification of normal vs. abnormal images; wax plugs were detected with a sensitivity of 100% (94.6-100) and specificity of 97.7% (95.0-99.1); other class-specific estimates of sensitivity and specificity ranged from 33.3% to 92.3% and 96.0% to 100%, respectively. We present an end-to-end DL-enabled system able to achieve expert-level diagnostic accuracy for identifying normal tympanic aspects and wax plugs within digital otoscopic images. However, the system's performance varied for other middle-ear conditions. Further prospective validation is necessary before wider clinical deployment.
Collapse
Affiliation(s)
| | | | - François Simon
- Department of Pediatric Otolaryngology, Necker-Enfants malades Hospital, APHP, Université Paris Cité, Paris, France
| | - Vincent Couloigner
- Department of Pediatric Otolaryngology, Necker-Enfants malades Hospital, APHP, Université Paris Cité, Paris, France
| | | | - Martin Chalumeau
- Inserm UMR1153 (CRESS), Université Paris Cité, Paris, France
- Department of General Pediatrics and Pediatric Infectious Diseases, Necker-Enfants malades Hospital, APHP, Université Paris Cité, Paris, France
| | | | - Jérémie F Cohen
- Inserm UMR1153 (CRESS), Université Paris Cité, Paris, France.
- Department of General Pediatrics and Pediatric Infectious Diseases, Necker-Enfants malades Hospital, APHP, Université Paris Cité, Paris, France.
| |
Collapse
|
5
|
Peach R, Friedrich M, Fronemann L, Muthuraman M, Schreglmann SR, Zeller D, Schrader C, Krauss JK, Schnitzler A, Wittstock M, Helmers AK, Paschen S, Kühn A, Skogseid IM, Eisner W, Mueller J, Matthies C, Reich M, Volkmann J, Ip CW. Head movement dynamics in dystonia: a multi-centre retrospective study using visual perceptive deep learning. NPJ Digit Med 2024; 7:160. [PMID: 38890413 PMCID: PMC11189529 DOI: 10.1038/s41746-024-01140-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2023] [Accepted: 05/22/2024] [Indexed: 06/20/2024] Open
Abstract
Dystonia is a neurological movement disorder characterised by abnormal involuntary movements and postures, particularly affecting the head and neck. However, current clinical assessment methods for dystonia rely on simplified rating scales which lack the ability to capture the intricate spatiotemporal features of dystonic phenomena, hindering clinical management and limiting understanding of the underlying neurobiology. To address this, we developed a visual perceptive deep learning framework that utilizes standard clinical videos to comprehensively evaluate and quantify disease states and the impact of therapeutic interventions, specifically deep brain stimulation. This framework overcomes the limitations of traditional rating scales and offers an efficient and accurate method that is rater-independent for evaluating and monitoring dystonia patients. To evaluate the framework, we leveraged semi-standardized clinical video data collected in three retrospective, longitudinal cohort studies across seven academic centres. We extracted static head angle excursions for clinical validation and derived kinematic variables reflecting naturalistic head dynamics to predict dystonia severity, subtype, and neuromodulation effects. The framework was also applied to a fully independent cohort of generalised dystonia patients for comparison between dystonia sub-types. Computer vision-derived measurements of head angle excursions showed a strong correlation with clinically assigned scores. Across comparisons, we identified consistent kinematic features from full video assessments encoding information critical to disease severity, subtype, and effects of neural circuit interventions, independent of static head angle deviations used in scoring. Our visual perceptive machine learning framework reveals kinematic pathosignatures of dystonia, potentially augmenting clinical management, facilitating scientific translation, and informing personalized precision neurology approaches.
Collapse
Affiliation(s)
- Robert Peach
- Department of Neurology, University Hospital Würzburg, Würzburg, 97080, Germany.
- Department of Brain Sciences, Imperial College London, London, UK.
| | - Maximilian Friedrich
- Department of Neurology, University Hospital Würzburg, Würzburg, 97080, Germany
- Center for Brain Circuit Therapeutics, Brigham & Women's Hospital, Boston, USA
- Harvard Medical School, Boston, USA
| | - Lara Fronemann
- Department of Neurology, University Hospital Würzburg, Würzburg, 97080, Germany
| | | | | | - Daniel Zeller
- Department of Neurology, University Hospital Würzburg, Würzburg, 97080, Germany
| | - Christoph Schrader
- Department of Neurology and Clinical Neurophysiology, Hannover Medical School, Hannover, Germany
| | - Joachim K Krauss
- Department of Neurosurgery, Hannover Medical School, Hannover, Germany
| | - Alfons Schnitzler
- Institute of Clinical Neuroscience and Medical Psychology, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
| | | | - Ann-Kristin Helmers
- Department of Neurology, UKSH, Kiel Campus Christian-Albrechts-University, Kiel, Germany
| | - Steffen Paschen
- Department of Neurology, Christian-Albrechts-University, Kiel, Germany
| | - Andrea Kühn
- Department of Neurology, Movement Disorder and Neuromodulation Unit, Charité - Universitätsmedizin, Berlin, Germany
| | - Inger Marie Skogseid
- Movement Disorders Unit, Department of Neurology, Oslo University Hospital, Rikshospitalet, Oslo, Norway
| | - Wilhelm Eisner
- Department of Neurology, Innsbruck Medical University, 6020, Innsbruck, Austria
| | - Joerg Mueller
- Klinik für Neurologie mit Stroke Unit, Vivantes Klinikum Spandau, Berlin, Germany
| | - Cordula Matthies
- Department of Neurology, University Hospital Würzburg, Würzburg, 97080, Germany
| | - Martin Reich
- Department of Neurology, University Hospital Würzburg, Würzburg, 97080, Germany
| | - Jens Volkmann
- Department of Neurology, University Hospital Würzburg, Würzburg, 97080, Germany
| | - Chi Wang Ip
- Department of Neurology, University Hospital Würzburg, Würzburg, 97080, Germany.
| |
Collapse
|
6
|
Liu J, Zhang Y, Wang K, Yavuz MC, Chen X, Yuan Y, Li H, Yang Y, Yuille A, Tang Y, Zhou Z. Universal and extensible language-vision models for organ segmentation and tumor detection from abdominal computed tomography. Med Image Anal 2024; 97:103226. [PMID: 38852215 DOI: 10.1016/j.media.2024.103226] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2023] [Revised: 03/30/2024] [Accepted: 05/27/2024] [Indexed: 06/11/2024]
Abstract
The advancement of artificial intelligence (AI) for organ segmentation and tumor detection is propelled by the growing availability of computed tomography (CT) datasets with detailed, per-voxel annotations. However, these AI models often struggle with flexibility for partially annotated datasets and extensibility for new classes due to limitations in the one-hot encoding, architectural design, and learning scheme. To overcome these limitations, we propose a universal, extensible framework enabling a single model, termed Universal Model, to deal with multiple public datasets and adapt to new classes (e.g., organs/tumors). Firstly, we introduce a novel language-driven parameter generator that leverages language embeddings from large language models, enriching semantic encoding compared with one-hot encoding. Secondly, the conventional output layers are replaced with lightweight, class-specific heads, allowing Universal Model to simultaneously segment 25 organs and six types of tumors and ease the addition of new classes. We train our Universal Model on 3410 CT volumes assembled from 14 publicly available datasets and then test it on 6173 CT volumes from four external datasets. Universal Model achieves first place on six CT tasks in the Medical Segmentation Decathlon (MSD) public leaderboard and leading performance on the Beyond The Cranial Vault (BTCV) dataset. In summary, Universal Model exhibits remarkable computational efficiency (6× faster than other dataset-specific models), demonstrates strong generalization across different hospitals, transfers well to numerous downstream tasks, and more importantly, facilitates the extensibility to new classes while alleviating the catastrophic forgetting of previously learned classes. Codes, models, and datasets are available at https://github.com/ljwztc/CLIP-Driven-Universal-Model.
Collapse
Affiliation(s)
- Jie Liu
- City University of Hong Kong, Hong Kong
| | - Yixiao Zhang
- Johns Hopkins University, United States of America
| | - Kang Wang
- University of California, San Francisco, United States of America
| | - Mehmet Can Yavuz
- University of California, San Francisco, United States of America
| | - Xiaoxi Chen
- University of Illinois Urbana-Champaign, United States of America
| | | | | | - Yang Yang
- University of California, San Francisco, United States of America
| | - Alan Yuille
- Johns Hopkins University, United States of America
| | | | - Zongwei Zhou
- Johns Hopkins University, United States of America.
| |
Collapse
|
7
|
Daneshvar N, Pandita D, Erickson S, Sulmasy LS, DeCamp M. Artificial Intelligence in the Provision of Health Care: An American College of Physicians Policy Position Paper. Ann Intern Med 2024. [PMID: 38830215 DOI: 10.7326/m24-0146] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 06/05/2024] Open
Abstract
Internal medicine physicians are increasingly interacting with systems that implement artificial intelligence (AI) and machine learning (ML) technologies. Some physicians and health care systems are even developing their own AI models, both within and outside of electronic health record (EHR) systems. These technologies have various applications throughout the provision of health care, such as clinical documentation, diagnostic image processing, and clinical decision support. With the growing availability of vast amounts of patient data and unprecedented levels of clinician burnout, the proliferation of these technologies is cautiously welcomed by some physicians. Others think it presents challenges to the patient-physician relationship and the professional integrity of physicians. These dispositions are understandable, given the "black box" nature of some AI models, for which specifications and development methods can be closely guarded or proprietary, along with the relative lagging or absence of appropriate regulatory scrutiny and validation. This American College of Physicians (ACP) position paper describes the College's foundational positions and recommendations regarding the use of AI- and ML-enabled tools and systems in the provision of health care. Many of the College's positions and recommendations, such as those related to patient-centeredness, privacy, and transparency, are founded on principles in the ACP Ethics Manual. They are also derived from considerations for the clinical safety and effectiveness of the tools as well as their potential consequences regarding health disparities. The College calls for more research on the clinical and ethical implications of these technologies and their effects on patient health and well-being.
Collapse
Affiliation(s)
| | - Deepti Pandita
- University of California Irvine Health, Laguna Niguel, California (D.P.)
| | - Shari Erickson
- American College of Physicians, Washington, DC (N.D., S.E.)
| | | | - Matthew DeCamp
- University of Colorado Anschutz Medical Campus, Aurora, Colorado (M.D.)
| |
Collapse
|
8
|
Cheng CT, Lin HH, Hsu CP, Chen HW, Huang JF, Hsieh CH, Fu CY, Chung IF, Liao CH. Deep Learning for Automated Detection and Localization of Traumatic Abdominal Solid Organ Injuries on CT Scans. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024; 37:1113-1123. [PMID: 38366294 PMCID: PMC11169164 DOI: 10.1007/s10278-024-01038-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/25/2023] [Revised: 01/31/2024] [Accepted: 02/01/2024] [Indexed: 02/18/2024]
Abstract
Computed tomography (CT) is the most commonly used diagnostic modality for blunt abdominal trauma (BAT), significantly influencing management approaches. Deep learning models (DLMs) have shown great promise in enhancing various aspects of clinical practice. There is limited literature available on the use of DLMs specifically for trauma image evaluation. In this study, we developed a DLM aimed at detecting solid organ injuries to assist medical professionals in rapidly identifying life-threatening injuries. The study enrolled patients from a single trauma center who received abdominal CT scans between 2008 and 2017. Patients with spleen, liver, or kidney injury were categorized as the solid organ injury group, while others were considered negative cases. Only images acquired from the trauma center were enrolled. A subset of images acquired in the last year was designated as the test set, and the remaining images were utilized to train and validate the detection models. The performance of each model was assessed using metrics such as the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity, positive predictive value, and negative predictive value based on the best Youden index operating point. The study developed the models using 1302 (87%) scans for training and tested them on 194 (13%) scans. The spleen injury model demonstrated an accuracy of 0.938 and a specificity of 0.952. The accuracy and specificity of the liver injury model were reported as 0.820 and 0.847, respectively. The kidney injury model showed an accuracy of 0.959 and a specificity of 0.989. We developed a DLM that can automate the detection of solid organ injuries by abdominal CT scans with acceptable diagnostic accuracy. It cannot replace the role of clinicians, but we can expect it to be a potential tool to accelerate the process of therapeutic decisions for trauma care.
Collapse
Affiliation(s)
- Chi-Tung Cheng
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan, Taiwan
| | - Hou-Hsien Lin
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan, Taiwan
| | - Chih-Po Hsu
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan, Taiwan
| | - Huan-Wu Chen
- Department of Medical Imaging & Intervention, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan, Taiwan
| | - Jen-Fu Huang
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan, Taiwan
| | - Chi-Hsun Hsieh
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan, Taiwan
| | - Chih-Yuan Fu
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan, Taiwan
| | - I-Fang Chung
- Institute of Biomedical Informatics, National Yang Ming Chiao Tung University, Taipei, Taiwan
| | - Chien-Hung Liao
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan, Taiwan.
| |
Collapse
|
9
|
Liu J, Li J, Duan Y, Zhou Y, Fan X, Li S, Chang S. MA-MIL: Sampling point-level abnormal ECG location method via weakly supervised learning. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 250:108164. [PMID: 38718709 DOI: 10.1016/j.cmpb.2024.108164] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/11/2023] [Revised: 03/21/2024] [Accepted: 04/04/2024] [Indexed: 05/15/2024]
Abstract
BACKGROUND AND OBJECTIVE Current automatic electrocardiogram (ECG) diagnostic systems could provide classification outcomes but often lack explanations for these results. This limitation hampers their application in clinical diagnoses. Previous supervised learning could not highlight abnormal segmentation output accurately enough for clinical application without manual labeling of large ECG datasets. METHOD In this study, we present a multi-instance learning framework called MA-MIL, which has designed a multi-layer and multi-instance structure that is aggregated step by step at different scales. We evaluated our method using the public MIT-BIH dataset and our private dataset. RESULTS The results show that our model performed well in both ECG classification output and heartbeat level, sub-heartbeat level abnormal segment detection, with accuracy and F1 scores of 0.987 and 0.986 for ECG classification and 0.968 and 0.949 for heartbeat level abnormal detection, respectively. Compared to visualization methods, the IoU values of MA-MIL improved by at least 17 % and at most 31 % across all categories. CONCLUSIONS MA-MIL could accurately locate the abnormal ECG segment, offering more trustworthy results for clinical application.
Collapse
Affiliation(s)
- Jin Liu
- Division of Biomedical Engineering, China Medical University, China
| | - Jiadong Li
- Division of Biomedical Engineering, China Medical University, China
| | - Yuxin Duan
- Division of Biomedical Engineering, China Medical University, China
| | - Yang Zhou
- Division of Biomedical Engineering, China Medical University, China
| | - Xiaoxue Fan
- Division of Biomedical Engineering, China Medical University, China
| | - Shuo Li
- School of Life Sciences, China Medical University, Shenyang, China
| | - Shijie Chang
- Division of Biomedical Engineering, China Medical University, China.
| |
Collapse
|
10
|
Nolin-Lapalme A, Theriault-Lauzier P, Corbin D, Tastet O, Sharma A, Hussin JG, Kadoury S, Jiang R, Krahn AD, Gallo R, Avram R. Maximizing Large Language Model Utility in Cardiovascular Care: A Practical Guide. Can J Cardiol 2024:S0828-282X(24)00415-X. [PMID: 38825181 DOI: 10.1016/j.cjca.2024.05.024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2024] [Revised: 05/03/2024] [Accepted: 05/16/2024] [Indexed: 06/04/2024] Open
Abstract
Large language models (LLMs) have emerged as powerful tools in artificial intelligence, demonstrating remarkable capabilities in natural language processing and generation. In this article, we explore the potential applications of LLMs in enhancing cardiovascular care and research. We discuss how LLMs can be utilized to simplify complex medical information, improve patient-physician communication, and automate tasks such as summarizing medical articles and extracting key information. Additionally, we highlight the role of LLMs in categorizing and analyzing unstructured data, such as medical notes and test results, which could revolutionize data handling and interpretation in cardiovascular research. However, we also emphasize the limitations and challenges associated with LLMs, including potential biases, reasoning opacity, and the need for rigorous validation in medical contexts. This article provides a practical guide for cardiovascular professionals to understand and harness the power of LLMs while navigating their limitations. We conclude by discussing the future directions and implications of LLMs in transforming cardiovascular care and research.
Collapse
Affiliation(s)
- Alexis Nolin-Lapalme
- Department of Medicine, Montreal Heart Institute, Université de Montréal, Montreal, Canada; Mila - Québec AI Institute, Montréal, Canada; Heartwise (heartwise.ai), Montreal Heart Institute, Montreal, Canada
| | - Pascal Theriault-Lauzier
- Division of Cardiovascular Medicine, Stanford University School of Medicine, California, United States of America
| | - Denis Corbin
- Department of Medicine, Montreal Heart Institute, Université de Montréal, Montreal, Canada; Heartwise (heartwise.ai), Montreal Heart Institute, Montreal, Canada
| | - Olivier Tastet
- Heartwise (heartwise.ai), Montreal Heart Institute, Montreal, Canada
| | - Abhinav Sharma
- DREAM-CV Lab, Department of Cardiology, McGill University, Montreal, Canada
| | - Julie G Hussin
- Department of Medicine, Montreal Heart Institute, Université de Montréal, Montreal, Canada
| | | | - River Jiang
- Center for Cardiovascular Innovation, Division of Cardiology, University of British Columbia, Vancouver, Canada
| | - Andrew D Krahn
- Center for Cardiovascular Innovation, Division of Cardiology, University of British Columbia, Vancouver, Canada
| | - Richard Gallo
- Department of Medicine, Montreal Heart Institute, Université de Montréal, Montreal, Canada
| | - Robert Avram
- Department of Medicine, Montreal Heart Institute, Université de Montréal, Montreal, Canada; Polytechnique Montréal, Montréal, Canada.
| |
Collapse
|
11
|
Brahmi Z, Mahyoob M, Al-Sarem M, Algaraady J, Bousselmi K, Alblwi A. Exploring the Role of Machine Learning in Diagnosing and Treating Speech Disorders: A Systematic Literature Review. Psychol Res Behav Manag 2024; 17:2205-2232. [PMID: 38835654 PMCID: PMC11149643 DOI: 10.2147/prbm.s460283] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2024] [Accepted: 05/07/2024] [Indexed: 06/06/2024] Open
Abstract
Purpose Speech disorders profoundly impact the overall quality of life by impeding social operations and hindering effective communication. This study addresses the gap in systematic reviews concerning machine learning-based assistive technology for individuals with speech disorders. The overarching purpose is to offer a comprehensive overview of the field through a Systematic Literature Review (SLR) and provide valuable insights into the landscape of ML-based solutions and related studies. Methods The research employs a systematic approach, utilizing a Systematic Literature Review (SLR) methodology. The study extensively examines the existing literature on machine learning-based assistive technology for speech disorders. Specific attention is given to ML techniques, characteristics of exploited datasets in the training phase, speaker languages, feature extraction techniques, and the features employed by ML algorithms. Originality This study contributes to the existing literature by systematically exploring the machine learning landscape in assistive technology for speech disorders. The originality lies in the focused investigation of ML-speech recognition for impaired speech disorder users over ten years (2014-2023). The emphasis on systematic research questions related to ML techniques, dataset characteristics, languages, feature extraction techniques, and feature sets adds a unique and comprehensive perspective to the current discourse. Findings The systematic literature review identifies significant trends and critical studies published between 2014 and 2023. In the analysis of the 65 papers from prestigious journals, support vector machines and neural networks (CNN, DNN) were the most utilized ML technique (20%, 16.92%), with the most studied disease being Dysarthria (35/65, 54% studies). Furthermore, an upsurge in using neural network-based architectures, mainly CNN and DNN, was observed after 2018. Almost half of the included studies were published between 2021 and 2022).
Collapse
Affiliation(s)
- Zaki Brahmi
- Department of Computer Science, Taibah University, Madina, Kingdom of Saudi Arabia
| | - Mohammad Mahyoob
- Department of Languages and Translation, Taibah University, Madina, Kingdom of Saudi Arabia
| | - Mohammed Al-Sarem
- Department of Computer Science, Taibah University, Madina, Kingdom of Saudi Arabia
| | | | - Khadija Bousselmi
- Department of Computer Science, LISTIC, University of Savoie Mont Blanc, Chambéry, France
| | - Abdulaziz Alblwi
- Department of Computer Science, Taibah University, Madina, Kingdom of Saudi Arabia
| |
Collapse
|
12
|
Ma Y, He J, Tan D, Han X, Feng R, Xiong H, Peng X, Pu X, Zhang L, Li Y, Chen S. The clinical and imaging data fusion model for single-period cerebral CTA collateral circulation assessment. JOURNAL OF X-RAY SCIENCE AND TECHNOLOGY 2024:XST240083. [PMID: 38820061 DOI: 10.3233/xst-240083] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/02/2024]
Abstract
Background The Chinese population ranks among the highest globally in terms of stroke prevalence. In the clinical diagnostic process, radiologists utilize computed tomography angiography (CTA) images for diagnosis, enabling a precise assessment of collateral circulation in the brains of stroke patients. Recent studies frequently combine imaging and machine learning methods to develop computer-aided diagnostic algorithms. However, in studies concerning collateral circulation assessment, the extracted imaging features are primarily composed of manually designed statistical features, which exhibit significant limitations in their representational capacity. Accurately assessing collateral circulation using image features in brain CTA images still presents challenges. Methods To tackle this issue, considering the scarcity of publicly accessible medical datasets, we combined clinical data with imaging data to establish a dataset named RadiomicsClinicCTA. Moreover, we devised two collateral circulation assessment models to exploit the synergistic potential of patients' clinical information and imaging data for a more accurate assessment of collateral circulation: data-level fusion and feature-level fusion. To remove redundant features from the dataset, we employed Levene's test and T-test methods for feature pre-screening. Subsequently, we performed feature dimensionality reduction using the LASSO and random forest algorithms and trained classification models with various machine learning algorithms on the data-level fusion dataset after feature engineering. Results Experimental results on the RadiomicsClinicCTA dataset demonstrate that the optimized data-level fusion model achieves an accuracy and AUC value exceeding 86% . Subsequently, we trained and assessed the performance of the feature-level fusion classification model. The results indicate the feature-level fusion classification model outperforms the optimized data-level fusion model. Comparative experiments show that the fused dataset better differentiates between good and bad side branch features relative to the pure radiomics dataset. Conclusions Our study underscores the efficacy of integrating clinical and imaging data through fusion models, significantly enhancing the accuracy of collateral circulation assessment in stroke patients.
Collapse
Affiliation(s)
- Yuqi Ma
- College of Computer and Information Science, Southwest University, Chongqing, China
| | - Jingliu He
- College of Computer and Information Science, Southwest University, Chongqing, China
| | - Duo Tan
- The Second People's Hospital of Guizhou Province, Guizhou, China
| | - Xu Han
- School of Electrical and Information Engineering, Tianjin University, Tianjin, China
| | - Ruiqi Feng
- College of Computer and Information Science, Southwest University, Chongqing, China
| | - Hailing Xiong
- College of Electronic and Information Engineering, Southwest University, Chongqing, China
| | - Xihua Peng
- College of Computer and Information Science, Southwest University, Chongqing, China
| | - Xun Pu
- College of Computer and Information Science, Southwest University, Chongqing, China
| | - Lin Zhang
- College of Computer and Information Science, Southwest University, Chongqing, China
| | - Yongmei Li
- Department of Radiology, The First Affiliated Hospital of Chongqing Medical University, Chongqing, China
| | - Shanxiong Chen
- College of Computer and Information Science, Southwest University, Chongqing, China
- Big Data & Intelligence Engineering School, Chongqing College of International Business and Economics, Chongqing, China
| |
Collapse
|
13
|
Karkehabadi H, Khoshbin E, Ghasemi N, Mahavi A, Mohammad-Rahimi H, Sadr S. Deep learning for determining the difficulty of endodontic treatment: a pilot study. BMC Oral Health 2024; 24:574. [PMID: 38760686 PMCID: PMC11102254 DOI: 10.1186/s12903-024-04235-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2023] [Accepted: 04/08/2024] [Indexed: 05/19/2024] Open
Abstract
BACKGROUND To develop and validate a deep learning model for automated assessment of endodontic case difficulty from periapical radiographs. METHODS A dataset of 1,386 periapical radiographs was compiled from two clinical sites. Two dentists and two endodontists annotated the radiographs for difficulty using the "simple assessment" criteria from the American Association of Endodontists' case difficulty assessment form in the Endocase application. A classification task labeled cases as "easy" or "hard", while regression predicted overall difficulty scores. Convolutional neural networks (i.e. VGG16, ResNet18, ResNet50, ResNext50, and Inception v2) were used, with a baseline model trained via transfer learning from ImageNet weights. Other models was pre-trained using self-supervised contrastive learning (i.e. BYOL, SimCLR, MoCo, and DINO) on 20,295 unlabeled dental radiographs to learn representation without manual labels. Both models were evaluated using 10-fold cross-validation, with performance compared to seven human examiners (three general dentists and four endodontists) on a hold-out test set. RESULTS The baseline VGG16 model attained 87.62% accuracy in classifying difficulty. Self-supervised pretraining did not improve performance. Regression predicted scores with ± 3.21 score error. All models outperformed human raters, with poor inter-examiner reliability. CONCLUSION This pilot study demonstrated the feasibility of automated endodontic difficulty assessment via deep learning models.
Collapse
Affiliation(s)
- Hamed Karkehabadi
- Department of Endodontics, Dental School, Hamadan University of Medical Sciences, Hamadan, Iran
- Department of Endodontics, Dental Research Center, Hamadan University of Medical Sciences, Hamadan, Iran
| | - Elham Khoshbin
- Department of Endodontics, Dental School, Hamadan University of Medical Sciences, Hamadan, Iran
| | - Nikoo Ghasemi
- Faculty of Dentistry, Zanjan University of Medical Sciences, Zanjan, Iran
| | - Amal Mahavi
- Department of Endodontics, Dental School, Hamadan University of Medical Sciences, Hamadan, Iran
| | - Hossein Mohammad-Rahimi
- Topic Group Dental Diagnostics and Digital Dentistry, ITU/WHO Focus Group AI on Health, Berlin, Federal Republic of Germany
| | - Soroush Sadr
- Department of Endodontics, Dental School, Hamadan University of Medical Sciences, Hamadan, Iran.
- Dental School, Hamadan University of Medical Sciences, Shahid Fahmideh Street, PO Box 6517838677, Hamadan, Iran.
| |
Collapse
|
14
|
Mamidi IS, Dunham ME, Adkins LK, McWhorter AJ, Fang Z, Banh BT. Laryngeal Cancer Screening During Flexible Video Laryngoscopy Using Large Computer Vision Models. Ann Otol Rhinol Laryngol 2024:34894241253376. [PMID: 38755974 DOI: 10.1177/00034894241253376] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/18/2024]
Abstract
OBJECTIVE Develop an artificial intelligence assisted computer vision model to screen for laryngeal cancer during flexible laryngoscopy. METHODS Using laryngeal images and flexible laryngoscopy video recordings, we developed computer vision models to classify video frames for usability and cancer screening. A separate model segments any identified lesions on the frames. We used these computer vision models to construct a video stream annotation system. This system classifies findings from flexible laryngoscopy as "potentially malignant" or "probably benign" and segments any detected lesions. Additionally, the model provides a confidence level for each classification. RESULTS The overall accuracy of the flexible laryngoscopy cancer screening model was 92%. For cancer screening, it achieved a sensitivity of 97.7% and a specificity of 76.9%. The segmentation model attained an average precision at a 0.50 intersection-over-union of 0.595. The confidence level for positive screening results can assist clinicians in counseling patients regarding the findings. CONCLUSION Our model is highly sensitive and adequately specific for laryngeal cancer screening. Segmentation helps endoscopists identify and describe potential lesions. Further optimization is required to enable the model's deployment in clinical settings for real-time annotation during flexible laryngoscopy.
Collapse
Affiliation(s)
- Ishwarya S Mamidi
- Department of Otolaryngology-Head and Neck Surgery, Louisiana State University Health Sciences Center, New Orleans, LA, USA
| | - Michael E Dunham
- Department of Otolaryngology-Head and Neck Surgery, Louisiana State University Health Sciences Center, New Orleans, LA, USA
| | - Lacey K Adkins
- Department of Otolaryngology-Head and Neck Surgery, Louisiana State University Health Sciences Center, New Orleans, LA, USA
| | - Andrew J McWhorter
- Department of Otolaryngology-Head and Neck Surgery, Louisiana State University Health Sciences Center, New Orleans, LA, USA
| | - Zhide Fang
- Biostatistics Program, School of Public Health, Louisiana State University Health Sciences Center, New Orleans LA, USA
| | - Britney T Banh
- Our Lady of the Lake Voice Center, Our Lady of the Lake Regional Medical Center, Baton Rouge, LA, USA
| |
Collapse
|
15
|
Gosavi AA, Nandgude TD, Mishra RK, Puri DB. Exploring the Potential of Artificial Intelligence as a Facilitating Tool for Formulation Development in Fluidized Bed Processor: a Comprehensive Review. AAPS PharmSciTech 2024; 25:111. [PMID: 38740666 DOI: 10.1208/s12249-024-02816-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2024] [Accepted: 04/23/2024] [Indexed: 05/16/2024] Open
Abstract
This in-depth study looks into how artificial intelligence (AI) could be used to make formulation development easier in fluidized bed processes (FBP). FBP is complex and involves numerous variables, making optimization challenging. Various AI techniques have addressed this challenge, including machine learning, neural networks, genetic algorithms, and fuzzy logic. By integrating AI with experimental design, process modeling, and optimization strategies, intelligent systems for FBP can be developed. The advantages of AI in this context include improved process understanding, reduced time and cost, enhanced product quality, and robust formulation optimization. However, data availability, model interpretability, and regulatory compliance challenges must be addressed. Case studies demonstrate successful applications of AI in decision-making, process outcome prediction, and scale-up. AI can improve efficiency, quality, and cost-effectiveness in significant ways. Still, it is important to think carefully about data quality, how easy it is to understand, and how to follow the rules. Future research should focus on fully harnessing the potential of AI to advance formulation development in FBP.
Collapse
Affiliation(s)
- Aachal A Gosavi
- Department of Pharmaceutics, Dr. D. Y. Patil Institute of Pharmaceutical Sciences and Research, Pimpri, Pune, India
| | - Tanaji D Nandgude
- Department of Pharmaceutics, JSPM University's School of Pharmaceutical Sciences, Wagholi, Pune, India
| | - Rakesh K Mishra
- Department of Pharmaceutics, Dr. D. Y. Patil Institute of Pharmaceutical Sciences and Research, Pimpri, Pune, India.
| | - Dhiraj B Puri
- Department of Mechanical Engineering, Birla Institute of Technology and Science-Pilani, K K Birla Goa Campus, Zuarinagar, Sancoale, Goa, India
| |
Collapse
|
16
|
Song AH, Williams M, Williamson DFK, Chow SSL, Jaume G, Gao G, Zhang A, Chen B, Baras AS, Serafin R, Colling R, Downes MR, Farré X, Humphrey P, Verrill C, True LD, Parwani AV, Liu JTC, Mahmood F. Analysis of 3D pathology samples using weakly supervised AI. Cell 2024; 187:2502-2520.e17. [PMID: 38729110 PMCID: PMC11168832 DOI: 10.1016/j.cell.2024.03.035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Revised: 01/15/2024] [Accepted: 03/25/2024] [Indexed: 05/12/2024]
Abstract
Human tissue, which is inherently three-dimensional (3D), is traditionally examined through standard-of-care histopathology as limited two-dimensional (2D) cross-sections that can insufficiently represent the tissue due to sampling bias. To holistically characterize histomorphology, 3D imaging modalities have been developed, but clinical translation is hampered by complex manual evaluation and lack of computational platforms to distill clinical insights from large, high-resolution datasets. We present TriPath, a deep-learning platform for processing tissue volumes and efficiently predicting clinical outcomes based on 3D morphological features. Recurrence risk-stratification models were trained on prostate cancer specimens imaged with open-top light-sheet microscopy or microcomputed tomography. By comprehensively capturing 3D morphologies, 3D volume-based prognostication achieves superior performance to traditional 2D slice-based approaches, including clinical/histopathological baselines from six certified genitourinary pathologists. Incorporating greater tissue volume improves prognostic performance and mitigates risk prediction variability from sampling bias, further emphasizing the value of capturing larger extents of heterogeneous morphology.
Collapse
Affiliation(s)
- Andrew H Song
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Mane Williams
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA; Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Drew F K Williamson
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Sarah S L Chow
- Department of Mechanical Engineering, Bioengineering, and Laboratory Medicine & Pathology, University of Washington, Seattle, WA, USA
| | - Guillaume Jaume
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Gan Gao
- Department of Mechanical Engineering, Bioengineering, and Laboratory Medicine & Pathology, University of Washington, Seattle, WA, USA
| | - Andrew Zhang
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA; Harvard-MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Bowen Chen
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Alexander S Baras
- Department of Pathology, Johns Hopkins University School of Medicine, Baltimore, MD, USA; Department of Biomedical Engineering, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Robert Serafin
- Department of Mechanical Engineering, Bioengineering, and Laboratory Medicine & Pathology, University of Washington, Seattle, WA, USA
| | - Richard Colling
- Nuffield Department of Surgical Sciences, University of Oxford, UK; Department of Cellular Pathology, Oxford University Hospitals NHS Foundations Trust, John Radcliffe Hospital, Oxford, UK
| | - Michelle R Downes
- Sunnybrook Health Sciences Centre, University of Toronto, Toronto, ON, Canada
| | - Xavier Farré
- Public Health Agency of Catalonia, Lleida, Spain
| | - Peter Humphrey
- Department of Pathology, Yale School of Medicine, New Haven, CT, USA
| | - Clare Verrill
- Nuffield Department of Surgical Sciences, University of Oxford, UK; Department of Cellular Pathology, Oxford University Hospitals NHS Foundations Trust, John Radcliffe Hospital, Oxford, UK; NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Lawrence D True
- Department of Laboratory Medicine & Pathology, University of Washington School of Medicine, Seattle, WA, USA
| | - Anil V Parwani
- Department of Pathology, The Ohio State University, Columbus, OH, USA
| | - Jonathan T C Liu
- Department of Mechanical Engineering, Bioengineering, and Laboratory Medicine & Pathology, University of Washington, Seattle, WA, USA.
| | - Faisal Mahmood
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA.
| |
Collapse
|
17
|
Kim Y, Park H, Yoon HJ, Suh J, Kang SH, Lim YH, Jang DH, Park JH, Shin ES, Bae JW, Lee JH, Oh JH, Kang DY, Kweon J, Jo MW, Park DW, Kim YH, Ahn JM. Fully automated quantitative coronary angiography versus optical coherence tomography guidance for coronary stent implantation (FLASH): Study protocol for a randomized controlled noninferiority trial. Am Heart J 2024; 275:86-95. [PMID: 38723880 DOI: 10.1016/j.ahj.2024.05.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/18/2024] [Revised: 05/05/2024] [Accepted: 05/05/2024] [Indexed: 07/01/2024]
Abstract
BACKGROUND Artificial intelligence-based quantitative coronary angiography (AI-QCA) has been developed to provide a more objective and reproducible data about the severity of coronary artery stenosis and the dimensions of the vessel for intervention in real-time, overcoming the limitations of significant inter- and intraobserver variability, and time-consuming nature of on-site QCA, without requiring extra time and effort. Compared with the subjective nature of visually estimated conventional CAG guidance, AI-QCA guidance provides a more practical and standardized angiography-based approach. Although the advantage of intravascular imaging-guided PCI is increasingly recognized, their broader adoption is limited by clinical and economic barriers in many catheterization laboratories. METHODS The FLASH (fully automated quantitative coronary angiography versus optical coherence tomography guidance for coronary stent implantation) trial is a randomized, investigator-initiated, multicenter, open-label, noninferiority trial comparing the AI-QCA-assisted PCI strategy with optical coherence tomography-guided PCI strategy in patients with significant coronary artery disease. All operators will utilize a novel, standardized AI-QCA software and PCI protocol in the AI-QCA-assisted group. A total of 400 patients will be randomized to either group at a 1:1 ratio. The primary endpoint is the minimal stent area (mm2), determined by the final OCT run after completion of PCI. Clinical follow-up and cost-effectiveness evaluations are planned at 1 month and 6 months for all patients enrolled in the study. RESULTS Enrollment of a total of 400 patients from the 13 participating centers in South Korea will be completed in February 2024. Follow-up of the last enrolled patients will be completed in August 2024, and primary results will be available by late 2024. CONCLUSION The FLASH is the first clinical trial to evaluate the feasibility of AI-QCA-assisted PCI, and will provide the clinical evidence on AI-QCA assistance in the field of coronary intervention. CLINICAL TRIAL REGISTRATION URL: https://www. CLINICALTRIALS gov. Unique identifier: NCT05388357.
Collapse
Affiliation(s)
- Yongcheol Kim
- Division of Cardiology, Department of Internal Medicine, Yonsei University College of Medicine and Cardiovascular Center, Yongin Severance Hospital, Yongin, Korea
| | - Hanbit Park
- Department of Medicine, Division of Cardiology, Gangneung Asan Hospital, University of Ulsan College of Medicine, Gangneung, Korea
| | - Hyuck-Jun Yoon
- Department of Internal Medicine and Cardiovascular Research Institute, Keimyung University Dongsan Hospital, Daegu, Korea
| | - Jon Suh
- Department of Cardiology, Soon Chun Hyang University Hospital Bucheon, Bucheon, Korea
| | - Si-Hyuck Kang
- Department of Internal Medicine, Cardiovascular Center, Seoul National University Bundang Hospital, Seongnam, Republic of Korea
| | - Young-Hyo Lim
- Department of Internal Medicine, Division of Cardiology, Hanyang University College of Medicine, Seoul, Korea
| | - Duck Hyun Jang
- Department of Internal Medicine, Division of Cardiology, Sejong General Hospital, Bucheon, Korea
| | - Jae Hyoung Park
- Department of Cardiology, Cardiovascular Center, Korea University Anam Hospital, Korea University College of Medicine, Seoul, Korea
| | - Eun-Seok Shin
- Department of Cardiology, Ulsan University Hospital, University of Ulsan College of Medicine, Ulsan, Korea
| | - Jang-Whan Bae
- Department of Internal Medicine, Division of Cardiology, Chungbuk National University College of Medicine, Cheongju, Korea
| | - Jang Hoon Lee
- Department of Internal Medicine, Kyungpook National University Hospital, School of Medicine, Kyungpook National University, Daegu, Korea
| | - Jun-Hyok Oh
- Department of Cardiology and Medical Research Institute, Pusan, Pusan National University, National University Hospital, Busan, Korea
| | - Do-Yoon Kang
- Division of Cardiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Korea
| | - Jihoon Kweon
- Department of Biomedical Engineering, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Korea
| | - Min-Woo Jo
- Department of Preventive Medicine, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Duk-Woo Park
- Division of Cardiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Korea
| | - Young-Hak Kim
- Division of Cardiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Korea
| | - Jung-Min Ahn
- Division of Cardiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Korea
| |
Collapse
|
18
|
Barmpas K, Panagakis Y, Zoumpourlis G, Adamos DA, Laskaris N, Zafeiriou S. A causal perspective on brainwave modeling for brain-computer interfaces. J Neural Eng 2024; 21:036001. [PMID: 38621380 DOI: 10.1088/1741-2552/ad3eb5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2023] [Accepted: 04/15/2024] [Indexed: 04/17/2024]
Abstract
Objective. Machine learning (ML) models have opened up enormous opportunities in the field of brain-computer Interfaces (BCIs). Despite their great success, they usually face severe limitations when they are employed in real-life applications outside a controlled laboratory setting.Approach. Mixing causal reasoning, identifying causal relationships between variables of interest, with brainwave modeling can change one's viewpoint on some of these major challenges which can be found in various stages in the ML pipeline, ranging from data collection and data pre-processing to training methods and techniques.Main results. In this work, we employ causal reasoning and present a framework aiming to breakdown and analyze important challenges of brainwave modeling for BCIs.Significance. Furthermore, we present how general ML practices as well as brainwave-specific techniques can be utilized and solve some of these identified challenges. And finally, we discuss appropriate evaluation schemes in order to measure these techniques' performance and efficiently compare them with other methods that will be developed in the future.
Collapse
Affiliation(s)
- Konstantinos Barmpas
- Department of Computing, Imperial College London, London SW7 2RH, United Kingdom
- Cogitat Ltd, London, United Kingdom
| | - Yannis Panagakis
- Department of Informatics and Telecommunications, National and Kapodistrian University of Athens, Athens 15784, Greece
- Archimedes Research Unit, Research Center Athena, Athens 15125, Greece
- Cogitat Ltd, London, United Kingdom
| | | | - Dimitrios A Adamos
- Department of Computing, Imperial College London, London SW7 2RH, United Kingdom
- Cogitat Ltd, London, United Kingdom
| | - Nikolaos Laskaris
- School of Informatics, Aristotle University of Thessaloniki, Thessaloniki 54124, Greece
- Cogitat Ltd, London, United Kingdom
| | - Stefanos Zafeiriou
- Department of Computing, Imperial College London, London SW7 2RH, United Kingdom
- Cogitat Ltd, London, United Kingdom
| |
Collapse
|
19
|
Wolcott ZC, English SW. Artificial intelligence to enhance prehospital stroke diagnosis and triage: a perspective. Front Neurol 2024; 15:1389056. [PMID: 38756217 PMCID: PMC11096539 DOI: 10.3389/fneur.2024.1389056] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2024] [Accepted: 04/22/2024] [Indexed: 05/18/2024] Open
Abstract
As health systems organize to deliver the highest quality stroke care to their patients, there is increasing emphasis being placed on prehospital stroke recognition, accurate diagnosis, and efficient triage to improve outcomes after stroke. Emergency medical services (EMS) personnel currently rely heavily on dispatch accuracy, stroke screening tools, bypass protocols and prehospital notification to care for patients with suspected stroke, but novel tools including mobile stroke units and telemedicine-enabled ambulances are already changing the landscape of prehospital stroke care. Herein, the authors provide our perspective on the current state of prehospital stroke diagnosis and triage including several of these emerging trends. Then, we provide commentary to highlight potential artificial intelligence (AI) applications to improve stroke detection, improve accurate and timely dispatch, enhance EMS training and performance, and develop novel stroke diagnostic tools for prehospital use.
Collapse
|
20
|
Kaji ES, Grove AF, Taunton MJ. Present and Future Optimization of Orthopaedic Care Through Machine Learning Algorithms. J Arthroplasty 2024; 39:1171-1172. [PMID: 38642965 DOI: 10.1016/j.arth.2024.03.043] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 04/22/2024] Open
|
21
|
Farabi Maleki S, Yousefi M, Afshar S, Pedrammehr S, Lim CP, Jafarizadeh A, Asadi H. Artificial Intelligence for Multiple Sclerosis Management Using Retinal Images: Pearl, Peaks, and Pitfalls. Semin Ophthalmol 2024; 39:271-288. [PMID: 38088176 DOI: 10.1080/08820538.2023.2293030] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/27/2023] [Accepted: 11/23/2023] [Indexed: 03/28/2024]
Abstract
Multiple sclerosis (MS) is a complex autoimmune disease characterized by inflammatory processes, demyelination, neurodegeneration, and axonal damage within the central nervous system (CNS). Retinal imaging, particularly Optical coherence tomography (OCT), has emerged as a crucial tool for investigating MS-related retinal injury. The integration of artificial intelligence(AI) has shown promise in enhancing OCT analysis for MS. Researchers are actively utilizing AI algorithms to accurately detect and classify MS-related abnormalities, leading to improved efficiency in diagnosis, monitoring, and personalized treatment planning. The prognostic value of AI in predicting MS disease progression has garnered substantial attention. Machine learning (ML) and deep learning (DL) algorithms can analyze longitudinal OCT data to forecast the course of the disease, providing critical information for personalized treatment planning and improved patient outcomes. Early detection of high-risk patients allows for targeted interventions to mitigate disability progression effectively. As such, AI-driven approaches yielded remarkable abilities in classifying distinct MS subtypes based on retinal features, aiding in disease characterization and guiding tailored therapeutic strategies. Additionally, these algorithms have enhanced the accuracy and efficiency of OCT image segmentation, streamlined diagnostic processes, and reduced human error. This study reviews the current research studies on the integration of AI,including ML and DL algorithms, with OCT in the context of MS. It examines the advancements, challenges, potential prospects, and ethical concerns of AI-powered techniques in enhancing MS diagnosis, monitoring disease progression, revolutionizing patient care, the development of patient screening tools, and supported clinical decision-making based on OCT images.
Collapse
Affiliation(s)
| | - Milad Yousefi
- Faculty of Mathematics, Statistics and Computer Sciences, University of Tabriz, Tabriz, Iran
| | - Sayeh Afshar
- Nikookari Eye Center, Tabriz University of Medical Sciences, Tabriz, Iran
| | | | - Chee Peng Lim
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Burwood, Australia
| | - Ali Jafarizadeh
- Nikookari Eye Center, Tabriz University of Medical Sciences, Tabriz, Iran
| | - Houshyar Asadi
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Burwood, Australia
| |
Collapse
|
22
|
Yang CN, Chen WL, Yeh HH, Chu HS, Wu JH, Hsieh YT. Convolutional Neural Network-Based Prediction of Axial Length Using Color Fundus Photography. Transl Vis Sci Technol 2024; 13:23. [PMID: 38809531 DOI: 10.1167/tvst.13.5.23] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/30/2024] Open
Abstract
Purpose To develop convolutional neural network (CNN)-based models for predicting the axial length (AL) using color fundus photography (CFP) and explore associated clinical and structural characteristics. Methods This study enrolled 1105 fundus images from 467 participants with ALs ranging from 19.91 to 32.59 mm, obtained at National Taiwan University Hospital between 2020 and 2021. The AL measurements obtained from a scanning laser interferometer served as the gold standard. The accuracy of prediction was compared among CNN-based models with different inputs, including CFP, age, and/or sex. Heatmaps were interpreted by integrated gradients. Results Using age, sex, and CFP as input, the mean ± standard deviation absolute error (MAE) for AL prediction by the model was 0.771 ± 0.128 mm, outperforming models that used age and sex alone (1.263 ± 0.115 mm; P < 0.001) and CFP alone (0.831 ± 0.216 mm; P = 0.016) by 39.0% and 7.31%, respectively. The removal of relatively poor-quality CFPs resulted in a slight MAE reduction to 0.759 ± 0.120 mm without statistical significance (P = 0.24). The inclusion of age and CFP improved prediction accuracy by 5.59% (P = 0.043), while adding sex had no significant improvement (P = 0.41). The optic disc and temporal peripapillary area were highlighted as the focused areas on the heatmaps. Conclusions Deep learning-based prediction of AL using CFP was fairly accurate and enhanced by age inclusion. The optic disc and temporal peripapillary area may contain crucial structural information for AL prediction in CFP. Translational Relevance This study might aid AL assessments and the understanding of the morphologic characteristics of the fundus related to AL.
Collapse
Affiliation(s)
- Che-Ning Yang
- School of Medicine, National Taiwan University, Taipei, Taiwan
| | - Wei-Li Chen
- School of Medicine, National Taiwan University, Taipei, Taiwan
- Department of Ophthalmology, National Taiwan University Hospital, Taipei, Taiwan
| | - Hsu-Hang Yeh
- Department of Ophthalmology, National Taiwan University Hospital, Taipei, Taiwan
| | - Hsiao-Sang Chu
- Department of Ophthalmology, National Taiwan University Hospital, Taipei, Taiwan
- Graduate Institute of Clinical Medicine, College of Medicine, National Taiwan University, Taipei, Taiwan
| | - Jo-Hsuan Wu
- Shiley Eye Institute and Viterbi Family Department of Ophthalmology, University of California San Diego, La Jolla, CA, USA
| | - Yi-Ting Hsieh
- School of Medicine, National Taiwan University, Taipei, Taiwan
- Department of Ophthalmology, National Taiwan University Hospital, Taipei, Taiwan
| |
Collapse
|
23
|
Ayhan MS, Neubauer J, Uzel MM, Gelisken F, Berens P. Interpretable detection of epiretinal membrane from optical coherence tomography with deep neural networks. Sci Rep 2024; 14:8484. [PMID: 38605115 PMCID: PMC11009346 DOI: 10.1038/s41598-024-57798-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2022] [Accepted: 03/21/2024] [Indexed: 04/13/2024] Open
Abstract
This study aimed to automatically detect epiretinal membranes (ERM) in various OCT-scans of the central and paracentral macula region and classify them by size using deep-neural-networks (DNNs). To this end, 11,061 OCT-images were included and graded according to the presence of an ERM and its size (small 100-1000 µm, large > 1000 µm). The data set was divided into training, validation and test sets (75%, 10%, 15% of the data, respectively). An ensemble of DNNs was trained and saliency maps were generated using Guided-Backprob. OCT-scans were also transformed into a one-dimensional-value using t-SNE analysis. The DNNs' receiver-operating-characteristics on the test set showed a high performance for no-ERM, small-ERM and large-ERM cases (AUC: 0.99, 0.92, 0.99, respectively; 3-way accuracy: 89%), with small-ERMs being the most difficult ones to detect. t-SNE analysis sorted cases by size and, in particular, revealed increased classification uncertainty at the transitions between groups. Saliency maps reliably highlighted ERM, regardless of the presence of other OCT features (i.e. retinal-thickening, intraretinal pseudo-cysts, epiretinal-proliferation) and entities such as ERM-retinoschisis, macular-pseudohole and lamellar-macular-hole. This study showed therefore that DNNs can reliably detect and grade ERMs according to their size not only in the fovea but also in the paracentral region. This is also achieved in cases of hard-to-detect, small-ERMs. In addition, the generated saliency maps can be used to highlight small-ERMs that might otherwise be missed. The proposed model could be used for screening-programs or decision-support-systems in the future.
Collapse
Affiliation(s)
- Murat Seçkin Ayhan
- Institute for Ophthalmic Research, University of Tübingen, Elfriede Aulhorn Str. 7, 72076, Tübingen, Germany
| | - Jonas Neubauer
- University Eye Clinic, University of Tübingen, Tübingen, Germany
| | - Mehmet Murat Uzel
- University Eye Clinic, University of Tübingen, Tübingen, Germany
- Department of Ophthalmology, Balıkesir University School of Medicine, Balıkesir, Turkey
| | - Faik Gelisken
- University Eye Clinic, University of Tübingen, Tübingen, Germany.
| | - Philipp Berens
- Institute for Ophthalmic Research, University of Tübingen, Elfriede Aulhorn Str. 7, 72076, Tübingen, Germany.
- Tübingen AI Center, Tübingen, Germany.
| |
Collapse
|
24
|
Walter C, Weissert C, Gizewski E, Burckhardt I, Mannsperger H, Hänselmann S, Busch W, Zimmermann S, Nolte O. Performance evaluation of machine-assisted interpretation of Gram stains from positive blood cultures. J Clin Microbiol 2024; 62:e0087623. [PMID: 38506525 PMCID: PMC11005413 DOI: 10.1128/jcm.00876-23] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2023] [Accepted: 02/24/2024] [Indexed: 03/21/2024] Open
Abstract
Manual microscopy of Gram stains from positive blood cultures (PBCs) is crucial for diagnosing bloodstream infections but remains labor intensive, time consuming, and subjective. This study aimed to evaluate a scan and analysis system that combines fully automated digital microscopy with deep convolutional neural networks (CNNs) to assist the interpretation of Gram stains from PBCs for routine laboratory use. The CNN was trained to classify images of Gram stains based on staining and morphology into seven different classes: background/false-positive, Gram-positive cocci in clusters (GPCCL), Gram-positive cocci in pairs (GPCP), Gram-positive cocci in chains (GPCC), rod-shaped bacilli (RSB), yeasts, and polymicrobial specimens. A total of 1,555 Gram-stained slides of PBCs were scanned, pre-classified, and reviewed by medical professionals. The results of assisted Gram stain interpretation were compared to those of manual microscopy and cultural species identification by matrix-assisted laser desorption/ionization time-of-flight mass spectrometry (MALDI-TOF MS). The comparison of assisted Gram stain interpretation and manual microscopy yielded positive/negative percent agreement values of 95.8%/98.0% (GPCCL), 87.6%/99.3% (GPCP/GPCC), 97.4%/97.8% (RSB), 83.3%/99.3% (yeasts), and 87.0%/98.5% (negative/false positive). The assisted Gram stain interpretation, when compared to MALDI-TOF MS species identification, also yielded similar results. During the analytical performance study, assisted interpretation showed excellent reproducibility and repeatability. Any microorganism in PBCs should be detectable at the determined limit of detection of 105 CFU/mL. Although the CNN-based interpretation of Gram stains from PBCs is not yet ready for clinical implementation, it has potential for future integration and advancement.
Collapse
Affiliation(s)
- Christian Walter
- Department of Infectious Diseases, Medical Microbiology and Hygiene, Medical Faculty Heidelberg, Heidelberg University, Heidelberg, Germany
- University Hospital Heidelberg, Heidelberg, Germany
| | - Christoph Weissert
- Division of Human Microbiology, Centre for Laboratory Medicine, St. Gall, Switzerland
| | - Eve Gizewski
- MetaSystems Hard & Software GmbH, Altlussheim, Germany
| | - Irene Burckhardt
- Department of Infectious Diseases, Medical Microbiology and Hygiene, Medical Faculty Heidelberg, Heidelberg University, Heidelberg, Germany
- University Hospital Heidelberg, Heidelberg, Germany
| | | | | | | | - Stefan Zimmermann
- Department of Infectious Diseases, Medical Microbiology and Hygiene, Medical Faculty Heidelberg, Heidelberg University, Heidelberg, Germany
- University Hospital Heidelberg, Heidelberg, Germany
| | - Oliver Nolte
- Division of Human Microbiology, Centre for Laboratory Medicine, St. Gall, Switzerland
| |
Collapse
|
25
|
Ahn S, Lee HS. Applicability of Spatial Technology in Cancer Research. Cancer Res Treat 2024; 56:343-356. [PMID: 38291743 PMCID: PMC11016655 DOI: 10.4143/crt.2023.1302] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2023] [Accepted: 01/29/2024] [Indexed: 02/01/2024] Open
Abstract
This review explores spatial mapping technologies in cancer research, highlighting their crucial role in understanding the complexities of the tumor microenvironment (TME). The TME, which is an intricate ecosystem of diverse cell types, has a significant impact on tumor dynamics and treatment outcomes. This review closely examines cutting-edge spatial mapping technologies, categorizing them into capture-, imaging-, and antibody-based approaches. Each technology was scrutinized for its advantages and disadvantages, factoring in aspects such as spatial profiling area, multiplexing capabilities, and resolution. Additionally, we draw attention to the nuanced choices researchers face, with capture-based methods lending themselves to hypothesis generation, and imaging/antibody-based methods that fit neatly into hypothesis testing. Looking ahead, we anticipate a scenario in which multi-omics data are seamlessly integrated, artificial intelligence enhances data analysis, and spatiotemporal profiling opens up new dimensions.
Collapse
Affiliation(s)
- Sangjeong Ahn
- Department of Pathology, Korea University Anam Hospital, Korea University College of Medicine, Seoul, Korea
- Artificial Intelligence Center, Korea University Anam Hospital, Korea University College of Medicine, Seoul, Korea
- Department of Medical Informatics, Korea University College of Medicine, Seoul, Korea
| | - Hye Seung Lee
- Department of Pathology, Seoul National University Hospital, Seoul National University College of Medicine, Seoul, Korea
- Cancer Research Institute, Seoul National University College of Medicine, Seoul, Korea
| |
Collapse
|
26
|
Roth CJ, Petersilge C, Clunie D, Towbin AJ, Cram D, Primo R, Li X, Berkowitz SJ, Barnosky V, Krupinski EA. HIMSS-SIIM Enterprise Imaging Community White Papers: Reflections and Future Directions. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024; 37:429-443. [PMID: 38336948 PMCID: PMC11031499 DOI: 10.1007/s10278-024-00992-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 01/23/2024] [Indexed: 02/12/2024]
Affiliation(s)
| | - Cheryl Petersilge
- Vidagos University of Pittsburgh School of Medicine, Pittsburgh, PA, USA
| | | | - Alexander J Towbin
- Department of Radiology, Cincinnati Children's Hospital, Cincinnati, OH, USA.
- Department of Radiology, University of Cincinnati College of Medicine, Cincinnati, OH, USA.
| | - Dawn Cram
- PaxeraHealth, 85 Wells Ave Suite 120, Newton, MA, 02459, USA
| | - Rik Primo
- Primo Medical Imaging Informatics Inc, Chicago, IL, USA
| | - Xin Li
- Department of Radiology, Hospital of the University of Pennsylvania, Philadelphia, PA, USA
| | - Seth J Berkowitz
- Department of Radiology, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, 02115, USA
| | - Victoria Barnosky
- Robert Morris University, Moon Township, Suazio, Philadelphia, PA, USA
| | | |
Collapse
|
27
|
Quistberg DA. Potential of artificial intelligence in injury prevention research and practice. Inj Prev 2024; 30:89-91. [PMID: 38307714 PMCID: PMC11003389 DOI: 10.1136/ip-2023-045203] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2023] [Accepted: 01/18/2024] [Indexed: 02/04/2024]
Abstract
There is increasing interest and use of artificial Intelligence algorithms and methods in biomedical research and practice, particularly as the technology has made significant advances in the past decade and become more accessible to more disciplines. This editorial briefly reviews this technology and its potential for injury prevention research and practice, proposing ways that it can be used to advance the discipline, as well as the potential pitfalls, concerns and biases that accompany it.
Collapse
Affiliation(s)
- D Alex Quistberg
- Urban Health Collaborative, Drexel University, Philadelphia, Pennsylvania, USA
- Environmental & Occupational Health, Drexel University, Philadelphia, Pennsylvania, USA
| |
Collapse
|
28
|
Wei ML, Tada M, So A, Torres R. Artificial intelligence and skin cancer. Front Med (Lausanne) 2024; 11:1331895. [PMID: 38566925 PMCID: PMC10985205 DOI: 10.3389/fmed.2024.1331895] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2023] [Accepted: 02/26/2024] [Indexed: 04/04/2024] Open
Abstract
Artificial intelligence is poised to rapidly reshape many fields, including that of skin cancer screening and diagnosis, both as a disruptive and assistive technology. Together with the collection and availability of large medical data sets, artificial intelligence will become a powerful tool that can be leveraged by physicians in their diagnoses and treatment plans for patients. This comprehensive review focuses on current progress toward AI applications for patients, primary care providers, dermatologists, and dermatopathologists, explores the diverse applications of image and molecular processing for skin cancer, and highlights AI's potential for patient self-screening and improving diagnostic accuracy for non-dermatologists. We additionally delve into the challenges and barriers to clinical implementation, paths forward for implementation and areas of active research.
Collapse
Affiliation(s)
- Maria L. Wei
- Department of Dermatology, University of California, San Francisco, San Francisco, CA, United States
- Dermatology Service, San Francisco VA Health Care System, San Francisco, CA, United States
| | - Mikio Tada
- Institute for Neurodegenerative Diseases, University of California, San Francisco, San Francisco, CA, United States
| | - Alexandra So
- School of Medicine, University of California, San Francisco, San Francisco, CA, United States
| | - Rodrigo Torres
- Dermatology Service, San Francisco VA Health Care System, San Francisco, CA, United States
| |
Collapse
|
29
|
Shahid MS, French AP, Valstar MF, Yakubov GE. Research in methodologies for modelling the oral cavity. Biomed Phys Eng Express 2024; 10:032001. [PMID: 38350128 DOI: 10.1088/2057-1976/ad28cc] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2023] [Accepted: 02/13/2024] [Indexed: 02/15/2024]
Abstract
The paper aims to explore the current state of understanding surrounding in silico oral modelling. This involves exploring methodologies, technologies and approaches pertaining to the modelling of the whole oral cavity; both internally and externally visible structures that may be relevant or appropriate to oral actions. Such a model could be referred to as a 'complete model' which includes consideration of a full set of facial features (i.e. not only mouth) as well as synergistic stimuli such as audio and facial thermal data. 3D modelling technologies capable of accurately and efficiently capturing a complete representation of the mouth for an individual have broad applications in the study of oral actions, due to their cost-effectiveness and time efficiency. This review delves into the field of clinical phonetics to classify oral actions pertaining to both speech and non-speech movements, identifying how the various vocal organs play a role in the articulatory and masticatory process. Vitaly, it provides a summation of 12 articulatory recording methods, forming a tool to be used by researchers in identifying which method of recording is appropriate for their work. After addressing the cost and resource-intensive limitations of existing methods, a new system of modelling is proposed that leverages external to internal correlation modelling techniques to create a more efficient models of the oral cavity. The vision is that the outcomes will be applicable to a broad spectrum of oral functions related to physiology, health and wellbeing, including speech, oral processing of foods as well as dental health. The applications may span from speech correction, designing foods for the aging population, whilst in the dental field we would be able to gain information about patient's oral actions that would become part of creating a personalised dental treatment plan.
Collapse
Affiliation(s)
| | - Andrew P French
- School of Computer Science, University of Nottingham, NG8 1BB, United Kingdom
- School of Biosciences, University of Nottingham, LE12 5RD, United Kingdom
| | - Michel F Valstar
- School of Computer Science, University of Nottingham, NG8 1BB, United Kingdom
| | - Gleb E Yakubov
- School of Biosciences, University of Nottingham, LE12 5RD, United Kingdom
| |
Collapse
|
30
|
Luo H, Yin W, Wang J, Zhang G, Liang W, Luo J, Yan C. Drug-drug interactions prediction based on deep learning and knowledge graph: A review. iScience 2024; 27:109148. [PMID: 38405609 PMCID: PMC10884936 DOI: 10.1016/j.isci.2024.109148] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/27/2024] Open
Abstract
Drug-drug interactions (DDIs) can produce unpredictable pharmacological effects and lead to adverse events that have the potential to cause irreversible damage to the organism. Traditional methods to detect DDIs through biological or pharmacological analysis are time-consuming and expensive, therefore, there is an urgent need to develop computational methods to effectively predict drug-drug interactions. Currently, deep learning and knowledge graph techniques which can effectively extract features of entities have been widely utilized to develop DDI prediction methods. In this research, we aim to systematically review DDI prediction researches applying deep learning and graph knowledge. The available biomedical data and public databases related to drugs are firstly summarized in this review. Then, we discuss the existing drug-drug interactions prediction methods which have utilized deep learning and knowledge graph techniques and group them into three main classes: deep learning-based methods, knowledge graph-based methods, and methods that combine deep learning with knowledge graph. We comprehensively analyze the commonly used drug related data and various DDI prediction methods, and compare these prediction methods on benchmark datasets. Finally, we briefly discuss the challenges related to drug-drug interactions prediction, including asymmetric DDIs prediction and high-order DDI prediction.
Collapse
Affiliation(s)
- Huimin Luo
- School of Computer and Information Engineering, Henan University, Kaifeng, China
- Henan Key Laboratory of Big Data Analysis and Processing, Henan University, Kaifeng, China
| | - Weijie Yin
- School of Computer and Information Engineering, Henan University, Kaifeng, China
| | - Jianlin Wang
- School of Computer and Information Engineering, Henan University, Kaifeng, China
- Academy for Advanced Interdisciplinary Studies, Zhengzhou, China
| | - Ge Zhang
- School of Computer and Information Engineering, Henan University, Kaifeng, China
- Henan Key Laboratory of Big Data Analysis and Processing, Henan University, Kaifeng, China
| | - Wenjuan Liang
- School of Computer and Information Engineering, Henan University, Kaifeng, China
| | - Junwei Luo
- College of Computer Science and Technology, Henan Polytechnic University, Jiaozuo, China
| | - Chaokun Yan
- School of Computer and Information Engineering, Henan University, Kaifeng, China
- Academy for Advanced Interdisciplinary Studies, Zhengzhou, China
| |
Collapse
|
31
|
Wang S, Han J, Huang J, Islam K, Shi Y, Zhou Y, Kim D, Zhou J, Lian Z, Liu Y, Huang J. Deep learning-based predictive classification of functional subpopulations of hematopoietic stem cells and multipotent progenitors. Stem Cell Res Ther 2024; 15:74. [PMID: 38475857 PMCID: PMC10935795 DOI: 10.1186/s13287-024-03682-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2023] [Accepted: 02/23/2024] [Indexed: 03/14/2024] Open
Abstract
BACKGROUND Hematopoietic stem cells (HSCs) and multipotent progenitors (MPPs) play a pivotal role in maintaining lifelong hematopoiesis. The distinction between stem cells and other progenitors, as well as the assessment of their functions, has long been a central focus in stem cell research. In recent years, deep learning has emerged as a powerful tool for cell image analysis and classification/prediction. METHODS In this study, we explored the feasibility of employing deep learning techniques to differentiate murine HSCs and MPPs based solely on their morphology, as observed through light microscopy (DIC) images. RESULTS After rigorous training and validation using extensive image datasets, we successfully developed a three-class classifier, referred to as the LSM model, capable of reliably distinguishing long-term HSCs, short-term HSCs, and MPPs. The LSM model extracts intrinsic morphological features unique to different cell types, irrespective of the methods used for cell identification and isolation, such as surface markers or intracellular GFP markers. Furthermore, employing the same deep learning framework, we created a two-class classifier that effectively discriminates between aged HSCs and young HSCs. This discovery is particularly significant as both cell types share identical surface markers yet serve distinct functions. This classifier holds the potential to offer a novel, rapid, and efficient means of assessing the functional states of HSCs, thus obviating the need for time-consuming transplantation experiments. CONCLUSION Our study represents the pioneering use of deep learning to differentiate HSCs and MPPs under steady-state conditions. This novel and robust deep learning-based platform will provide a basis for the future development of a new generation stem cell identification and separation system. It may also provide new insight into the molecular mechanisms underlying stem cell self-renewal.
Collapse
Affiliation(s)
- Shen Wang
- Department of Mechanical Engineering and Mechanics, Lehigh University, Bethlehem, PA, USA
| | - Jianzhong Han
- Coriell Institute for Medical Research, Camden, NJ, USA
| | - Jingru Huang
- Shanghai Key Laboratory of Medical Epigenetics, Laboratory of Cancer Epigenetics, Institutes of Biomedical Sciences, Medical College of Fudan University, Chinese Academy of Medical Sciences, Shanghai, People's Republic of China
| | - Khayrul Islam
- Department of Mechanical Engineering and Mechanics, Lehigh University, Bethlehem, PA, USA
| | - Yuheng Shi
- Shanghai Key Laboratory of Medical Epigenetics, Laboratory of Cancer Epigenetics, Institutes of Biomedical Sciences, Medical College of Fudan University, Chinese Academy of Medical Sciences, Shanghai, People's Republic of China
| | - Yuyuan Zhou
- Department of Bioengineering, Lehigh University, Bethlehem, PA, USA
| | - Dongwook Kim
- Coriell Institute for Medical Research, Camden, NJ, USA
| | - Jane Zhou
- Health and Human Biology, Brown University, Providence, RI, USA
| | - Zhaorui Lian
- Coriell Institute for Medical Research, Camden, NJ, USA
| | - Yaling Liu
- Department of Mechanical Engineering and Mechanics, Lehigh University, Bethlehem, PA, USA.
- Department of Bioengineering, Lehigh University, Bethlehem, PA, USA.
| | - Jian Huang
- Coriell Institute for Medical Research, Camden, NJ, USA.
- Cooper Medical School of Rowan University, Camden, NJ, USA.
- Center for Metabolic Disease Research, Temple University Lewis Katz School of Medicine, Philadelphia, PA, USA.
| |
Collapse
|
32
|
Luo X, Li P, Chen H, Zhou K, Piao S, Yang L, Hu B, Geng D. Automatic segmentation of hepatocellular carcinoma on dynamic contrast-enhanced MRI based on deep learning. Phys Med Biol 2024; 69:065008. [PMID: 38330492 DOI: 10.1088/1361-6560/ad2790] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2023] [Accepted: 02/08/2024] [Indexed: 02/10/2024]
Abstract
Objective. Precise hepatocellular carcinoma (HCC) detection is crucial for clinical management. While studies focus on computed tomography-based automatic algorithms, there is a rareness of research on automatic detection based on dynamic contrast enhanced (DCE) magnetic resonance imaging. This study is to develop an automatic detection and segmentation deep learning model for HCC using DCE.Approach: DCE images acquired from 2016 to 2021 were retrospectively collected. Then, 382 patients (301 male; 81 female) with 466 lesions pathologically confirmed were included and divided into an 80% training-validation set and a 20% independent test set. For external validation, 51 patients (42 male; 9 female) in another hospital from 2018 to 2021 were included. The U-net architecture was modified to accommodate multi-phasic DCE input. The model was trained with the training-validation set using five-fold cross-validation, and furtherly evaluated with the independent test set using comprehensive metrics for segmentation and detection performance. The proposed automatic segmentation model consisted of five main steps: phase registration, automatic liver region extraction using a pre-trained model, automatic HCC lesion segmentation using the multi-phasic deep learning model, ensemble of five-fold predictions, and post-processing using connected component analysis to enhance the performance to refine predictions and eliminate false positives.Main results. The proposed model achieved a mean dice similarity coefficient (DSC) of 0.81 ± 0.11, a sensitivity of 94.41 ± 15.50%, a precision of 94.19 ± 17.32%, and 0.14 ± 0.48 false positive lesions per patient in the independent test set. The model detected 88% (80/91) HCC lesions in the condition of DSC > 0.5, and the DSC per tumor was 0.80 ± 0.13. In the external set, the model detected 92% (58/62) lesions with 0.12 ± 0.33 false positives per patient, and the DSC per tumor was 0.75 ± 0.10.Significance.This study developed an automatic detection and segmentation deep learning model for HCC using DCE, which yielded promising post-processed results in accurately identifying and delineating HCC lesions.
Collapse
Affiliation(s)
- Xiao Luo
- Academy for Engineering and Technology, Fudan University, Shanghai, People's Republic of China
| | - Peiwen Li
- Department of Radiology, Huashan Hospital, Fudan University, Shanghai, People's Republic of China
| | - Hongyi Chen
- Academy for Engineering and Technology, Fudan University, Shanghai, People's Republic of China
| | - Kun Zhou
- Academy for Engineering and Technology, Fudan University, Shanghai, People's Republic of China
| | - Sirong Piao
- Department of Radiology, Huashan Hospital, Fudan University, Shanghai, People's Republic of China
- Department of Radiology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, People's Republic China
| | - Liqin Yang
- Department of Radiology, Huashan Hospital, Fudan University, Shanghai, People's Republic of China
- Shanghai Engineering Research Center of Intelligent Imaging for Critical Brain Diseases, Shanghai, People's Republic China
- Institute of Functional and Molecular Medical Imaging, Fudan University, Shanghai, People's Republic of China
| | - Bin Hu
- Department of Radiology, Huashan Hospital, Fudan University, Shanghai, People's Republic of China
| | - Daoying Geng
- Academy for Engineering and Technology, Fudan University, Shanghai, People's Republic of China
- Department of Radiology, Huashan Hospital, Fudan University, Shanghai, People's Republic of China
- Shanghai Engineering Research Center of Intelligent Imaging for Critical Brain Diseases, Shanghai, People's Republic China
- Institute of Functional and Molecular Medical Imaging, Fudan University, Shanghai, People's Republic of China
| |
Collapse
|
33
|
Wang P, Leong QY, Lau NY, Ng WY, Kwek SP, Tan L, Song SW, You K, Chong LM, Zhuang I, Ong YH, Foo N, Tadeo X, Kumar KS, Vijayakumar S, Sapanel Y, Raczkowska MN, Remus A, Blasiak A, Ho D. N-of-1 medicine. Singapore Med J 2024; 65:167-175. [PMID: 38527301 PMCID: PMC11060644 DOI: 10.4103/singaporemedj.smj-2023-243] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2023] [Accepted: 01/19/2024] [Indexed: 03/27/2024]
Abstract
ABSTRACT The fields of precision and personalised medicine have led to promising advances in tailoring treatment to individual patients. Examples include genome/molecular alteration-guided drug selection, single-patient gene therapy design and synergy-based drug combination development, and these approaches can yield substantially diverse recommendations. Therefore, it is important to define each domain and delineate their commonalities and differences in an effort to develop novel clinical trial designs, streamline workflow development, rethink regulatory considerations, create value in healthcare and economics assessments, and other factors. These and other segments are essential to recognise the diversity within these domains to accelerate their respective workflows towards practice-changing healthcare. To emphasise these points, this article elaborates on the concept of digital health and digital medicine-enabled N-of-1 medicine, which individualises combination regimen and dosing using a patient's own data. We will conclude with recommendations for consideration when developing novel workflows based on emerging digital-based platforms.
Collapse
Affiliation(s)
- Peter Wang
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore
| | - Qiao Ying Leong
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
| | - Ni Yin Lau
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
| | - Wei Ying Ng
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
| | - Siong Peng Kwek
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
| | - Lester Tan
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore
| | - Shang-Wei Song
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore
| | - Kui You
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore
| | - Li Ming Chong
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore
| | - Isaiah Zhuang
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
| | - Yoong Hun Ong
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
| | - Nigel Foo
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore
| | - Xavier Tadeo
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
| | - Kirthika Senthil Kumar
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
| | - Smrithi Vijayakumar
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
| | - Yoann Sapanel
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
- Singapore’s Health District @ Queenstown, Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| | - Marlena Natalia Raczkowska
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
| | - Alexandria Remus
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore
- Heat Resilience Performance Centre (HRPC), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| | - Agata Blasiak
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore
- Department of Pharmacology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| | - Dean Ho
- Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore
- Singapore’s Health District @ Queenstown, Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- Department of Pharmacology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The Bia-Echo Asia Centre for Reproductive Longevity and Equality, Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| |
Collapse
|
34
|
Wang ZC, Fan ZZ, Liu XY, Zhu MJ, Jiang SS, Tian S, Chen BH, Wu LM. Deep Learning for Discrimination of Hypertrophic Cardiomyopathy and Hypertensive Heart Disease on MRI Native T1 Maps. J Magn Reson Imaging 2024; 59:837-848. [PMID: 37431848 DOI: 10.1002/jmri.28904] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2023] [Revised: 06/05/2023] [Accepted: 06/06/2023] [Indexed: 07/12/2023] Open
Abstract
BACKGROUND Native T1 and radiomics were used for hypertrophic cardiomyopathy (HCM) and hypertensive heart disease (HHD) differentiation previously. The current problem is that global native T1 remains modest discrimination performance and radiomics requires feature extraction beforehand. Deep learning (DL) is a promising technique in differential diagnosis. However, its feasibility for discriminating HCM and HHD has not been investigated. PURPOSE To examine the feasibility of DL in differentiating HCM and HHD based on T1 images and compare its diagnostic performance with other methods. STUDY TYPE Retrospective. POPULATION 128 HCM patients (men, 75; age, 50 years ± 16) and 59 HHD patients (men, 40; age, 45 years ± 17). FIELD STRENGTH/SEQUENCE 3.0T; Balanced steady-state free precession, phase-sensitive inversion recovery (PSIR) and multislice native T1 mapping. ASSESSMENT Compare HCM and HHD patients baseline data. Myocardial T1 values were extracted from native T1 images. Radiomics was implemented through feature extraction and Extra Trees Classifier. The DL network is ResNet32. Different input including myocardial ring (DL-myo), myocardial ring bounding box (DL-box) and the surrounding tissue without myocardial ring (DL-nomyo) were tested. We evaluate diagnostic performance through AUC of ROC curve. STATISTICAL TESTS Accuracy, sensitivity, specificity, ROC, and AUC were calculated. Independent t test, Mann-Whitney U-test and Chi-square test were adopted for HCM and HHD comparison. P < 0.05 was considered statistically significant. RESULTS DL-myo, DL-box, and DL-nomyo models showed an AUC (95% confidential interval) of 0.830 (0.702-0.959), 0.766 (0.617-0.915), 0.795 (0.654-0.936) in the testing set. AUC of native T1 and radiomics were 0.545 (0.352-0.738) and 0.800 (0.655-0.944) in the testing set. DATA CONCLUSION The DL method based on T1 mapping seems capable of discriminating HCM and HHD. Considering diagnostic performance, the DL network outperformed the native T1 method. Compared with radiomics, DL won an advantage for its high specificity and automated working mode. LEVEL OF EVIDENCE 4 TECHNICAL EFFICACY STAGE: 2.
Collapse
Affiliation(s)
- Zi-Chen Wang
- Ottawa-Shanghai Joint School of Medicine, School of Medicine, Shanghai Jiao Tong University, Shanghai, China
| | - Zhang-Zhengyi Fan
- Ottawa-Shanghai Joint School of Medicine, School of Medicine, Shanghai Jiao Tong University, Shanghai, China
| | - Xi-Yuan Liu
- Ottawa-Shanghai Joint School of Medicine, School of Medicine, Shanghai Jiao Tong University, Shanghai, China
| | - Ming-Jie Zhu
- Ottawa-Shanghai Joint School of Medicine, School of Medicine, Shanghai Jiao Tong University, Shanghai, China
| | | | | | - Bing-Hua Chen
- Department of Radiology, Renji Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China
| | - Lian-Ming Wu
- Department of Radiology, Renji Hospital, School of Medicine, Shanghai Jiao Tong University, Shanghai, China
| |
Collapse
|
35
|
Benboujja F, Hartnick E, Zablah E, Hersh C, Callans K, Villamor P, Yager PH, Hartnick C. Overcoming language barriers in pediatric care: a multilingual, AI-driven curriculum for global healthcare education. Front Public Health 2024; 12:1337395. [PMID: 38454985 PMCID: PMC10917955 DOI: 10.3389/fpubh.2024.1337395] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2023] [Accepted: 01/22/2024] [Indexed: 03/09/2024] Open
Abstract
Background Online medical education often faces challenges related to communication and comprehension barriers, particularly when the instructional language differs from the healthcare providers' and caregivers' native languages. Our study addresses these challenges within pediatric healthcare by employing generative language models to produce a linguistically tailored, multilingual curriculum that covers the topics of team training, surgical procedures, perioperative care, patient journeys, and educational resources for healthcare providers and caregivers. Methods An interdisciplinary group formulated a video curriculum in English, addressing the nuanced challenges of pediatric healthcare. Subsequently, it was translated into Spanish, primarily emphasizing Latin American demographics, utilizing OpenAI's GPT-4. Videos were enriched with synthetic voice profiles of native speakers to uphold the consistency of the narrative. Results We created a collection of 45 multilingual video modules, each ranging from 3 to 8 min in length and covering essential topics such as teamwork, how to improve interpersonal communication, "How I Do It" surgical procedures, as well as focused topics in anesthesia, intensive care unit care, ward nursing, and transitions from hospital to home. Through AI-driven translation, this comprehensive collection ensures global accessibility and offers healthcare professionals and caregivers a linguistically inclusive resource for elevating standards of pediatric care worldwide. Conclusion This development of multilingual educational content marks a progressive step toward global standardization of pediatric care. By utilizing advanced language models for translation, we ensure that the curriculum is inclusive and accessible. This initiative aligns well with the World Health Organization's Digital Health Guidelines, advocating for digitally enabled healthcare education.
Collapse
Affiliation(s)
- Fouzi Benboujja
- Department of Otolaryngology, Mass Eye and Ear, Harvard Medical School, Boston, MA, United States
| | | | - Evelyn Zablah
- Department of Otolaryngology, Mass Eye and Ear, Harvard Medical School, Boston, MA, United States
| | - Cheryl Hersh
- Pediatric Airway, Voice and Swallowing Center, Massachusetts General Hospital for Children, Boston, MA, United States
| | - Kevin Callans
- Department of Otolaryngology, Mass Eye and Ear, Harvard Medical School, Boston, MA, United States
- Massachusetts General Hospital for Children, Boston, MA, United States
| | - Perla Villamor
- Hospital Serena del Mar, Cartagena, Colombia
- Hospital Infantil Napoleón Franco Pareja, Cartagena, Colombia
| | - Phoebe H. Yager
- Pediatric Intensive Care Unit, Massachusetts General Hospital, Harvard Medical School, Boston, MA, United States
| | - Christopher Hartnick
- Department of Otolaryngology, Mass Eye and Ear, Harvard Medical School, Boston, MA, United States
| |
Collapse
|
36
|
Wu X, Zhang D, Li G, Gao X, Metcalfe B, Chen L. Data augmentation for invasive brain-computer interfaces based on stereo-electroencephalography (SEEG). J Neural Eng 2024; 21:016026. [PMID: 38237174 DOI: 10.1088/1741-2552/ad200e] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2023] [Accepted: 01/18/2024] [Indexed: 02/23/2024]
Abstract
Objective.Deep learning is increasingly used for brain-computer interfaces (BCIs). However, the quantity of available data is sparse, especially for invasive BCIs. Data augmentation (DA) methods, such as generative models, can help to address this sparseness. However, all the existing studies on brain signals were based on convolutional neural networks and ignored the temporal dependence. This paper attempted to enhance generative models by capturing the temporal relationship from a time-series perspective.Approach. A conditional generative network (conditional transformer-based generative adversarial network (cTGAN)) based on the transformer model was proposed. The proposed method was tested using a stereo-electroencephalography (SEEG) dataset which was recorded from eight epileptic patients performing five different movements. Three other commonly used DA methods were also implemented: noise injection (NI), variational autoencoder (VAE), and conditional Wasserstein generative adversarial network with gradient penalty (cWGANGP). Using the proposed method, the artificial SEEG data was generated, and several metrics were used to compare the data quality, including visual inspection, cosine similarity (CS), Jensen-Shannon distance (JSD), and the effect on the performance of a deep learning-based classifier.Main results. Both the proposed cTGAN and the cWGANGP methods were able to generate realistic data, while NI and VAE outputted inferior samples when visualized as raw sequences and in a lower dimensional space. The cTGAN generated the best samples in terms of CS and JSD and outperformed cWGANGP significantly in enhancing the performance of a deep learning-based classifier (each of them yielding a significant improvement of 6% and 3.4%, respectively).Significance. This is the first time that DA methods have been applied to invasive BCIs based on SEEG. In addition, this study demonstrated the advantages of the model that preserves the temporal dependence from a time-series perspective.
Collapse
Affiliation(s)
- Xiaolong Wu
- The Centre for Autonomous Robotics (CENTAUR), Department of Electronic & Electrical Engineering, University of Bath, Bath, United Kingdom
| | - Dingguo Zhang
- The Centre for Autonomous Robotics (CENTAUR), Department of Electronic & Electrical Engineering, University of Bath, Bath, United Kingdom
| | - Guangye Li
- School of Mechanical Engineering, Shanghai Jiao Tong University, People's Republic of China
| | - Xin Gao
- The Centre for Autonomous Robotics (CENTAUR), Department of Electronic & Electrical Engineering, University of Bath, Bath, United Kingdom
| | - Benjamin Metcalfe
- The Centre for Autonomous Robotics (CENTAUR), Department of Electronic & Electrical Engineering, University of Bath, Bath, United Kingdom
| | - Liang Chen
- Liang Chen is with Huashan Hospital, Fudan University, People's Republic of China
| |
Collapse
|
37
|
Pareek A, Ro DH, Karlsson J, Martin RK. Machine learning/artificial intelligence in sports medicine: state of the art and future directions. J ISAKOS 2024:S2059-7754(24)00013-0. [PMID: 38336099 DOI: 10.1016/j.jisako.2024.01.013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2022] [Revised: 12/30/2023] [Accepted: 01/25/2024] [Indexed: 02/12/2024]
Abstract
Machine learning (ML) is changing the way health care is practiced and recent applications of these novel statistical techniques have started to impact orthopaedic sports medicine. Machine learning enables the analysis of large volumes of data to establish complex relationships between "input" and "output" variables. These relationships may be more complex than could be established through traditional statistical analysis and can lead to the ability to predict the "output" with high levels of accuracy. Supervised learning is the most common ML approach for healthcare data and recent studies have developed algorithms to predict patient-specific outcome after surgical procedures such as hip arthroscopy and anterior cruciate ligament reconstruction. Deep learning is a higher-level ML approach that facilitates the processing and interpretation of complex datasets through artificial neural networks that are inspired by the way the human brain processes information. In orthopaedic sports medicine, deep learning has primarily been used for automatic image (computer vision) and text (natural language processing) interpretation. While applications in orthopaedic sports medicine have been increasing exponentially, one significant barrier to widespread adoption of ML remains clinician unfamiliarity with the associated methods and concepts. The goal of this review is to introduce these concepts, review current machine learning models in orthopaedic sport medicine, and discuss future opportunities for innovation within the specialty.
Collapse
Affiliation(s)
- Ayoosh Pareek
- Sports Medicine and Shoulder Service, Hospital for Special Surgery, New York, 10021, USA; Department of Orthopaedics, Institute of Clinical Sciences, Sahlgrenska Academy, Gothenburg University, Gothenburg, 43180, Sweden.
| | - Du Hyun Ro
- Department of Orthopedic Surgery, Seoul National University Hospital, Seoul, 03080, South Korea; CONNECTEVE Co., Ltd, Seoul, 03080, South Korea
| | - Jón Karlsson
- Department of Orthopaedics, Institute of Clinical Sciences, Sahlgrenska Academy, Gothenburg University, Gothenburg, 43180, Sweden
| | - R Kyle Martin
- Department of Orthopedic Surgery, University of Minnesota, Minneapolis, MN, 55454, USA; Department of Orthopedic Surgery, CentraCare, Saint Cloud, MN, 56303, USA; Oslo Sports Trauma Research Center, Norwegian School of Sport Sciences, Oslo, 0806, Norway
| |
Collapse
|
38
|
Burguete-Lopez A, Makarenko M, Bonifazi M, Menezes de Oliveira BN, Getman F, Tian Y, Mazzone V, Li N, Giammona A, Liberale C, Fratalocchi A. Real-time simultaneous refractive index and thickness mapping of sub-cellular biology at the diffraction limit. Commun Biol 2024; 7:154. [PMID: 38321111 PMCID: PMC10847501 DOI: 10.1038/s42003-024-05839-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2023] [Accepted: 01/20/2024] [Indexed: 02/08/2024] Open
Abstract
Mapping the cellular refractive index (RI) is a central task for research involving the composition of microorganisms and the development of models providing automated medical screenings with accuracy beyond 95%. These models require significantly enhancing the state-of-the-art RI mapping capabilities to provide large amounts of accurate RI data at high throughput. Here, we present a machine-learning-based technique that obtains a biological specimen's real-time RI and thickness maps from a single image acquired with a conventional color camera. This technology leverages a suitably engineered nanostructured membrane that stretches a biological analyte over its surface and absorbs transmitted light, generating complex reflection spectra from each sample point. The technique does not need pre-existing sample knowledge. It achieves 10-4 RI sensitivity and sub-nanometer thickness resolution on diffraction-limited spatial areas. We illustrate practical application by performing sub-cellular segmentation of HCT-116 colorectal cancer cells, obtaining complete three-dimensional reconstruction of the cellular regions with a characteristic length of 30 μm. These results can facilitate the development of real-time label-free technologies for biomedical studies on microscopic multicellular dynamics.
Collapse
Affiliation(s)
- Arturo Burguete-Lopez
- PRIMALIGHT, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE), King Abdullah University of Science and Technology (KAUST), Thuwal, 23955-6900, Saudi Arabia
| | - Maksim Makarenko
- PRIMALIGHT, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE), King Abdullah University of Science and Technology (KAUST), Thuwal, 23955-6900, Saudi Arabia
| | - Marcella Bonifazi
- PRIMALIGHT, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE), King Abdullah University of Science and Technology (KAUST), Thuwal, 23955-6900, Saudi Arabia
- Physik-Institut, University of Zurich, Winterthurerstrasse 190, Zurich, 8057, Switzerland
| | - Barbara Nicoly Menezes de Oliveira
- PRIMALIGHT, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE), King Abdullah University of Science and Technology (KAUST), Thuwal, 23955-6900, Saudi Arabia
| | - Fedor Getman
- PRIMALIGHT, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE), King Abdullah University of Science and Technology (KAUST), Thuwal, 23955-6900, Saudi Arabia
| | - Yi Tian
- PRIMALIGHT, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE), King Abdullah University of Science and Technology (KAUST), Thuwal, 23955-6900, Saudi Arabia
| | - Valerio Mazzone
- PRIMALIGHT, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE), King Abdullah University of Science and Technology (KAUST), Thuwal, 23955-6900, Saudi Arabia
- Physik-Institut, University of Zurich, Winterthurerstrasse 190, Zurich, 8057, Switzerland
| | - Ning Li
- PRIMALIGHT, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE), King Abdullah University of Science and Technology (KAUST), Thuwal, 23955-6900, Saudi Arabia
| | - Alessandro Giammona
- Biological and Environmental Science and Engineering Division (BESE), King Abdullah University of Science and Technology (KAUST), Thuwal, 23955-6900, Saudi Arabia
- Institute of Molecular Bioimaging and Physiology (IBFM), National Research Council (CNR), Segrate, Italy
| | - Carlo Liberale
- Biological and Environmental Science and Engineering Division (BESE), King Abdullah University of Science and Technology (KAUST), Thuwal, 23955-6900, Saudi Arabia
| | - Andrea Fratalocchi
- PRIMALIGHT, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE), King Abdullah University of Science and Technology (KAUST), Thuwal, 23955-6900, Saudi Arabia.
| |
Collapse
|
39
|
Zheng F, Li M, Wang Y, Yu W, Wang R, Chen Z, Xiao N, Lu Y. Intensive vision-guided network for radiology report generation. Phys Med Biol 2024; 69:045008. [PMID: 38157546 DOI: 10.1088/1361-6560/ad1995] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2023] [Accepted: 12/29/2023] [Indexed: 01/03/2024]
Abstract
Objective.Automatic radiology report generation is booming due to its huge application potential for the healthcare industry. However, existing computer vision and natural language processing approaches to tackle this problem are limited in two aspects. First, when extracting image features, most of them neglect multi-view reasoning in vision and model single-view structure of medical images, such as space-view or channel-view. However, clinicians rely on multi-view imaging information for comprehensive judgment in daily clinical diagnosis. Second, when generating reports, they overlook context reasoning with multi-modal information and focus on pure textual optimization utilizing retrieval-based methods. We aim to address these two issues by proposing a model that better simulates clinicians perspectives and generates more accurate reports.Approach.Given the above limitation in feature extraction, we propose a globally-intensive attention (GIA) module in the medical image encoder to simulate and integrate multi-view vision perception. GIA aims to learn three types of vision perception: depth view, space view, and pixel view. On the other hand, to address the above problem in report generation, we explore how to involve multi-modal signals to generate precisely matched reports, i.e. how to integrate previously predicted words with region-aware visual content in next word prediction. Specifically, we design a visual knowledge-guided decoder (VKGD), which can adaptively consider how much the model needs to rely on visual information and previously predicted text to assist next word prediction. Hence, our final intensive vision-guided network framework includes a GIA-guided visual encoder and the VKGD.Main results.Experiments on two commonly-used datasets IU X-RAY and MIMIC-CXR demonstrate the superior ability of our method compared with other state-of-the-art approaches.Significance.Our model explores the potential of simulating clinicians perspectives and automatically generates more accurate reports, which promotes the exploration of medical automation and intelligence.
Collapse
Affiliation(s)
- Fudan Zheng
- Sun Yat-Sen University, No. 132 Waihuandong Road, Guangzhou Higher Education Mega Center, Guangzhou, 510006, People's Republic of China
| | - Mengfei Li
- Sun Yat-Sen University, No. 132 Waihuandong Road, Guangzhou Higher Education Mega Center, Guangzhou, 510006, People's Republic of China
| | - Ying Wang
- National SuperComputer Center in Guangzhou, No. 132 Waihuandong Road, Guangzhou Higher Education Mega Center, Guangzhou, 510006, People's Republic of China
| | - Weijiang Yu
- Huawei Technologies Co., Ltd, Huawei Industrial Park, Bantian, Longgang District, Shenzhen, 518129, People's Republic of China
| | - Ruixuan Wang
- Sun Yat-Sen University, No. 132 Waihuandong Road, Guangzhou Higher Education Mega Center, Guangzhou, 510006, People's Republic of China
| | - Zhiguang Chen
- Sun Yat-Sen University, No. 132 Waihuandong Road, Guangzhou Higher Education Mega Center, Guangzhou, 510006, People's Republic of China
- National SuperComputer Center in Guangzhou, No. 132 Waihuandong Road, Guangzhou Higher Education Mega Center, Guangzhou, 510006, People's Republic of China
| | - Nong Xiao
- Sun Yat-Sen University, No. 132 Waihuandong Road, Guangzhou Higher Education Mega Center, Guangzhou, 510006, People's Republic of China
- National SuperComputer Center in Guangzhou, No. 132 Waihuandong Road, Guangzhou Higher Education Mega Center, Guangzhou, 510006, People's Republic of China
| | - Yutong Lu
- Sun Yat-Sen University, No. 132 Waihuandong Road, Guangzhou Higher Education Mega Center, Guangzhou, 510006, People's Republic of China
- National SuperComputer Center in Guangzhou, No. 132 Waihuandong Road, Guangzhou Higher Education Mega Center, Guangzhou, 510006, People's Republic of China
| |
Collapse
|
40
|
Courtman M, Kim D, Wit H, Wang H, Sun L, Ifeachor E, Mullin S, Thurston M. Deep Learning Detection of Aneurysm Clips for Magnetic Resonance Imaging Safety. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024; 37:72-80. [PMID: 38343241 DOI: 10.1007/s10278-023-00932-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/05/2023] [Revised: 10/03/2023] [Accepted: 10/19/2023] [Indexed: 03/02/2024]
Abstract
Flagging the presence of metal devices before a head MRI scan is essential to allow appropriate safety checks. There is an unmet need for an automated system which can flag aneurysm clips prior to MRI appointments. We assess the accuracy with which a machine learning model can classify the presence or absence of an aneurysm clip on CT images. A total of 280 CT head scans were collected, 140 with aneurysm clips visible and 140 without. The data were used to retrain a pre-trained image classification neural network to classify CT localizer images. Models were developed using fivefold cross-validation and then tested on a holdout test set. A mean sensitivity of 100% and a mean accuracy of 82% were achieved. Predictions were explained using SHapley Additive exPlanations (SHAP), which highlighted that appropriate regions of interest were informing the models. Models were also trained from scratch to classify three-dimensional CT head scans. These did not exceed the sensitivity of the localizer models. This work illustrates an application of computer vision image classification to enhance current processes and improve patient safety.
Collapse
Affiliation(s)
- Megan Courtman
- Faculty of Science and Engineering, School of Engineering, Computing and Mathematics, University of Plymouth, Plymouth, PL4 8AA, UK.
| | - Daniel Kim
- Department of Radiology, Royal Cornwall Hospitals NHS Trust, Truro, TR1 3LJ, UK
| | - Huub Wit
- Department of Radiology, Torbay and South Devon NHS Trust, Torquay, TQ2 7AA, UK
| | - Hongrui Wang
- Department of Radiology, University Hospitals Plymouth NHS Trust, Plymouth, PL6 8DH, UK
| | - Lingfen Sun
- Faculty of Science and Engineering, School of Engineering, Computing and Mathematics, University of Plymouth, Plymouth, PL4 8AA, UK
| | - Emmanuel Ifeachor
- Faculty of Science and Engineering, School of Engineering, Computing and Mathematics, University of Plymouth, Plymouth, PL4 8AA, UK
| | - Stephen Mullin
- Plymouth Institute of Health and Care Research, University of Plymouth, Plymouth, PL4 8AA, UK
| | - Mark Thurston
- Department of Radiology, University Hospitals Plymouth NHS Trust, Plymouth, PL6 8DH, UK
| |
Collapse
|
41
|
Gerbasi A, Dagliati A, Albi G, Chiesa M, Andreini D, Baggiano A, Mushtaq S, Pontone G, Bellazzi R, Colombo G. CAD-RADS scoring of coronary CT angiography with Multi-Axis Vision Transformer: A clinically-inspired deep learning pipeline. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 244:107989. [PMID: 38141455 DOI: 10.1016/j.cmpb.2023.107989] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/26/2023] [Revised: 11/10/2023] [Accepted: 12/17/2023] [Indexed: 12/25/2023]
Abstract
BACKGROUND AND OBJECTIVE The standard non-invasive imaging technique used to assess the severity and extent of Coronary Artery Disease (CAD) is Coronary Computed Tomography Angiography (CCTA). However, manual grading of each patient's CCTA according to the CAD-Reporting and Data System (CAD-RADS) scoring is time-consuming and operator-dependent, especially in borderline cases. This work proposes a fully automated, and visually explainable, deep learning pipeline to be used as a decision support system for the CAD screening procedure. The pipeline performs two classification tasks: firstly, identifying patients who require further clinical investigations and secondly, classifying patients into subgroups based on the degree of stenosis, according to commonly used CAD-RADS thresholds. METHODS The pipeline pre-processes multiplanar projections of the coronary arteries, extracted from the original CCTAs, and classifies them using a fine-tuned Multi-Axis Vision Transformer architecture. With the aim of emulating the current clinical practice, the model is trained to assign a per-patient score by stacking the bi-dimensional longitudinal cross-sections of the three main coronary arteries along channel dimension. Furthermore, it generates visually interpretable maps to assess the reliability of the predictions. RESULTS When run on a database of 1873 three-channel images of 253 patients collected at the Monzino Cardiology Center in Milan, the pipeline obtained an AUC of 0.87 and 0.93 for the two classification tasks, respectively. CONCLUSION According to our knowledge, this is the first model trained to assign CAD-RADS scores learning solely from patient scores and not requiring finer imaging annotation steps that are not part of the clinical routine.
Collapse
Affiliation(s)
- Alessia Gerbasi
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Via Ferrata 5, Pavia, Italy.
| | - Arianna Dagliati
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Via Ferrata 5, Pavia, Italy
| | - Giuseppe Albi
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Via Ferrata 5, Pavia, Italy
| | | | - Daniele Andreini
- Centro Cardiologico Monzino IRCCS, Milan, Italy; Department of Biomedical and Clinical Sciences, University of Milan, Milan, Italy
| | - Andrea Baggiano
- Centro Cardiologico Monzino IRCCS, Milan, Italy; Department of Clinical Sciences and Community Health, University of Milan, Milan, Italy
| | | | - Gianluca Pontone
- Centro Cardiologico Monzino IRCCS, Milan, Italy; Department of Biomedical, Surgical and Dental Sciences, University of Milan, Milan, Italy
| | - Riccardo Bellazzi
- Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Via Ferrata 5, Pavia, Italy; IRCCS Istituti Clinici Scientifici Maugeri, Pavia, Pavia, Italy
| | | |
Collapse
|
42
|
Goodman ED, Patel KK, Zhang Y, Locke W, Kennedy CJ, Mehrotra R, Ren S, Guan M, Zohar O, Downing M, Chen HW, Clark JZ, Berrigan MT, Brat GA, Yeung-Levy S. Analyzing Surgical Technique in Diverse Open Surgical Videos With Multitask Machine Learning. JAMA Surg 2024; 159:185-192. [PMID: 38055227 PMCID: PMC10701669 DOI: 10.1001/jamasurg.2023.6262] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Accepted: 09/04/2023] [Indexed: 12/07/2023]
Abstract
Objective To overcome limitations of open surgery artificial intelligence (AI) models by curating the largest collection of annotated videos and to leverage this AI-ready data set to develop a generalizable multitask AI model capable of real-time understanding of clinically significant surgical behaviors in prospectively collected real-world surgical videos. Design, Setting, and Participants The study team programmatically queried open surgery procedures on YouTube and manually annotated selected videos to create the AI-ready data set used to train a multitask AI model for 2 proof-of-concept studies, one generating surgical signatures that define the patterns of a given procedure and the other identifying kinematics of hand motion that correlate with surgeon skill level and experience. The Annotated Videos of Open Surgery (AVOS) data set includes 1997 videos from 23 open-surgical procedure types uploaded to YouTube from 50 countries over the last 15 years. Prospectively recorded surgical videos were collected from a single tertiary care academic medical center. Deidentified videos were recorded of surgeons performing open surgical procedures and analyzed for correlation with surgical training. Exposures The multitask AI model was trained on the AI-ready video data set and then retrospectively applied to the prospectively collected video data set. Main Outcomes and Measures Analysis of open surgical videos in near real-time, performance on AI-ready and prospectively collected videos, and quantification of surgeon skill. Results Using the AI-ready data set, the study team developed a multitask AI model capable of real-time understanding of surgical behaviors-the building blocks of procedural flow and surgeon skill-across space and time. Through principal component analysis, a single compound skill feature was identified, composed of a linear combination of kinematic hand attributes. This feature was a significant discriminator between experienced surgeons and surgical trainees across 101 prospectively collected surgical videos of 14 operators. For each unit increase in the compound feature value, the odds of the operator being an experienced surgeon were 3.6 times higher (95% CI, 1.67-7.62; P = .001). Conclusions and Relevance In this observational study, the AVOS-trained model was applied to analyze prospectively collected open surgical videos and identify kinematic descriptors of surgical skill related to efficiency of hand motion. The ability to provide AI-deduced insights into surgical structure and skill is valuable in optimizing surgical skill acquisition and ultimately improving surgical care.
Collapse
Affiliation(s)
- Emmett D. Goodman
- Department of Computer Science, Stanford University, Stanford, California
- Department of Biomedical Data Science, Stanford University, Stanford, California
| | - Krishna K. Patel
- Department of Computer Science, Stanford University, Stanford, California
- Department of Biomedical Data Science, Stanford University, Stanford, California
| | - Yilun Zhang
- Department of Surgery, Beth Israel Deaconess Medical Center, Boston, Massachusetts
| | - William Locke
- Department of Computer Science, Stanford University, Stanford, California
- Department of Biomedical Data Science, Stanford University, Stanford, California
| | - Chris J. Kennedy
- Department of Surgery, Beth Israel Deaconess Medical Center, Boston, Massachusetts
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts
| | - Rohan Mehrotra
- Department of Computer Science, Stanford University, Stanford, California
- Department of Biomedical Data Science, Stanford University, Stanford, California
| | - Stephen Ren
- Department of Computer Science, Stanford University, Stanford, California
- Department of Biomedical Data Science, Stanford University, Stanford, California
| | - Melody Guan
- Department of Computer Science, Stanford University, Stanford, California
- Department of Biomedical Data Science, Stanford University, Stanford, California
| | - Orr Zohar
- Department of Biomedical Data Science, Stanford University, Stanford, California
- Department of Electrical Engineering, Stanford University, Stanford, California
| | - Maren Downing
- Department of Surgery, Beth Israel Deaconess Medical Center, Boston, Massachusetts
| | - Hao Wei Chen
- Department of Surgery, Beth Israel Deaconess Medical Center, Boston, Massachusetts
| | - Jevin Z. Clark
- Department of Surgery, Beth Israel Deaconess Medical Center, Boston, Massachusetts
| | - Margaret T. Berrigan
- Department of Surgery, Beth Israel Deaconess Medical Center, Boston, Massachusetts
| | - Gabriel A. Brat
- Department of Surgery, Beth Israel Deaconess Medical Center, Boston, Massachusetts
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts
| | - Serena Yeung-Levy
- Department of Computer Science, Stanford University, Stanford, California
- Department of Biomedical Data Science, Stanford University, Stanford, California
- Department of Electrical Engineering, Stanford University, Stanford, California
- Clinical Excellence Research Center, Stanford University School of Medicine, Stanford, California
| |
Collapse
|
43
|
Ross AE, Zhang J, Huang HC, Yamashita R, Keim-Malpass J, Simko JP, DeVries S, Morgan TM, Souhami L, Dobelbower MC, McGinnis LS, Jones CU, Dess RT, Zeitzer KL, Choi K, Hartford AC, Michalski JM, Raben A, Gomella LG, Sartor AO, Rosenthal SA, Sandler HM, Spratt DE, Pugh SL, Mohamad O, Esteva A, Chen E, Schaeffer EM, Tran PT, Feng FY. External Validation of a Digital Pathology-based Multimodal Artificial Intelligence Architecture in the NRG/RTOG 9902 Phase 3 Trial. Eur Urol Oncol 2024:S2588-9311(24)00029-4. [PMID: 38302323 DOI: 10.1016/j.euo.2024.01.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2023] [Revised: 12/02/2023] [Accepted: 01/05/2024] [Indexed: 02/03/2024]
Abstract
BACKGROUND Accurate risk stratification is critical to guide management decisions in localized prostate cancer (PCa). Previously, we had developed and validated a multimodal artificial intelligence (MMAI) model generated from digital histopathology and clinical features. Here, we externally validate this model on men with high-risk or locally advanced PCa treated and followed as part of a phase 3 randomized control trial. OBJECTIVE To externally validate the MMAI model on men with high-risk or locally advanced PCa treated and followed as part of a phase 3 randomized control trial. DESIGN, SETTING, AND PARTICIPANTS Our validation cohort included 318 localized high-risk PCa patients from NRG/RTOG 9902 with available histopathology (337 [85%] of the 397 patients enrolled into the trial had available slides, of which 19 [5.6%] failed due to poor image quality). OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS Two previously locked prognostic MMAI models were validated for their intended endpoint: distant metastasis (DM) and PCa-specific mortality (PCSM). Individual clinical factors and the number of National Comprehensive Cancer Network (NCCN) high-risk features served as comparators. Subdistribution hazard ratio (sHR) was reported per standard deviation increase of the score with corresponding 95% confidence interval (CI) using Fine-Gray or Cox proportional hazards models. RESULTS AND LIMITATIONS The DM and PCSM MMAI algorithms were significantly and independently associated with the risk of DM (sHR [95% CI] = 2.33 [1.60-3.38], p < 0.001) and PCSM, respectively (sHR [95% CI] = 3.54 [2.38-5.28], p < 0.001) when compared against other prognostic clinical factors and NCCN high-risk features. The lower 75% of patients by DM MMAI had estimated 5- and 10-yr DM rates of 4% and 7%, and the highest quartile had average 5- and 10-yr DM rates of 19% and 32%, respectively (p < 0.001). Similar results were observed for the PCSM MMAI algorithm. CONCLUSIONS We externally validated the prognostic ability of MMAI models previously developed among men with localized high-risk disease. MMAI prognostic models further risk stratify beyond the clinical and pathological variables for DM and PCSM in a population of men already at a high risk for disease progression. This study provides evidence for consistent validation of our deep learning MMAI models to improve prognostication and enable more informed decision-making for patient care. PATIENT SUMMARY This paper presents a novel approach using images from pathology slides along with clinical variables to validate artificial intelligence (computer-generated) prognostic models. When implemented, clinicians can offer a more personalized and tailored prognostic discussion for men with localized prostate cancer.
Collapse
Affiliation(s)
- Ashley E Ross
- Department of Urology, Northwestern Medicine, Chicago, IL, USA.
| | | | | | | | | | - Jeffry P Simko
- University of California San Francisco, San Francisco, CA, USA
| | - Sandy DeVries
- University of California San Francisco, San Francisco, CA, USA
| | | | - Luis Souhami
- The Research Institute of the McGill University Health Centre (MUHC), Montreal, QC, Canada
| | | | | | | | | | | | - Kwang Choi
- Brooklyn MB-CCOP/SUNY Downstate, Brooklyn, NY, USA
| | | | | | - Adam Raben
- Christiana Care Health Services, Inc. CCOP, Wilmington, DE, USA
| | | | - A Oliver Sartor
- Tulane University Health Sciences Center, New Orleans, LA, USA
| | | | | | - Daniel E Spratt
- UH Seidman Cancer Center, Case Western Reserve University, Cleveland, OH, USA
| | - Stephanie L Pugh
- NRG Oncology Statistics and Data Management Center and American College of Radiology, Philadelphia, PA, USA
| | - Osama Mohamad
- University of California San Francisco, San Francisco, CA, USA
| | | | | | | | | | - Felix Y Feng
- University of California San Francisco, San Francisco, CA, USA
| |
Collapse
|
44
|
Tabja Bortesi JP, Ranisau J, Di S, McGillion M, Rosella L, Johnson A, Devereaux PJ, Petch J. Machine Learning Approaches for the Image-Based Identification of Surgical Wound Infections: Scoping Review. J Med Internet Res 2024; 26:e52880. [PMID: 38236623 PMCID: PMC10835585 DOI: 10.2196/52880] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2023] [Revised: 11/09/2023] [Accepted: 12/12/2023] [Indexed: 01/19/2024] Open
Abstract
BACKGROUND Surgical site infections (SSIs) occur frequently and impact patients and health care systems. Remote surveillance of surgical wounds is currently limited by the need for manual assessment by clinicians. Machine learning (ML)-based methods have recently been used to address various aspects of the postoperative wound healing process and may be used to improve the scalability and cost-effectiveness of remote surgical wound assessment. OBJECTIVE The objective of this review was to provide an overview of the ML methods that have been used to identify surgical wound infections from images. METHODS We conducted a scoping review of ML approaches for visual detection of SSIs following the JBI (Joanna Briggs Institute) methodology. Reports of participants in any postoperative context focusing on identification of surgical wound infections were included. Studies that did not address SSI identification, surgical wounds, or did not use image or video data were excluded. We searched MEDLINE, Embase, CINAHL, CENTRAL, Web of Science Core Collection, IEEE Xplore, Compendex, and arXiv for relevant studies in November 2022. The records retrieved were double screened for eligibility. A data extraction tool was used to chart the relevant data, which was described narratively and presented using tables. Employment of TRIPOD (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis) guidelines was evaluated and PROBAST (Prediction Model Risk of Bias Assessment Tool) was used to assess risk of bias (RoB). RESULTS In total, 10 of the 715 unique records screened met the eligibility criteria. In these studies, the clinical contexts and surgical procedures were diverse. All papers developed diagnostic models, though none performed external validation. Both traditional ML and deep learning methods were used to identify SSIs from mostly color images, and the volume of images used ranged from under 50 to thousands. Further, 10 TRIPOD items were reported in at least 4 studies, though 15 items were reported in fewer than 4 studies. PROBAST assessment led to 9 studies being identified as having an overall high RoB, with 1 study having overall unclear RoB. CONCLUSIONS Research on the image-based identification of surgical wound infections using ML remains novel, and there is a need for standardized reporting. Limitations related to variability in image capture, model building, and data sources should be addressed in the future.
Collapse
Affiliation(s)
| | - Jonathan Ranisau
- Centre for Data Science and Digital Health, Hamilton Health Sciences, Hamilton, ON, Canada
| | - Shuang Di
- Centre for Data Science and Digital Health, Hamilton Health Sciences, Hamilton, ON, Canada
- Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
| | | | - Laura Rosella
- Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
| | | | - P J Devereaux
- Population Health Research Institute, Hamilton, ON, Canada
| | - Jeremy Petch
- Centre for Data Science and Digital Health, Hamilton Health Sciences, Hamilton, ON, Canada
- Population Health Research Institute, Hamilton, ON, Canada
- Institute for Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada
- Division of Cardiology, McMaster University, Hamilton, ON, Canada
| |
Collapse
|
45
|
Balu A, Kugener G, Pangal DJ, Lee H, Lasky S, Han J, Buchanan I, Liu J, Zada G, Donoho DA. Simulated outcomes for durotomy repair in minimally invasive spine surgery. Sci Data 2024; 11:62. [PMID: 38200013 PMCID: PMC10781746 DOI: 10.1038/s41597-023-02744-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2023] [Accepted: 11/13/2023] [Indexed: 01/12/2024] Open
Abstract
Minimally invasive spine surgery (MISS) is increasingly performed using endoscopic and microscopic visualization, and the captured video can be used for surgical education and development of predictive artificial intelligence (AI) models. Video datasets depicting adverse event management are also valuable, as predictive models not exposed to adverse events may exhibit poor performance when these occur. Given that no dedicated spine surgery video datasets for AI model development are publicly available, we introduce Simulated Outcomes for Durotomy Repair in Minimally Invasive Spine Surgery (SOSpine). A validated MISS cadaveric dural repair simulator was used to educate neurosurgery residents, and surgical microscope video recordings were paired with outcome data. Objects including durotomy, needle, grasper, needle driver, and nerve hook were then annotated. Altogether, SOSpine contains 15,698 frames with 53,238 annotations and associated durotomy repair outcomes. For validation, an AI model was fine-tuned on SOSpine video and detected surgical instruments with a mean average precision of 0.77. In summary, SOSpine depicts spine surgeons managing a common complication, providing opportunities to develop surgical AI models.
Collapse
Affiliation(s)
- Alan Balu
- Department of Neurosurgery, Georgetown University School of Medicine, 3900 Reservoir Rd NW, Washington, D.C., 20007, USA.
| | - Guillaume Kugener
- Department of Neurological Surgery, Keck School of Medicine of University of Southern California, 1200 North State St., Suite 3300, Los Angeles, CA, 90033, USA
| | - Dhiraj J Pangal
- Department of Neurological Surgery, Keck School of Medicine of University of Southern California, 1200 North State St., Suite 3300, Los Angeles, CA, 90033, USA
| | - Heewon Lee
- University of Southern California, 3709 Trousdale Pkwy., Los Angeles, CA, 90089, USA
| | - Sasha Lasky
- University of Southern California, 3709 Trousdale Pkwy., Los Angeles, CA, 90089, USA
| | - Jane Han
- University of Southern California, 3709 Trousdale Pkwy., Los Angeles, CA, 90089, USA
| | - Ian Buchanan
- Department of Neurological Surgery, Keck School of Medicine of University of Southern California, 1200 North State St., Suite 3300, Los Angeles, CA, 90033, USA
| | - John Liu
- Department of Neurological Surgery, Keck School of Medicine of University of Southern California, 1200 North State St., Suite 3300, Los Angeles, CA, 90033, USA
| | - Gabriel Zada
- Department of Neurological Surgery, Keck School of Medicine of University of Southern California, 1200 North State St., Suite 3300, Los Angeles, CA, 90033, USA
| | - Daniel A Donoho
- Department of Neurosurgery, Children's National Hospital, 111 Michigan Avenue NW, Washington, DC, 20010, USA
| |
Collapse
|
46
|
Liu X, Flanagan C, Li G, Lei Y, Zeng L, Fang J, Guo X, McGrath S, Han Y. Identification of difficult laryngoscopy using an optimized hybrid architecture. BMC Med Res Methodol 2024; 24:4. [PMID: 38177983 PMCID: PMC10765670 DOI: 10.1186/s12874-023-02115-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2023] [Accepted: 12/01/2023] [Indexed: 01/06/2024] Open
Abstract
BACKGROUND Identification of difficult laryngoscopy is a frequent demand in cervical spondylosis clinical surgery. This work aims to develop a hybrid architecture for identifying difficult laryngoscopy based on new indexes. METHODS Initially, two new indexes for identifying difficult laryngoscopy are proposed, and their efficacy for predicting difficult laryngoscopy is compared to that of two conventional indexes. Second, a hybrid adaptive architecture with convolutional layers, spatial extraction, and a vision transformer is proposed for predicting difficult laryngoscopy. The proposed adaptive hybrid architecture is then optimized by determining the optimal location for extracting spatial information. RESULTS The test accuracy of four indexes using simple model is 0.8320. The test accuracy of optimized hybrid architecture using four indexes is 0.8482. CONCLUSION The newly proposed two indexes, the angle between the lower margins of the second and sixth cervical spines and the vertical direction, are validated to be effective for recognizing difficult laryngoscopy. In addition, the optimized hybrid architecture employing four indexes demonstrates improved efficacy in detecting difficult laryngoscopy. TRIAL REGISTRATION Ethics permission for this research was obtained from the Medical Scientific Research Ethics Committee of Peking University Third Hospital (IRB00006761-2015021) on 30 March 2015. A well-informed agreement has been received from all participants. Patients were enrolled in this research at the Chinese Clinical Trial Registry ( http://www.chictr.org.cn , identifier: ChiCTR-ROC-16008598) on 6 June 2016.
Collapse
Affiliation(s)
- XiaoXiao Liu
- College of Mathematics and Information Science, Hebei University, Baoding, China
- Electronic and Computer Engineering, University of Limerick, Limerick, Ireland
| | - Colin Flanagan
- Electronic and Computer Engineering, University of Limerick, Limerick, Ireland
| | - Gang Li
- Department of General Surgery (GL), Peking University Third Hospital, Beijing, China
| | - Yiming Lei
- Ministry of Education Engineering Research Centre on Mobile Digital Hospital Systems, School of Electronics, Peking University, Beijing, China.
| | - Liaoyuan Zeng
- School of Communications, University of Electronic Science and Technology of China, Chengdu, China
| | - Jingchao Fang
- Department of Radiology (JCF), Peking University Third Hospital, Beijing, China
| | - Xiangyang Guo
- Department of Anaesthesiology, Peking University Third Hospital, Beijing, China
| | - Sean McGrath
- Electronic and Computer Engineering, University of Limerick, Limerick, Ireland.
| | - Yongzheng Han
- Department of Anaesthesiology, Peking University Third Hospital, Beijing, China.
| |
Collapse
|
47
|
Zhang J, Liu L, Xiang P, Fang Q, Nie X, Ma H, Hu J, Xiong R, Wang Y, Lu H. AI co-pilot bronchoscope robot. Nat Commun 2024; 15:241. [PMID: 38172095 PMCID: PMC10764930 DOI: 10.1038/s41467-023-44385-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2023] [Accepted: 12/12/2023] [Indexed: 01/05/2024] Open
Abstract
The unequal distribution of medical resources and scarcity of experienced practitioners confine access to bronchoscopy primarily to well-equipped hospitals in developed regions, contributing to the unavailability of bronchoscopic services in underdeveloped areas. Here, we present an artificial intelligence (AI) co-pilot bronchoscope robot that empowers novice doctors to conduct lung examinations as safely and adeptly as experienced colleagues. The system features a user-friendly, plug-and-play catheter, devised for robot-assisted steering, facilitating access to bronchi beyond the fifth generation in average adult patients. Drawing upon historical bronchoscopic videos and expert imitation, our AI-human shared control algorithm enables novice doctors to achieve safe steering in the lung, mitigating misoperations. Both in vitro and in vivo results underscore that our system equips novice doctors with the skills to perform lung examinations as expertly as seasoned practitioners. This study offers innovative strategies to address the pressing issue of medical resource disparities through AI assistance.
Collapse
Affiliation(s)
- Jingyu Zhang
- State Key Laboratory of Industrial Control and Technology, Zhejiang University, 310027, Hangzhou, China
- Institute of Cyber-Systems and Control, Department of Control Science and Engineering, Zhejiang University, 310027, Hangzhou, China
| | - Lilu Liu
- State Key Laboratory of Industrial Control and Technology, Zhejiang University, 310027, Hangzhou, China
- Institute of Cyber-Systems and Control, Department of Control Science and Engineering, Zhejiang University, 310027, Hangzhou, China
| | - Pingyu Xiang
- State Key Laboratory of Industrial Control and Technology, Zhejiang University, 310027, Hangzhou, China
- Institute of Cyber-Systems and Control, Department of Control Science and Engineering, Zhejiang University, 310027, Hangzhou, China
| | - Qin Fang
- State Key Laboratory of Industrial Control and Technology, Zhejiang University, 310027, Hangzhou, China
- Institute of Cyber-Systems and Control, Department of Control Science and Engineering, Zhejiang University, 310027, Hangzhou, China
| | - Xiuping Nie
- State Key Laboratory of Industrial Control and Technology, Zhejiang University, 310027, Hangzhou, China
- Institute of Cyber-Systems and Control, Department of Control Science and Engineering, Zhejiang University, 310027, Hangzhou, China
| | - Honghai Ma
- Department of Thoracic Surgery, First Affiliated Hospital, School of Medicine, Zhejiang University, 310009, Hangzhou, China
| | - Jian Hu
- Department of Thoracic Surgery, First Affiliated Hospital, School of Medicine, Zhejiang University, 310009, Hangzhou, China
| | - Rong Xiong
- State Key Laboratory of Industrial Control and Technology, Zhejiang University, 310027, Hangzhou, China.
- Institute of Cyber-Systems and Control, Department of Control Science and Engineering, Zhejiang University, 310027, Hangzhou, China.
| | - Yue Wang
- State Key Laboratory of Industrial Control and Technology, Zhejiang University, 310027, Hangzhou, China.
- Institute of Cyber-Systems and Control, Department of Control Science and Engineering, Zhejiang University, 310027, Hangzhou, China.
| | - Haojian Lu
- State Key Laboratory of Industrial Control and Technology, Zhejiang University, 310027, Hangzhou, China.
- Institute of Cyber-Systems and Control, Department of Control Science and Engineering, Zhejiang University, 310027, Hangzhou, China.
| |
Collapse
|
48
|
Guo Y, Guo J, Li Y, Zhang P, Zhao YD, Qiao Y, Liu B, Wang G. Rapid detection of non-normal teeth on dental X-ray images using improved Mask R-CNN with attention mechanism. Int J Comput Assist Radiol Surg 2024:10.1007/s11548-023-03047-1. [PMID: 38170416 DOI: 10.1007/s11548-023-03047-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2023] [Accepted: 12/08/2023] [Indexed: 01/05/2024]
Abstract
PURPOSE Dental health has been getting increased attention. Timely detection of non-normal teeth (caries, residual root, retainer, teeth filling, etc.) is of great importance for people's health, well-being, and quality of life. This work proposes a rapid detection of non-normal teeth based on improved Mask R-CNN, aiming to achieve comprehensive screening of non-normal teeth on dental X-ray images. METHODS An improved Mask R-CNN based on attention mechanism was used to develop a non-normal teeth detection method trained on a high-quality annotated dataset, which can segment the whole mask of each non-normal tooth on the dental X-ray image immediately. RESULTS The average precision (AP) of the proposed non-normal teeth detection was 0.795 with an intersection-over-union of 0.5 and max detections (maxDets) of 32, which was higher than that of the typical Mask R-CNN method (AP = 0.750). In addition, validation experiments showed that the evaluation metrics (AP, recall, precision-recall (P-R) curve) of the proposed method were superior to those of the Mask R-CNN method. Furthermore, the experimental results indicated that proposed method exhibited a high sensitivity (95.65%) in detecting secondary caries. The proposed method took about 0.12 s to segment non-normal teeth on one dental X-ray image using the laptop (8G memory, NVIDIA RTX 3060 graphics processing unit), which was much faster than conventional manual methods. CONCLUSION The proposed method enhances the accuracy and efficiency of abnormal tooth diagnosis for practitioners, while also facilitating early detection and treatment of dental caries to substantially lower patient costs. Additionally, it can enable rapid and objective evaluation of student performance in dental examinations.
Collapse
Affiliation(s)
- Yanbin Guo
- Hubei Bioinformatics and Molecular Imaging Key Laboratory, Department of Biomedical Engineering, College of Life Science and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China
| | - Jing Guo
- Department of Dental General and Emergency, The Affiliated Stomatological Hospital, Jiangxi Medical College, Nanchang University, Jiangxi Province Key Laboratory of Oral Biomedicine, Jiangxi Province Clinical Research Center for Oral Diseases, Nanchang, 330038, Jiangxi Province, China
| | - Yong Li
- Department of Anesthesiology, The First Affiliated Hospital of Nanchang University, Nanchang, 330006, China
| | - Peng Zhang
- Department of Dental General and Emergency, The Affiliated Stomatological Hospital, Jiangxi Medical College, Nanchang University, Jiangxi Province Key Laboratory of Oral Biomedicine, Jiangxi Province Clinical Research Center for Oral Diseases, Nanchang, 330038, Jiangxi Province, China
| | - Yuan-Di Zhao
- Hubei Bioinformatics and Molecular Imaging Key Laboratory, Department of Biomedical Engineering, College of Life Science and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China
| | - Yundi Qiao
- Department of Dental General and Emergency, The Affiliated Stomatological Hospital, Jiangxi Medical College, Nanchang University, Jiangxi Province Key Laboratory of Oral Biomedicine, Jiangxi Province Clinical Research Center for Oral Diseases, Nanchang, 330038, Jiangxi Province, China
| | - Benyuan Liu
- Department of Computer Science, University of Massachusetts Lowell, Lowell, MA, USA.
| | - Guoping Wang
- Hubei Bioinformatics and Molecular Imaging Key Laboratory, Department of Biomedical Engineering, College of Life Science and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China.
| |
Collapse
|
49
|
Kim KM, Kwak JW. PVS-GEN: Systematic Approach for Universal Synthetic Data Generation Involving Parameterization, Verification, and Segmentation. SENSORS (BASEL, SWITZERLAND) 2024; 24:266. [PMID: 38203126 PMCID: PMC10781314 DOI: 10.3390/s24010266] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/27/2023] [Revised: 12/06/2023] [Accepted: 12/14/2023] [Indexed: 01/12/2024]
Abstract
Synthetic data generation addresses the challenges of obtaining extensive empirical datasets, offering benefits such as cost-effectiveness, time efficiency, and robust model development. Nonetheless, synthetic data-generation methodologies still encounter significant difficulties, including a lack of standardized metrics for modeling different data types and comparing generated results. This study introduces PVS-GEN, an automated, general-purpose process for synthetic data generation and verification. The PVS-GEN method parameterizes time-series data with minimal human intervention and verifies model construction using a specific metric derived from extracted parameters. For complex data, the process iteratively segments the empirical dataset until an extracted parameter can reproduce synthetic data that reflects the empirical characteristics, irrespective of the sensor data type. Moreover, we introduce the PoR metric to quantify the quality of the generated data by evaluating its time-series characteristics. Consequently, the proposed method can automatically generate diverse time-series data that covers a wide range of sensor types. We compared PVS-GEN with existing synthetic data-generation methodologies, and PVS-GEN demonstrated a superior performance. It generated data with a similarity of up to 37.1% across multiple data types and by 19.6% on average using the proposed metric, irrespective of the data type.
Collapse
Affiliation(s)
| | - Jong Wook Kwak
- Department of Computer Engineering, Yeungnam University, Gyeongsan 38541, Republic of Korea;
| |
Collapse
|
50
|
Hase T, Ghosh S, Aisaki KI, Kitajima S, Kanno J, Kitano H, Yachie A. DTox: A deep neural network-based in visio lens for large scale toxicogenomics data. J Toxicol Sci 2024; 49:105-115. [PMID: 38432953 DOI: 10.2131/jts.49.105] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/05/2024]
Abstract
With the advancement of large-scale omics technologies, particularly transcriptomics data sets on drug and treatment response repositories available in public domain, toxicogenomics has emerged as a key field in safety pharmacology and chemical risk assessment. Traditional statistics-based bioinformatics analysis poses challenges in its application across multidimensional toxicogenomic data, including administration time, dosage, and gene expression levels. Motivated by the visual inspection workflow of field experts to augment their efficiency of screening significant genes to derive meaningful insights, together with the ability of deep neural architectures to learn the image signals, we developed DTox, a deep neural network-based in visio approach. Using the Percellome toxicogenomics database, instead of utilizing the numerical gene expression values of the transcripts (gene probes of the microarray) for dose-time combinations, DTox learned the image representation of 3D surface plots of distinct time and dosage data points to train the classifier on the experts' labels of gene probe significance. DTox outperformed statistical threshold-based bioinformatics and machine learning approaches based on numerical expression values. This result shows the ability of image-driven neural networks to overcome the limitations of classical numeric value-based approaches. Further, by augmenting the model with explainability modules, our study showed the potential to reveal the visual analysis process of human experts in toxicogenomics through the model weights. While the current work demonstrates the application of the DTox model in toxicogenomic studies, it can be further generalized as an in visio approach for multi-dimensional numeric data with applications in various fields in medical data sciences.
Collapse
Affiliation(s)
- Takeshi Hase
- The Systems Biology Institute, Saisei Ikedayama Bldg
- SBX BioSciences, Inc, Canada
- Institute of Education, Tokyo Medical and Dental University
- Faculty of Pharmacy, Keio University
- Center for Mathematical Modelling and Data Science, Osaka University
| | - Samik Ghosh
- The Systems Biology Institute, Saisei Ikedayama Bldg
| | - Ken-Ichi Aisaki
- Division of Cellular and Molecular Toxicology, Center for Biological Safety and Research (CBSR), National Institute of Health Sciences (NIHS)
| | - Satoshi Kitajima
- Division of Cellular and Molecular Toxicology, Center for Biological Safety and Research (CBSR), National Institute of Health Sciences (NIHS)
| | - Jun Kanno
- The Systems Biology Institute, Saisei Ikedayama Bldg
- Division of Cellular and Molecular Toxicology, Center for Biological Safety and Research (CBSR), National Institute of Health Sciences (NIHS)
- Faculty of Medicine, University of Tsukuba
| | - Hiroaki Kitano
- The Systems Biology Institute, Saisei Ikedayama Bldg
- Integrated Open Systems Unit, Okinawa Institute of Science and Technology (OIST)
| | - Ayako Yachie
- The Systems Biology Institute, Saisei Ikedayama Bldg
- SBX BioSciences, Inc, Canada
| |
Collapse
|