1
|
Xie Y, Gu L, Harada T, Zhang J, Xia Y, Wu Q. Rethinking masked image modelling for medical image representation. Med Image Anal 2024; 98:103304. [PMID: 39173412 DOI: 10.1016/j.media.2024.103304] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2023] [Revised: 08/06/2024] [Accepted: 08/09/2024] [Indexed: 08/24/2024]
Abstract
Masked Image Modelling (MIM), a form of self-supervised learning, has garnered significant success in computer vision by improving image representations using unannotated data. Traditional MIMs typically employ a strategy of random sampling across the image. However, this random masking technique may not be ideally suited for medical imaging, which possesses distinct characteristics divergent from natural images. In medical imaging, particularly in pathology, disease-related features are often exceedingly sparse and localized, while the remaining regions appear normal and undifferentiated. Additionally, medical images frequently accompany reports, directly pinpointing pathological changes' location. Inspired by this, we propose Masked medical Image Modelling (MedIM), a novel approach, to our knowledge, the first research that employs radiological reports to guide the masking and restore the informative areas of images, encouraging the network to explore the stronger semantic representations from medical images. We introduce two mutual comprehensive masking strategies, knowledge-driven masking (KDM), and sentence-driven masking (SDM). KDM uses Medical Subject Headings (MeSH) words unique to radiology reports to identify symptom clues mapped to MeSH words (e.g., cardiac, edema, vascular, pulmonary) and guide the mask generation. Recognizing that radiological reports often comprise several sentences detailing varied findings, SDM integrates sentence-level information to identify key regions for masking. MedIM reconstructs images informed by this masking from the KDM and SDM modules, promoting a comprehensive and enriched medical image representation. Our extensive experiments on seven downstream tasks covering multi-label/class image classification, pneumothorax segmentation, and medical image-report analysis, demonstrate that MedIM with report-guided masking achieves competitive performance. Our method substantially outperforms ImageNet pre-training, MIM-based pre-training, and medical image-report pre-training counterparts. Codes are available at https://github.com/YtongXie/MedIM.
Collapse
Affiliation(s)
| | - Lin Gu
- RIKEN AIP, Japan; RCAST, The University of Tokyo, Japan
| | | | - Jianpeng Zhang
- College of Computer Science and Technology, Zhejiang University, China
| | - Yong Xia
- School of Computer Science and Engineering, Northwestern Polytechnical University, Xi'an 710072, China; Ningbo Institute of Northwestern Polytechnical University, Ningbo 315048, China
| | - Qi Wu
- University of Adelaide, Australia.
| |
Collapse
|
2
|
Huang W, Li C, Yang H, Liu J, Liang Y, Zheng H, Wang S. Enhancing the vision-language foundation model with key semantic knowledge-emphasized report refinement. Med Image Anal 2024; 97:103299. [PMID: 39146702 DOI: 10.1016/j.media.2024.103299] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2024] [Revised: 07/05/2024] [Accepted: 08/06/2024] [Indexed: 08/17/2024]
Abstract
Recently, vision-language representation learning has made remarkable advancements in building up medical foundation models, holding immense potential for transforming the landscape of clinical research and medical care. The underlying hypothesis is that the rich knowledge embedded in radiology reports can effectively assist and guide the learning process, reducing the need for additional labels. However, these reports tend to be complex and sometimes even consist of redundant descriptions that make the representation learning too challenging to capture the key semantic information. This paper develops a novel iterative vision-language representation learning framework by proposing a key semantic knowledge-emphasized report refinement method. Particularly, raw radiology reports are refined to highlight the key information according to a constructed clinical dictionary and two model-optimized knowledge-enhancement metrics. The iterative framework is designed to progressively learn, starting from gaining a general understanding of the patient's condition based on raw reports and gradually refines and extracts critical information essential to the fine-grained analysis tasks. The effectiveness of the proposed framework is validated on various downstream medical image analysis tasks, including disease classification, region-of-interest segmentation, and phrase grounding. Our framework surpasses seven state-of-the-art methods in both fine-tuning and zero-shot settings, demonstrating its encouraging potential for different clinical applications.
Collapse
Affiliation(s)
- Weijian Huang
- Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China; Peng Cheng Laboratory, Shenzhen 518066, China; University of Chinese Academy of Sciences, Beijing 100049, China
| | - Cheng Li
- Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
| | - Hao Yang
- Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China; Peng Cheng Laboratory, Shenzhen 518066, China; University of Chinese Academy of Sciences, Beijing 100049, China
| | - Jiarun Liu
- Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China; Peng Cheng Laboratory, Shenzhen 518066, China; University of Chinese Academy of Sciences, Beijing 100049, China
| | - Yong Liang
- Peng Cheng Laboratory, Shenzhen 518066, China
| | - Hairong Zheng
- Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
| | - Shanshan Wang
- Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China.
| |
Collapse
|
3
|
Zhang Y, Ma X, Huang K, Li M, Heng PA. Semantic-Oriented Visual Prompt Learning for Diabetic Retinopathy Grading on Fundus Images. IEEE TRANSACTIONS ON MEDICAL IMAGING 2024; 43:2960-2969. [PMID: 38564346 DOI: 10.1109/tmi.2024.3383827] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/04/2024]
Abstract
Diabetic retinopathy (DR) is a serious ocular condition that requires effective monitoring and treatment by ophthalmologists. However, constructing a reliable DR grading model remains a challenging and costly task, heavily reliant on high-quality training sets and adequate hardware resources. In this paper, we investigate the knowledge transferability of large-scale pre-trained models (LPMs) to fundus images based on prompt learning to construct a DR grading model efficiently. Unlike full-tuning which fine-tunes all parameters of LPMs, prompt learning only involves a minimal number of additional learnable parameters while achieving a competitive effect as full-tuning. Inspired by visual prompt tuning, we propose Semantic-oriented Visual Prompt Learning (SVPL) to enhance the semantic perception ability for better extracting task-specific knowledge from LPMs, without any additional annotations. Specifically, SVPL assigns a group of learnable prompts for each DR level to fit the complex pathological manifestations and then aligns each prompt group to task-specific semantic space via a contrastive group alignment (CGA) module. We also propose a plug-and-play adapter module, Hierarchical Semantic Delivery (HSD), which allows the semantic transition of prompt groups from shallow to deep layers to facilitate efficient knowledge mining and model convergence. Our extensive experiments on three public DR grading datasets demonstrate that SVPL achieves superior results compared to other transfer tuning and DR grading methods. Further analysis suggests that the generalized knowledge from LPMs is advantageous for constructing the DR grading model on fundus images.
Collapse
|
4
|
Guo R, Wei J, Sun L, Yu B, Chang G, Liu D, Zhang S, Yao Z, Xu M, Bu L. A survey on advancements in image-text multimodal models: From general techniques to biomedical implementations. Comput Biol Med 2024; 178:108709. [PMID: 38878398 DOI: 10.1016/j.compbiomed.2024.108709] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2023] [Revised: 06/01/2024] [Accepted: 06/03/2024] [Indexed: 07/24/2024]
Abstract
With the significant advancements of Large Language Models (LLMs) in the field of Natural Language Processing (NLP), the development of image-text multimodal models has garnered widespread attention. Current surveys on image-text multimodal models mainly focus on representative models or application domains, but lack a review on how general technical models influence the development of domain-specific models, which is crucial for domain researchers. Based on this, this paper first reviews the technological evolution of image-text multimodal models, from early explorations of feature space to visual language encoding structures, and then to the latest large model architectures. Next, from the perspective of technological evolution, we explain how the development of general image-text multimodal technologies promotes the progress of multimodal technologies in the biomedical field, as well as the importance and complexity of specific datasets in the biomedical domain. Then, centered on the tasks of image-text multimodal models, we analyze their common components and challenges. After that, we summarize the architecture, components, and data of general image-text multimodal models, and introduce the applications and improvements of image-text multimodal models in the biomedical field. Finally, we categorize the challenges faced in the development and application of general models into external factors and intrinsic factors, further refining them into 2 external factors and 5 intrinsic factors, and propose targeted solutions, providing guidance for future research directions. For more details and data, please visit our GitHub page: https://github.com/i2vec/A-survey-on-image-text-multimodal-models.
Collapse
Affiliation(s)
- Ruifeng Guo
- Shenyang Institute of Computing Technology, Chinese Academy of Sciences, Shenyang, 110168, China; University of Chinese Academy of Sciences, Beijing, 100049, China.
| | - Jingxuan Wei
- Shenyang Institute of Computing Technology, Chinese Academy of Sciences, Shenyang, 110168, China; University of Chinese Academy of Sciences, Beijing, 100049, China.
| | - Linzhuang Sun
- Shenyang Institute of Computing Technology, Chinese Academy of Sciences, Shenyang, 110168, China; University of Chinese Academy of Sciences, Beijing, 100049, China.
| | - Bihui Yu
- Shenyang Institute of Computing Technology, Chinese Academy of Sciences, Shenyang, 110168, China; University of Chinese Academy of Sciences, Beijing, 100049, China.
| | - Guiyong Chang
- Shenyang Institute of Computing Technology, Chinese Academy of Sciences, Shenyang, 110168, China; University of Chinese Academy of Sciences, Beijing, 100049, China.
| | - Dawei Liu
- Shenyang Institute of Computing Technology, Chinese Academy of Sciences, Shenyang, 110168, China; University of Chinese Academy of Sciences, Beijing, 100049, China.
| | - Sibo Zhang
- Shenyang Institute of Computing Technology, Chinese Academy of Sciences, Shenyang, 110168, China; University of Chinese Academy of Sciences, Beijing, 100049, China.
| | - Zhengbing Yao
- Shenyang Institute of Computing Technology, Chinese Academy of Sciences, Shenyang, 110168, China; University of Chinese Academy of Sciences, Beijing, 100049, China.
| | - Mingjun Xu
- Shenyang Institute of Computing Technology, Chinese Academy of Sciences, Shenyang, 110168, China; University of Chinese Academy of Sciences, Beijing, 100049, China.
| | - Liping Bu
- Shenyang Institute of Computing Technology, Chinese Academy of Sciences, Shenyang, 110168, China; University of Chinese Academy of Sciences, Beijing, 100049, China.
| |
Collapse
|
5
|
Wang G, Wang K, Gao Y, Chen L, Gao T, Ma Y, Jiang Z, Yang G, Feng F, Zhang S, Gu Y, Liu G, Chen L, Ma LS, Sang Y, Xu Y, Lin G, Liu X. A generalized AI system for human embryo selection covering the entire IVF cycle via multi-modal contrastive learning. PATTERNS (NEW YORK, N.Y.) 2024; 5:100985. [PMID: 39081572 PMCID: PMC11284500 DOI: 10.1016/j.patter.2024.100985] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/10/2023] [Revised: 03/12/2024] [Accepted: 04/10/2024] [Indexed: 08/02/2024]
Abstract
In vitro fertilization (IVF) has revolutionized infertility treatment, benefiting millions of couples worldwide. However, current clinical practices for embryo selection rely heavily on visual inspection of morphology, which is highly variable and experience dependent. Here, we propose a comprehensive artificial intelligence (AI) system that can interpret embryo-developmental knowledge encoded in vast unlabeled multi-modal datasets and provide personalized embryo selection. This AI platform consists of a transformer-based network backbone named IVFormer and a self-supervised learning framework, VTCLR (visual-temporal contrastive learning of representations), for training multi-modal embryo representations pre-trained on large and unlabeled data. When evaluated on clinical scenarios covering the entire IVF cycle, our pre-trained AI model demonstrates accurate and reliable performance on euploidy ranking and live-birth occurrence prediction. For AI vs. physician for euploidy ranking, our model achieved superior performance across all score categories. The results demonstrate the potential of the AI system as a non-invasive, efficient, and cost-effective tool to improve embryo selection and IVF outcomes.
Collapse
Affiliation(s)
- Guangyu Wang
- State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China
| | - Kai Wang
- College of Future Technology, Peking University and Peking-Tsinghua Center for Life Sciences, Beijing 100871, China
| | - Yuanxu Gao
- College of Future Technology, Peking University and Peking-Tsinghua Center for Life Sciences, Beijing 100871, China
| | - Longbin Chen
- Institute of Reproductive and Stem Cells, School of Basic Medicine, Central South University, Changsha, China
| | - Tianrun Gao
- State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China
| | - Yuanlin Ma
- Reproductive Medicine Center, the First Affiliated Hospital, Sun Yat-sen University, Guangdong, China
| | - Zeyu Jiang
- State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China
| | - Guoxing Yang
- State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China
| | - Fajin Feng
- State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China
| | - Shuoping Zhang
- Research Department, CITIC Xiangya Reproductive and Genetic Hospital, Changsha, China
| | - Yifan Gu
- Research Department, CITIC Xiangya Reproductive and Genetic Hospital, Changsha, China
| | - Guangdong Liu
- Department of Gynaecology and Obstetrics, The Sixth Medical Center of the General Hospital of the People’s Liberation Army, Beijing, China
| | - Lei Chen
- Department of Gynaecology and Obstetrics, The Sixth Medical Center of the General Hospital of the People’s Liberation Army, Beijing, China
| | - Li-Shuang Ma
- Capital Institute of Pediatrics, Affiliated Children’s Hospital, Beijing, China
| | - Ye Sang
- The First College of Clinical Medical Science, China Three Gorges University & Yichang Central People’s Hospital, Yichang 443003, China
| | - Yanwen Xu
- Reproductive Medicine Center, the First Affiliated Hospital, Sun Yat-sen University, Guangdong, China
| | - Ge Lin
- Institute of Reproductive and Stem Cells, School of Basic Medicine, Central South University, Changsha, China
- Research Department, CITIC Xiangya Reproductive and Genetic Hospital, Changsha, China
| | - Xiaohong Liu
- State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China
- UCL Cancer Institute, University College London, London WC1E 6BT, UK
| |
Collapse
|
6
|
Reale-Nosei G, Amador-Domínguez E, Serrano E. From vision to text: A comprehensive review of natural image captioning in medical diagnosis and radiology report generation. Med Image Anal 2024; 97:103264. [PMID: 39013207 DOI: 10.1016/j.media.2024.103264] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2023] [Revised: 04/25/2024] [Accepted: 07/01/2024] [Indexed: 07/18/2024]
Abstract
Natural Image Captioning (NIC) is an interdisciplinary research area that lies within the intersection of Computer Vision (CV) and Natural Language Processing (NLP). Several works have been presented on the subject, ranging from the early template-based approaches to the more recent deep learning-based methods. This paper conducts a survey in the area of NIC, especially focusing on its applications for Medical Image Captioning (MIC) and Diagnostic Captioning (DC) in the field of radiology. A review of the state-of-the-art is conducted summarizing key research works in NIC and DC to provide a wide overview on the subject. These works include existing NIC and MIC models, datasets, evaluation metrics, and previous reviews in the specialized literature. The revised work is thoroughly analyzed and discussed, highlighting the limitations of existing approaches and their potential implications in real clinical practice. Similarly, future potential research lines are outlined on the basis of the detected limitations.
Collapse
Affiliation(s)
- Gabriel Reale-Nosei
- ETSI Informáticos, Universidad Politécnica de Madrid, 28660 Boadilla del Monte, Madrid, Spain.
| | - Elvira Amador-Domínguez
- Ontology Engineering Group, Departamento de Inteligencia Artificial, ETSI Informáticos, Universidad Politécnica de Madrid, 28660 Boadilla del Monte, Madrid, Spain; Departamento de Sistemas Informáticos, ETSI Sistemas Informáticos, Universidad Politécnica de Madrid, 28031 Madrid, Spain.
| | - Emilio Serrano
- Ontology Engineering Group, Departamento de Inteligencia Artificial, ETSI Informáticos, Universidad Politécnica de Madrid, 28660 Boadilla del Monte, Madrid, Spain.
| |
Collapse
|
7
|
Hu D, Jiang Z, Shi J, Xie F, Wu K, Tang K, Cao M, Huai J, Zheng Y. Histopathology language-image representation learning for fine-grained digital pathology cross-modal retrieval. Med Image Anal 2024; 95:103163. [PMID: 38626665 DOI: 10.1016/j.media.2024.103163] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2023] [Revised: 03/09/2024] [Accepted: 04/02/2024] [Indexed: 04/18/2024]
Abstract
Large-scale digital whole slide image (WSI) datasets analysis have gained significant attention in computer-aided cancer diagnosis. Content-based histopathological image retrieval (CBHIR) is a technique that searches a large database for data samples matching input objects in both details and semantics, offering relevant diagnostic information to pathologists. However, the current methods are limited by the difficulty of gigapixels, the variable size of WSIs, and the dependence on manual annotations. In this work, we propose a novel histopathology language-image representation learning framework for fine-grained digital pathology cross-modal retrieval, which utilizes paired diagnosis reports to learn fine-grained semantics from the WSI. An anchor-based WSI encoder is built to extract hierarchical region features and a prompt-based text encoder is introduced to learn fine-grained semantics from the diagnosis reports. The proposed framework is trained with a multivariate cross-modal loss function to learn semantic information from the diagnosis report at both the instance level and region level. After training, it can perform four types of retrieval tasks based on the multi-modal database to support diagnostic requirements. We conducted experiments on an in-house dataset and a public dataset to evaluate the proposed method. Extensive experiments have demonstrated the effectiveness of the proposed method and its advantages to the present histopathology retrieval methods. The code is available at https://github.com/hudingyi/FGCR.
Collapse
Affiliation(s)
- Dingyi Hu
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Engineering Medicine, Beihang University, Beijing, 100191, China; Image Processing Center, School of Astronautics, Beihang University, Beijing, 100191, China
| | - Zhiguo Jiang
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Engineering Medicine, Beihang University, Beijing, 100191, China; Image Processing Center, School of Astronautics, Beihang University, Beijing, 100191, China
| | - Jun Shi
- School of Software, Hefei University of Technology, Hefei 230601, China
| | - Fengying Xie
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Engineering Medicine, Beihang University, Beijing, 100191, China; Image Processing Center, School of Astronautics, Beihang University, Beijing, 100191, China
| | - Kun Wu
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Engineering Medicine, Beihang University, Beijing, 100191, China; Image Processing Center, School of Astronautics, Beihang University, Beijing, 100191, China
| | - Kunming Tang
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Engineering Medicine, Beihang University, Beijing, 100191, China; Image Processing Center, School of Astronautics, Beihang University, Beijing, 100191, China
| | - Ming Cao
- Department of Pathology, the First People's Hospital of Wuhu, China
| | - Jianguo Huai
- Department of Pathology, the First People's Hospital of Wuhu, China
| | - Yushan Zheng
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Engineering Medicine, Beihang University, Beijing, 100191, China.
| |
Collapse
|
8
|
Yang J. Multi-task learning for medical foundation models. NATURE COMPUTATIONAL SCIENCE 2024; 4:473-474. [PMID: 39030385 DOI: 10.1038/s43588-024-00658-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/21/2024]
Affiliation(s)
- Jiancheng Yang
- Computer Vision Laboratory, Swiss Federal Institute of Technology Lausanne (EPFL), Lausanne, Switzerland.
| |
Collapse
|
9
|
Liu A, Guo Y, Yong JH, Xu F. Multi-Grained Radiology Report Generation With Sentence-Level Image-Language Contrastive Learning. IEEE TRANSACTIONS ON MEDICAL IMAGING 2024; 43:2657-2669. [PMID: 38437149 DOI: 10.1109/tmi.2024.3372638] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/06/2024]
Abstract
The automatic generation of accurate radiology reports is of great clinical importance and has drawn growing research interest. However, it is still a challenging task due to the imbalance between normal and abnormal descriptions and the multi-sentence and multi-topic nature of radiology reports. These features result in significant challenges to generating accurate descriptions for medical images, especially the important abnormal findings. Previous methods to tackle these problems rely heavily on extra manual annotations, which are expensive to acquire. We propose a multi-grained report generation framework incorporating sentence-level image-sentence contrastive learning, which does not require any extra labeling but effectively learns knowledge from the image-report pairs. We first introduce contrastive learning as an auxiliary task for image feature learning. Different from previous contrastive methods, we exploit the multi-topic nature of imaging reports and perform fine-grained contrastive learning by extracting sentence topics and contents and contrasting between sentence contents and refined image contents guided by sentence topics. This forces the model to learn distinct abnormal image features for each specific topic. During generation, we use two decoders to first generate coarse sentence topics and then the fine-grained text of each sentence. We directly supervise the intermediate topics using sentence topics learned by our contrastive objective. This strengthens the generation constraint and enables independent fine-tuning of the decoders using reinforcement learning, which further boosts model performance. Experiments on two large-scale datasets MIMIC-CXR and IU-Xray demonstrate that our approach outperforms existing state-of-the-art methods, evaluated by both language generation metrics and clinical accuracy.
Collapse
|
10
|
Schäfer R, Nicke T, Höfener H, Lange A, Merhof D, Feuerhake F, Schulz V, Lotz J, Kiessling F. Overcoming data scarcity in biomedical imaging with a foundational multi-task model. NATURE COMPUTATIONAL SCIENCE 2024; 4:495-509. [PMID: 39030386 PMCID: PMC11288886 DOI: 10.1038/s43588-024-00662-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/21/2023] [Accepted: 06/17/2024] [Indexed: 07/21/2024]
Abstract
Foundational models, pretrained on a large scale, have demonstrated substantial success across non-medical domains. However, training these models typically requires large, comprehensive datasets, which contrasts with the smaller and more specialized datasets common in biomedical imaging. Here we propose a multi-task learning strategy that decouples the number of training tasks from memory requirements. We trained a universal biomedical pretrained model (UMedPT) on a multi-task database including tomographic, microscopic and X-ray images, with various labeling strategies such as classification, segmentation and object detection. The UMedPT foundational model outperformed ImageNet pretraining and previous state-of-the-art models. For classification tasks related to the pretraining database, it maintained its performance with only 1% of the original training data and without fine-tuning. For out-of-domain tasks it required only 50% of the original training data. In an external independent validation, imaging features extracted using UMedPT proved to set a new standard for cross-center transferability.
Collapse
Affiliation(s)
- Raphael Schäfer
- Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany
| | - Till Nicke
- Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany
| | - Henning Höfener
- Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany
| | - Annkristin Lange
- Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany
| | - Dorit Merhof
- Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany
- Institute of Image Analysis and Computer Vision, Faculty of Informatics and Data Science, University of Regensburg, Regensburg, Germany
| | - Friedrich Feuerhake
- Institute for Pathology, Hannover Medical School, Hanover, Germany
- Institute for Neuropathology, Medical Center, University of Freiburg, Freiburg, Germany
| | - Volkmar Schulz
- Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany
- Institute for Experimental Molecular Imaging, RWTH Aachen University, Aachen, Germany
| | - Johannes Lotz
- Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany.
| | - Fabian Kiessling
- Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany.
- Institute for Experimental Molecular Imaging, RWTH Aachen University, Aachen, Germany.
| |
Collapse
|
11
|
Zhang S, Metaxas D. On the challenges and perspectives of foundation models for medical image analysis. Med Image Anal 2024; 91:102996. [PMID: 37857067 DOI: 10.1016/j.media.2023.102996] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2023] [Revised: 09/24/2023] [Accepted: 10/04/2023] [Indexed: 10/21/2023]
Abstract
This article discusses the opportunities, applications and future directions of large-scale pretrained models, i.e., foundation models, which promise to significantly improve the analysis of medical images. Medical foundation models have immense potential in solving a wide range of downstream tasks, as they can help to accelerate the development of accurate and robust models, reduce the dependence on large amounts of labeled data, preserve the privacy and confidentiality of patient data. Specifically, we illustrate the "spectrum" of medical foundation models, ranging from general imaging models, modality-specific models, to organ/task-specific models, and highlight their challenges, opportunities and applications. We also discuss how foundation models can be leveraged in downstream medical tasks to enhance the accuracy and efficiency of medical image analysis, leading to more precise diagnosis and treatment decisions.
Collapse
Affiliation(s)
- Shaoting Zhang
- University of Electronic Science and Technology of China, Chengdu, Sichuan, China; Shanghai Artificial Intelligence Laboratory, Shanghai, China.
| | | |
Collapse
|
12
|
Liu F, Zhu T, Wu X, Yang B, You C, Wang C, Lu L, Liu Z, Zheng Y, Sun X, Yang Y, Clifton L, Clifton DA. A medical multimodal large language model for future pandemics. NPJ Digit Med 2023; 6:226. [PMID: 38042919 PMCID: PMC10693607 DOI: 10.1038/s41746-023-00952-2] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2023] [Accepted: 10/24/2023] [Indexed: 12/04/2023] Open
Abstract
Deep neural networks have been integrated into the whole clinical decision procedure which can improve the efficiency of diagnosis and alleviate the heavy workload of physicians. Since most neural networks are supervised, their performance heavily depends on the volume and quality of available labels. However, few such labels exist for rare diseases (e.g., new pandemics). Here we report a medical multimodal large language model (Med-MLLM) for radiograph representation learning, which can learn broad medical knowledge (e.g., image understanding, text semantics, and clinical phenotypes) from unlabelled data. As a result, when encountering a rare disease, our Med-MLLM can be rapidly deployed and easily adapted to them with limited labels. Furthermore, our model supports medical data across visual modality (e.g., chest X-ray and CT) and textual modality (e.g., medical report and free-text clinical note); therefore, it can be used for clinical tasks that involve both visual and textual data. We demonstrate the effectiveness of our Med-MLLM by showing how it would perform using the COVID-19 pandemic "in replay". In the retrospective setting, we test the model on the early COVID-19 datasets; and in the prospective setting, we test the model on the new variant COVID-19-Omicron. The experiments are conducted on 1) three kinds of input data; 2) three kinds of downstream tasks, including disease reporting, diagnosis, and prognosis; 3) five COVID-19 datasets; and 4) three different languages, including English, Chinese, and Spanish. All experiments show that our model can make accurate and robust COVID-19 decision-support with little labelled data.
Collapse
Affiliation(s)
- Fenglin Liu
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK.
| | - Tingting Zhu
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK
| | - Xian Wu
- Jarvis Research Center, Tencent YouTu Lab, Beijing, China
| | - Bang Yang
- School of Computer Science, Peking University, Beijing, China
| | | | - Chenyang Wang
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK
| | - Lei Lu
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK
| | - Zhangdaihong Liu
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK
- Oxford-Suzhou Centre for Advanced Research, Suzhou, China
| | - Yefeng Zheng
- Jarvis Research Center, Tencent YouTu Lab, Beijing, China
| | - Xu Sun
- School of Computer Science, Peking University, Beijing, China
| | - Yang Yang
- School of Public Health, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Lei Clifton
- Nuffield Department of Population Health, University of Oxford, Oxford, UK
| | - David A Clifton
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK.
- Oxford-Suzhou Centre for Advanced Research, Suzhou, China.
| |
Collapse
|
13
|
Li F, Zhai P, Yang C, Feng G, Yang J, Yuan Y. Automated diagnosis of anterior cruciate ligament via a weighted multi-view network. Front Bioeng Biotechnol 2023; 11:1268543. [PMID: 37885456 PMCID: PMC10598377 DOI: 10.3389/fbioe.2023.1268543] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2023] [Accepted: 09/22/2023] [Indexed: 10/28/2023] Open
Abstract
Objective: To build a three-dimensional (3D) deep learning-based computer-aided diagnosis (CAD) system and investigate its applicability for automatic detection of anterior cruciate ligament (ACL) of the knee joint in magnetic resonance imaging (MRI). Methods: In this study, we develop a 3D weighted multi-view convolutional neural network by fusing different views of MRI to detect ACL. The network is evaluated on two MRI datasets, the in-house MRI-ACL dataset and the publicly available MRNet-v1.0 dataset. In the MRI-ACL dataset, the retrospective study collects 100 cases, and four views per patient are included. There are 50 ACL patients and 50 normal patients, respectively. The MRNet-v1.0 dataset contains 1,250 cases with three views, of which 208 are ACL patients, and the rest are normal or other abnormal patients. Results: The area under the receiver operating characteristic curve (AUC) of the ACL diagnosis system is 97.00% and 92.86% at the optimal threshold for the MRI-ACL dataset and the MRNet-v1.0 dataset, respectively, indicating a high overall diagnostic accuracy. In comparison, the best AUC of the single-view diagnosis methods are 96.00% (MRI-ACL dataset) and 91.78% (MRNet-v1.0 dataset), and our method improves by about 1.00% and 1.08%. Furthermore, our method also improves by about 1.00% (MRI-ACL dataset) and 0.28% (MRNet-v1.0 dataset) compared with the multi-view network (i.e., MRNet). Conclusion: The presented 3D weighted multi-view network achieves superior AUC in diagnosing ACL, not only in the in-house MRI-ACL dataset but also in the publicly available MRNet-v1.0 dataset, which demonstrates its clinical applicability for the automatic detection of ACL.
Collapse
Affiliation(s)
- Feng Li
- Orthopedic Department, Ningbo No. 2 Hospital, Ningbo, China
| | - Penghua Zhai
- Center for Pattern Recognition and Intelligent Medicine, Guoke Ningbo Life science and Health industry Research Institute, Ningbo, China
| | - Chao Yang
- Orthopedic Department, Ningbo No. 2 Hospital, Ningbo, China
| | - Gong Feng
- Orthopedic Department, Ningbo No. 2 Hospital, Ningbo, China
| | - Ji Yang
- Orthopedic Department, Ningbo No. 2 Hospital, Ningbo, China
| | - Yi Yuan
- Orthopedic Department, Ningbo No. 2 Hospital, Ningbo, China
| |
Collapse
|
14
|
Messiou C, Lee R, Salto-Tellez M. Multimodal analysis and the oncology patient: Creating a hospital system for integrated diagnostics and discovery. Comput Struct Biotechnol J 2023; 21:4536-4539. [PMID: 37767106 PMCID: PMC10520501 DOI: 10.1016/j.csbj.2023.09.014] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2023] [Revised: 09/13/2023] [Accepted: 09/13/2023] [Indexed: 09/29/2023] Open
Abstract
We propose that an information technology and computational framework that would unify access to hospital digital information silos, and enable integration of this information using machine learning methods, would bring a new paradigm to patient management and research. This is the core principle of Integrated Diagnostics (ID): the amalgamation of multiple analytical modalities, with evolved information technology, applied to a defined patient cohort, and resulting in a synergistic effect in the clinical value of the individual diagnostic tools. This has the potential to transform the practice of personalized oncology at a time at which it is very much needed. In this article we present different models from the literature that contribute to the vision of ID and we provide published exemplars of ID tools. We briefly describe ongoing efforts within a universal healthcare system to create national clinical datasets. Following this, we argue the case to create "hospital units" to leverage this multi-modal analysis, data integration and holistic clinical decision-making. Finally, we describe the joint model created in our institutions.
Collapse
Affiliation(s)
- Christina Messiou
- Imaging and Data Science Theme lead and Director of the Imaging AI hub at The Royal Marsden and Institute of Cancer Research, National Institute for Health Research Biomedical Research Centre, Sutton SM2 5PT, UK
| | - Richard Lee
- Consultant Respiratory Physician & Champion for Early Diagnosis Early Diagnosis and Detection Centre, NIHR Biomedical Research Centre at the Royal Marsden and ICR, National Heart and Lung Institute, Imperial College London, UK
| | - Manuel Salto-Tellez
- The Integrated Pathology Unit, the Institute of Cancer Research & The Royal Marsden Hospital, Sutton SM2 5PT, UK
- Precision Medicine Centre, Queen’s University Belfast, UK
| |
Collapse
|
15
|
Yu F, Endo M, Krishnan R, Pan I, Tsai A, Reis EP, Fonseca EKUN, Lee HMH, Abad ZSH, Ng AY, Langlotz CP, Venugopal VK, Rajpurkar P. Evaluating progress in automatic chest X-ray radiology report generation. PATTERNS (NEW YORK, N.Y.) 2023; 4:100802. [PMID: 37720336 PMCID: PMC10499844 DOI: 10.1016/j.patter.2023.100802] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/08/2022] [Revised: 02/03/2023] [Accepted: 06/29/2023] [Indexed: 09/19/2023]
Abstract
Artificial intelligence (AI) models for automatic generation of narrative radiology reports from images have the potential to enhance efficiency and reduce the workload of radiologists. However, evaluating the correctness of these reports requires metrics that can capture clinically pertinent differences. In this study, we investigate the alignment between automated metrics and radiologists' scoring of errors in report generation. We address the limitations of existing metrics by proposing new metrics, RadGraph F1 and RadCliQ, which demonstrate stronger correlation with radiologists' evaluations. In addition, we analyze the failure modes of the metrics to understand their limitations and provide guidance for metric selection and interpretation. This study establishes RadGraph F1 and RadCliQ as meaningful metrics for guiding future research in radiology report generation.
Collapse
Affiliation(s)
- Feiyang Yu
- Department of Computer Science, Stanford University, Stanford, CA 94305, USA
| | - Mark Endo
- Department of Computer Science, Stanford University, Stanford, CA 94305, USA
| | - Rayan Krishnan
- Department of Computer Science, Stanford University, Stanford, CA 94305, USA
| | - Ian Pan
- Department of Radiology, Brigham and Women’s Hospital, Boston, MA 02115, USA
| | - Andy Tsai
- Department of Radiology, Boston Children’s Hospital, Harvard Medical School, Boston, MA 02115, USA
| | - Eduardo Pontes Reis
- Cardiothoracic Radiology Group, Hospital Israelita Albert Einstein, São Paulo, São Paulo 05652, Brazil
| | | | - Henrique Min Ho Lee
- Cardiothoracic Radiology Group, Hospital Israelita Albert Einstein, São Paulo, São Paulo 05652, Brazil
| | | | - Andrew Y. Ng
- Department of Computer Science, Stanford University, Stanford, CA 94305, USA
| | | | | | - Pranav Rajpurkar
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA 02115, USA
| |
Collapse
|
16
|
Zhang X, Wu C, Zhang Y, Xie W, Wang Y. Knowledge-enhanced visual-language pre-training on chest radiology images. Nat Commun 2023; 14:4542. [PMID: 37507376 PMCID: PMC10382552 DOI: 10.1038/s41467-023-40260-7] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2023] [Accepted: 07/19/2023] [Indexed: 07/30/2023] Open
Abstract
While multi-modal foundation models pre-trained on large-scale data have been successful in natural language understanding and vision recognition, their use in medical domains is still limited due to the fine-grained nature of medical tasks and the high demand for domain knowledge. To address this challenge, we propose an approach called Knowledge-enhanced Auto Diagnosis (KAD) which leverages existing medical domain knowledge to guide vision-language pre-training using paired chest X-rays and radiology reports. We evaluate KAD on four external X-ray datasets and demonstrate that its zero-shot performance is not only comparable to that of fully supervised models but also superior to the average of three expert radiologists for three (out of five) pathologies with statistical significance. Moreover, when few-shot annotation is available, KAD outperforms all existing approaches in fine-tuning settings, demonstrating its potential for application in different clinical scenarios.
Collapse
Affiliation(s)
- Xiaoman Zhang
- Cooperative Medianet Innovation Center, Shanghai Jiao Tong University, 200240, Shanghai, China
- Shanghai Artificial Intelligence Laboratory, 200232, Shanghai, China
| | - Chaoyi Wu
- Cooperative Medianet Innovation Center, Shanghai Jiao Tong University, 200240, Shanghai, China
- Shanghai Artificial Intelligence Laboratory, 200232, Shanghai, China
| | - Ya Zhang
- Cooperative Medianet Innovation Center, Shanghai Jiao Tong University, 200240, Shanghai, China
- Shanghai Artificial Intelligence Laboratory, 200232, Shanghai, China
| | - Weidi Xie
- Cooperative Medianet Innovation Center, Shanghai Jiao Tong University, 200240, Shanghai, China.
- Shanghai Artificial Intelligence Laboratory, 200232, Shanghai, China.
| | - Yanfeng Wang
- Cooperative Medianet Innovation Center, Shanghai Jiao Tong University, 200240, Shanghai, China.
- Shanghai Artificial Intelligence Laboratory, 200232, Shanghai, China.
| |
Collapse
|
17
|
Sato J, Suzuki Y, Wataya T, Nishigaki D, Kita K, Yamagata K, Tomiyama N, Kido S. Anatomy-aware self-supervised learning for anomaly detection in chest radiographs. iScience 2023; 26:107086. [PMID: 37434699 PMCID: PMC10331430 DOI: 10.1016/j.isci.2023.107086] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2023] [Revised: 04/17/2023] [Accepted: 06/06/2023] [Indexed: 07/13/2023] Open
Abstract
In this study, we present a self-supervised learning (SSL)-based model that enables anatomical structure-based unsupervised anomaly detection (UAD). The model employs an anatomy-aware pasting (AnatPaste) augmentation tool that uses a threshold-based lung segmentation pretext task to create anomalies in normal chest radiographs used for model pretraining. These anomalies are similar to real anomalies and help the model recognize them. We evaluate our model using three open-source chest radiograph datasets. Our model exhibits area under curves of 92.1%, 78.7%, and 81.9%, which are the highest among those of existing UAD models. To the best of our knowledge, this is the first SSL model to employ anatomical information from segmentation as a pretext task. The performance of AnatPaste shows that incorporating anatomical information into SSL can effectively improve accuracy.
Collapse
Affiliation(s)
- Junya Sato
- Department of Artificial Intelligence Diagnostic Radiology, Osaka University Graduate School of Medicine, 2-2, Yamadaoka, Suita, Osaka 565-0871, Japan
- Department of Radiology, Osaka University Graduate School of Medicine, 2-2, Yamadaoka, Suita, Osaka 565-0871, Japan
- Graduate School of Information Science and Technology, Osaka University, Yamadaoka, 1-5 Suita, Osaka 565-0871, Japan
| | - Yuki Suzuki
- Department of Artificial Intelligence Diagnostic Radiology, Osaka University Graduate School of Medicine, 2-2, Yamadaoka, Suita, Osaka 565-0871, Japan
| | - Tomohiro Wataya
- Department of Artificial Intelligence Diagnostic Radiology, Osaka University Graduate School of Medicine, 2-2, Yamadaoka, Suita, Osaka 565-0871, Japan
- Department of Radiology, Osaka University Graduate School of Medicine, 2-2, Yamadaoka, Suita, Osaka 565-0871, Japan
| | - Daiki Nishigaki
- Department of Artificial Intelligence Diagnostic Radiology, Osaka University Graduate School of Medicine, 2-2, Yamadaoka, Suita, Osaka 565-0871, Japan
- Department of Radiology, Osaka University Graduate School of Medicine, 2-2, Yamadaoka, Suita, Osaka 565-0871, Japan
| | - Kosuke Kita
- Department of Artificial Intelligence Diagnostic Radiology, Osaka University Graduate School of Medicine, 2-2, Yamadaoka, Suita, Osaka 565-0871, Japan
| | - Kazuki Yamagata
- Department of Artificial Intelligence Diagnostic Radiology, Osaka University Graduate School of Medicine, 2-2, Yamadaoka, Suita, Osaka 565-0871, Japan
- Department of Radiology, Osaka University Graduate School of Medicine, 2-2, Yamadaoka, Suita, Osaka 565-0871, Japan
| | - Noriyuki Tomiyama
- Department of Radiology, Osaka University Graduate School of Medicine, 2-2, Yamadaoka, Suita, Osaka 565-0871, Japan
| | - Shoji Kido
- Department of Artificial Intelligence Diagnostic Radiology, Osaka University Graduate School of Medicine, 2-2, Yamadaoka, Suita, Osaka 565-0871, Japan
| |
Collapse
|
18
|
Zhou HY, Yu Y, Wang C, Zhang S, Gao Y, Pan J, Shao J, Lu G, Zhang K, Li W. A transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics. Nat Biomed Eng 2023:10.1038/s41551-023-01045-x. [PMID: 37308585 DOI: 10.1038/s41551-023-01045-x] [Citation(s) in RCA: 29] [Impact Index Per Article: 29.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2022] [Accepted: 04/26/2023] [Indexed: 06/14/2023]
Abstract
During the diagnostic process, clinicians leverage multimodal information, such as the chief complaint, medical images and laboratory test results. Deep-learning models for aiding diagnosis have yet to meet this requirement of leveraging multimodal information. Here we report a transformer-based representation-learning model as a clinical diagnostic aid that processes multimodal input in a unified manner. Rather than learning modality-specific features, the model leverages embedding layers to convert images and unstructured and structured text into visual tokens and text tokens, and uses bidirectional blocks with intramodal and intermodal attention to learn holistic representations of radiographs, the unstructured chief complaint and clinical history, and structured clinical information such as laboratory test results and patient demographic information. The unified model outperformed an image-only model and non-unified multimodal diagnosis models in the identification of pulmonary disease (by 12% and 9%, respectively) and in the prediction of adverse clinical outcomes in patients with COVID-19 (by 29% and 7%, respectively). Unified multimodal transformer-based models may help streamline the triaging of patients and facilitate the clinical decision-making process.
Collapse
Affiliation(s)
- Hong-Yu Zhou
- Department of Computer Science, The University of Hong Kong, Pokfulam, China
| | - Yizhou Yu
- Department of Computer Science, The University of Hong Kong, Pokfulam, China.
| | - Chengdi Wang
- Department of Pulmonary and Critical Care Medicine, Med-X Center for Manufacturing, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University, Chengdu, China.
| | - Shu Zhang
- AI Lab, Deepwise Healthcare, Beijing, China
| | | | - Jia Pan
- Department of Computer Science, The University of Hong Kong, Pokfulam, China
| | - Jun Shao
- Department of Pulmonary and Critical Care Medicine, Med-X Center for Manufacturing, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University, Chengdu, China
| | - Guangming Lu
- Department of Medical Imaging, Jinling Hospital, Nanjing University School of Medicine, Nanjing, China
| | - Kang Zhang
- Zhuhai International Eye Center and Provincial Key Laboratory of Tumor Interventional Diagnosis and Treatment, Zhuhai People's Hospital and the First Affiliated Hospital of Faculty of Medicine, Macau University of Science and Technology and University Hospital, Guangdong, China.
- Department of Big Data and Biomedical Artificial Intelligence, National Biomedical Imaging Center, College of Future Technology, Peking University, Beijing, China.
- Clinical Translational Research Center, West China Hospital, Sichuan University, Chengdu, China.
| | - Weimin Li
- Department of Pulmonary and Critical Care Medicine, Med-X Center for Manufacturing, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University, Chengdu, China.
| |
Collapse
|
19
|
|
20
|
When artificial intelligence meets PD-1/PD-L1 inhibitors: Population screening, response prediction and efficacy evaluation. Comput Biol Med 2022; 145:105499. [DOI: 10.1016/j.compbiomed.2022.105499] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2022] [Revised: 03/26/2022] [Accepted: 04/03/2022] [Indexed: 02/07/2023]
|