1
|
Ruitenbeek HC, Oei EHG, Visser JJ, Kijowski R. Artificial intelligence in musculoskeletal imaging: realistic clinical applications in the next decade. Skeletal Radiol 2024; 53:1849-1868. [PMID: 38902420 DOI: 10.1007/s00256-024-04684-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/01/2024] [Revised: 04/06/2024] [Accepted: 04/15/2024] [Indexed: 06/22/2024]
Abstract
This article will provide a perspective review of the most extensively investigated deep learning (DL) applications for musculoskeletal disease detection that have the best potential to translate into routine clinical practice over the next decade. Deep learning methods for detecting fractures, estimating pediatric bone age, calculating bone measurements such as lower extremity alignment and Cobb angle, and grading osteoarthritis on radiographs have been shown to have high diagnostic performance with many of these applications now commercially available for use in clinical practice. Many studies have also documented the feasibility of using DL methods for detecting joint pathology and characterizing bone tumors on magnetic resonance imaging (MRI). However, musculoskeletal disease detection on MRI is difficult as it requires multi-task, multi-class detection of complex abnormalities on multiple image slices with different tissue contrasts. The generalizability of DL methods for musculoskeletal disease detection on MRI is also challenging due to fluctuations in image quality caused by the wide variety of scanners and pulse sequences used in routine MRI protocols. The diagnostic performance of current DL methods for musculoskeletal disease detection must be further evaluated in well-designed prospective studies using large image datasets acquired at different institutions with different imaging parameters and imaging hardware before they can be fully implemented in clinical practice. Future studies must also investigate the true clinical benefits of current DL methods and determine whether they could enhance quality, reduce error rates, improve workflow, and decrease radiologist fatigue and burnout with all of this weighed against the costs.
Collapse
Affiliation(s)
- Huibert C Ruitenbeek
- Department of Radiology and Nuclear Medicine, Erasmus MC, University Medical Center, P.O. Box 2040, 3000 CA, Rotterdam, The Netherlands
| | - Edwin H G Oei
- Department of Radiology and Nuclear Medicine, Erasmus MC, University Medical Center, P.O. Box 2040, 3000 CA, Rotterdam, The Netherlands
| | - Jacob J Visser
- Department of Radiology and Nuclear Medicine, Erasmus MC, University Medical Center, P.O. Box 2040, 3000 CA, Rotterdam, The Netherlands
| | - Richard Kijowski
- Department of Radiology, New York University Grossman School of Medicine, 660 First Avenue, 3rd Floor, New York, NY, 10016, USA.
| |
Collapse
|
2
|
Mostafa RR, Khedr AM, Aghbari ZA, Afyouni I, Kamel I, Ahmed N. Medical image segmentation approach based on hybrid adaptive differential evolution and crayfish optimizer. Comput Biol Med 2024; 180:109011. [PMID: 39146840 DOI: 10.1016/j.compbiomed.2024.109011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2024] [Revised: 07/18/2024] [Accepted: 08/07/2024] [Indexed: 08/17/2024]
Abstract
Image segmentation plays a pivotal role in medical image analysis, particularly for accurately isolating tumors and lesions. Effective segmentation improves diagnostic precision and facilitates quantitative analysis, which is vital for medical professionals. However, traditional segmentation methods often struggle with multilevel thresholding due to the associated computational complexity. Therefore, determining the optimal threshold set is an NP-hard problem, highlighting the pressing need for efficient optimization strategies to overcome these challenges. This paper introduces a multi-threshold image segmentation (MTIS) method that integrates a hybrid approach combining Differential Evolution (DE) and the Crayfish Optimization Algorithm (COA), known as HADECO. Utilizing two-dimensional (2D) Kapur's entropy and a 2D histogram, this method aims to enhance the efficiency and accuracy of subsequent image analysis and diagnosis. HADECO is a hybrid algorithm that combines DE and COA by exchanging information based on predefined rules, leveraging the strengths of both for superior optimization results. It employs Latin Hypercube Sampling (LHS) to generate a high-quality initial population. HADECO introduces an improved DE algorithm (IDE) with adaptive and dynamic adjustments to key DE parameters and new mutation strategies to enhance its search capability. In addition, it incorporates an adaptive COA (ACOA) with dynamic adjustments to the switching probability parameter, effectively balancing exploration and exploitation. To evaluate the effectiveness of HADECO, its performance is initially assessed using CEC'22 benchmark functions. HADECO is evaluated against several contemporary algorithms using the Wilcoxon signed rank test (WSRT) and the Friedman test (FT) to integrate the results. The findings highlight HADECO's superior optimization abilities, demonstrated by its lowest average Friedman ranking of 1.08. Furthermore, the HADECO-based MTIS method is evaluated using MRI images for knee and CT scans for brain intracranial hemorrhage (ICH). Quantitative results in brain hemorrhage image segmentation show that the proposed method achieves a superior average peak signal-to-noise ratio (PSNR) and feature similarity index (FSIM) of 1.5 and 1.7 at the 6-level threshold. In knee image segmentation, it attains an average PSNR and FSIM of 1.3 and 1.2 at the 5-level threshold, demonstrating the method's effectiveness in solving image segmentation problems.
Collapse
Affiliation(s)
- Reham R Mostafa
- Big Data Mining and Multimedia Research Group, Centre for Data Analytics and Cybersecurity (CDAC), Research Institute of Sciences and Engineering (RISE), University of Sharjah, Sharjah 27272, United Arab Emirates; Information Systems Department, Faculty of Computers and Information Sciences, Mansoura University, Mansoura 35516, Egypt.
| | - Ahmed M Khedr
- Computer Science Department, University of Sharjah, Sharjah 27272, United Arab Emirates.
| | - Zaher Al Aghbari
- Computer Science Department, University of Sharjah, Sharjah 27272, United Arab Emirates.
| | - Imad Afyouni
- Computer Science Department, University of Sharjah, Sharjah 27272, United Arab Emirates.
| | - Ibrahim Kamel
- Electrical & Computer Engineering Department, University of Sharjah, Sharjah 27272, United Arab Emirates.
| | - Naveed Ahmed
- Computer Science Department, University of Sharjah, Sharjah 27272, United Arab Emirates.
| |
Collapse
|
3
|
Arafa MA, Omar I, Farhat KH, Elshinawy M, Khan F, Alkhathami FA, Mokhtar A, Althunayan A, Rabah DM, Badawy AHA. A Comparison of Systematic, Targeted, and Combined Biopsy Using Machine Learning for Prediction of Prostate Cancer Risk: A Multi-Center Study. Med Princ Pract 2024:1-10. [PMID: 39047698 DOI: 10.1159/000540425] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/17/2024] [Accepted: 07/17/2024] [Indexed: 07/27/2024] Open
Abstract
OBJECTIVES The aims of the study were to construct a new prognostic prediction model for detecting prostate cancer (PCa) patients using machine-learning (ML) techniques and to compare those models across systematic and target biopsy detection techniques. METHODS The records of the two main hospitals in Riyadh, Saudi Arabia, were analyzed for data on diagnosed PCa from 2019 to 2023. Four ML algorithms were utilized for the prediction and classification of PCa. RESULTS A total of 528 patients with prostate-specific antigen (PSA) greater than 3.5 ng/mL who had undergone transrectal ultrasound-guided prostate biopsy were evaluated. The total number of confirmed PCa cases was 234. Age, prostate volume, PSA, body mass index (BMI), multiparametric magnetic resonance imaging (mpMRI) score, number of regions of interest detected in MRI, and the diameter of the largest size lesion were significantly associated with PCa. Random Forest (RF) and XGBoost (XGB) (ML algorithms) accurately predicted PCa. Yet, their performance for classification and prediction of PCa was higher and more accurate for cases detected by targeted and combined biopsy (systematic and targeted together) compared to systematic biopsy alone. F1, the area under the curve (AUC), and the accuracy of XGB and RF models for targeted biopsy and combined biopsy ranged from 0.94 to 0.97 compared to the AUC of systematic biopsy for RF and XGB algorithms, respectively. CONCLUSIONS The RF model generated and presented an excellent prediction capability for the risk of PCa detected by targeted and combined biopsy compared to systematic biopsy alone. ML models can prevent missed PCa diagnoses by serving as a screening tool.
Collapse
Affiliation(s)
- Mostafa A Arafa
- The Cancer Research Chair, Surgery Department, College of Medicine, King Saud University, Riyadh, Saudi Arabia
- Department of Epidemiology, High Institute of Public Health, Alexandria University, Alexandria, Egypt
| | - Islam Omar
- Klipsch School of Electrical and Computer Engineering, New Mexico State University, Las Cruces, New Mexico, USA
| | - Karim H Farhat
- The Cancer Research Chair, Surgery Department, College of Medicine, King Saud University, Riyadh, Saudi Arabia
| | - Mona Elshinawy
- Engineering Technology and Surveying Engineering Department, New Mexico State University, Las Cruces, New Mexico, USA
| | - Farrukh Khan
- Department of Surgery, College of Medicine, King Saud University, Riyadh, Saudi Arabia
| | - Faisal A Alkhathami
- Department of Surgery, College of Medicine, King Saud University, Riyadh, Saudi Arabia
| | - Alaa Mokhtar
- Department of Urology, King Faisal Specialist Hospital and Research Center, Riyadh, Saudi Arabia
| | - Abdulaziz Althunayan
- The Cancer Research Chair, Surgery Department, College of Medicine, King Saud University, Riyadh, Saudi Arabia
- Department of Surgery, College of Medicine, King Saud University, Riyadh, Saudi Arabia
| | - Danny M Rabah
- The Cancer Research Chair, Surgery Department, College of Medicine, King Saud University, Riyadh, Saudi Arabia
- Department of Surgery, College of Medicine, King Saud University, Riyadh, Saudi Arabia
- Department of Urology, King Faisal Specialist Hospital and Research Center, Riyadh, Saudi Arabia
| | - Abdel-Hameed A Badawy
- Klipsch School of Electrical and Computer Engineering, New Mexico State University, Las Cruces, New Mexico, USA
| |
Collapse
|
4
|
Güngör E, Vehbi H, Cansın A, Ertan MB. Achieving high accuracy in meniscus tear detection using advanced deep learning models with a relatively small data set. Knee Surg Sports Traumatol Arthrosc 2024. [PMID: 39015056 DOI: 10.1002/ksa.12369] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/12/2024] [Revised: 07/06/2024] [Accepted: 07/06/2024] [Indexed: 07/18/2024]
Abstract
PURPOSE This study aims to evaluate the effectiveness of advanced deep learning models, specifically YOLOv8 and EfficientNetV2, in detecting meniscal tears on magnetic resonance imaging (MRI) using a relatively small data set. METHOD Our data set consisted of MRI studies from 642 knees-two orthopaedic surgeons labelled and annotated the MR images. The training pipeline included MRI scans of these knees. It was divided into two stages: initially, a deep learning algorithm called YOLO was employed to identify the meniscus location, and subsequently, the EfficientNetV2 deep learning architecture was utilized to detect meniscal tears. A concise report indicating the location and detection of a torn meniscus is provided at the end. RESULT The YOLOv8 model achieved mean average precision at 50% threshold (mAP@50) scores of 0.98 in the sagittal view and 0.985 in the coronal view. Similarly, the EfficientNetV2 model obtained area under the curve scores of 0.97 and 0.98 in the sagittal and coronal views, respectively. These outstanding results demonstrate exceptional performance in meniscus localization and tear detection. CONCLUSION Despite a relatively small data set, state-of-the-art models like YOLOv8 and EfficientNetV2 yielded promising results. This artificial intelligence system enhances meniscal injury diagnosis by generating instant structured reports, facilitating faster image interpretation and reducing physician workload. LEVEL OF EVIDENCE Level III.
Collapse
Affiliation(s)
- Erdal Güngör
- Department of Orthopaedics and Traumatology, Medipol University Esenler Hospital, Istanbul, Turkey
| | - Husam Vehbi
- Department of Radiology, Medipol University Esenler Hospital, Istanbul, Turkey
| | - Ahmetcan Cansın
- International School of Medicine, İstanbul Medipol University, Istanbul, Turkey
| | - Mehmet Batu Ertan
- Department of Orthopaedics and Traumatology, Medicana International Ankara Hospital, Ankara, Turkey
| |
Collapse
|
5
|
Jiang K, Xie Y, Zhang X, Zhang X, Zhou B, Li M, Chen Y, Hu J, Zhang Z, Chen S, Yu K, Qiu C, Zhang X. Fully and Weakly Supervised Deep Learning for Meniscal Injury Classification, and Location Based on MRI. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024:10.1007/s10278-024-01198-4. [PMID: 39020156 DOI: 10.1007/s10278-024-01198-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/17/2024] [Revised: 06/14/2024] [Accepted: 07/08/2024] [Indexed: 07/19/2024]
Abstract
Meniscal injury is a common cause of knee joint pain and a precursor to knee osteoarthritis (KOA). The purpose of this study is to develop an automatic pipeline for meniscal injury classification and localization using fully and weakly supervised networks based on MRI images. In this retrospective study, data were from the osteoarthritis initiative (OAI). The MR images were reconstructed using a sagittal intermediate-weighted fat-suppressed turbo spin-echo sequence. (1) We used 130 knees from the OAI to develop the LGSA-UNet model which fuses the features of adjacent slices and adjusts the blocks in Siam to enable the central slice to obtain rich contextual information. (2) One thousand seven hundred and fifty-six knees from the OAI were included to establish segmentation and classification models. The segmentation model achieved a DICE coefficient ranging from 0.84 to 0.93. The AUC values ranged from 0.85 to 0.95 in the binary models. The accuracy for the three types of menisci (normal, tear, and maceration) ranged from 0.60 to 0.88. Furthermore, 206 knees from the orthopedic hospital were used as an external validation data set to evaluate the performance of the model. The segmentation and classification models still performed well on the external validation set. To compare the diagnostic performances between the deep learning (DL) models and radiologists, the external validation sets were sent to two radiologists. The binary classification model outperformed the diagnostic performance of the junior radiologist (0.82-0.87 versus 0.74-0.88). This study highlights the potential of DL in knee meniscus segmentation and injury classification which can help improve diagnostic efficiency.
Collapse
Affiliation(s)
- Kexin Jiang
- Department of Medical Imaging, The Third Affiliated Hospital, Southern Medical University (Academy of Orthopedics Guangdong Province), 183 Zhongshan Ave W, Guangzhou, 510630, China
| | - Yuhan Xie
- School of Electronics and Communication Engineering, Sun Yat-sen University, Guangzhou, China
| | - Xintao Zhang
- Department of Medical Imaging, The Third Affiliated Hospital, Southern Medical University (Academy of Orthopedics Guangdong Province), 183 Zhongshan Ave W, Guangzhou, 510630, China
| | - Xinru Zhang
- Department of Medical Imaging, The Third Affiliated Hospital, Southern Medical University (Academy of Orthopedics Guangdong Province), 183 Zhongshan Ave W, Guangzhou, 510630, China
| | - Beibei Zhou
- Department of Medical Imaging, The Third Affiliated Hospital, Southern Medical University (Academy of Orthopedics Guangdong Province), 183 Zhongshan Ave W, Guangzhou, 510630, China
| | - Mianwen Li
- Department of Medical Imaging, The Third Affiliated Hospital, Southern Medical University (Academy of Orthopedics Guangdong Province), 183 Zhongshan Ave W, Guangzhou, 510630, China
| | - Yanjun Chen
- Department of Medical Imaging, The Third Affiliated Hospital, Southern Medical University (Academy of Orthopedics Guangdong Province), 183 Zhongshan Ave W, Guangzhou, 510630, China
| | - Jiaping Hu
- Department of Medical Imaging, The Third Affiliated Hospital, Southern Medical University (Academy of Orthopedics Guangdong Province), 183 Zhongshan Ave W, Guangzhou, 510630, China
| | - Zhiyong Zhang
- School of Electronics and Communication Engineering, Sun Yat-sen University, Guangzhou, China
| | - Shaolong Chen
- School of Electronics and Communication Engineering, Sun Yat-sen University, Guangzhou, China
| | - Keyan Yu
- Department of Medical Imaging, The Third Affiliated Hospital, Southern Medical University (Academy of Orthopedics Guangdong Province), 183 Zhongshan Ave W, Guangzhou, 510630, China
| | - Changzhen Qiu
- School of Electronics and Communication Engineering, Sun Yat-sen University, Guangzhou, China.
| | - Xiaodong Zhang
- Department of Medical Imaging, The Third Affiliated Hospital, Southern Medical University (Academy of Orthopedics Guangdong Province), 183 Zhongshan Ave W, Guangzhou, 510630, China.
| |
Collapse
|
6
|
Yang R, Xiao T, Cheng Y, Li A, Qu J, Liang R, Bao S, Wang X, Wang J, Suo J, Luo Q, Dai Q. Sharing massive biomedical data at magnitudes lower bandwidth using implicit neural function. Proc Natl Acad Sci U S A 2024; 121:e2320870121. [PMID: 38959033 PMCID: PMC11252806 DOI: 10.1073/pnas.2320870121] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2023] [Accepted: 05/21/2024] [Indexed: 07/04/2024] Open
Abstract
Efficient storage and sharing of massive biomedical data would open up their wide accessibility to different institutions and disciplines. However, compressors tailored for natural photos/videos are rapidly limited for biomedical data, while emerging deep learning-based methods demand huge training data and are difficult to generalize. Here, we propose to conduct Biomedical data compRession with Implicit nEural Function (BRIEF) by representing the target data with compact neural networks, which are data specific and thus have no generalization issues. Benefiting from the strong representation capability of implicit neural function, BRIEF achieves 2[Formula: see text]3 orders of magnitude compression on diverse biomedical data at significantly higher fidelity than existing techniques. Besides, BRIEF is of consistent performance across the whole data volume, and supports customized spatially varying fidelity. BRIEF's multifold advantageous features also serve reliable downstream tasks at low bandwidth. Our approach will facilitate low-bandwidth data sharing and promote collaboration and progress in the biomedical field.
Collapse
Affiliation(s)
- Runzhao Yang
- Department of Automation, Tsinghua University, Beijing100084, China
- Institute of Brain and Cognitive Sciences, Tsinghua University, Beijing100084, China
- Shanghai Artificial Intelligence Laboratory, Shanghai200232, China
| | - Tingxiong Xiao
- Department of Automation, Tsinghua University, Beijing100084, China
| | - Yuxiao Cheng
- Department of Automation, Tsinghua University, Beijing100084, China
| | - Anan Li
- Britton Chance Center and MoE Key Laboratory for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan430074, China
- Huazhong University of Science and Technology-Suzhou Institute for Brainsmatics, Jiangsu Industrial Technology Research Institute, Suzhou215123, China
| | - Jinyuan Qu
- Department of Automation, Tsinghua University, Beijing100084, China
| | - Rui Liang
- School of Biomedical Engineering, Hainan University, Haikou570228, China
| | - Shengda Bao
- Britton Chance Center and MoE Key Laboratory for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan430074, China
| | - Xiaofeng Wang
- Britton Chance Center and MoE Key Laboratory for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan430074, China
| | - Jue Wang
- Department of Automation, Tsinghua University, Beijing100084, China
| | - Jinli Suo
- Department of Automation, Tsinghua University, Beijing100084, China
- Institute of Brain and Cognitive Sciences, Tsinghua University, Beijing100084, China
- Shanghai Artificial Intelligence Laboratory, Shanghai200232, China
| | - Qingming Luo
- School of Biomedical Engineering, Hainan University, Haikou570228, China
| | - Qionghai Dai
- Department of Automation, Tsinghua University, Beijing100084, China
- Institute of Brain and Cognitive Sciences, Tsinghua University, Beijing100084, China
| |
Collapse
|
7
|
Li Y, El Habib Daho M, Conze PH, Zeghlache R, Le Boité H, Tadayoni R, Cochener B, Lamard M, Quellec G. A review of deep learning-based information fusion techniques for multimodal medical image classification. Comput Biol Med 2024; 177:108635. [PMID: 38796881 DOI: 10.1016/j.compbiomed.2024.108635] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2023] [Revised: 03/18/2024] [Accepted: 05/18/2024] [Indexed: 05/29/2024]
Abstract
Multimodal medical imaging plays a pivotal role in clinical diagnosis and research, as it combines information from various imaging modalities to provide a more comprehensive understanding of the underlying pathology. Recently, deep learning-based multimodal fusion techniques have emerged as powerful tools for improving medical image classification. This review offers a thorough analysis of the developments in deep learning-based multimodal fusion for medical classification tasks. We explore the complementary relationships among prevalent clinical modalities and outline three main fusion schemes for multimodal classification networks: input fusion, intermediate fusion (encompassing single-level fusion, hierarchical fusion, and attention-based fusion), and output fusion. By evaluating the performance of these fusion techniques, we provide insight into the suitability of different network architectures for various multimodal fusion scenarios and application domains. Furthermore, we delve into challenges related to network architecture selection, handling incomplete multimodal data management, and the potential limitations of multimodal fusion. Finally, we spotlight the promising future of Transformer-based multimodal fusion techniques and give recommendations for future research in this rapidly evolving field.
Collapse
Affiliation(s)
- Yihao Li
- LaTIM UMR 1101, Inserm, Brest, France; University of Western Brittany, Brest, France
| | - Mostafa El Habib Daho
- LaTIM UMR 1101, Inserm, Brest, France; University of Western Brittany, Brest, France.
| | | | - Rachid Zeghlache
- LaTIM UMR 1101, Inserm, Brest, France; University of Western Brittany, Brest, France
| | - Hugo Le Boité
- Sorbonne University, Paris, France; Ophthalmology Department, Lariboisière Hospital, AP-HP, Paris, France
| | - Ramin Tadayoni
- Ophthalmology Department, Lariboisière Hospital, AP-HP, Paris, France; Paris Cité University, Paris, France
| | - Béatrice Cochener
- LaTIM UMR 1101, Inserm, Brest, France; University of Western Brittany, Brest, France; Ophthalmology Department, CHRU Brest, Brest, France
| | - Mathieu Lamard
- LaTIM UMR 1101, Inserm, Brest, France; University of Western Brittany, Brest, France
| | | |
Collapse
|
8
|
Ozhinsky E, Liu F, Pedoia V, Majumdar S. Machine learning-based automated scan prescription of lumbar spine MRI acquisitions. Magn Reson Imaging 2024; 110:29-34. [PMID: 38574982 DOI: 10.1016/j.mri.2024.03.041] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2024] [Revised: 03/24/2024] [Accepted: 03/31/2024] [Indexed: 04/06/2024]
Abstract
PURPOSE High quality scan prescription that optimally covers the area of interest with scan planes aligned to relevant anatomical structures is crucial for error-free radiologic interpretation. The goal of this project was to develop a machine learning pipeline for oblique scan prescription that could be trained on localizer images and metadata from previously acquired MR exams. METHODS A novel Multislice Rotational Region-based Convolutional Neural Network (MS-R2CNN) architecture was developed. Based on this architecture, models for automated prescription sagittal lumbar spine acquisitions from axial, sagittal, and coronal localizer slices were trained. The automated prescription pipeline was integrated with the scanner console software and evaluated in experiments with healthy volunteers (N = 3) and patients with lower-back pain (N = 20). RESULTS Experiments in healthy volunteers demonstrated high accuracy of automated prescription in all subjects. There was good agreement between alignment and coverage of manual and automated prescriptions, as well as consistent views of the lumbar spine at different positions of the subjects within the scanner bore. In patients with lower-back pain, the generated prescription was applied in 18 cases (90% of the total number). None of the cases required major adjustment, while in 11 cases (55%) there were minor manual adjustments to the generated prescription. CONCLUSIONS This study demonstrates the ability of oriented object detection-based models to be trained to prescribe oblique lumbar spine MRI acquisitions without the need of manual annotation or feature engineering and the feasibility of using machine learning-based pipelines on the scanner for automated prescription of MRI acquisitions.
Collapse
Affiliation(s)
- Eugene Ozhinsky
- Department of Radiology and Biomedical Imaging, University of California San Francisco, 1700 4(th) St, San Francisco, CA 94143, United States.
| | - Felix Liu
- Department of Radiology and Biomedical Imaging, University of California San Francisco, 1700 4(th) St, San Francisco, CA 94143, United States.
| | - Valentina Pedoia
- Department of Radiology and Biomedical Imaging, University of California San Francisco, 1700 4(th) St, San Francisco, CA 94143, United States.
| | - Sharmila Majumdar
- Department of Radiology and Biomedical Imaging, University of California San Francisco, 1700 4(th) St, San Francisco, CA 94143, United States.
| |
Collapse
|
9
|
Belton N, Hagos MT, Lawlor A, Curran KM. Towards a unified approach for unsupervised brain MRI Motion Artefact Detection with few shot Anomaly Detection. Comput Med Imaging Graph 2024; 115:102391. [PMID: 38718561 DOI: 10.1016/j.compmedimag.2024.102391] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2023] [Revised: 03/19/2024] [Accepted: 04/26/2024] [Indexed: 06/03/2024]
Abstract
Automated Motion Artefact Detection (MAD) in Magnetic Resonance Imaging (MRI) is a field of study that aims to automatically flag motion artefacts in order to prevent the requirement for a repeat scan. In this paper, we identify and tackle the three current challenges in the field of automated MAD; (1) reliance on fully-supervised training, meaning they require specific examples of Motion Artefacts (MA), (2) inconsistent use of benchmark datasets across different works and use of private datasets for testing and training of newly proposed MAD techniques and (3) a lack of sufficiently large datasets for MRI MAD. To address these challenges, we demonstrate how MAs can be identified by formulating the problem as an unsupervised Anomaly Detection (AD) task. We compare the performance of three State-of-the-Art AD algorithms DeepSVDD, Interpolated Gaussian Descriptor and FewSOME on two open-source Brain MRI datasets on the task of MAD and MA severity classification, with FewSOME achieving a MAD AUC >90% on both datasets and a Spearman Rank Correlation Coefficient of 0.8 on the task of MA severity classification. These models are trained in the few shot setting, meaning large Brain MRI datasets are not required to build robust MAD algorithms. This work also sets a standard protocol for testing MAD algorithms on open-source benchmark datasets. In addition to addressing these challenges, we demonstrate how our proposed 'anomaly-aware' scoring function improves FewSOME's MAD performance in the setting where one and two shots of the anomalous class are available for training. Code available at https://github.com/niamhbelton/Unsupervised-Brain-MRI-Motion-Artefact-Detection/.
Collapse
Affiliation(s)
- Niamh Belton
- Science Foundation Ireland Centre for Research Training in Machine Learning, Ireland; School of Medicine, University College Dublin, Ireland.
| | - Misgina Tsighe Hagos
- Science Foundation Ireland Centre for Research Training in Machine Learning, Ireland; School of Computer Science, University College Dublin, Ireland
| | - Aonghus Lawlor
- School of Computer Science, University College Dublin, Ireland; Insight Centre for Data Analytics, University College Dublin, Dublin, Ireland
| | - Kathleen M Curran
- Science Foundation Ireland Centre for Research Training in Machine Learning, Ireland; School of Medicine, University College Dublin, Ireland
| |
Collapse
|
10
|
Yin R, Chen H, Wang C, Qin C, Tao T, Hao Y, Wu R, Jiang Y, Gui J. Transformer-Based Multilabel Deep Learning Model Is Efficient for Detecting Ankle Lateral and Medial Ligament Injuries on Magnetic Resonance Imaging and Improving Clinicians' Diagnostic Accuracy for Rotational Chronic Ankle Instability. Arthroscopy 2024:S0749-8063(24)00409-2. [PMID: 38876447 DOI: 10.1016/j.arthro.2024.05.027] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/10/2023] [Revised: 05/11/2024] [Accepted: 05/19/2024] [Indexed: 06/16/2024]
Abstract
PURPOSE To develop a deep learning (DL) model that can simultaneously detect lateral and medial collateral ligament injuries of the ankle, aiding in the diagnosis of chronic ankle instability (CAI), and assess its impact on clinicians' diagnostic performance. METHODS DL models were developed and externally validated on retrospectively collected ankle magnetic resonance imaging (MRI) between April 2016 and March 2022 respectively at 3 centers. Included patients had confirmed diagnoses of CAI through arthroscopy, as well as individuals who had undergone MRI and physical examinations that ruled out ligament injuries. DL models were constructed based on a multilabel paradigm. A transformer-based multilabel DL model (AnkleNet) was developed and compared with 4 convolution neural network (CNN) models. Subsequently, a reader study was conducted to evaluate the impact of model assistance on clinicians when diagnosing challenging cases: identifying rotational CAI (RCAI). Diagnostic performance was assessed using area under the receiver operating characteristic curve (AUC). RESULTS Our transformer-based model achieved AUCs of 0.910 and 0.892 for detecting lateral and medial collateral ligament injury, respectively, both of which were significantly higher than those of CNN-based models (all P < .001). In terms of further CAI diagnosis, there was a macro-average AUC of 0.870 and a balanced accuracy of 0.805. The reader study indicated that incorporation with our model significantly enhanced the diagnostic accuracy of clinicians (P = .042), particularly junior clinicians, and led to a reduction in diagnostic variability. The code of the model can be accessed at https://github.com/ChiariRay/AnkleNet. CONCLUSIONS Our transformer-based model was able to detect lateral and medial collateral ligament injuries based on MRI and outperformed CNN-based models, demonstrating a promising performance in diagnosing CAI, especially patients with RCAI. CLINICAL RELEVANCE Developing such an algorithm can improve the diagnostic performance of clinicians, aiding in identifying patients who would benefit from arthroscopy, such as patients with RCAI.
Collapse
Affiliation(s)
- Rui Yin
- Department of Sports Medicine and Joint Surgery, Nanjing First Hospital, Nanjing Medical University, Nanjing, China
| | - Hao Chen
- Department of Clinical Neuroscience, Cambridge University, Cambridge, U.K; School of Computer Science, University of Birmingham, Birmingham, U.K
| | - Changjiang Wang
- Department of Sports Medicine and Joint Surgery, Nanjing First Hospital, Nanjing Medical University, Nanjing, China
| | - Chaoren Qin
- Department of Sports Medicine and Joint Surgery, Nanjing First Hospital, Nanjing Medical University, Nanjing, China
| | - Tianqi Tao
- Department of Sports Medicine and Joint Surgery, Nanjing First Hospital, Nanjing Medical University, Nanjing, China
| | - Yunjia Hao
- Department of Sports Medicine and Joint Surgery, Nanjing First Hospital, Nanjing Medical University, Nanjing, China; Department of Hand and Foot Microsurgery, Xuzhou Central Hospital, Xuzhou, China
| | - Rui Wu
- Department of Sports Medicine and Joint Surgery, Nanjing First Hospital, Nanjing Medical University, Nanjing, China; Department of Orthopedics, The Second People's Hospital of Lianyungang, Lianyungang, China
| | - Yiqiu Jiang
- Department of Sports Medicine and Joint Surgery, Nanjing First Hospital, Nanjing Medical University, Nanjing, China
| | - Jianchao Gui
- Department of Sports Medicine and Joint Surgery, Nanjing First Hospital, Nanjing Medical University, Nanjing, China.
| |
Collapse
|
11
|
Zhao X, Yang T, Li B, Yang A, Yan Y, Jiao C. DiffGAN: An adversarial diffusion model with local transformer for MRI reconstruction. Magn Reson Imaging 2024; 109:108-119. [PMID: 38492787 DOI: 10.1016/j.mri.2024.03.017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2023] [Revised: 03/13/2024] [Accepted: 03/13/2024] [Indexed: 03/18/2024]
Abstract
Magnetic resonance imaging (MRI) is non-invasive and crucial for clinical diagnosis, but it has long acquisition time and aliasing artifacts. Accelerated imaging techniques can effectively reduce the scanning time of MRI, thereby decreasing the anxiety and discomfort of patients. Vision Transformer (ViT) based methods have greatly improved MRI image reconstruction, but their computational complexity and memory requirements for the self-attention mechanism grow quadratically with image resolution, which limits their use for high resolution images. In addition, the current generative adversarial networks in MRI reconstruction are difficult to train stably. To address these problems, we propose a Local Vision Transformer (LVT) based adversarial Diffusion model (Diff-GAN) for accelerating MRI reconstruction. We employ a generative adversarial network (GAN) as the reverse diffusion model to enable large diffusion steps. In the forward diffusion module, we use a diffusion process to generate Gaussian mixture distribution noise, which mitigates the gradient vanishing issue in GAN training. This network leverages the LVT module with the local self-attention, which can capture high-quality local features and detailed information. We evaluate our method on four datasets: IXI, MICCAI 2013, MRNet and FastMRI, and demonstrate that Diff-GAN can outperform several state-of-the-art GAN-based methods for MRI reconstruction.
Collapse
Affiliation(s)
- Xiang Zhao
- School of Information Science and Engineering, Henan University of Technology, Zhengzhou 450001, China
| | - Tiejun Yang
- School of Artificial Intelligence and Big Data, Henan University of Technology, Zhengzhou 450001, China; Key Laboratory of Grain Information Processing and Control (HAUT), Ministry of Education, Zhengzhou, China; Henan Key Laboratory of Grain Photoelectric Detection and Control (HAUT), Zhengzhou, Henan, China.
| | - Bingjie Li
- School of Information Science and Engineering, Henan University of Technology, Zhengzhou 450001, China
| | - Aolin Yang
- School of Information Science and Engineering, Henan University of Technology, Zhengzhou 450001, China
| | - Yanghui Yan
- School of Information Science and Engineering, Henan University of Technology, Zhengzhou 450001, China
| | - Chunxia Jiao
- School of Information Science and Engineering, Henan University of Technology, Zhengzhou 450001, China
| |
Collapse
|
12
|
Chen Y, Zhang S, Tang N, George DM, Huang T, Tang J. Using Google web search to analyze and evaluate the application of ChatGPT in femoroacetabular impingement syndrome. Front Public Health 2024; 12:1412063. [PMID: 38883198 PMCID: PMC11176516 DOI: 10.3389/fpubh.2024.1412063] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2024] [Accepted: 05/23/2024] [Indexed: 06/18/2024] Open
Abstract
Background Chat Generative Pre-trained Transformer (ChatGPT) is a new machine learning tool that allows patients to access health information online, specifically compared to Google, the most commonly used search engine in the United States. Patients can use ChatGPT to better understand medical issues. This study compared the two search engines based on: (i) frequently asked questions (FAQs) about Femoroacetabular Impingement Syndrome (FAI), (ii) the corresponding answers to these FAQs, and (iii) the most FAQs yielding a numerical response. Purpose To assess the suitability of ChatGPT as an online health information resource for patients by replicating their internet searches. Study design Cross-sectional study. Methods The same keywords were used to search the 10 most common questions about FAI on both Google and ChatGPT. The responses from both search engines were recorded and analyzed. Results Of the 20 questions, 8 (40%) were similar. Among the 10 questions searched on Google, 7 were provided by a medical practice. For numerical questions, there was a notable difference in answers between Google and ChatGPT for 3 out of the top 5 most common questions (60%). Expert evaluation indicated that 67.5% of experts were satisfied or highly satisfied with the accuracy of ChatGPT's descriptions of both conservative and surgical treatment options for FAI. Additionally, 62.5% of experts were satisfied or highly satisfied with the safety of the information provided. Regarding the etiology of FAI, including cam and pincer impingements, 52.5% of experts expressed satisfaction or high satisfaction with ChatGPT's explanations. Overall, 62.5% of experts affirmed that ChatGPT could serve effectively as a reliable medical resource for initial information retrieval. Conclusion This study confirms that ChatGPT, despite being a new tool, shows significant potential as a supplementary resource for health information on FAI. Expert evaluations commend its capacity to provide accurate and comprehensive responses, valued by medical professionals for relevance and safety. Nonetheless, continuous improvements in its medical content's depth and precision are recommended for ongoing reliability. While ChatGPT offers a promising alternative to traditional search engines, meticulous validation is imperative before it can be fully embraced as a trusted medical resource.
Collapse
Affiliation(s)
- Yifan Chen
- Orthopaedic Department, The Second Xiangya Hospital of Central South University, Changsha, China
| | - Shengqun Zhang
- Orthopaedic Department, The Second Xiangya Hospital of Central South University, Changsha, China
| | - Ning Tang
- Orthopaedic Department, The Third Xiangya Hospital of Central South University, Changsha, China
| | | | - Tianlong Huang
- Orthopaedic Department, The Second Xiangya Hospital of Central South University, Changsha, China
| | - JinPing Tang
- Department of Orthopaedics, The Third People's Hospital of Chenzhou, Chenzhou, Hunan, China
| |
Collapse
|
13
|
Botnari A, Kadar M, Patrascu JM. A Comprehensive Evaluation of Deep Learning Models on Knee MRIs for the Diagnosis and Classification of Meniscal Tears: A Systematic Review and Meta-Analysis. Diagnostics (Basel) 2024; 14:1090. [PMID: 38893617 PMCID: PMC11172202 DOI: 10.3390/diagnostics14111090] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2024] [Revised: 05/19/2024] [Accepted: 05/21/2024] [Indexed: 06/21/2024] Open
Abstract
OBJECTIVES This study delves into the cutting-edge field of deep learning techniques, particularly deep convolutional neural networks (DCNNs), which have demonstrated unprecedented potential in assisting radiologists and orthopedic surgeons in precisely identifying meniscal tears. This research aims to evaluate the effectiveness of deep learning models in recognizing, localizing, describing, and categorizing meniscal tears in magnetic resonance images (MRIs). MATERIALS AND METHODS This systematic review was rigorously conducted, strictly following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Extensive searches were conducted on MEDLINE (PubMed), Web of Science, Cochrane Library, and Google Scholar. All identified articles underwent a comprehensive risk of bias analysis. Predictive performance values were either extracted or calculated for quantitative analysis, including sensitivity and specificity. The meta-analysis was performed for all prediction models that identified the presence and location of meniscus tears. RESULTS This study's findings underscore that a range of deep learning models exhibit robust performance in detecting and classifying meniscal tears, in one case surpassing the expertise of musculoskeletal radiologists. Most studies in this review concentrated on identifying tears in the medial or lateral meniscus and even precisely locating tears-whether in the anterior or posterior horn-with exceptional accuracy, as demonstrated by AUC values ranging from 0.83 to 0.94. CONCLUSIONS Based on these findings, deep learning models have showcased significant potential in analyzing knee MR images by learning intricate details within images. They offer precise outcomes across diverse tasks, including segmenting specific anatomical structures and identifying pathological regions. Contributions: This study focused exclusively on DL models for identifying and localizing meniscus tears. It presents a meta-analysis that includes eight studies for detecting the presence of a torn meniscus and a meta-analysis of three studies with low heterogeneity that localize and classify the menisci. Another novelty is the analysis of arthroscopic surgery as ground truth. The quality of the studies was assessed against the CLAIM checklist, and the risk of bias was determined using the QUADAS-2 tool.
Collapse
Affiliation(s)
- Alexei Botnari
- Department of Orthopedics, Faculty of Medicine, “Victor Babes” University of Medicine and Pharmacy, 300041 Timisoara, Romania
| | - Manuella Kadar
- Department of Computer Science, Faculty of Informatics and Engineering, “1 Decembrie 1918” University of Alba Iulia, 510009 Alba Iulia, Romania
| | - Jenel Marian Patrascu
- Department of Orthopedics-Traumatology, Faculty of Medicine, “Victor Babes” University of Medicine and Pharmacy, 300041 Timisoara, Romania;
| |
Collapse
|
14
|
Venkatesh K, Mutasa S, Moore F, Sulam J, Yi PH. Gradient-Based Saliency Maps Are Not Trustworthy Visual Explanations of Automated AI Musculoskeletal Diagnoses. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024:10.1007/s10278-024-01136-4. [PMID: 38710971 DOI: 10.1007/s10278-024-01136-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/08/2024] [Revised: 04/30/2024] [Accepted: 05/01/2024] [Indexed: 05/08/2024]
Abstract
Saliency maps are popularly used to "explain" decisions made by modern machine learning models, including deep convolutional neural networks (DCNNs). While the resulting heatmaps purportedly indicate important image features, their "trustworthiness," i.e., utility and robustness, has not been evaluated for musculoskeletal imaging. The purpose of this study was to systematically evaluate the trustworthiness of saliency maps used in disease diagnosis on upper extremity X-ray images. The underlying DCNNs were trained using the Stanford MURA dataset. We studied four trustworthiness criteria-(1) localization accuracy of abnormalities, (2) repeatability, (3) reproducibility, and (4) sensitivity to underlying DCNN weights-across six different gradient-based saliency methods (Grad-CAM (GCAM), gradient explanation (GRAD), integrated gradients (IG), Smoothgrad (SG), smooth IG (SIG), and XRAI). Ground-truth was defined by the consensus of three fellowship-trained musculoskeletal radiologists who each placed bounding boxes around abnormalities on a holdout saliency test set. Compared to radiologists, all saliency methods showed inferior localization (AUPRCs: 0.438 (SG)-0.590 (XRAI); average radiologist AUPRC: 0.816), repeatability (IoUs: 0.427 (SG)-0.551 (IG); average radiologist IOU: 0.613), and reproducibility (IoUs: 0.250 (SG)-0.502 (XRAI); average radiologist IOU: 0.613) on abnormalities such as fractures, orthopedic hardware insertions, and arthritis. Five methods (GCAM, GRAD, IG, SG, XRAI) passed the sensitivity test. Ultimately, no saliency method met all four trustworthiness criteria; therefore, we recommend caution and rigorous evaluation of saliency maps prior to their clinical use.
Collapse
Affiliation(s)
- Kesavan Venkatesh
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA
| | - Simukayi Mutasa
- University of Maryland Medical Intelligent Imaging (UM2ii) Center, Department of Diagnostic Radiology and Nuclear Medicine, University of Maryland School of Medicine, 520 W Lombard St, Baltimore, MD, USA
| | - Fletcher Moore
- University of Maryland Medical Intelligent Imaging (UM2ii) Center, Department of Diagnostic Radiology and Nuclear Medicine, University of Maryland School of Medicine, 520 W Lombard St, Baltimore, MD, USA
| | - Jeremias Sulam
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA
| | - Paul H Yi
- University of Maryland Medical Intelligent Imaging (UM2ii) Center, Department of Diagnostic Radiology and Nuclear Medicine, University of Maryland School of Medicine, 520 W Lombard St, Baltimore, MD, USA.
| |
Collapse
|
15
|
Wang M, Yu C, Li M, Zhang X, Jiang K, Zhang Z, Zhang X. One-stop detection of anterior cruciate ligament injuries on magnetic resonance imaging using deep learning with multicenter validation. Quant Imaging Med Surg 2024; 14:3405-3416. [PMID: 38720839 PMCID: PMC11074745 DOI: 10.21037/qims-23-1539] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2023] [Accepted: 03/14/2024] [Indexed: 05/12/2024]
Abstract
Background Anterior cruciate ligament (ACL) injuries are closely associated with knee osteoarthritis (OA). However, diagnosing ACL injuries based on knee magnetic resonance imaging (MRI) has been subjective and time-consuming for clinical doctors. Therefore, we aimed to devise a deep learning (DL) model leveraging MRI to enable a comprehensive and automated approach for the detection of ACL injuries. Methods A retrospective study was performed extracting data from the Osteoarthritis Initiative (OAI). A total of 1,589 knees (comprising 1,443 intact, 90 with partial tears, and 56 with full tears) were enrolled to construct the classification model. This one-stop detection pipeline was developed using a tailored YOLOv5m architecture and a ResNet-18 convolutional neural network (CNN) to facilitate tasks based on sagittal 2-dimensional (2D) intermediate-weighted fast spin-echo sequence at 3.0T. To ensure the reliability and robustness of the classification system, it was subjected to external validation across 3 distinct datasets. The accuracy, sensitivity, specificity, and the mean average precision (mAP) were utilized as the evaluation metric for the model performance by employing a 5-fold cross-validation approach. The radiologist's interpretations were employed as the reference for conducting the evaluation. Results The localization model demonstrated an accuracy of 0.89 and a sensitivity of 0.93, achieving a mAP score of 0.96. The classification model demonstrated strong performance in detecting intact, partial tears, and full tears at the optimal threshold on the internal dataset, with sensitivities of 0.941, 0.833, and 0.929, specificities of 0.925, 0.947, and 0.991, and accuracies of 0.940, 0.941, and 0.989, respectively. In comparison, on a subset consisting of 171 randomly selected knees from the OAI, the radiologists demonstrated a sensitivity ranging between 0.660 and 1.000, specificity ranging between 0.691 and 1.000, and accuracy ranging between 0.689 and 1.000. On a subset consisting of 170 randomly selected knees from the Chinese dataset, the radiologists exhibited a sensitivity ranging between 0.711 and 0.948, specificity ranging between 0.768 and 0.977, and accuracy ranging between 0.683 and 0.917. After retraining, the model achieved sensitivities ranging between 0.630 and 0.961, specificities ranging between 0.860 and 0.961, and accuracies ranging between 0.832 and 0.951, respectively, on the external validation dataset. Conclusions The proposed model utilizing knee MRI showcases robust performance in the domains of ACL localization and classification.
Collapse
Affiliation(s)
- Mei Wang
- Department of Medical Imaging, The Third Affiliated Hospital of Southern Medical University (Academy of Orthopedics Guangdong Province), Guangzhou, China
- Department of Radiology, Guangzhou First People’s Hospital, Guangzhou, China
| | - Congjing Yu
- School of Electronics and Communication Engineering, Sun Yat-sen University, Shenzhen, China
| | - Mianwen Li
- Department of Medical Imaging, The Third Affiliated Hospital of Southern Medical University (Academy of Orthopedics Guangdong Province), Guangzhou, China
| | - Xinru Zhang
- Department of Medical Imaging, The Third Affiliated Hospital of Southern Medical University (Academy of Orthopedics Guangdong Province), Guangzhou, China
| | - Kexin Jiang
- Department of Medical Imaging, The Third Affiliated Hospital of Southern Medical University (Academy of Orthopedics Guangdong Province), Guangzhou, China
| | - Zhiyong Zhang
- School of Electronics and Communication Engineering, Sun Yat-sen University, Shenzhen, China
| | - Xiaodong Zhang
- Department of Medical Imaging, The Third Affiliated Hospital of Southern Medical University (Academy of Orthopedics Guangdong Province), Guangzhou, China
| |
Collapse
|
16
|
Zhang T, Wei D, Zhu M, Gu S, Zheng Y. Self-supervised learning for medical image data with anatomy-oriented imaging planes. Med Image Anal 2024; 94:103151. [PMID: 38527405 DOI: 10.1016/j.media.2024.103151] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2023] [Revised: 12/29/2023] [Accepted: 03/20/2024] [Indexed: 03/27/2024]
Abstract
Self-supervised learning has emerged as a powerful tool for pretraining deep networks on unlabeled data, prior to transfer learning of target tasks with limited annotation. The relevance between the pretraining pretext and target tasks is crucial to the success of transfer learning. Various pretext tasks have been proposed to utilize properties of medical image data (e.g., three dimensionality), which are more relevant to medical image analysis than generic ones for natural images. However, previous work rarely paid attention to data with anatomy-oriented imaging planes, e.g., standard cardiac magnetic resonance imaging views. As these imaging planes are defined according to the anatomy of the imaged organ, pretext tasks effectively exploiting this information can pretrain the networks to gain knowledge on the organ of interest. In this work, we propose two complementary pretext tasks for this group of medical image data based on the spatial relationship of the imaging planes. The first is to learn the relative orientation between the imaging planes and implemented as regressing their intersecting lines. The second exploits parallel imaging planes to regress their relative slice locations within a stack. Both pretext tasks are conceptually straightforward and easy to implement, and can be combined in multitask learning for better representation learning. Thorough experiments on two anatomical structures (heart and knee) and representative target tasks (semantic segmentation and classification) demonstrate that the proposed pretext tasks are effective in pretraining deep networks for remarkably boosted performance on the target tasks, and superior to other recent approaches.
Collapse
Affiliation(s)
- Tianwei Zhang
- School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China
| | - Dong Wei
- Jarvis Research Center, Tencent YouTu Lab, Shenzhen 518057, China
| | - Mengmeng Zhu
- School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China
| | - Shi Gu
- School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China.
| | - Yefeng Zheng
- Jarvis Research Center, Tencent YouTu Lab, Shenzhen 518057, China
| |
Collapse
|
17
|
Berenguer AD, Kvasnytsia M, Bossa MN, Mukherjee T, Deligiannis N, Sahli H. Semi-supervised medical image classification via distance correlation minimization and graph attention regularization. Med Image Anal 2024; 94:103107. [PMID: 38401269 DOI: 10.1016/j.media.2024.103107] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2023] [Revised: 12/11/2023] [Accepted: 02/13/2024] [Indexed: 02/26/2024]
Abstract
We propose a novel semi-supervised learning method to leverage unlabeled data alongside minimal annotated data and improve medical imaging classification performance in realistic scenarios with limited labeling budgets to afford data annotations. Our method introduces distance correlation to minimize correlations between feature representations from different views of the same image encoded with non-coupled deep neural networks architectures. In addition, it incorporates a data-driven graph-attention based regularization strategy to model affinities among images within the unlabeled data by exploiting their inherent relational information in the feature space. We conduct extensive experiments on four medical imaging benchmark data sets involving X-ray, dermoscopic, magnetic resonance, and computer tomography imaging on single and multi-label medical imaging classification scenarios. Our experiments demonstrate the effectiveness of our method in achieving very competitive performance and outperforming several state-of-the-art semi-supervised learning methods. Furthermore, they confirm the suitability of distance correlation as a versatile dependence measure and the benefits of the proposed graph-attention based regularization for semi-supervised learning in medical imaging analysis.
Collapse
Affiliation(s)
- Abel Díaz Berenguer
- Vrije Universiteit Brussel (VUB), Department of Electronics and Informatics (ETRO), Pleinlaan 2, 1050 Brussels, Belgium.
| | - Maryna Kvasnytsia
- Vrije Universiteit Brussel (VUB), Department of Electronics and Informatics (ETRO), Pleinlaan 2, 1050 Brussels, Belgium
| | - Matías Nicolás Bossa
- Vrije Universiteit Brussel (VUB), Department of Electronics and Informatics (ETRO), Pleinlaan 2, 1050 Brussels, Belgium
| | - Tanmoy Mukherjee
- Vrije Universiteit Brussel (VUB), Department of Electronics and Informatics (ETRO), Pleinlaan 2, 1050 Brussels, Belgium
| | - Nikos Deligiannis
- Vrije Universiteit Brussel (VUB), Department of Electronics and Informatics (ETRO), Pleinlaan 2, 1050 Brussels, Belgium; Interuniversity Microelectronics Centre (IMEC), Kapeldreef 75, 3001 Heverlee, Belgium
| | - Hichem Sahli
- Vrije Universiteit Brussel (VUB), Department of Electronics and Informatics (ETRO), Pleinlaan 2, 1050 Brussels, Belgium; Interuniversity Microelectronics Centre (IMEC), Kapeldreef 75, 3001 Heverlee, Belgium
| |
Collapse
|
18
|
Reis FJJ, Alaiti RK, Vallio CS, Hespanhol L. Artificial intelligence and Machine Learning approaches in sports: Concepts, applications, challenges, and future perspectives. Braz J Phys Ther 2024; 28:101083. [PMID: 38838418 PMCID: PMC11215955 DOI: 10.1016/j.bjpt.2024.101083] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/12/2023] [Revised: 04/09/2024] [Accepted: 05/08/2024] [Indexed: 06/07/2024] Open
Abstract
BACKGROUND The development and application of Artificial Intelligence (AI) and Machine Learning (ML) in healthcare have gained attention as a promising and powerful resource to change the landscape of healthcare. The potential of these technologies for injury prediction, performance analysis, personalized training, and treatment comes with challenges related to the complexity of sports dynamics and the multidimensional aspects of athletic performance. OBJECTIVES We aimed to present the current state of AI and ML applications in sports science, specifically in the areas of injury prediction, performance enhancement, and rehabilitation. We also examine the challenges of incorporating AI and ML into sports and suggest directions for future research. METHOD We conducted a comprehensive literature review, focusing on publications related to AI and ML applications in sports. This review encompassed studies on injury prediction, performance analysis, and personalized training, emphasizing the AI and ML models applied in sports. RESULTS The findings highlight significant advancements in injury prediction accuracy, performance analysis precision, and the customization of training programs through AI and ML. However, future studies need to address challenges such as ethical considerations, data quality, interpretability of ML models, and the integration of complex data. CONCLUSION AI and ML may be useful for the prevention, detection, diagnosis, and treatment of health conditions. In this Masterclass paper, we introduce AI and ML concepts, outline recent breakthroughs in AI technologies and their applications, identify the challenges for further progress of AI systems, and discuss ethical issues, clinical and research opportunities, and future perspectives.
Collapse
Affiliation(s)
- Felipe J J Reis
- Department of Physical Therapy, Federal Institute of Rio de Janeiro, Rio de Janeiro, Brazil; Pain in Motion Research Group, Department of Physical Therapy, Human Physiology and Anatomy, Faculty of Physical Education & Physiotherapy, Vrije Universiteit Brussel, Brussels, Belgium; School of Physical and Occupational Therapy, McGill University, Montreal, Canada.
| | - Rafael Krasic Alaiti
- Nucleus of Neuroscience and Behavior and Nucleus of Applied Neuroscience, Universidade de Sao Paulo (USP), Sao Paulo, Brazil; Research, Technology, and Data Science Office, Grupo Superador, Sao Paulo, Brazil
| | - Caio Sain Vallio
- Health Innovation, Data Science, and MLOps Semantics, São Paulo, Brazil
| | - Luiz Hespanhol
- Department of Physical Therapy, Faculty of Medicine, University of Sao Paulo (USP), Sao Paulo, Brazil; Amsterdam Collaboration on Health & Safety in Sports, Department of Public and Occupational Health, Amsterdam Movement Sciences, Amsterdam University Medical Centers (UMC) location VU University Medical Center Amsterdam (VUmc), Amsterdam, the Netherlands
| |
Collapse
|
19
|
Fan F, Liu H, Dai X, Liu G, Liu J, Deng X, Peng Z, Wang C, Zhang K, Chen H, Yin C, Zhan M, Deng Z. Automated bone age assessment from knee joint by integrating deep learning and MRI-based radiomics. Int J Legal Med 2024; 138:927-938. [PMID: 38129687 DOI: 10.1007/s00414-023-03148-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2023] [Accepted: 12/09/2023] [Indexed: 12/23/2023]
Abstract
Bone age assessment (BAA) is a crucial task in clinical, forensic, and athletic fields. Since traditional age estimation methods are suffered from potential radiation damage, this study aimed to develop and evaluate a deep learning radiomics method based on multiparametric knee MRI for noninvasive and automatic BAA. This retrospective study enrolled 598 patients (age range,10.00-29.99 years) who underwent MR examinations of the knee joint (T1/T2*/PD-weighted imaging). Three-dimensional convolutional neural networks (3D CNNs) were trained to extract and fuse multimodal and multiscale MRI radiomic features for age estimation and compared to traditional machine learning models based on hand-crafted features. The age estimation error was greater in individuals aged 25-30 years; thus, this method may not be suitable for individuals over 25 years old. In the test set aged 10-25 years (n = 95), the 3D CNN (a fusion of T1WI, T2*WI, and PDWI) demonstrated the lowest mean absolute error of 1.32 ± 1.01 years, which is higher than that of other MRI modalities and the hand-crafted models. In the classification for 12-, 14-, 16-, and 18- year thresholds, accuracies and the areas under the ROC curves were all over 0.91 and 0.96, which is similar to the manual methods. Visualization of important features showed that 3D CNN estimated age by focusing on the epiphyseal plates. The deep learning radiomics method enables non-invasive and automated BAA from multimodal knee MR images. The use of 3D CNN and MRI-based radiomics has the potential to assist radiologists or medicolegists in age estimation.
Collapse
Affiliation(s)
- Fei Fan
- West China School of Basic Medical Sciences & Forensic Medicine, Sichuan University, Chengdu, 610041, People's Republic of China
| | - Han Liu
- College of Computer Science, Sichuan University, Chengdu, 610064, People's Republic of China
| | - Xinhua Dai
- Department of Laboratory Medicine, West China Hospital, Sichuan University, Chengdu, 610041, People's Republic of China
| | - Guangfeng Liu
- West China School of Basic Medical Sciences & Forensic Medicine, Sichuan University, Chengdu, 610041, People's Republic of China
| | - Junhong Liu
- West China School of Basic Medical Sciences & Forensic Medicine, Sichuan University, Chengdu, 610041, People's Republic of China
| | - Xiaodong Deng
- West China School of Basic Medical Sciences & Forensic Medicine, Sichuan University, Chengdu, 610041, People's Republic of China
| | - Zhao Peng
- Department of Radiology, West China Hospital, Sichuan University, Chengdu, 610041, People's Republic of China
| | - Chang Wang
- Department of Radiology, Anhui Provincial Children's Hospital, Hefei, 230054, People's Republic of China
| | - Kui Zhang
- West China School of Basic Medical Sciences & Forensic Medicine, Sichuan University, Chengdu, 610041, People's Republic of China
| | - Hu Chen
- College of Computer Science, Sichuan University, Chengdu, 610064, People's Republic of China
| | - Chuangao Yin
- Department of Radiology, Anhui Provincial Children's Hospital, Hefei, 230054, People's Republic of China.
| | - Mengjun Zhan
- West China School of Basic Medical Sciences & Forensic Medicine, Sichuan University, Chengdu, 610041, People's Republic of China.
| | - Zhenhua Deng
- West China School of Basic Medical Sciences & Forensic Medicine, Sichuan University, Chengdu, 610041, People's Republic of China.
| |
Collapse
|
20
|
Wang D, Yan Y. Improving inceptionV4 model based on fractional-order snow leopard optimization algorithm for diagnosing of ACL tears. Sci Rep 2024; 14:9843. [PMID: 38684782 PMCID: PMC11059154 DOI: 10.1038/s41598-024-60419-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2023] [Accepted: 04/23/2024] [Indexed: 05/02/2024] Open
Abstract
In the current research study, a new method is presented to diagnose Anterior Cruciate Ligament (ACL) tears by introducing an optimized version of the InceptionV4 model. Our proposed methodology utilizes a custom-made variant of the Snow Leopard Optimization Algorithm, known as the Fractional-order Snow Leopard Optimization Algorithm (FO-LOA), to extract essential features from knee magnetic resonance imaging (MRI) images. This results in a substantial improvement in the accuracy of ACL tear detection. By effectively extracting critical features from knee MRI images, our proposed methodology significantly enhances diagnostic accuracy, potentially reducing false negatives and false positives. The enhanced model based on FO-LOA underwent thorough testing using the MRNet dataset, demonstrating exceptional performance metrics including an accuracy rate of 98.00%, sensitivity of 98.00%, precision of 97.00%, specificity of 98.00%, F1-score of 98.00%, and Matthews Correlation Coefficient (MCC) of 88.00%. These findings surpass current methodologies like Convolutional Neural Network (CNN), Inception-v3, Deep Belief Networks and Improved Honey Badger Algorithm (DBN/IHBA), integration of the CNN with an Amended Cooking Training-based Optimizer version (CNN/ACTO), Self-Supervised Representation Learning (SSRL), signifying a significant breakthrough in ACL injury diagnosis. Using FO-SLO to optimize the InceptionV4 framework shows promise in improving the accuracy of ACL tear identification, enabling prompt and efficient treatment interventions.
Collapse
Affiliation(s)
- Delei Wang
- Zhejiang Pharmaceutical University, Ningbo, 315500, Zhejiang, China
| | - Yanqing Yan
- Guangdong University of Science and Technology, Dongguan, 523000, Guangdong, China.
| |
Collapse
|
21
|
Turgut N, Beyaz S. The 100 most cited articles in artificial intelligence related to orthopedics. Front Surg 2024; 11:1370335. [PMID: 38712339 PMCID: PMC11072182 DOI: 10.3389/fsurg.2024.1370335] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/14/2024] [Accepted: 04/04/2024] [Indexed: 05/08/2024] Open
Abstract
Background This bibliometric study aimed to identify and analyze the top 100 articles related to artificial intelligence in the field of orthopedics. Methods The articles were assessed based on their number of citations, publication years, countries, journals, authors, affiliations, and funding agencies. Additionally, they were analyzed in terms of their themes and objectives. Keyword co-occurrence, co-citation of authors, and co-citation of references analyses were conducted using VOSviewer (version 1.6.19). Results The number of citations of these articles ranged from 32 to 272, with six papers having more than 200 citations The years of 2019 (n: 37) and 2020 (n: 19) together constituted 56% of the list. The USA was the leading contributor country to this field (n: 61). The most frequently used keywords were "machine learning" (n: 26), "classification" (n: 18), "deep learning" (n: 16), "artificial intelligence" (n: 14), respectively. The most common themes were decision support (n: 25), fracture detection (n: 24), and osteoarthrtitis staging (n: 21). The majority of the studies were diagnostic in nature (n: 85), with only two articles focused on treatment. Conclusions This study provides valuable insights and presents the historical perspective of scientific development on artificial intelligence in the field of orthopedics. The literature in this field is expanding rapidly. Currently, research is generally done for diagnostic purposes and predominantly focused on decision support systems, fracture detection, and osteoarthritis classification.
Collapse
Affiliation(s)
- Necmettin Turgut
- Department of Orthopedics and Traumatology, Adana Turgut Noyan Research and Training Centre, Başkent University, Adana, Türkiye
| | | |
Collapse
|
22
|
Luo A, Gou S, Tong N, Liu B, Jiao L, Xu H, Wang Y, Ding T. Visual interpretable MRI fine grading of meniscus injury for intelligent assisted diagnosis and treatment. NPJ Digit Med 2024; 7:97. [PMID: 38622284 PMCID: PMC11018801 DOI: 10.1038/s41746-024-01082-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2023] [Accepted: 03/22/2024] [Indexed: 04/17/2024] Open
Abstract
Meniscal injury represents a common type of knee injury, accounting for over 50% of all knee injuries. The clinical diagnosis and treatment of meniscal injury heavily rely on magnetic resonance imaging (MRI). However, accurately diagnosing the meniscus from a comprehensive knee MRI is challenging due to its limited and weak signal, significantly impeding the precise grading of meniscal injuries. In this study, a visual interpretable fine grading (VIFG) diagnosis model has been developed to facilitate intelligent and quantified grading of meniscal injuries. Leveraging a multilevel transfer learning framework, it extracts comprehensive features and incorporates an attributional attention module to precisely locate the injured positions. Moreover, the attention-enhancing feedback module effectively concentrates on and distinguishes regions with similar grades of injury. The proposed method underwent validation on FastMRI_Knee and Xijing_Knee dataset, achieving mean grading accuracies of 0.8631 and 0.8502, surpassing the state-of-the-art grading methods notably in error-prone Grade 1 and Grade 2 cases. Additionally, the visually interpretable heatmaps generated by VIFG provide accurate depictions of actual or potential meniscus injury areas beyond human visual capability. Building upon this, a novel fine grading criterion was introduced for subtypes of meniscal injury, further classifying Grade 2 into 2a, 2b, and 2c, aligning with the anatomical knowledge of meniscal blood supply. It can provide enhanced injury-specific details, facilitating the development of more precise surgical strategies. The efficacy of this subtype classification was evidenced in 20 arthroscopic cases, underscoring the potential enhancement brought by intelligent-assisted diagnosis and treatment for meniscal injuries.
Collapse
Affiliation(s)
- Anlin Luo
- Key Laboratory of Intelligent Perception an Image Understanding of Ministry of Education, School of Artificial Intelligence, Xidian University, 710071, Xi'an, China
| | - Shuiping Gou
- Key Laboratory of Intelligent Perception an Image Understanding of Ministry of Education, School of Artificial Intelligence, Xidian University, 710071, Xi'an, China.
- AI-based Big Medical lmaging Data Frontier Research Center, Academy of Advanced Interdisciplinary Research, Xidian University, 710071, Xi'an, Shaanxi, China.
| | - Nuo Tong
- AI-based Big Medical lmaging Data Frontier Research Center, Academy of Advanced Interdisciplinary Research, Xidian University, 710071, Xi'an, Shaanxi, China
| | - Bo Liu
- Key Laboratory of Intelligent Perception an Image Understanding of Ministry of Education, School of Artificial Intelligence, Xidian University, 710071, Xi'an, China
| | - Licheng Jiao
- Key Laboratory of Intelligent Perception an Image Understanding of Ministry of Education, School of Artificial Intelligence, Xidian University, 710071, Xi'an, China
| | - Hu Xu
- Xijing Orthopaedics Hospital, The Fourth Military Medical University, 710032, Xi'an, Shaanxi, China
| | - Yingchun Wang
- Xijing Orthopaedics Hospital, The Fourth Military Medical University, 710032, Xi'an, Shaanxi, China
| | - Tan Ding
- Xijing Orthopaedics Hospital, The Fourth Military Medical University, 710032, Xi'an, Shaanxi, China.
| |
Collapse
|
23
|
Lu T, Lu M, Liu H, Song D, Wang Z, Guo Y, Fang Y, Chen Q, Li T. Establishment of a prognostic model for gastric cancer patients who underwent radical gastrectomy using machine learning: a two-center study. Front Oncol 2024; 13:1282042. [PMID: 38665864 PMCID: PMC11043579 DOI: 10.3389/fonc.2023.1282042] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2023] [Accepted: 12/21/2023] [Indexed: 04/28/2024] Open
Abstract
Objective Gastric cancer is a prevalent gastrointestinal malignancy worldwide. In this study, a prognostic model was developed for gastric cancer patients who underwent radical gastrectomy using machine learning, employing advanced computational techniques to investigate postoperative mortality risk factors in such patients. Methods Data of 295 patients with gastric cancer who underwent radical gastrectomy at the Department of General Surgery of Affiliated Hospital of Xuzhou Medical University (Xuzhou, China) between March 2016 and November 2019 were retrospectively analyzed as the training group. Additionally, 109 patients who underwent radical gastrectomy at the Department of General Surgery Affiliated to Jining First People's Hospital (Jining, China) were included for external validation. Four machine learning models, including logistic regression (LR), decision tree (DT), random forest (RF), and gradient boosting machine (GBM), were utilized. Model performance was assessed by comparing the area under the curve (AUC) for each model. An LR-based nomogram model was constructed to assess patients' clinical prognosis. Results Lasso regression identified eight associated factors: age, sex, maximum tumor diameter, nerve or vascular invasion, TNM stage, gastrectomy type, lymphocyte count, and carcinoembryonic antigen (CEA) level. The performance of these models was evaluated using the AUC. In the training group, the AUC values were 0.795, 0.759, 0.873, and 0.853 for LR, DT, RF, and GBM, respectively. In the validation group, the AUC values were 0.734, 0.708, 0.746, and 0.707 for LR, DT, RF, and GBM, respectively. The nomogram model, constructed based on LR, demonstrated excellent clinical prognostic evaluation capabilities. Conclusion Machine learning algorithms are robust performance assessment tools for evaluating the prognosis of gastric cancer patients who have undergone radical gastrectomy. The LR-based nomogram model can aid clinicians in making more reliable clinical decisions.
Collapse
Affiliation(s)
- Tong Lu
- Department of Emergency Medicine, Jining No.1 People’s Hospital, Jining, China
| | - Miao Lu
- Wuxi Mental Health Center, Wuxi, China
| | - Haonan Liu
- Department of Oncology, Affiliated Hospital of Xuzhou Medical University, Xuzhou, China
| | - Daqing Song
- Department of Emergency Medicine, Jining No.1 People’s Hospital, Jining, China
| | - Zhengzheng Wang
- Department of Gastroenterology, The Second Affiliated Hospital of Xuzhou Medical University, Xuzhou, China
| | - Yahui Guo
- Department of Gastroenterology, Xuzhou First People’s Hospital, Xuzhou, China
| | - Yu Fang
- Jiangsu Normal University, Xuzhou, China
| | - Qi Chen
- Department of Gastroenterology, Jining First People’s Hospital, Jining, China
| | - Tao Li
- Department of Emergency Medicine, Jining No.1 People’s Hospital, Jining, China
| |
Collapse
|
24
|
Cohen I, Sorin V, Lekach R, Raskin D, Segev M, Klang E, Eshed I, Barash Y. Artificial intelligence for detection of effusion and lipo-hemarthrosis in X-rays and CT of the knee. Eur J Radiol 2024; 175:111460. [PMID: 38608501 DOI: 10.1016/j.ejrad.2024.111460] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2024] [Revised: 03/29/2024] [Accepted: 04/08/2024] [Indexed: 04/14/2024]
Abstract
BACKGROUND Traumatic knee injuries are challenging to diagnose accurately through radiography and to a lesser extent, through CT, with fractures sometimes overlooked. Ancillary signs like joint effusion or lipo-hemarthrosis are indicative of fractures, suggesting the need for further imaging. Artificial Intelligence (AI) can automate image analysis, improving diagnostic accuracy and help prioritizing clinically important X-ray or CT studies. OBJECTIVE To develop and evaluate an AI algorithm for detecting effusion of any kind in knee X-rays and selected CT images and distinguishing between simple effusion and lipo-hemarthrosis indicative of intra-articular fractures. METHODS This retrospective study analyzed post traumatic knee imaging from January 2016 to February 2023, categorizing images into lipo-hemarthrosis, simple effusion, or normal. It utilized the FishNet-150 algorithm for image classification, with class activation maps highlighting decision-influential regions. The AI's diagnostic accuracy was validated against a gold standard, based on the evaluations made by a radiologist with at least four years of experience. RESULTS Analysis included CT images from 515 patients and X-rays from 637 post traumatic patients, identifying lipo-hemarthrosis, simple effusion, and normal findings. The AI showed an AUC of 0.81 for detecting any effusion, 0.78 for simple effusion, and 0.83 for lipo-hemarthrosis in X-rays; and 0.89, 0.89, and 0.91, respectively, in CTs. CONCLUSION The AI algorithm effectively detects knee effusion and differentiates between simple effusion and lipo-hemarthrosis in post-traumatic patients for both X-rays and selected CT images further studies are needed to validate these results.
Collapse
Affiliation(s)
- Israel Cohen
- Department of Diagnostic Imaging, Sheba Medical Center, Tel Hashomer, Israel; Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel.
| | - Vera Sorin
- Department of Diagnostic Imaging, Sheba Medical Center, Tel Hashomer, Israel; Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel.
| | - Ruth Lekach
- Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel; Department of Nuclear Medicine, Sourasky Medical Center, Tel-Aviv, Israel.
| | - Daniel Raskin
- Department of Diagnostic Imaging, Sheba Medical Center, Tel Hashomer, Israel; Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel.
| | - Maria Segev
- Department of Diagnostic Imaging, Sheba Medical Center, Tel Hashomer, Israel; Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel.
| | - Eyal Klang
- Department of Diagnostic Imaging, Sheba Medical Center, Tel Hashomer, Israel; Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel.
| | - Iris Eshed
- Department of Diagnostic Imaging, Sheba Medical Center, Tel Hashomer, Israel; Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel.
| | - Yiftach Barash
- Department of Diagnostic Imaging, Sheba Medical Center, Tel Hashomer, Israel; Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel.
| |
Collapse
|
25
|
Voinea ȘV, Gheonea IA, Teică RV, Florescu LM, Roman M, Selișteanu D. Refined Detection and Classification of Knee Ligament Injury Based on ResNet Convolutional Neural Networks. Life (Basel) 2024; 14:478. [PMID: 38672749 PMCID: PMC11051415 DOI: 10.3390/life14040478] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2024] [Revised: 03/20/2024] [Accepted: 04/03/2024] [Indexed: 04/28/2024] Open
Abstract
Currently, medical imaging has largely supplanted traditional methods in the realm of diagnosis and treatment planning. This shift is primarily attributable to the non-invasive nature, rapidity, and user-friendliness of medical-imaging techniques. The widespread adoption of medical imaging, however, has shifted the bottleneck to healthcare professionals who must analyze each case post-image acquisition. This process is characterized by its sluggishness and subjectivity, making it susceptible to errors. The anterior cruciate ligament (ACL), a frequently injured knee ligament, predominantly affects a youthful and sports-active demographic. ACL injuries often leave patients with substantial disabilities and alter knee mechanics. Since some of these cases necessitate surgery, it is crucial to accurately classify and detect ACL injury. This paper investigates the utilization of pre-trained convolutional neural networks featuring residual connections (ResNet) along with image-processing methods to identify ACL injury and differentiate between various tear levels. The ResNet employed in this study is not the standard ResNet but rather an adapted version capable of processing 3D volumes constructed from 2D image slices. Achieving a peak accuracy of 97.15% with a custom split, 96.32% through Monte-Carlo cross-validation, and 93.22% via five-fold cross-validation, our approach enhances the performance of three-class classifiers by over 7% in terms of raw accuracy. Moreover, we achieved an improvement of more than 1% across all types of evaluation. It is quite clear that the model's output can effectively serve as an initial diagnostic baseline for radiologists with minimal effort and nearly instantaneous results. This advancement underscores the paper's focus on harnessing deep learning for the nuanced detection and classification of ACL tears, demonstrating a significant leap toward automating and refining diagnostic accuracy in sports medicine and orthopedics.
Collapse
Affiliation(s)
- Ștefan-Vlad Voinea
- Department of Automatic Control and Electronics, University of Craiova, 200585 Craiova, Romania; (Ș.-V.V.); (M.R.)
| | - Ioana Andreea Gheonea
- Department of Radiology and Medical Imaging, University of Medicine and Pharmacy of Craiova, 200349 Craiova, Romania; (I.A.G.); (L.M.F.)
| | - Rossy Vlăduț Teică
- Doctoral School, University of Medicine and Pharmacy of Craiova, 200349 Craiova, Romania;
| | - Lucian Mihai Florescu
- Department of Radiology and Medical Imaging, University of Medicine and Pharmacy of Craiova, 200349 Craiova, Romania; (I.A.G.); (L.M.F.)
| | - Monica Roman
- Department of Automatic Control and Electronics, University of Craiova, 200585 Craiova, Romania; (Ș.-V.V.); (M.R.)
| | - Dan Selișteanu
- Department of Automatic Control and Electronics, University of Craiova, 200585 Craiova, Romania; (Ș.-V.V.); (M.R.)
| |
Collapse
|
26
|
Wang DY, Liu SG, Ding J, Sun AL, Jiang D, Jiang J, Zhao JZ, Chen DS, Ji G, Li N, Yuan HS, Yu JK. A Deep Learning Model Enhances Clinicians' Diagnostic Accuracy to More Than 96% for Anterior Cruciate Ligament Ruptures on Magnetic Resonance Imaging. Arthroscopy 2024; 40:1197-1205. [PMID: 37597705 DOI: 10.1016/j.arthro.2023.08.010] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/08/2023] [Revised: 08/02/2023] [Accepted: 08/03/2023] [Indexed: 08/21/2023]
Abstract
PURPOSE To develop a deep learning model to accurately detect anterior cruciate ligament (ACL) ruptures on magnetic resonance imaging (MRI) and to evaluate its effect on the diagnostic accuracy and efficiency of clinicians. METHODS A training dataset was built from MRIs acquired from January 2017 to June 2021, including patients with knee symptoms, irrespective of ACL ruptures. An external validation dataset was built from MRIs acquired from January 2021 to June 2022, including patients who underwent knee arthroscopy or arthroplasty. Patients with fractures or prior knee surgeries were excluded in both datasets. Subsequently, a deep learning model was developed and validated using these datasets. Clinicians of varying expertise levels in sports medicine and radiology were recruited, and their capacities in diagnosing ACL injuries in terms of accuracy and diagnosing time were evaluated both with and without artificial intelligence (AI) assistance. RESULTS A deep learning model was developed based on the training dataset of 22,767 MRIs from 5 centers and verified with external validation dataset of 4,086 MRIs from 6 centers. The model achieved an area under the receiver operating characteristic curve of 0.987 and a sensitivity and specificity of 95.1%. Thirty-eight clinicians from 25 centers were recruited to diagnose 3,800 MRIs. The AI assistance significantly improved the accuracy of all clinicians, exceeding 96%. Additionally, a notable reduction in diagnostic time was observed. The most significant improvements in accuracy and time efficiency were observed in the trainee groups, suggesting that AI support is particularly beneficial for clinicians with moderately limited diagnostic expertise. CONCLUSIONS This deep learning model demonstrated expert-level diagnostic performance for ACL ruptures, serving as a valuable tool to assist clinicians of various specialties and experience levels in making accurate and efficient diagnoses. LEVEL OF EVIDENCE Level III, retrospective comparative case series.
Collapse
Affiliation(s)
- Ding-Yu Wang
- Department of Sports Medicine, Peking University Third Hospital, Institute of Sports Medicine of Peking University, Beijing, China; Beijing Key Laboratory of Sports Injuries, Beijing, China; Engineering Research Center of Sports Trauma Treatment Technology and Devices, Ministry of Education, Beijing, China
| | - Shang-Gui Liu
- Department of Sports Medicine, Peking University Third Hospital, Institute of Sports Medicine of Peking University, Beijing, China; Beijing Key Laboratory of Sports Injuries, Beijing, China; Engineering Research Center of Sports Trauma Treatment Technology and Devices, Ministry of Education, Beijing, China
| | - Jia Ding
- Beijing Yizhun Medical AI Co., Ltd, Beijing, China
| | - An-Lan Sun
- Beijing Yizhun Medical AI Co., Ltd, Beijing, China
| | - Dong Jiang
- Department of Sports Medicine, Peking University Third Hospital, Institute of Sports Medicine of Peking University, Beijing, China; Beijing Key Laboratory of Sports Injuries, Beijing, China; Engineering Research Center of Sports Trauma Treatment Technology and Devices, Ministry of Education, Beijing, China
| | - Jia Jiang
- Department of Sports Medicine, Shanghai Jiaotong University Affiliated Sixth People's Hospital, Shanghai, China
| | - Jin-Zhong Zhao
- Department of Sports Medicine, Shanghai Jiaotong University Affiliated Sixth People's Hospital, Shanghai, China
| | - De-Sheng Chen
- Department of Sports Medicine and Arthroscopy, Tianjin Hospital of Tianjin University, Tianjin, China
| | - Gang Ji
- Department of Orthopaedic Surgery, Third Hospital of Hebei Medical University, Hebei, China
| | - Nan Li
- Research Center of Clinical Epidemiology, Peking University Third Hospital, Beijing, China
| | - Hui-Shu Yuan
- Department of Radiology, Peking University Third Hospital, Beijing, China
| | - Jia-Kuo Yu
- Department of Sports Medicine, Peking University Third Hospital, Institute of Sports Medicine of Peking University, Beijing, China; Beijing Key Laboratory of Sports Injuries, Beijing, China; Engineering Research Center of Sports Trauma Treatment Technology and Devices, Ministry of Education, Beijing, China.
| |
Collapse
|
27
|
Tieu A, Kroen E, Kadish Y, Liu Z, Patel N, Zhou A, Yilmaz A, Lee S, Deyer T. The Role of Artificial Intelligence in the Identification and Evaluation of Bone Fractures. Bioengineering (Basel) 2024; 11:338. [PMID: 38671760 PMCID: PMC11047896 DOI: 10.3390/bioengineering11040338] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2024] [Revised: 03/23/2024] [Accepted: 03/26/2024] [Indexed: 04/28/2024] Open
Abstract
Artificial intelligence (AI), particularly deep learning, has made enormous strides in medical imaging analysis. In the field of musculoskeletal radiology, deep-learning models are actively being developed for the identification and evaluation of bone fractures. These methods provide numerous benefits to radiologists such as increased diagnostic accuracy and efficiency while also achieving standalone performances comparable or superior to clinician readers. Various algorithms are already commercially available for integration into clinical workflows, with the potential to improve healthcare delivery and shape the future practice of radiology. In this systematic review, we explore the performance of current AI methods in the identification and evaluation of fractures, particularly those in the ankle, wrist, hip, and ribs. We also discuss current commercially available products for fracture detection and provide an overview of the current limitations of this technology and future directions of the field.
Collapse
Affiliation(s)
- Andrew Tieu
- BioMedical Engineering and Imaging Institute, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Ezriel Kroen
- New York Medical College, Valhalla, NY 10595, USA
| | | | - Zelong Liu
- BioMedical Engineering and Imaging Institute, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Nikhil Patel
- BioMedical Engineering and Imaging Institute, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Alexander Zhou
- BioMedical Engineering and Imaging Institute, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | | | | | - Timothy Deyer
- East River Medical Imaging, New York, NY 10021, USA
- Department of Radiology, Cornell Medicine, New York, NY 10021, USA
| |
Collapse
|
28
|
Huffman N, Pasqualini I, Khan ST, Klika AK, Deren ME, Jin Y, Kunze KN, Piuzzi NS. Enabling Personalized Medicine in Orthopaedic Surgery Through Artificial Intelligence: A Critical Analysis Review. JBJS Rev 2024; 12:01874474-202403000-00006. [PMID: 38466797 DOI: 10.2106/jbjs.rvw.23.00232] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/13/2024]
Abstract
» The application of artificial intelligence (AI) in the field of orthopaedic surgery holds potential for revolutionizing health care delivery across 3 crucial domains: (I) personalized prediction of clinical outcomes and adverse events, which may optimize patient selection, surgical planning, and enhance patient safety and outcomes; (II) diagnostic automated and semiautomated imaging analyses, which may reduce time burden and facilitate precise and timely diagnoses; and (III) forecasting of resource utilization, which may reduce health care costs and increase value for patients and institutions.» Computer vision is one of the most highly studied areas of AI within orthopaedics, with applications pertaining to fracture classification, identification of the manufacturer and model of prosthetic implants, and surveillance of prosthesis loosening and failure.» Prognostic applications of AI within orthopaedics include identifying patients who will likely benefit from a specified treatment, predicting prosthetic implant size, postoperative length of stay, discharge disposition, and surgical complications. Not only may these applications be beneficial to patients but also to institutions and payors because they may inform potential cost expenditure, improve overall hospital efficiency, and help anticipate resource utilization.» AI infrastructure development requires institutional financial commitment and a team of clinicians and data scientists with expertise in AI that can complement skill sets and knowledge. Once a team is established and a goal is determined, teams (1) obtain, curate, and label data; (2) establish a reference standard; (3) develop an AI model; (4) evaluate the performance of the AI model; (5) externally validate the model, and (6) reinforce, improve, and evaluate the model's performance until clinical implementation is possible.» Understanding the implications of AI in orthopaedics may eventually lead to wide-ranging improvements in patient care. However, AI, while holding tremendous promise, is not without methodological and ethical limitations that are essential to address. First, it is important to ensure external validity of programs before their use in a clinical setting. Investigators should maintain high quality data records and registry surveillance, exercise caution when evaluating others' reported AI applications, and increase transparency of the methodological conduct of current models to improve external validity and avoid propagating bias. By addressing these challenges and responsibly embracing the potential of AI, the medical field may eventually be able to harness its power to improve patient care and outcomes.
Collapse
Affiliation(s)
- Nickelas Huffman
- Cleveland Clinic, Department of Orthopaedic Surgery, Cleveland, Ohio
| | | | - Shujaa T Khan
- Cleveland Clinic, Department of Orthopaedic Surgery, Cleveland, Ohio
| | - Alison K Klika
- Cleveland Clinic, Department of Orthopaedic Surgery, Cleveland, Ohio
| | - Matthew E Deren
- Cleveland Clinic, Department of Orthopaedic Surgery, Cleveland, Ohio
| | - Yuxuan Jin
- Cleveland Clinic, Department of Orthopaedic Surgery, Cleveland, Ohio
| | - Kyle N Kunze
- Department of Orthopaedic Surgery, Hospital for Special Surgery, New York, New York
| | - Nicolas S Piuzzi
- Cleveland Clinic, Department of Orthopaedic Surgery, Cleveland, Ohio
- Department of Biomedical Engineering, Cleveland Clinic Foundation, Cleveland, Ohio
| |
Collapse
|
29
|
Santomartino SM, Kung J, Yi PH. Systematic review of artificial intelligence development and evaluation for MRI diagnosis of knee ligament or meniscus tears. Skeletal Radiol 2024; 53:445-454. [PMID: 37584757 DOI: 10.1007/s00256-023-04416-2] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/28/2023] [Revised: 07/24/2023] [Accepted: 07/24/2023] [Indexed: 08/17/2023]
Abstract
OBJECTIVE The purpose of this systematic review was to summarize the results of original research studies evaluating the characteristics and performance of deep learning models for detection of knee ligament and meniscus tears on MRI. MATERIALS AND METHODS We searched PubMed for studies published as of February 2, 2022 for original studies evaluating development and evaluation of deep learning models for MRI diagnosis of knee ligament or meniscus tears. We summarized study details according to multiple criteria including baseline article details, model creation, deep learning details, and model evaluation. RESULTS 19 studies were included with radiology departments leading the publications in deep learning development and implementation for detecting knee injuries via MRI. Among the studies, there was a lack of standard reporting and inconsistently described development details. However, all included studies reported consistently high model performance that significantly supplemented human reader performance. CONCLUSION From our review, we found radiology departments have been leading deep learning development for injury detection on knee MRIs. Although studies inconsistently described DL model development details, all reported high model performance, indicating great promise for DL in knee MRI analysis.
Collapse
Affiliation(s)
- Samantha M Santomartino
- Drexel University College of Medicine, Philadelphia, PA, USA
- University of Maryland Medical Intelligent Imaging (UM2ii) Center, Department of Diagnostic Radiology and Nuclear Medicine, University of Maryland, University of Maryland School of Medicine, Baltimore, MD, USA
| | - Justin Kung
- Department of Orthopaedic Surgery, University of South Carolina, Columbia, SC, USA
| | - Paul H Yi
- University of Maryland Medical Intelligent Imaging (UM2ii) Center, Department of Diagnostic Radiology and Nuclear Medicine, University of Maryland, University of Maryland School of Medicine, Baltimore, MD, USA.
- Malone Center for Engineering in Healthcare, Johns Hopkins University, Baltimore Street First Floor Rm. 1172, Baltimore, MD, 21201, USA.
| |
Collapse
|
30
|
Yi PH, Garner HW, Hirschmann A, Jacobson JA, Omoumi P, Oh K, Zech JR, Lee YH. Clinical Applications, Challenges, and Recommendations for Artificial Intelligence in Musculoskeletal and Soft-Tissue Ultrasound: AJR Expert Panel Narrative Review. AJR Am J Roentgenol 2024; 222:e2329530. [PMID: 37436032 DOI: 10.2214/ajr.23.29530] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/13/2023]
Abstract
Artificial intelligence (AI) is increasingly used in clinical practice for musculoskeletal imaging tasks, such as disease diagnosis and image reconstruction. AI applications in musculoskeletal imaging have focused primarily on radiography, CT, and MRI. Although musculoskeletal ultrasound stands to benefit from AI in similar ways, such applications have been relatively underdeveloped. In comparison with other modalities, ultrasound has unique advantages and disadvantages that must be considered in AI algorithm development and clinical translation. Challenges in developing AI for musculoskeletal ultrasound involve both clinical aspects of image acquisition and practical limitations in image processing and annotation. Solutions from other radiology subspecialties (e.g., crowdsourced annotations coordinated by professional societies), along with use cases (most commonly rotator cuff tendon tears and palpable soft-tissue masses), can be applied to musculoskeletal ultrasound to help develop AI. To facilitate creation of high-quality imaging datasets for AI model development, technologists and radiologists should focus on increasing uniformity in musculoskeletal ultrasound performance and increasing annotations of images for specific anatomic regions. This Expert Panel Narrative Review summarizes available evidence regarding AI's potential utility in musculoskeletal ultrasound and challenges facing its development. Recommendations for future AI advancement and clinical translation in musculoskeletal ultrasound are discussed.
Collapse
Affiliation(s)
- Paul H Yi
- University of Maryland Medical Intelligent Imaging Center, University of Maryland School of Medicine, Baltimore, MD
- Department of Diagnostic Radiology and Nuclear Medicine, University of Maryland School of Medicine, Baltimore, MD
| | | | - Anna Hirschmann
- Imamed Radiology Nordwest, Basel, Switzerland
- Department of Radiology, University of Basel, Basel, Switzerland
| | - Jon A Jacobson
- Lenox Hill Radiology, New York, NY
- Department of Radiology, University of California, San Diego Medical Center, San Diego, CA
| | - Patrick Omoumi
- Department of Radiology, Lausanne University Hospital, Lausanne, Switzerland
- Department of Radiology, University of Lausanne, Lausanne, Switzerland
| | - Kangrok Oh
- Department of Radiology, Research Institute of Radiological Science and Center for Clinical Imaging Data Science, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul 03722, South Korea
| | - John R Zech
- Department of Radiology, Columbia University Irving Medical Center, New York-Presbyterian Hospital, New York, NY
| | - Young Han Lee
- Department of Radiology, Research Institute of Radiological Science and Center for Clinical Imaging Data Science, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul 03722, South Korea
| |
Collapse
|
31
|
Zhao Y, Coppola A, Karamchandani U, Amiras D, Gupte CM. Artificial intelligence applied to magnetic resonance imaging reliably detects the presence, but not the location, of meniscus tears: a systematic review and meta-analysis. Eur Radiol 2024:10.1007/s00330-024-10625-7. [PMID: 38386028 DOI: 10.1007/s00330-024-10625-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2023] [Revised: 12/24/2023] [Accepted: 01/13/2024] [Indexed: 02/23/2024]
Abstract
OBJECTIVES To review and compare the accuracy of convolutional neural networks (CNN) for the diagnosis of meniscal tears in the current literature and analyze the decision-making processes utilized by these CNN algorithms. MATERIALS AND METHODS PubMed, MEDLINE, EMBASE, and Cochrane databases up to December 2022 were searched in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) statement. Risk of analysis was used for all identified articles. Predictive performance values, including sensitivity and specificity, were extracted for quantitative analysis. The meta-analysis was divided between AI prediction models identifying the presence of meniscus tears and the location of meniscus tears. RESULTS Eleven articles were included in the final review, with a total of 13,467 patients and 57,551 images. Heterogeneity was statistically significantly large for the sensitivity of the tear identification analysis (I2 = 79%). A higher level of accuracy was observed in identifying the presence of a meniscal tear over locating tears in specific regions of the meniscus (AUC, 0.939 vs 0.905). Pooled sensitivity and specificity were 0.87 (95% confidence interval (CI) 0.80-0.91) and 0.89 (95% CI 0.83-0.93) for meniscus tear identification and 0.88 (95% CI 0.82-0.91) and 0.84 (95% CI 0.81-0.85) for locating the tears. CONCLUSIONS AI prediction models achieved favorable performance in the diagnosis, but not location, of meniscus tears. Further studies on the clinical utilities of deep learning should include standardized reporting, external validation, and full reports of the predictive performances of these models, with a view to localizing tears more accurately. CLINICAL RELEVANCE STATEMENT Meniscus tears are hard to diagnose in the knee magnetic resonance images. AI prediction models may play an important role in improving the diagnostic accuracy of clinicians and radiologists. KEY POINTS • Artificial intelligence (AI) provides great potential in improving the diagnosis of meniscus tears. • The pooled diagnostic performance for artificial intelligence (AI) in identifying meniscus tears was better (sensitivity 87%, specificity 89%) than locating the tears (sensitivity 88%, specificity 84%). • AI is good at confirming the diagnosis of meniscus tears, but future work is required to guide the management of the disease.
Collapse
Affiliation(s)
- Yi Zhao
- Imperial College London School of Medicine, Exhibition Rd, South Kensington, London, SW7 2BU, UK.
| | - Andrew Coppola
- Imperial College London School of Medicine, Exhibition Rd, South Kensington, London, SW7 2BU, UK
| | | | - Dimitri Amiras
- Imperial College London School of Medicine, Exhibition Rd, South Kensington, London, SW7 2BU, UK
- Imperial College London NHS Trust, London, UK
| | - Chinmay M Gupte
- Imperial College London School of Medicine, Exhibition Rd, South Kensington, London, SW7 2BU, UK
- Imperial College London NHS Trust, London, UK
| |
Collapse
|
32
|
Gitto S, Serpi F, Albano D, Risoleo G, Fusco S, Messina C, Sconfienza LM. AI applications in musculoskeletal imaging: a narrative review. Eur Radiol Exp 2024; 8:22. [PMID: 38355767 PMCID: PMC10866817 DOI: 10.1186/s41747-024-00422-8] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2023] [Accepted: 12/29/2023] [Indexed: 02/16/2024] Open
Abstract
This narrative review focuses on clinical applications of artificial intelligence (AI) in musculoskeletal imaging. A range of musculoskeletal disorders are discussed using a clinical-based approach, including trauma, bone age estimation, osteoarthritis, bone and soft-tissue tumors, and orthopedic implant-related pathology. Several AI algorithms have been applied to fracture detection and classification, which are potentially helpful tools for radiologists and clinicians. In bone age assessment, AI methods have been applied to assist radiologists by automatizing workflow, thus reducing workload and inter-observer variability. AI may potentially aid radiologists in identifying and grading abnormal findings of osteoarthritis as well as predicting the onset or progression of this disease. Either alone or combined with radiomics, AI algorithms may potentially improve diagnosis and outcome prediction of bone and soft-tissue tumors. Finally, information regarding appropriate positioning of orthopedic implants and related complications may be obtained using AI algorithms. In conclusion, rather than replacing radiologists, the use of AI should instead help them to optimize workflow, augment diagnostic performance, and keep up with ever-increasing workload.Relevance statement This narrative review provides an overview of AI applications in musculoskeletal imaging. As the number of AI technologies continues to increase, it will be crucial for radiologists to play a role in their selection and application as well as to fully understand their potential value in clinical practice. Key points • AI may potentially assist musculoskeletal radiologists in several interpretative tasks.• AI applications to trauma, age estimation, osteoarthritis, tumors, and orthopedic implants are discussed.• AI should help radiologists to optimize workflow and augment diagnostic performance.
Collapse
Affiliation(s)
- Salvatore Gitto
- Department of Biomedical Sciences for Health, Università degli Studi di Milano, Via Cristina Belgioioso 173, Milan, 20157, Italy
- IRCCS Istituto Ortopedico Galeazzi, Milan, Italy
| | - Francesca Serpi
- Department of Biomedical Sciences for Health, Università degli Studi di Milano, Via Cristina Belgioioso 173, Milan, 20157, Italy
| | - Domenico Albano
- IRCCS Istituto Ortopedico Galeazzi, Milan, Italy
- Dipartimento di Scienze Biomediche, Chirurgiche ed Odontoiatriche, Università degli Studi di Milano, Milan, Italy
| | - Giovanni Risoleo
- Scuola di Specializzazione in Radiodiagnostica, Università degli Studi di Milano, Milan, Italy
| | - Stefano Fusco
- Department of Biomedical Sciences for Health, Università degli Studi di Milano, Via Cristina Belgioioso 173, Milan, 20157, Italy
| | - Carmelo Messina
- Department of Biomedical Sciences for Health, Università degli Studi di Milano, Via Cristina Belgioioso 173, Milan, 20157, Italy
- IRCCS Istituto Ortopedico Galeazzi, Milan, Italy
| | - Luca Maria Sconfienza
- Department of Biomedical Sciences for Health, Università degli Studi di Milano, Via Cristina Belgioioso 173, Milan, 20157, Italy.
- IRCCS Istituto Ortopedico Galeazzi, Milan, Italy.
| |
Collapse
|
33
|
Hanneman K, Playford D, Dey D, van Assen M, Mastrodicasa D, Cook TS, Gichoya JW, Williamson EE, Rubin GD. Value Creation Through Artificial Intelligence and Cardiovascular Imaging: A Scientific Statement From the American Heart Association. Circulation 2024; 149:e296-e311. [PMID: 38193315 DOI: 10.1161/cir.0000000000001202] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/10/2024]
Abstract
Multiple applications for machine learning and artificial intelligence (AI) in cardiovascular imaging are being proposed and developed. However, the processes involved in implementing AI in cardiovascular imaging are highly diverse, varying by imaging modality, patient subtype, features to be extracted and analyzed, and clinical application. This article establishes a framework that defines value from an organizational perspective, followed by value chain analysis to identify the activities in which AI might produce the greatest incremental value creation. The various perspectives that should be considered are highlighted, including clinicians, imagers, hospitals, patients, and payers. Integrating the perspectives of all health care stakeholders is critical for creating value and ensuring the successful deployment of AI tools in a real-world setting. Different AI tools are summarized, along with the unique aspects of AI applications to various cardiac imaging modalities, including cardiac computed tomography, magnetic resonance imaging, and positron emission tomography. AI is applicable and has the potential to add value to cardiovascular imaging at every step along the patient journey, from selecting the more appropriate test to optimizing image acquisition and analysis, interpreting the results for classification and diagnosis, and predicting the risk for major adverse cardiac events.
Collapse
|
34
|
Hoy MK, Desai V, Mutasa S, Hoy RC, Gorniak R, Belair JA. Deep Learning-Assisted Identification of Femoroacetabular Impingement (FAI) on Routine Pelvic Radiographs. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024; 37:339-346. [PMID: 38343231 DOI: 10.1007/s10278-023-00920-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/04/2023] [Revised: 08/08/2023] [Accepted: 08/22/2023] [Indexed: 03/02/2024]
Abstract
To use a novel deep learning system to localize the hip joints and detect findings of cam-type femoroacetabular impingement (FAI). A retrospective search of hip/pelvis radiographs obtained in patients to evaluate for FAI yielded 3050 total studies. Each hip was classified separately by the original interpreting radiologist in the following manner: 724 hips had severe cam-type FAI morphology, 962 moderate cam-type FAI morphology, 846 mild cam-type FAI morphology, and 518 hips were normal. The anteroposterior (AP) view from each study was anonymized and extracted. After localization of the hip joints by a novel convolutional neural network (CNN) based on the focal loss principle, a second CNN classified the images of the hip as cam positive, or no FAI. Accuracy was 74% for diagnosing normal vs. abnormal cam-type FAI morphology, with aggregate sensitivity and specificity of 0.821 and 0.669, respectively, at the chosen operating point. The aggregate AUC was 0.736. A deep learning system can be applied to detect FAI-related changes on single view pelvic radiographs. Deep learning is useful for quickly identifying and categorizing pathology on imaging, which may aid the interpreting radiologist.
Collapse
Affiliation(s)
| | - Vishal Desai
- Thomas Jefferson University, Philadelphia, PA, USA
| | | | - Robert C Hoy
- Temple University Hospital, Philadelphia, PA, USA
| | | | | |
Collapse
|
35
|
Jin W, Fatehi M, Guo R, Hamarneh G. Evaluating the clinical utility of artificial intelligence assistance and its explanation on the glioma grading task. Artif Intell Med 2024; 148:102751. [PMID: 38325929 DOI: 10.1016/j.artmed.2023.102751] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2023] [Revised: 11/06/2023] [Accepted: 12/21/2023] [Indexed: 02/09/2024]
Abstract
Clinical evaluation evidence and model explainability are key gatekeepers to ensure the safe, accountable, and effective use of artificial intelligence (AI) in clinical settings. We conducted a clinical user-centered evaluation with 35 neurosurgeons to assess the utility of AI assistance and its explanation on the glioma grading task. Each participant read 25 brain MRI scans of patients with gliomas, and gave their judgment on the glioma grading without and with the assistance of AI prediction and explanation. The AI model was trained on the BraTS dataset with 88.0% accuracy. The AI explanation was generated using the explainable AI algorithm of SmoothGrad, which was selected from 16 algorithms based on the criterion of being truthful to the AI decision process. Results showed that compared to the average accuracy of 82.5±8.7% when physicians performed the task alone, physicians' task performance increased to 87.7±7.3% with statistical significance (p-value = 0.002) when assisted by AI prediction, and remained at almost the same level of 88.5±7.0% (p-value = 0.35) with the additional assistance of AI explanation. Based on quantitative and qualitative results, the observed improvement in physicians' task performance assisted by AI prediction was mainly because physicians' decision patterns converged to be similar to AI, as physicians only switched their decisions when disagreeing with AI. The insignificant change in physicians' performance with the additional assistance of AI explanation was because the AI explanations did not provide explicit reasons, contexts, or descriptions of clinical features to help doctors discern potentially incorrect AI predictions. The evaluation showed the clinical utility of AI to assist physicians on the glioma grading task, and identified the limitations and clinical usage gaps of existing explainable AI techniques for future improvement.
Collapse
Affiliation(s)
- Weina Jin
- School of Computing Science, Simon Fraser University, Burnaby, Canada.
| | - Mostafa Fatehi
- Division of Neurosurgery, The University of British Columbia, Vancouver, Canada.
| | - Ru Guo
- Division of Neurosurgery, The University of British Columbia, Vancouver, Canada.
| | - Ghassan Hamarneh
- School of Computing Science, Simon Fraser University, Burnaby, Canada.
| |
Collapse
|
36
|
Tian Y, Nayak KS. New clinical opportunities of low-field MRI: heart, lung, body, and musculoskeletal. MAGMA (NEW YORK, N.Y.) 2024; 37:1-14. [PMID: 37902898 PMCID: PMC10876830 DOI: 10.1007/s10334-023-01123-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/26/2023] [Revised: 09/28/2023] [Accepted: 10/05/2023] [Indexed: 11/01/2023]
Abstract
Contemporary whole-body low-field MRI scanners (< 1 T) present new and exciting opportunities for improved body imaging. The fundamental reason is that the reduced off-resonance and reduced SAR provide substantially increased flexibility in the design of MRI pulse sequences. Promising body applications include lung parenchyma imaging, imaging adjacent to metallic implants, cardiac imaging, and dynamic imaging in general. The lower cost of such systems may make MRI favorable for screening high-risk populations and population health research, and the more open configurations allowed may prove favorable for obese subjects and for pregnant women. This article summarizes promising body applications for contemporary whole-body low-field MRI systems, with a focus on new platforms developed within the past 5 years. This is an active area of research, and one can expect many improvements as MRI physicists fully explore the landscape of pulse sequences that are feasible, and as clinicians apply these to patient populations.
Collapse
Affiliation(s)
- Ye Tian
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, 3740 McClintock Ave, EEB 406, Los Angeles, CA, 90089-2564, USA.
| | - Krishna S Nayak
- Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, 3740 McClintock Ave, EEB 406, Los Angeles, CA, 90089-2564, USA
| |
Collapse
|
37
|
Cheng Q, Lin H, Zhao J, Lu X, Wang Q. Application of machine learning-based multi-sequence MRI radiomics in diagnosing anterior cruciate ligament tears. J Orthop Surg Res 2024; 19:99. [PMID: 38297322 PMCID: PMC10829177 DOI: 10.1186/s13018-024-04602-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/29/2023] [Accepted: 01/28/2024] [Indexed: 02/02/2024] Open
Abstract
OBJECTIVE To compare the diagnostic power among various machine learning algorithms utilizing multi-sequence magnetic resonance imaging (MRI) radiomics in detecting anterior cruciate ligament (ACL) tears. Additionally, this research aimed to create and validate the optimal diagnostic model. METHODS In this retrospective analysis, 526 patients were included, comprising 178 individuals with ACL tears and 348 with a normal ACL. Radiomics features were derived from multi-sequence MRI scans, encompassing T1-weighted imaging and proton density (PD)-weighted imaging. The process of selecting the most reliable radiomics features involved using interclass correlation coefficient (ICC) testing, t tests, and the least absolute shrinkage and selection operator (LASSO) technique. After the feature selection process, five machine learning classifiers were created. These classifiers comprised logistic regression (LR), support vector machine (SVM), K-nearest neighbors (KNN), light gradient boosting machine (LightGBM), and multilayer perceptron (MLP). A thorough performance evaluation was carried out, utilizing diverse metrics like the area under the receiver operating characteristic curve (ROC), specificity, accuracy, sensitivity positive predictive value, and negative predictive value. The classifier exhibiting the best performance was chosen. Subsequently, three models were developed: the PD model, the T1 model, and the combined model, all based on the optimal classifier. The diagnostic performance of these models was assessed by employing AUC values, calibration curves, and decision curve analysis. RESULTS Out of 2032 features, 48 features were selected. The SVM-based multi-sequence radiomics outperformed all others, achieving AUC values of 0.973 and 0.927, sensitivities of 0.933 and 0.857, and specificities of 0.930 and 0.829, in the training and validation cohorts, respectively. CONCLUSION The multi-sequence MRI radiomics model, which is based on machine learning, exhibits exceptional performance in diagnosing ACL tears. It provides valuable insights crucial for the diagnosis and treatment of knee joint injuries, serving as an accurate and objective supplementary diagnostic tool for clinical practitioners.
Collapse
Affiliation(s)
- Qi Cheng
- Department of Orthopedic Surgery, The First Affiliated Hospital of Wannan Medical College, Yijishan Hospital, Wuhu, 241001, Anhui, People's Republic of China
| | - Haoran Lin
- Department of Orthopedic Surgery, The First Affiliated Hospital of Wannan Medical College, Yijishan Hospital, Wuhu, 241001, Anhui, People's Republic of China
| | - Jie Zhao
- Department of Orthopedic Surgery, The First Affiliated Hospital of Wannan Medical College, Yijishan Hospital, Wuhu, 241001, Anhui, People's Republic of China
| | - Xiao Lu
- Department of Orthopedic Surgery, The First Affiliated Hospital of Wannan Medical College, Yijishan Hospital, Wuhu, 241001, Anhui, People's Republic of China
| | - Qiang Wang
- Department of Orthopedic Surgery, The First Affiliated Hospital of Wannan Medical College, Yijishan Hospital, Wuhu, 241001, Anhui, People's Republic of China.
| |
Collapse
|
38
|
Lu T, Lu M, Wu D, Ding YY, Liu HN, Li TT, Song DQ. Predictive value of machine learning models for lymph node metastasis in gastric cancer: A two-center study. World J Gastrointest Surg 2024; 16:85-94. [PMID: 38328326 PMCID: PMC10845275 DOI: 10.4240/wjgs.v16.i1.85] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Revised: 11/24/2023] [Accepted: 12/21/2023] [Indexed: 01/25/2024] Open
Abstract
BACKGROUND Gastric cancer is one of the most common malignant tumors in the digestive system, ranking sixth in incidence and fourth in mortality worldwide. Since 42.5% of metastatic lymph nodes in gastric cancer belong to nodule type and peripheral type, the application of imaging diagnosis is restricted. AIM To establish models for predicting the risk of lymph node metastasis in gastric cancer patients using machine learning (ML) algorithms and to evaluate their predictive performance in clinical practice. METHODS Data of a total of 369 patients who underwent radical gastrectomy at the Department of General Surgery of Affiliated Hospital of Xuzhou Medical University (Xuzhou, China) from March 2016 to November 2019 were collected and retrospectively analyzed as the training group. In addition, data of 123 patients who underwent radical gastrectomy at the Department of General Surgery of Jining First People's Hospital (Jining, China) were collected and analyzed as the verification group. Seven ML models, including decision tree, random forest, support vector machine (SVM), gradient boosting machine, naive Bayes, neural network, and logistic regression, were developed to evaluate the occurrence of lymph node metastasis in patients with gastric cancer. The ML models were established following ten cross-validation iterations using the training dataset, and subsequently, each model was assessed using the test dataset. The models' performance was evaluated by comparing the area under the receiver operating characteristic curve of each model. RESULTS Among the seven ML models, except for SVM, the other ones exhibited higher accuracy and reliability, and the influences of various risk factors on the models are intuitive. CONCLUSION The ML models developed exhibit strong predictive capabilities for lymph node metastasis in gastric cancer, which can aid in personalized clinical diagnosis and treatment.
Collapse
Affiliation(s)
- Tong Lu
- Department of Emergency Medicine, Jining No. 1 People’s Hospital, Jining 272000, Shandong Province, China
| | - Miao Lu
- Wuxi Mental Health Center, Wuxi 214000, Jiangsu Province, China
| | - Dong Wu
- Department of Emergency Medicine, Jining No. 1 People’s Hospital, Jining 272000, Shandong Province, China
| | - Yuan-Yuan Ding
- Department of Gastroenterology, Jining No. 1 People’s Hospital, Jining 272000, Shandong Province, China
| | - Hao-Nan Liu
- Department of Oncology, The Affiliated Hospital of Xuzhou Medical University, Xuzhou 221002, Jiangsu Province, China
| | - Tao-Tao Li
- Department of Emergency Medicine, Jining No. 1 People’s Hospital, Jining 272000, Shandong Province, China
| | - Da-Qing Song
- Department of Emergency Medicine, Jining No. 1 People’s Hospital, Jining 272000, Shandong Province, China
| |
Collapse
|
39
|
Andriollo L, Picchi A, Sangaletti R, Perticarini L, Rossi SMP, Logroscino G, Benazzo F. The Role of Artificial Intelligence in Anterior Cruciate Ligament Injuries: Current Concepts and Future Perspectives. Healthcare (Basel) 2024; 12:300. [PMID: 38338185 PMCID: PMC10855330 DOI: 10.3390/healthcare12030300] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2023] [Revised: 01/19/2024] [Accepted: 01/22/2024] [Indexed: 02/12/2024] Open
Abstract
The remarkable progress in data aggregation and deep learning algorithms has positioned artificial intelligence (AI) and machine learning (ML) to revolutionize the field of medicine. AI is becoming more and more prevalent in the healthcare sector, and its impact on orthopedic surgery is already evident in several fields. This review aims to examine the literature that explores the comprehensive clinical relevance of AI-based tools utilized before, during, and after anterior cruciate ligament (ACL) reconstruction. The review focuses on current clinical applications and future prospects in preoperative management, encompassing risk prediction and diagnostics; intraoperative tools, specifically navigation, identifying complex anatomic landmarks during surgery; and postoperative applications in terms of postoperative care and rehabilitation. Additionally, AI tools in educational and training settings are presented. Orthopedic surgeons are showing a growing interest in AI, as evidenced by the applications discussed in this review, particularly those related to ACL injury. The exponential increase in studies on AI tools applicable to the management of ACL tears promises a significant future impact in its clinical application, with growing attention from orthopedic surgeons.
Collapse
Affiliation(s)
- Luca Andriollo
- Robotic Prosthetic Surgery Unit—Sports Traumatology Unit, Fondazione Poliambulanza Istituto Ospedaliero, 25124 Brescia, Italy; (R.S.); (L.P.); (S.M.P.R.); (F.B.)
- Department of Orthopedics, Catholic University of the Sacred Heart, 00168 Rome, Italy
| | - Aurelio Picchi
- Unit of Orthopedics, Department of Life, Health and Environmental Sciences, University of L’Aquila, 67100 L’Aquila, Italy; (A.P.); (G.L.)
| | - Rudy Sangaletti
- Robotic Prosthetic Surgery Unit—Sports Traumatology Unit, Fondazione Poliambulanza Istituto Ospedaliero, 25124 Brescia, Italy; (R.S.); (L.P.); (S.M.P.R.); (F.B.)
| | - Loris Perticarini
- Robotic Prosthetic Surgery Unit—Sports Traumatology Unit, Fondazione Poliambulanza Istituto Ospedaliero, 25124 Brescia, Italy; (R.S.); (L.P.); (S.M.P.R.); (F.B.)
| | - Stefano Marco Paolo Rossi
- Robotic Prosthetic Surgery Unit—Sports Traumatology Unit, Fondazione Poliambulanza Istituto Ospedaliero, 25124 Brescia, Italy; (R.S.); (L.P.); (S.M.P.R.); (F.B.)
| | - Giandomenico Logroscino
- Unit of Orthopedics, Department of Life, Health and Environmental Sciences, University of L’Aquila, 67100 L’Aquila, Italy; (A.P.); (G.L.)
| | - Francesco Benazzo
- Robotic Prosthetic Surgery Unit—Sports Traumatology Unit, Fondazione Poliambulanza Istituto Ospedaliero, 25124 Brescia, Italy; (R.S.); (L.P.); (S.M.P.R.); (F.B.)
- Biomedical Sciences Area, IUSS University School for Advanced Studies, 27100 Pavia, Italy
| |
Collapse
|
40
|
Ying M, Wang Y, Yang K, Wang H, Liu X. A deep learning knowledge distillation framework using knee MRI and arthroscopy data for meniscus tear detection. Front Bioeng Biotechnol 2024; 11:1326706. [PMID: 38292305 PMCID: PMC10825958 DOI: 10.3389/fbioe.2023.1326706] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/23/2023] [Accepted: 12/22/2023] [Indexed: 02/01/2024] Open
Abstract
Purpose: To construct a deep learning knowledge distillation framework exploring the utilization of MRI alone or combing with distilled Arthroscopy information for meniscus tear detection. Methods: A database of 199 paired knee Arthroscopy-MRI exams was used to develop a multimodal teacher network and an MRI-based student network, which used residual neural networks architectures. A knowledge distillation framework comprising the multimodal teacher network T and the monomodal student network S was proposed. We optimized the loss functions of mean squared error (MSE) and cross-entropy (CE) to enable the student network S to learn arthroscopic information from the teacher network T through our deep learning knowledge distillation framework, ultimately resulting in a distilled student network S T. A coronal proton density (PD)-weighted fat-suppressed MRI sequence was used in this study. Fivefold cross-validation was employed, and the accuracy, sensitivity, specificity, F1-score, receiver operating characteristic (ROC) curves and area under the receiver operating characteristic curve (AUC) were used to evaluate the medial and lateral meniscal tears detection performance of the models, including the undistilled student model S, the distilled student model S T and the teacher model T. Results: The AUCs of the undistilled student model S, the distilled student model S T, the teacher model T for medial meniscus (MM) tear detection and lateral meniscus (LM) tear detection are 0.773/0.672, 0.792/0.751 and 0.834/0.746, respectively. The distilled student model S T had higher AUCs than the undistilled model S. After undergoing knowledge distillation processing, the distilled student model demonstrated promising results, with accuracy (0.764/0.734), sensitivity (0.838/0.661), and F1-score (0.680/0.754) for both medial and lateral tear detection better than the undistilled one with accuracy (0.734/0.648), sensitivity (0.733/0.607), and F1-score (0.620/0.673). Conclusion: Through the knowledge distillation framework, the student model S based on MRI benefited from the multimodal teacher model T and achieved an improved meniscus tear detection performance.
Collapse
Affiliation(s)
- Mengjie Ying
- Department of Orthopedics, Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Yufan Wang
- Engineering Research Center for Digital Medicine of the Ministry of Education, Shanghai, China
- School of Biomedical Engineering and Med-X Research Institute, Shanghai Jiao Tong University, Shanghai, China
| | - Kai Yang
- Department of Radiology, Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Haoyuan Wang
- Department of Orthopedics, Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Xudong Liu
- Department of Orthopedics, Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, China
| |
Collapse
|
41
|
Wang P, Liu Y, Zhou Z. Supraspinatus extraction from MRI based on attention-dense spatial pyramid UNet network. J Orthop Surg Res 2024; 19:60. [PMID: 38216968 PMCID: PMC10787409 DOI: 10.1186/s13018-023-04509-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/07/2023] [Accepted: 12/23/2023] [Indexed: 01/14/2024] Open
Abstract
BACKGROUND With potential of deep learning in musculoskeletal image interpretation being explored, this paper focuses on the common site of rotator cuff tears, the supraspinatus. It aims to propose and validate a deep learning model to automatically extract the supraspinatus, verifying its superiority through comparison with several classical image segmentation models. METHOD Imaging data were retrospectively collected from 60 patients who underwent inpatient treatment for rotator cuff tears at a hospital between March 2021 and May 2023. A dataset of the supraspinatus from MRI was constructed after collecting, filtering, and manually annotating at the pixel level. This paper proposes a novel A-DAsppUnet network that can automatically extract the supraspinatus after training and optimization. The analysis of model performance is based on three evaluation metrics: precision, intersection over union, and Dice coefficient. RESULTS The experimental results demonstrate that the precision, intersection over union, and Dice coefficients of the proposed model are 99.20%, 83.38%, and 90.94%, respectively. Furthermore, the proposed model exhibited significant advantages over the compared models. CONCLUSION The designed model in this paper accurately extracts the supraspinatus from MRI, and the extraction results are complete and continuous with clear boundaries. The feasibility of using deep learning methods for musculoskeletal extraction and assisting in clinical decision-making was verified. This research holds practical significance and application value in the field of utilizing artificial intelligence for assisting medical decision-making.
Collapse
Affiliation(s)
- Peng Wang
- Third Clinical Medical School, Nanjing University of Chinese Medicine, Nanjing, 210023, People's Republic of China
- Affiliated Hospital of Integrated Traditional Chinese and Western Medicine, Nanjing University of Chinese Medicine, No. 100 Maigaoqiao Cross Street, Qixia District, Nanjing City, 210028, Jiangsu Province, People's Republic of China
| | - Yang Liu
- School of Remote Sensing and Geomatics Engineering, Nanjing University of Information Science & Technology, Nanjing, 210044, People's Republic of China
| | - Zhong Zhou
- Affiliated Hospital of Integrated Traditional Chinese and Western Medicine, Nanjing University of Chinese Medicine, No. 100 Maigaoqiao Cross Street, Qixia District, Nanjing City, 210028, Jiangsu Province, People's Republic of China.
| |
Collapse
|
42
|
Xue Y, Yang S, Sun W, Tan H, Lin K, Peng L, Wang Z, Zhang J. Approaching expert-level accuracy for differentiating ACL tear types on MRI with deep learning. Sci Rep 2024; 14:938. [PMID: 38195977 PMCID: PMC10776725 DOI: 10.1038/s41598-024-51666-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2023] [Accepted: 01/08/2024] [Indexed: 01/11/2024] Open
Abstract
Treatment for anterior cruciate ligament (ACL) tears depends on the condition of the ligament. We aimed to identify different tear statuses from preoperative MRI using deep learning-based radiomics with sex and age. We reviewed 862 patients with preoperative MRI scans reflecting ACL status from Hunan Provincial People's Hospital. Based on sagittal proton density-weighted images, a fully automated approach was developed that consisted of a deep learning model for segmenting ACL tissue (ACL-DNet) and a deep learning-based recognizer for ligament status classification (ACL-SNet). The efficacy of the proposed approach was evaluated by using the sensitivity, specificity and area under the receiver operating characteristic curve (AUC) and compared with that of a group of three orthopedists in the holdout test set. The ACL-DNet model yielded a Dice coefficient of 98% ± 6% on the MRI datasets. Our proposed classification model yielded a sensitivity of 97% and a specificity of 97%. In comparison, the sensitivity of alternative models ranged from 84 to 90%, while the specificity was between 86 and 92%. The AUC of the ACL-SNet model was 99%, demonstrating high overall diagnostic accuracy. The diagnostic performance of the clinical experts as reflected in the AUC was 96%, 92% and 88%, respectively. The fully automated model shows potential as a highly reliable and reproducible tool that allows orthopedists to noninvasively identify the ACL status and may aid in optimizing different techniques, such as ACL remnant preservation, for ACL reconstruction.
Collapse
Affiliation(s)
- Yang Xue
- School of Computer Science, Hunan First Normal University, Changsha, 410205, China
- Hunan Provincial Key Laboratory of Information Technology for Basic Education, Changsha, 410205, China
| | - Shu Yang
- Department of Orthopaedic, Hunan Provincial People's Hospital (The First Affiliated Hospital of Hunan Normal University), Changsha, 410002, China
| | - Wenjie Sun
- Department of Radiology, Hunan Provincial People's Hospital (The First Affiliated Hospital of Hunan Normal University), Changsha, 410002, China
| | - Hui Tan
- Department of Radiology, Hunan Provincial People's Hospital (The First Affiliated Hospital of Hunan Normal University), Changsha, 410002, China
| | - Kaibin Lin
- School of Computer Science, Hunan First Normal University, Changsha, 410205, China
- Hunan Provincial Key Laboratory of Information Technology for Basic Education, Changsha, 410205, China
| | - Li Peng
- School of Computer Science, Hunan First Normal University, Changsha, 410205, China
- Hunan Provincial Key Laboratory of Information Technology for Basic Education, Changsha, 410205, China
| | - Zheng Wang
- School of Computer Science, Hunan First Normal University, Changsha, 410205, China.
- Hunan Provincial Key Laboratory of Information Technology for Basic Education, Changsha, 410205, China.
| | - Jianglin Zhang
- Department of Dermatology, Shenzhen People's Hospital (The Second Clinical Medical College, Jinan University; The First Affiliated Hospital, Southern University of Science and Technology), Shenzhen, 518020, Guangdong, China.
- Candidate Branch of National Clinical Research Center for Skin Diseases, Shenzhen, 518020, Guangdong, China.
- Department of Geriatrics, Shenzhen People's Hospital, (The Second Clinical Medical College, Jinan University; The First Affiliated Hospital, Southern University of Science and Technology), Shenzhen, 518020, Guangdong, China.
| |
Collapse
|
43
|
Shao L, Chen B, Zhang Z, Zhang Z, Chen X. Artificial intelligence generated content (AIGC) in medicine: A narrative review. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2024; 21:1672-1711. [PMID: 38303483 DOI: 10.3934/mbe.2024073] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/03/2024]
Abstract
Recently, artificial intelligence generated content (AIGC) has been receiving increased attention and is growing exponentially. AIGC is generated based on the intentional information extracted from human-provided instructions by generative artificial intelligence (AI) models. AIGC quickly and automatically generates large amounts of high-quality content. Currently, there is a shortage of medical resources and complex medical procedures in medicine. Due to its characteristics, AIGC can help alleviate these problems. As a result, the application of AIGC in medicine has gained increased attention in recent years. Therefore, this paper provides a comprehensive review on the recent state of studies involving AIGC in medicine. First, we present an overview of AIGC. Furthermore, based on recent studies, the application of AIGC in medicine is reviewed from two aspects: medical image processing and medical text generation. The basic generative AI models, tasks, target organs, datasets and contribution of studies are considered and summarized. Finally, we also discuss the limitations and challenges faced by AIGC and propose possible solutions with relevant studies. We hope this review can help readers understand the potential of AIGC in medicine and obtain some innovative ideas in this field.
Collapse
Affiliation(s)
- Liangjing Shao
- Academy for Engineering & Technology, Fudan University, Shanghai 200433, China
- Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention, Fudan University, Shanghai 200032, China
| | - Benshuang Chen
- Academy for Engineering & Technology, Fudan University, Shanghai 200433, China
- Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention, Fudan University, Shanghai 200032, China
| | - Ziqun Zhang
- Information office, Fudan University, Shanghai 200032, China
| | - Zhen Zhang
- Baoshan Branch of Ren Ji Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 200444, China
| | - Xinrong Chen
- Academy for Engineering & Technology, Fudan University, Shanghai 200433, China
- Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention, Fudan University, Shanghai 200032, China
| |
Collapse
|
44
|
Amin A, Cardoso SA, Suyambu J, Abdus Saboor H, Cardoso RP, Husnain A, Isaac NV, Backing H, Mehmood D, Mehmood M, Maslamani ANJ. Future of Artificial Intelligence in Surgery: A Narrative Review. Cureus 2024; 16:e51631. [PMID: 38318552 PMCID: PMC10839429 DOI: 10.7759/cureus.51631] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/03/2024] [Indexed: 02/07/2024] Open
Abstract
Artificial intelligence (AI) is the capability of a machine to execute cognitive processes that are typically considered to be functions of the human brain. It is the study of algorithms that enable machines to reason and perform mental tasks, including problem-solving, object and word recognition, and decision-making. Once considered science fiction, AI today is a fact and an increasingly prevalent subject in both academic and popular literature. It is expected to reshape medicine, benefiting both healthcare professionals and patients. Machine learning (ML) is a subset of AI that allows machines to learn and make predictions by recognizing patterns, thus empowering the medical team to deliver better care to patients through accurate diagnosis and treatment. ML is expanding its footprint in a variety of surgical specialties, including general surgery, ophthalmology, cardiothoracic surgery, and vascular surgery, to name a few. In recent years, we have seen AI make its way into the operating theatres. Though it has not yet been able to replace the surgeon, it has the potential to become a highly valuable surgical tool. Rest assured that the day is not far off when AI shall play a significant intraoperative role, a projection that is currently marred by safety concerns. This review aims to explore the present application of AI in various surgical disciplines and how it benefits both patients and physicians, as well as the current obstacles and limitations facing its seemingly unstoppable rise.
Collapse
Affiliation(s)
- Aamir Amin
- Cardiothoracic Surgery, Harefield Hospital, Guy's and St Thomas' NHS Foundation Trust, London, GBR
| | - Swizel Ann Cardoso
- Major Trauma Services, University Hospital Birmingham NHS Foundation Trust DC, Birmingham, GBR
| | - Jenisha Suyambu
- Medicine, University of Perpetual Help System Data - Jonelta Foundation School of Medicine, Las Piñas, PHL
| | | | - Rayner P Cardoso
- Medicine and Surgery, All India Institute of Medical Sciences, Jodhpur, Jodhpur, IND
| | - Ali Husnain
- Radiology, Northwestern University, Lahore, PAK
| | - Natasha Varghese Isaac
- Medicine and Surgery, St John's Medical College Hospital, Rajiv Gandhi University of Health Sciences, Bengaluru, IND
| | - Haydee Backing
- Medicine, Universidad de San Martin de Porres, Lima, PER
| | - Dalia Mehmood
- Community Medicine, Fatima Jinnah Medical University, Lahore, PAK
| | - Maria Mehmood
- Internal Medicine, Shalamar Medical and Dental College, Lahore, PAK
| | | |
Collapse
|
45
|
Lu T, Fang Y, Liu H, Chen C, Li T, Lu M, Song D. Comparison of Machine Learning and Logic Regression Algorithms for Predicting Lymph Node Metastasis in Patients with Gastric Cancer: A two-Center Study. Technol Cancer Res Treat 2024; 23:15330338231222331. [PMID: 38190617 PMCID: PMC10775719 DOI: 10.1177/15330338231222331] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2023] [Revised: 11/01/2023] [Accepted: 11/20/2023] [Indexed: 01/10/2024] Open
Abstract
OBJECTIVES This two-center study aimed to establish a model for predicting the risk of lymph node metastasis in gastric cancer patients using machine learning (ML) and logistic regression (LR) algorithms, and to evaluate its predictive performance in clinical practice. METHODS Data of a total of 369 patients who underwent radical gastrectomy in the Department of General Surgery of Affiliated Hospital of Xuzhou Medical University (Xuzhou, China) from March 2016 to November 2019 were collected and retrospectively analyzed as the training group. In addition, data of 123 patients who underwent radical gastrectomy in the Department of General Surgery of Jining First People's Hospital (Jining, China) were collected and analyzed as the verification group. Besides, 7 ML and logistic models were developed, including decision tree, random forest, support vector machine (SVM), gradient boosting machine (GBM), naive Bayes, neural network, and LR, in order to evaluate the occurrence of lymph node metastasis in patients with gastric cancer. The ML model was established following 10 cross-validation iterations within the training dataset, and subsequently, each model was assessed using the test dataset. The model's performance was evaluated by comparing the area under the receiver operating characteristic curve of each model. RESULTS Compared with the traditional logistic model, among the 7 ML algorithms, except for SVM, the other models exhibited higher accuracy and reliability, and the influences of various risk factors on the model were more intuitive. CONCLUSION For the prediction of lymph node metastasis in gastric cancer patients, the ML algorithm outperformed traditional LR, and the GBM algorithm exhibited the most robust predictive capability.
Collapse
Affiliation(s)
- Tong Lu
- Department of emergency medicine, Jining No.1 People's Hospital, Jining, China
| | - Yu Fang
- Jiangsu Normal University, Xuzhou, China
| | - Haonan Liu
- Department of Oncology, The Affiliated Hospital of Xuzhou Medical University, Xuzhou, China
| | - Chong Chen
- Department of Gastroenterology, Xuzhou No.1 People's Hospital, Xuzhou, China
| | - Taotao Li
- Department of emergency medicine, Jining No.1 People's Hospital, Jining, China
| | - Miao Lu
- Wuxi Mental Health Center, Wuxi, China
| | - Daqing Song
- Department of emergency medicine, Jining No.1 People's Hospital, Jining, China
| |
Collapse
|
46
|
Azad R, Kazerouni A, Heidari M, Aghdam EK, Molaei A, Jia Y, Jose A, Roy R, Merhof D. Advances in medical image analysis with vision Transformers: A comprehensive review. Med Image Anal 2024; 91:103000. [PMID: 37883822 DOI: 10.1016/j.media.2023.103000] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2023] [Revised: 09/30/2023] [Accepted: 10/11/2023] [Indexed: 10/28/2023]
Abstract
The remarkable performance of the Transformer architecture in natural language processing has recently also triggered broad interest in Computer Vision. Among other merits, Transformers are witnessed as capable of learning long-range dependencies and spatial correlations, which is a clear advantage over convolutional neural networks (CNNs), which have been the de facto standard in Computer Vision problems so far. Thus, Transformers have become an integral part of modern medical image analysis. In this review, we provide an encyclopedic review of the applications of Transformers in medical imaging. Specifically, we present a systematic and thorough review of relevant recent Transformer literature for different medical image analysis tasks, including classification, segmentation, detection, registration, synthesis, and clinical report generation. For each of these applications, we investigate the novelty, strengths and weaknesses of the different proposed strategies and develop taxonomies highlighting key properties and contributions. Further, if applicable, we outline current benchmarks on different datasets. Finally, we summarize key challenges and discuss different future research directions. In addition, we have provided cited papers with their corresponding implementations in https://github.com/mindflow-institue/Awesome-Transformer.
Collapse
Affiliation(s)
- Reza Azad
- Faculty of Electrical Engineering and Information Technology, RWTH Aachen University, Aachen, Germany
| | - Amirhossein Kazerouni
- School of Electrical Engineering, Iran University of Science and Technology, Tehran, Iran
| | - Moein Heidari
- School of Electrical Engineering, Iran University of Science and Technology, Tehran, Iran
| | | | - Amirali Molaei
- School of Computer Engineering, Iran University of Science and Technology, Tehran, Iran
| | - Yiwei Jia
- Faculty of Electrical Engineering and Information Technology, RWTH Aachen University, Aachen, Germany
| | - Abin Jose
- Faculty of Electrical Engineering and Information Technology, RWTH Aachen University, Aachen, Germany
| | - Rijo Roy
- Faculty of Electrical Engineering and Information Technology, RWTH Aachen University, Aachen, Germany
| | - Dorit Merhof
- Faculty of Informatics and Data Science, University of Regensburg, Regensburg, Germany; Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany.
| |
Collapse
|
47
|
Huang Z, Ding Y, Yang Y, Zhao S, Zhang S, Xiao J, Ding C, Guo N, Li Z, Zhou S, Cao G, Wang X. Performance of machine learning-based coronary computed tomography angiography for selecting revascularization candidates. Acta Radiol 2024; 65:123-132. [PMID: 36847335 DOI: 10.1177/02841851231158730] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/01/2023]
Abstract
BACKGROUND Limited studies have investigated the accuracy of therapeutic decision-making using machine learning-based coronary computed tomography angiography (ML-CCTA) compared with CCTA. PURPOSE To investigate the performance of ML-CCTA for therapeutic decision compared with CCTA. MATERIAL AND METHODS The study population consisted of 322 consecutive patients with stable coronary artery disease. The SYNTAX score was calculated with an online calculator based on ML-CCTA results. Therapeutic decision-making was determined by ML-CCTA results and the ML-CCTA-based SYNTAX score. The therapeutic strategy and the appropriate revascularization procedure were selected using ML-CCTA, CCTA, and invasive coronary angiography (ICA) independently. RESULTS The sensitivity, specificity, positive predictive value, negative predictive value, accuracy of ML-CCTA and CCTA for selecting revascularization candidates were 87.01%, 96.43%, 95.71%, 89.01%, 91.93%, and 85.71%, 87.50%, 86.27%, 86.98%, 86.65%, respectively, using ICA as the standard reference. The area under the receiver operating characteristic curve (AUC) of ML-CCTA for selecting revascularization candidates was significantly higher than CCTA (0.917 vs. 0.866, P = 0.016). Subgroup analysis showed the AUC of ML-CCTA for selecting percutaneous coronary intervention (PCI) or coronary artery bypass graft (CABG) candidates was significantly higher than CCTA (0.883 vs. 0.777, P < 0.001, 0.912 vs. 0.826, P = 0.003, respectively). CONCLUSION ML-CCTA could distinguish between patients who need revascularization and those who do not. In addition, ML-CCTA showed a slightly superior to CCTA in making an appropriate decision for patients and selecting a suitable revascularization strategy.
Collapse
Affiliation(s)
- Zengfa Huang
- Department of Radiology, The Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| | - Yi Ding
- Department of Radiology, The Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| | - Yang Yang
- Department of Radiology, The Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| | - Shengchao Zhao
- Department of Radiology, The Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| | - Shutong Zhang
- Department of Radiology, The Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| | - Jianwei Xiao
- Department of Radiology, The Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| | - Chengyu Ding
- Shukun (Beijing) Technology Co., Ltd., Beijing, China
| | - Ning Guo
- Shukun (Beijing) Technology Co., Ltd., Beijing, China
| | - Zuoqin Li
- Department of Radiology, The Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| | - Shiguang Zhou
- Department of Radiology, The Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| | - Guijuan Cao
- Department of Radiology, The Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| | - Xiang Wang
- Department of Radiology, The Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| |
Collapse
|
48
|
Wang C, Piao S, Huang Z, Gao Q, Zhang J, Li Y, Shan H. Joint learning framework of cross-modal synthesis and diagnosis for Alzheimer's disease by mining underlying shared modality information. Med Image Anal 2024; 91:103032. [PMID: 37995628 DOI: 10.1016/j.media.2023.103032] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2022] [Revised: 08/31/2023] [Accepted: 11/13/2023] [Indexed: 11/25/2023]
Abstract
Alzheimer's disease (AD) is one of the most common neurodegenerative disorders presenting irreversible progression of cognitive impairment. How to identify AD as early as possible is critical for intervention with potential preventive measures. Among various neuroimaging modalities used to diagnose AD, functional positron emission tomography (PET) has higher sensitivity than structural magnetic resonance imaging (MRI), but it is also costlier and often not available in many hospitals. How to leverage massive unpaired unlabeled PET to improve the diagnosis performance of AD from MRI becomes rather important. To address this challenge, this paper proposes a novel joint learning framework of unsupervised cross-modal synthesis and AD diagnosis by mining underlying shared modality information, improving the AD diagnosis from MRI while synthesizing more discriminative PET images. We mine underlying shared modality information in two aspects: diversifying modality information through the cross-modal synthesis network and locating critical diagnosis-related patterns through the AD diagnosis network. First, to diversify the modality information, we propose a novel unsupervised cross-modal synthesis network, which implements the inter-conversion between 3D PET and MRI in a single model modulated by the AdaIN module. Second, to locate shared critical diagnosis-related patterns, we propose an interpretable diagnosis network based on fully 2D convolutions, which takes either 3D synthesized PET or original MRI as input. Extensive experimental results on the ADNI dataset show that our framework can synthesize more realistic images, outperform the state-of-the-art AD diagnosis methods, and have better generalization on external AIBL and NACC datasets.
Collapse
Affiliation(s)
- Chenhui Wang
- Institute of Science and Technology for Brain-inspired Intelligence, Fudan University, Shanghai 200433, China
| | - Sirong Piao
- Department of Radiology, Huashan Hospital, Fudan University, Shanghai 200040, China
| | - Zhizhong Huang
- Shanghai Key Lab of Intelligent Information Processing, Fudan University, Shanghai 200433, China; School of Computer Science, Fudan University, Shanghai 200433, China
| | - Qi Gao
- Institute of Science and Technology for Brain-inspired Intelligence, Fudan University, Shanghai 200433, China
| | - Junping Zhang
- Shanghai Key Lab of Intelligent Information Processing, Fudan University, Shanghai 200433, China; School of Computer Science, Fudan University, Shanghai 200433, China
| | - Yuxin Li
- Department of Radiology, Huashan Hospital, Fudan University, Shanghai 200040, China.
| | - Hongming Shan
- Institute of Science and Technology for Brain-inspired Intelligence, Fudan University, Shanghai 200433, China; MOE Frontiers Center for Brain Science, Fudan University, Shanghai, 200433, China; MOE Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence, Fudan University, Shanghai, 200433, China; Shanghai Center for Brain Science and Brain-inspired Technology, Shanghai 201210, China.
| |
Collapse
|
49
|
Hammer S, Nunes DW, Hammer M, Zeman F, Akers M, Götz A, Balla A, Doppler MC, Fellner C, Platz Batista da Silva N, Thurn S, Verloh N, Stroszczynski C, Wohlgemuth WA, Palm C, Uller W. Deep learning-based differentiation of peripheral high-flow and low-flow vascular malformations in T2-weighted short tau inversion recovery MRI. Clin Hemorheol Microcirc 2024; 87:221-235. [PMID: 38306026 DOI: 10.3233/ch-232071] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/03/2024]
Abstract
BACKGROUND Differentiation of high-flow from low-flow vascular malformations (VMs) is crucial for therapeutic management of this orphan disease. OBJECTIVE A convolutional neural network (CNN) was evaluated for differentiation of peripheral vascular malformations (VMs) on T2-weighted short tau inversion recovery (STIR) MRI. METHODS 527 MRIs (386 low-flow and 141 high-flow VMs) were randomly divided into training, validation and test set for this single-center study. 1) Results of the CNN's diagnostic performance were compared with that of two expert and four junior radiologists. 2) The influence of CNN's prediction on the radiologists' performance and diagnostic certainty was evaluated. 3) Junior radiologists' performance after self-training was compared with that of the CNN. RESULTS Compared with the expert radiologists the CNN achieved similar accuracy (92% vs. 97%, p = 0.11), sensitivity (80% vs. 93%, p = 0.16) and specificity (97% vs. 100%, p = 0.50). In comparison to the junior radiologists, the CNN had a higher specificity and accuracy (97% vs. 80%, p < 0.001; 92% vs. 77%, p < 0.001). CNN assistance had no significant influence on their diagnostic performance and certainty. After self-training, the junior radiologists' specificity and accuracy improved and were comparable to that of the CNN. CONCLUSIONS Diagnostic performance of the CNN for differentiating high-flow from low-flow VM was comparable to that of expert radiologists. CNN did not significantly improve the simulated daily practice of junior radiologists, self-training was more effective.
Collapse
Affiliation(s)
- Simone Hammer
- Department of Radiology, Faculty of Medicine, Medical Center University of Regensburg, University of Regensburg, Regensburg, Germany
| | - Danilo Weber Nunes
- Regensburg Medical Image Computing (ReMIC), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Regensburg, Germany
| | - Michael Hammer
- Regensburg Medical Image Computing (ReMIC), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Regensburg, Germany
| | - Florian Zeman
- Faculty of Medicine, Center for Clinical Trials, Medical Center University of Regensburg, University of Regensburg, Regensburg, Germany
| | - Michael Akers
- Department of Radiology, Faculty of Medicine, Medical Center University of Regensburg, University of Regensburg, Regensburg, Germany
| | - Andrea Götz
- Department of Radiology, Faculty of Medicine, Medical Center University of Regensburg, University of Regensburg, Regensburg, Germany
| | - Annika Balla
- Department of Radiology, Faculty of Medicine, Medical Center University of Regensburg, University of Regensburg, Regensburg, Germany
| | - Michael Christian Doppler
- Department of Diagnostic and Interventional Radiology, Faculty of Medicine, Medical Center University of Freiburg, University of Freiburg, Freiburg, Germany
| | - Claudia Fellner
- Department of Radiology, Faculty of Medicine, Medical Center University of Regensburg, University of Regensburg, Regensburg, Germany
| | - Natascha Platz Batista da Silva
- Department of Radiology, Faculty of Medicine, Medical Center University of Regensburg, University of Regensburg, Regensburg, Germany
| | - Sylvia Thurn
- Department of Radiology, Faculty of Medicine, Medical Center University of Regensburg, University of Regensburg, Regensburg, Germany
| | - Niklas Verloh
- Department of Diagnostic and Interventional Radiology, Faculty of Medicine, Medical Center University of Freiburg, University of Freiburg, Freiburg, Germany
| | - Christian Stroszczynski
- Department of Radiology, Faculty of Medicine, Medical Center University of Regensburg, University of Regensburg, Regensburg, Germany
| | - Walter Alexander Wohlgemuth
- Department of Radiology, Faculty of Medicine, Medical Center University of Halle (Saale), University of Halle (Saale), Halle, Germany
| | - Christoph Palm
- Regensburg Medical Image Computing (ReMIC), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Regensburg, Germany
- Regensburg Center of Biomedical Engineering (RCBE), OTH Regensburg and University of Regensburg, Regensburg, Germany
| | - Wibke Uller
- Department of Diagnostic and Interventional Radiology, Faculty of Medicine, Medical Center University of Freiburg, University of Freiburg, Freiburg, Germany
| |
Collapse
|
50
|
Kasuya S, Inaoka T, Wada A, Nakatsuka T, Nakagawa K, Terada H. Feasibility of the fat-suppression image-subtraction method using deep learning for abnormality detection on knee MRI. Pol J Radiol 2023; 88:e562-e573. [PMID: 38362017 PMCID: PMC10867951 DOI: 10.5114/pjr.2023.133660] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2023] [Accepted: 09/04/2023] [Indexed: 02/17/2024] Open
Abstract
Purpose To evaluate the feasibility of using a deep learning (DL) model to generate fat-suppression images and detect abnormalities on knee magnetic resonance imaging (MRI) through the fat-suppression image-subtraction method. Material and methods A total of 45 knee MRI studies in patients with knee disorders and 12 knee MRI studies in healthy volunteers were enrolled. The DL model was developed using 2-dimensional convolutional neural networks for generating fat-suppression images and subtracting generated fat-suppression images without any abnormal findings from those with normal/abnormal findings and detecting/classifying abnormalities on knee MRI. The image qualities of the generated fat-suppression images and subtraction-images were assessed. The accuracy, average precision, average recall, F-measure, sensitivity, and area under the receiver operator characteristic curve (AUROC) of DL for each abnormality were calculated. Results A total of 2472 image datasets, each consisting of one slice of original T1WI, original intermediate-weighted images, generated fat-suppression (FS)-intermediate-weighted images without any abnormal findings, generated FS-intermediate-weighted images with normal/abnormal findings, and subtraction images between the generated FS-intermediate-weighted images at the same cross-section, were created. The generated fat-suppression images were of adequate image quality. Of the 2472 subtraction-images, 2203 (89.1%) were judged to be of adequate image quality. The accuracies for overall abnormalities, anterior cruciate ligament, bone marrow, cartilage, meniscus, and others were 89.5-95.1%. The average precision, average recall, and F-measure were 73.4-90.6%, 77.5-89.4%, and 78.4-89.4%, respectively. The sensitivity was 57.4-90.5%. The AUROCs were 0.910-0.979. Conclusions The DL model was able to generate fat-suppression images of sufficient quality to detect abnormalities on knee MRI through the fat-suppression image-subtraction method.
Collapse
Affiliation(s)
- Shusuke Kasuya
- Department of Radiology, Toho University Sakura Medical Center, Sakura, Japan
| | - Tsutomu Inaoka
- Department of Radiology, Toho University Sakura Medical Center, Sakura, Japan
| | - Akihiko Wada
- Department of Radiology, Juntendo University, Tokyo, Japan
| | - Tomoya Nakatsuka
- Department of Radiology, Toho University Sakura Medical Center, Sakura, Japan
| | - Koichi Nakagawa
- Department of Orthopaedic Surgery, Toho University Sakura Medical Center, Sakura, Japan
| | - Hitoshi Terada
- Department of Radiology, Toho University Sakura Medical Center, Sakura, Japan
| |
Collapse
|