1
|
Bozzo A, Tsui JMG, Bhatnagar S, Forsberg J. Deep Learning and Multimodal Artificial Intelligence in Orthopaedic Surgery. J Am Acad Orthop Surg 2024; 32:e523-e532. [PMID: 38652882 PMCID: PMC11075751 DOI: 10.5435/jaaos-d-23-00831] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/13/2023] [Revised: 02/13/2024] [Accepted: 03/01/2024] [Indexed: 04/25/2024] Open
Abstract
This review article focuses on the applications of deep learning with neural networks and multimodal neural networks in the orthopaedic domain. By providing practical examples of how artificial intelligence (AI) is being applied successfully in orthopaedic surgery, particularly in the realm of imaging data sets and the integration of clinical data, this study aims to provide orthopaedic surgeons with the necessary tools to not only evaluate existing literature but also to consider AI's potential in their own clinical or research pursuits. We first review standard deep neural networks which can analyze numerical clinical variables, then describe convolutional neural networks which can analyze image data, and then introduce multimodal AI models which analyze various types of different data. Then, we contrast these deep learning techniques with related but more limited techniques such as radiomics, describe how to interpret deep learning studies, and how to initiate such studies at your institution. Ultimately, by empowering orthopaedic surgeons with the knowledge and know-how of deep learning, this review aspires to facilitate the translation of research into clinical practice, thereby enhancing the efficacy and precision of real-world orthopaedic care for patients.
Collapse
Affiliation(s)
- Anthony Bozzo
- From the Division of Orthopaedic Surgery, McGill University, Canada (Bozzo), the Division of Radiation Oncology, McGill University, Canada (Tsui), the Department of Epidemiology and Biostatistics, Department of Diagnostic Radiology, McGill University, Canada (Bhatnagar), and the Memorial Sloan Kettering Cancer Center (Forsberg)
| | - James M. G. Tsui
- From the Division of Orthopaedic Surgery, McGill University, Canada (Bozzo), the Division of Radiation Oncology, McGill University, Canada (Tsui), the Department of Epidemiology and Biostatistics, Department of Diagnostic Radiology, McGill University, Canada (Bhatnagar), and the Memorial Sloan Kettering Cancer Center (Forsberg)
| | - Sahir Bhatnagar
- From the Division of Orthopaedic Surgery, McGill University, Canada (Bozzo), the Division of Radiation Oncology, McGill University, Canada (Tsui), the Department of Epidemiology and Biostatistics, Department of Diagnostic Radiology, McGill University, Canada (Bhatnagar), and the Memorial Sloan Kettering Cancer Center (Forsberg)
| | - Jonathan Forsberg
- From the Division of Orthopaedic Surgery, McGill University, Canada (Bozzo), the Division of Radiation Oncology, McGill University, Canada (Tsui), the Department of Epidemiology and Biostatistics, Department of Diagnostic Radiology, McGill University, Canada (Bhatnagar), and the Memorial Sloan Kettering Cancer Center (Forsberg)
| |
Collapse
|
2
|
Su Q, Wang N, Wang B, Wang Y, Dai Z, Zhao X, Li X, Li Q, Yang G, Nie P. Ct-based intratumoral and peritumoral radiomics for predicting prognosis in osteosarcoma: A multicenter study. Eur J Radiol 2024; 172:111350. [PMID: 38309216 DOI: 10.1016/j.ejrad.2024.111350] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2023] [Revised: 01/09/2024] [Accepted: 01/25/2024] [Indexed: 02/05/2024]
Abstract
PURPOSE To evaluate the performance of CT-based intratumoral, peritumoral and combined radiomics signatures in predicting prognosis in patients with osteosarcoma. METHODS The data of 202 patients (training cohort:102, testing cohort:100) with osteosarcoma admitted to the two hospitals from August 2008 to February 2022 were retrospectively analyzed. Progression free survival (PFS) and overall survival (OS) were used as the end points. The radiomics features were extracted from CT images, three radiomics signatures(RSintratumoral, RSperitumoral, RScombined)were constructed based on intratumoral, peritumoral and combined radiomics features, respectively, and the radiomics score (Rad-score) were calculated. Kaplan-Meier survival analysis was used to evaluate the relationship between the Rad-score with PFS and OS, the Harrell's concordance index (C-index) was used to evaluate the predictive performance of the radiomics signatures. RESULTS Finally, 8, 6, and 21 features were selected for the establishment of RSintratumoral, RSperitumoral, and RScombined, respectively. Kaplan-Meier survival analysis confirmed that the Rad-scores of the three RSs were significantly correlated with the PFS and OS of patients with osteosarcoma. Among the three radiomics signatures, RScombined had better predictive performance, the C-index of PSF prediction was 0.833 in the training cohort and 0.814 in the testing cohort, the C-index of OS prediction was 0.796 in the training cohort and 0.764 in the testing cohort. CONCLUSIONS CT-based intratumoral, peritumoral and combined radiomics signatures can predict the prognosis of patients with osteosarcoma, which may assist in individualized treatment and improving the prognosis of osteosarcoma patients.
Collapse
Affiliation(s)
- Qiushi Su
- Department of Radiology, the Affiliated Hospital of Qingdao University, Qingdao, Shandong, China
| | - Ning Wang
- Department of Radiology, Shandong Provincial Hospital Affiliated to Shandong First Medical University, Jinan, Shandong, China
| | - Bingyan Wang
- Department of Ultrasound, the Affiliated Hospital of Qingdao University, Qingdao, Shandong, China
| | | | - Zhengjun Dai
- Scientific Research Department, Huiying Medical Technology Co., Ltd, Beijing, China
| | - Xia Zhao
- Department of Radiology, the Affiliated Hospital of Shandong University of Traditional Chinese Medicine, Jinan, Shandong, China
| | - Xiaoli Li
- Department of Radiology, the Affiliated Hospital of Qingdao University, Qingdao, Shandong, China
| | - Qiyuan Li
- Department of Radiology, the Affiliated Hospital of Qingdao University, Qingdao, Shandong, China
| | - Guangjie Yang
- Department of Nuclear Medicine, the Affiliated Hospital of Qingdao University, Qingdao, Shandong, China.
| | - Pei Nie
- Department of Radiology, the Affiliated Hospital of Qingdao University, Qingdao, Shandong, China.
| |
Collapse
|
3
|
Gitto S, Cuocolo R, Huisman M, Messina C, Albano D, Omoumi P, Kotter E, Maas M, Van Ooijen P, Sconfienza LM. CT and MRI radiomics of bone and soft-tissue sarcomas: an updated systematic review of reproducibility and validation strategies. Insights Imaging 2024; 15:54. [PMID: 38411750 PMCID: PMC10899555 DOI: 10.1186/s13244-024-01614-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2023] [Accepted: 01/09/2024] [Indexed: 02/28/2024] Open
Abstract
OBJECTIVE To systematically review radiomic feature reproducibility and model validation strategies in recent studies dealing with CT and MRI radiomics of bone and soft-tissue sarcomas, thus updating a previous version of this review which included studies published up to 2020. METHODS A literature search was conducted on EMBASE and PubMed databases for papers published between January 2021 and March 2023. Data regarding radiomic feature reproducibility and model validation strategies were extracted and analyzed. RESULTS Out of 201 identified papers, 55 were included. They dealt with radiomics of bone (n = 23) or soft-tissue (n = 32) tumors. Thirty-two (out of 54 employing manual or semiautomatic segmentation, 59%) studies included a feature reproducibility analysis. Reproducibility was assessed based on intra/interobserver segmentation variability in 30 (55%) and geometrical transformations of the region of interest in 2 (4%) studies. At least one machine learning validation technique was used for model development in 34 (62%) papers, and K-fold cross-validation was employed most frequently. A clinical validation of the model was reported in 38 (69%) papers. It was performed using a separate dataset from the primary institution (internal test) in 22 (40%), an independent dataset from another institution (external test) in 14 (25%) and both in 2 (4%) studies. CONCLUSIONS Compared to papers published up to 2020, a clear improvement was noted with almost double publications reporting methodological aspects related to reproducibility and validation. Larger multicenter investigations including external clinical validation and the publication of databases in open-access repositories could further improve methodology and bring radiomics from a research area to the clinical stage. CRITICAL RELEVANCE STATEMENT An improvement in feature reproducibility and model validation strategies has been shown in this updated systematic review on radiomics of bone and soft-tissue sarcomas, highlighting efforts to enhance methodology and bring radiomics from a research area to the clinical stage. KEY POINTS • 2021-2023 radiomic studies on CT and MRI of musculoskeletal sarcomas were reviewed. • Feature reproducibility was assessed in more than half (59%) of the studies. • Model clinical validation was performed in 69% of the studies. • Internal (44%) and/or external (29%) test datasets were employed for clinical validation.
Collapse
Affiliation(s)
- Salvatore Gitto
- Dipartimento di Scienze Biomediche per la Salute, Università degli Studi di Milano, Milan, Italy
- IRCCS Istituto Ortopedico Galeazzi, Milan, Italy
| | - Renato Cuocolo
- Department of Medicine, Surgery and Dentistry, University of Salerno, Baronissi, Italy
| | - Merel Huisman
- Radboud University Medical Center, Department of Radiology and Nuclear Medicine, Nijmegen, The Netherlands
| | - Carmelo Messina
- Dipartimento di Scienze Biomediche per la Salute, Università degli Studi di Milano, Milan, Italy
- IRCCS Istituto Ortopedico Galeazzi, Milan, Italy
| | - Domenico Albano
- IRCCS Istituto Ortopedico Galeazzi, Milan, Italy
- Dipartimento di Scienze Biomediche, Chirurgiche ed Odontoiatriche, Università degli Studi di Milano, Milan, Italy
| | - Patrick Omoumi
- Department of Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| | - Elmar Kotter
- Department of Radiology, Freiburg University Medical Center, Freiburg, Germany
| | - Mario Maas
- Department of Radiology and Nuclear Medicine, Amsterdam UMC Location University of Amsterdam, Amsterdam, The Netherlands
| | - Peter Van Ooijen
- Department of Radiation Oncology, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands
| | - Luca Maria Sconfienza
- Dipartimento di Scienze Biomediche per la Salute, Università degli Studi di Milano, Milan, Italy.
- IRCCS Istituto Ortopedico Galeazzi, Milan, Italy.
| |
Collapse
|
4
|
Brandenberger D, White LM. Radiomics in Musculoskeletal Tumors. Semin Musculoskelet Radiol 2024; 28:49-61. [PMID: 38330970 DOI: 10.1055/s-0043-1776428] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/10/2024]
Abstract
Sarcomas are heterogeneous rare tumors predominantly affecting the musculoskeletal (MSK) system. Due to significant variations in their natural history and variable response to conventional treatments, the discovery of novel diagnostic and prognostic biomarkers to guide therapeutic decision-making is an active and ongoing field of research. As new cellular, molecular, and metabolic biomarkers continue to be discovered, quantitative radiologic imaging is becoming increasingly important in sarcoma management. Radiomics offers the potential for discovering novel imaging diagnostic and predictive biomarkers using standard-of-care medical imaging. In this review, we detail the core concepts of radiomics and the application of radiomics to date in MSK sarcoma research. Also described are specific challenges related to radiomic studies, as well as viewpoints on clinical adoption and future perspectives in the field.
Collapse
Affiliation(s)
- Daniel Brandenberger
- Department of Medical Imaging, Musculoskeletal Imaging, University of Toronto, Toronto, Ontario, Canada
- Institut für Radiologie und Nuklearmedizin, Kantonsspital Baselland, Liestal, Switzerland
- Toronto Joint Department of Medical Imaging, University Health Network, Sinai Health System, and Women's College Hospital, Mount Sinai Hospital, Toronto, Ontario, Canada
| | - Lawrence M White
- Department of Medical Imaging, Musculoskeletal Imaging, University of Toronto, Toronto, Ontario, Canada
- Toronto Joint Department of Medical Imaging, University Health Network, Sinai Health System, and Women's College Hospital, Mount Sinai Hospital, Toronto, Ontario, Canada
| |
Collapse
|
5
|
Recht MP, White LM, Fritz J, Resnick DL. Advances in Musculoskeletal Imaging: Recent Developments and Predictions for the Future. Radiology 2023; 308:e230615. [PMID: 37642575 DOI: 10.1148/radiol.230615] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/31/2023]
Affiliation(s)
- Michael P Recht
- From the Department of Radiology, NYU Grossman School of Medicine, 660 First Ave, 3rd Floor, New York, NY 10016 (M.P.R., J.F.); Department of Medical Imaging, University Health Network, Sinai Health System and Women's College Hospital, Toronto, Canada (L.M.W.); and Department of Radiology, UCSD Teleradiology and Education Center, La Jolla, Calif (D.L.R.)
| | - Lawrence M White
- From the Department of Radiology, NYU Grossman School of Medicine, 660 First Ave, 3rd Floor, New York, NY 10016 (M.P.R., J.F.); Department of Medical Imaging, University Health Network, Sinai Health System and Women's College Hospital, Toronto, Canada (L.M.W.); and Department of Radiology, UCSD Teleradiology and Education Center, La Jolla, Calif (D.L.R.)
| | - Jan Fritz
- From the Department of Radiology, NYU Grossman School of Medicine, 660 First Ave, 3rd Floor, New York, NY 10016 (M.P.R., J.F.); Department of Medical Imaging, University Health Network, Sinai Health System and Women's College Hospital, Toronto, Canada (L.M.W.); and Department of Radiology, UCSD Teleradiology and Education Center, La Jolla, Calif (D.L.R.)
| | - Donald L Resnick
- From the Department of Radiology, NYU Grossman School of Medicine, 660 First Ave, 3rd Floor, New York, NY 10016 (M.P.R., J.F.); Department of Medical Imaging, University Health Network, Sinai Health System and Women's College Hospital, Toronto, Canada (L.M.W.); and Department of Radiology, UCSD Teleradiology and Education Center, La Jolla, Calif (D.L.R.)
| |
Collapse
|