1
|
Weber MA. Easily missed pathologies of the musculoskeletal system in the emergency radiology setting. ROFO-FORTSCHR RONTG 2024. [PMID: 39094774 DOI: 10.1055/a-2369-8330] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/04/2024]
Abstract
The musculoskeletal region is the main area in terms of easily missed pathologies in the emergency radiology setting, because the majority of diagnoses missed in the emergency setting are fractures.A review of the literature was performed by searching the PubMed and ScienceDirect databases, using the keywords ('missed injuries' or 'missed fractures') and ('emergency radiology' or 'emergency room') and ('musculoskeletal' or 'bone' or 'skeleton') for the title and abstract query. The inclusion criteria were scientific papers presented in the English and German languages. Among the 347 relevant hits between 1980 and 2024 as identified by the author of this review article, there were 114 relevant articles from the years between 2018 and 2024. Based on this literature search and the author's personal experience, this study presents useful information for reducing the number of missed pathologies in the musculoskeletal system in the emergency radiology setting.Predominant factors that make up the majority of missed fractures are 'subtle but still visible fractures' and 'radiographically imperceptible fractures'. Radiologists are able to minimize the factors contributing to fractures being missed. For example, implementing a 'four-eyes principle', i.e., two readers read the radiographs, would help to overcome the missing of 'subtle but still visible fractures' and the additional use of cross-sectional imaging would help to overcome the missing of 'radiographically imperceptible fractures'. Knowledge of what is commonly missed and evaluation of high-risk areas with utmost care also increase the diagnostic performance of radiologists. · Radiological imaging in an emergency setting increases the likelihood of radiological diagnostic errors, such as missing musculoskeletal pathologies.. · The majority of diagnoses missed in the emergency setting are fractures.. · To lessen the number of easily missed pathologies in the musculoskeletal system in the emergency radiology setting, a systematic approach is necessary.. · Adequate training of radiologists in emergency radiology and close collaboration with clinical partners are important measures to decrease the number of missed musculoskeletal injuries.. · Weber MA. Easily missed pathologies of the musculoskeletal system in the emergency radiology setting. Fortschr Röntgenstr 2024; DOI 10.1055/a-2369-8330.
Collapse
Affiliation(s)
- Marc-André Weber
- Institute of Diagnostic and Interventional Radiology, Pediatric Radiology and Neuroradiology, University Medical Center Rostock, Rostock, Germany
| |
Collapse
|
2
|
Nowroozi A, Salehi MA, Shobeiri P, Agahi S, Momtazmanesh S, Kaviani P, Kalra MK. Artificial intelligence diagnostic accuracy in fracture detection from plain radiographs and comparing it with clinicians: a systematic review and meta-analysis. Clin Radiol 2024; 79:579-588. [PMID: 38772766 DOI: 10.1016/j.crad.2024.04.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2024] [Revised: 04/09/2024] [Accepted: 04/15/2024] [Indexed: 05/23/2024]
Abstract
PURPOSE Fracture detection is one of the most commonly used and studied aspects of artificial intelligence (AI) in medicine. In this systematic review and meta-analysis, we aimed to summarize available literature and data regarding AI performance in fracture detection on plain radiographs and various factors affecting it. METHODS We systematically reviewed studies evaluating AI algorithms in detecting bone fractures in plain radiographs, combined their performance using meta-analysis (a bivariate regression approach), and compared it with that of clinicians. We also analyzed the factors potentially affecting algorithm performance using meta-regression. RESULTS Our analysis included 100 studies. In 83 studies with confusion matrices, AI algorithms showed a sensitivity of 91.43% and a specificity of 92.12% (Area under the summary receiver operator curve = 0.968). After adjustment and false discovery rate correction, tibia/fibula (excluding ankle) fractures were associated with higher (7.0%, p=0.004) AI sensitivity, while more recent publications (5.5%, p=0.003) and Xception architecture (6.6%, p<0.001) were associated with higher specificity. Clinicians and AI showed similar specificity in fracture identification, although AI leaned to higher sensitivity (7.6%, p=0.07). Radiologists, on the other hand, were more specific than AI overall and in several subgroups, and more sensitive to hip fractures before FDR correction. CONCLUSIONS Currently available AI aids could result in a significant improvement in care where radiologists are not readily available. Moreover, identifying factors affecting algorithm performance could guide AI development teams in their process of optimizing their products.
Collapse
Affiliation(s)
- A Nowroozi
- School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | - M A Salehi
- School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | - P Shobeiri
- School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | - S Agahi
- School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | - S Momtazmanesh
- School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | - P Kaviani
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA 02114, USA
| | - M K Kalra
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA 02114, USA.
| |
Collapse
|
3
|
van Nistelrooij N, Schitter S, van Lierop P, Ghoul KE, König D, Hanisch M, Tel A, Xi T, Thiem DGE, Smeets R, Dubois L, Flügge T, van Ginneken B, Bergé S, Vinayahalingam S. Detecting Mandible Fractures in CBCT Scans Using a 3-Stage Neural Network. J Dent Res 2024:220345241256618. [PMID: 38910411 DOI: 10.1177/00220345241256618] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/25/2024] Open
Abstract
After nasal bone fractures, fractures of the mandible are the most frequently encountered injuries of the facial skeleton. Accurate identification of fracture locations is critical for effectively managing these injuries. To address this need, JawFracNet, an innovative artificial intelligence method, has been developed to enable automated detection of mandibular fractures in cone-beam computed tomography (CBCT) scans. JawFracNet employs a 3-stage neural network model that processes 3-dimensional patches from a CBCT scan. Stage 1 predicts a segmentation mask of the mandible in a patch, which is subsequently used in stage 2 to predict a segmentation of the fractures and in stage 3 to classify whether the patch contains any fracture. The final output of JawFracNet is the fracture segmentation of the entire scan, obtained by aggregating and unifying voxel-level and patch-level predictions. A total of 164 CBCT scans without mandibular fractures and 171 CBCT scans with mandibular fractures were included in this study. Evaluation of JawFracNet demonstrated a precision of 0.978 and a sensitivity of 0.956 in detecting mandibular fractures. The current study proposes the first benchmark for mandibular fracture detection in CBCT scans. Straightforward replication is promoted by publicly sharing the code and providing access to JawFracNet on grand-challenge.org.
Collapse
Affiliation(s)
- N van Nistelrooij
- Department of Oral and Maxillofacial Surgery, Radboud University Medical Center, Nijmegen, The Netherlands
- Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Department of Oral and Maxillofacial Surgery, Berlin, Germany
| | - S Schitter
- Department of Oral and Maxillofacial Surgery, Division of Regenerative, Orofacial Medicine, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - P van Lierop
- Department of Oral and Maxillofacial Surgery, Radboud University Medical Center, Nijmegen, The Netherlands
| | - K El Ghoul
- Department of Oral and Maxillofacial Surgery, Erasmus Medical Center, Rotterdam, The Netherlands
| | - D König
- Department of Oral and Maxillofacial Surgery, Division of Regenerative, Orofacial Medicine, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - M Hanisch
- Department of Oral and Maxillofacial Surgery, Universitätsklinikum, Münster, Münster, Germany
| | - A Tel
- Clinic of Maxillofacial Surgery, Head-Neck and NeuroScience Department University Hospital of Udine, Udine, Italy
| | - T Xi
- Department of Oral and Maxillofacial Surgery, Radboud University Medical Center, Nijmegen, The Netherlands
| | - D G E Thiem
- Department of Oral and Maxillofacial Surgery, Facial Plastic Surgery, University Medical Centre Mainz, Mainz, Germany
| | - R Smeets
- Department of Oral and Maxillofacial Surgery, Division of Regenerative, Orofacial Medicine, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - L Dubois
- Department of Oral and Maxillofacial Surgery, Amsterdam UMC, University of Amsterdam, Amsterdam, The Netherlands
| | - T Flügge
- Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Department of Oral and Maxillofacial Surgery, Berlin, Germany
| | - B van Ginneken
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, The Netherlands
| | - S Bergé
- Department of Oral and Maxillofacial Surgery, Radboud University Medical Center, Nijmegen, The Netherlands
| | - S Vinayahalingam
- Department of Oral and Maxillofacial Surgery, Radboud University Medical Center, Nijmegen, The Netherlands
| |
Collapse
|
4
|
Lassalle L, Regnard NE, Ventre J, Marty V, Clovis L, Zhang Z, Nitche N, Guermazi A, Laredo JD. Automated weight-bearing foot measurements using an artificial intelligence-based software. Skeletal Radiol 2024:10.1007/s00256-024-04726-z. [PMID: 38880791 DOI: 10.1007/s00256-024-04726-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/13/2024] [Revised: 06/05/2024] [Accepted: 06/05/2024] [Indexed: 06/18/2024]
Abstract
OBJECTIVE To assess the accuracy of an artificial intelligence (AI) software (BoneMetrics, Gleamer) in performing automated measurements on weight-bearing forefoot and lateral foot radiographs. METHODS Consecutive forefoot and lateral foot radiographs were retrospectively collected from three imaging institutions. Two senior musculoskeletal radiologists independently annotated key points to measure the hallux valgus, first-second metatarsal, and first-fifth metatarsal angles on forefoot radiographs and the talus-first metatarsal, medial arch, and calcaneus inclination angles on lateral foot radiographs. The ground truth was defined as the mean of their measurements. Statistical analysis included mean absolute error (MAE), bias assessed with Bland-Altman analysis between the ground truth and AI prediction, and intraclass coefficient (ICC) between the manual ratings. RESULTS Eighty forefoot radiographs were included (53 ± 17 years, 50 women), and 26 were excluded. Ninety-seven lateral foot radiographs were included (51 ± 20 years, 46 women), and 21 were excluded. MAE for the hallux valgus, first-second metatarsal, and first-fifth metatarsal angles on forefoot radiographs were respectively 1.2° (95% CI [1; 1.4], bias = - 0.04°, ICC = 0.98), 0.7° (95% CI [0.6; 0.9], bias = - 0.19°, ICC = 0.91) and 0.9° (95% CI [0.7; 1.1], bias = 0.44°, ICC = 0.96). MAE for the talus-first, medial arch, and calcaneal inclination angles on the lateral foot radiographs were respectively 3.9° (95% CI [3.4; 4.5], bias = 0.61° ICC = 0.88), 1.5° (95% CI [1.2; 1.8], bias = - 0.18°, ICC = 0.95) and 1° (95% CI [0.8; 1.2], bias = 0.74°, ICC = 0.99). Bias and MAE between the ground truth and the AI prediction were low across all measurements. ICC between the two manual ratings was excellent, except for the talus-first metatarsal angle. CONCLUSION AI demonstrated potential for accurate and automated measurements on weight-bearing forefoot and lateral foot radiographs.
Collapse
Affiliation(s)
- Louis Lassalle
- Réseau Imagerie Sud Francilien, Lieusaint, France.
- Clinique du Mousseau, Ramsay Santé, Evry, France.
- , Gleamer, Paris, France.
| | - Nor-Eddine Regnard
- Réseau Imagerie Sud Francilien, Lieusaint, France
- Clinique du Mousseau, Ramsay Santé, Evry, France
- , Gleamer, Paris, France
| | | | | | | | | | | | - Ali Guermazi
- Department of Radiology, Boston University School of Medicine, Boston, MA, USA
| | - Jean-Denis Laredo
- , Gleamer, Paris, France
- Service de Radiologie, Institut Mutualiste Montsouris, Paris, France
- Laboratoire (B3OA) de Biomécanique Et Biomatériaux Ostéo-Articulaires, Faculté de Médecine Paris-Cité, Paris, France
- Professeur Émérite d'Imagerie Médicale, Université Paris-Cité, Paris, France
| |
Collapse
|
5
|
Dell’Aria A, Tack D, Saddiki N, Makdoud S, Alexiou J, De Hemptinne FX, Berkenbaum I, Neugroschl C, Tacelli N. Radiographic Detection of Post-Traumatic Bone Fractures: Contribution of Artificial Intelligence Software to the Analysis of Senior and Junior Radiologists. J Belg Soc Radiol 2024; 108:44. [PMID: 38680721 PMCID: PMC11049681 DOI: 10.5334/jbsr.3574] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2024] [Accepted: 04/08/2024] [Indexed: 05/01/2024] Open
Abstract
Objectives The aims of this study were: (a) to evaluate the performance of an artificial intelligence (AI) software package (Boneview Trauma, Gleamer) for the detection of post-traumatic bone fractures in radiography as a standalone; (b) used by two radiologists (osteoarticular senior and junior); and (c) to determine to whom AI would be most helpful. Materials and Methods Within 14 days of a trauma, 101 consecutive patients underwent radiographic examination of the upper or lower limbs. The definite diagnosis for identifying fractures was: (a) radio-clinical consensus between the radiologist on-call who analyzed the images and the orthopedist (Group 1); (b) Cone Beam computed tomography (CBCT) exploration of the area of interest, in case of doubts or absence of consensus (Group 2). Independently of this diagnosis for both groups, the radiographic images were separately analyzed by two radiologists (osteoarticular senior: SR; junior: JR) prior without, and thereafter with the results of AI. Results AI performed better than the radiologists in detecting common fractures (Group 1), but not subtle fractures (Group 2). In association with AI, both radiologists increased their overall performances in both groups, whereas this increase was significantly higher for the JR (p < 0.05). Conclusion AI is reliable for common radiographic fracture identification and is a useful learning tool for radiologists in training. However, the software's overall performance does not exceed that of an osteoarticular senior radiologist, particularly in case of subtle lesions.
Collapse
Affiliation(s)
- Andrea Dell’Aria
- Department of Radiology, Hôpitaux Iris-Sud (HIS), Université libre de Bruxelles (ULB), Brussels, Belgium
| | - Denis Tack
- Department of Radiology, Centre Hospitalier EpiCURA, Université libre de Bruxelles (ULB), Brussels, Belgium
| | - Najat Saddiki
- Department of Radiology, Hôpitaux Iris-Sud (HIS), Université libre de Bruxelles (ULB), Brussels, Belgium
| | - Sonia Makdoud
- Department of Radiology, Hôpitaux Iris-Sud (HIS), Université d’Alger 1- Faculté de Médecine d’Alger-Ziania, Algiers, Algeria
| | - Jean Alexiou
- Department of Radiology, Hôpitaux Iris-Sud (HIS), Université libre de Bruxelles (ULB), Brussels, Belgium
| | | | - Ivan Berkenbaum
- Department of Orthopaedic Surgery, Hôpitaux Iris-Sud (HIS), Université libre de Bruxelles (ULB), Brussels, Belgium
| | - Carine Neugroschl
- Department of Radiology, Hôpitaux Iris-Sud (HIS), Université libre de Bruxelles (ULB), Brussels, Belgium
| | - Nunzia Tacelli
- Department of Radiology, Hôpitaux Iris-Sud (HIS), Université libre de Bruxelles (ULB), Brussels, Belgium
| |
Collapse
|
6
|
Dupuis M, Delbos L, Rouquette A, Adamsbaum C, Veil R. External validation of an artificial intelligence solution for the detection of elbow fractures and joint effusions in children. Diagn Interv Imaging 2024; 105:104-109. [PMID: 37813759 DOI: 10.1016/j.diii.2023.09.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2023] [Revised: 09/19/2023] [Accepted: 09/25/2023] [Indexed: 10/11/2023]
Abstract
PURPOSE The purpose of this study was to conduct an external validation of an artificial intelligence (AI) solution for the detection of elbow fractures and joint effusions using radiographs from a real-life cohort of children. MATERIALS AND METHODS This single-center retrospective study was conducted on 758 radiographic sets (1637 images) obtained from consecutive emergency room visits of 712 children (mean age, 7.27 ± 3.97 [standard deviation] years; age range, 7 months and 10 days to 15 years and 10 months), referred for a trauma of the elbow. For each set, fracture and/or effusion detection by eleven senior radiologists (reference standard) and AI solution was recorded. Diagnostic performance of the AI solution was measured via four different approaches: fracture detection (presence/absence of fracture as binary variable), fracture enumeration, fracture localization and lesion detection (fracture and/or a joint effusion used as constructed binary variable). RESULTS The sensitivity of the AI solution for each of the four approaches was >89%. Greatest sensitivity of the AI solution was obtained for lesion detection (95.0%; 95% confidence interval: 92.1-96.9). The specificity of the AI solution ranged between 63% (for lesion detection) and 77% (for fracture detection). For all four approaches, the negative predictive values were >92% and the positive predictive values ranged between 54% (for fracture enumeration and localization) and 73% (for lesion detection). Specificity was lower for plastered children for all approaches (P < 0.001). CONCLUSION The AI solution demonstrates high performances for detecting elbow's fracture and/or joint effusion in children. However, in our context of use, 8% of the radiographic sets ruled-out by the algorithm concerned children with a genuine traumatic elbow lesion.
Collapse
Affiliation(s)
- Michel Dupuis
- AP-HP, Bicêtre Hospital, Pediatric Imaging Department, 94270 Le Kremlin Bicêtre, France
| | - Léo Delbos
- AP-HP, Bicêtre Hospital, Epidemiology and Public Health Department, 94270 Le Kremlin Bicêtre, France
| | - Alexandra Rouquette
- AP-HP, Bicêtre Hospital, Epidemiology and Public Health Department, 94270 Le Kremlin Bicêtre, France
| | - Catherine Adamsbaum
- AP-HP, Bicêtre Hospital, Pediatric Imaging Department, 94270 Le Kremlin Bicêtre, France; Paris Saclay University, Faculté de Médicine, 94270 Le Kremlin Bicêtre, France.
| | - Raphaël Veil
- AP-HP, Bicêtre Hospital, Epidemiology and Public Health Department, 94270 Le Kremlin Bicêtre, France
| |
Collapse
|
7
|
Guenoun D, Zins M, Champsaur P, Thomassin-Naggara I. French community grid for the evaluation of radiological artificial intelligence solutions (DRIM France Artificial Intelligence Initiative). Diagn Interv Imaging 2024; 105:74-81. [PMID: 37749026 DOI: 10.1016/j.diii.2023.09.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2023] [Revised: 09/08/2023] [Accepted: 09/09/2023] [Indexed: 09/27/2023]
Abstract
PURPOSE The purpose of this study was to validate a national descriptive and analytical grid for artificial intelligence (AI) solutions in radiology. MATERIALS AND METHODS The RAND-UCLA Appropriateness Method was chosen by expert radiologists from the DRIM France IA group for this statement paper. The study, initiated by the radiology community, involved seven steps including literature review, template development, panel selection, pre-panel meeting survey, data extraction and analysis, second and final panel meeting, and data reporting. RESULTS The panel consisted of seven software vendors, three for bone fracture detection using conventional radiology and four for breast cancer detection using mammography. A consensus was reached on various aspects, including general target, main objective, certification marking, integration, expression of results, forensic aspects and cybersecurity, performance and scientific validation, description of the company and economic details, possible usage scenarios in the clinical workflow, database, specific objectives and targets of the AI tool. CONCLUSION The study validates a descriptive and analytical grid for radiological AI solutions consisting of ten items, using breast cancer and bone fracture as an experimental guide. This grid would assist radiologists in selecting relevant and validated AI solutions. Further developments of the grid are needed to include other organs and tasks.
Collapse
Affiliation(s)
- Daphné Guenoun
- APHM, Sainte-Marguerite Hospital, Institute for Locomotion, Department of Radiology, 13009, Marseille, France; Aix Marseille Univ, CNRS, ISM, Inst Movement Sci, 13009, Marseille, France.
| | - Marc Zins
- Department of Radiology and Medical Imaging, Saint-Joseph Hospital, 75014, Paris, France
| | - Pierre Champsaur
- APHM, Sainte-Marguerite Hospital, Institute for Locomotion, Department of Radiology, 13009, Marseille, France; Aix Marseille Univ, CNRS, ISM, Inst Movement Sci, 13009, Marseille, France
| | - Isabelle Thomassin-Naggara
- Sorbonne Université, 75005, Paris, France; Department of Diagnostic and Interventional Imaging, Assistance Publique-Hôpitaux de Paris, Hôpital Tenon, 75020 Paris, France
| |
Collapse
|
8
|
Pham TD, Holmes SB, Coulthard P. A review on artificial intelligence for the diagnosis of fractures in facial trauma imaging. Front Artif Intell 2024; 6:1278529. [PMID: 38249794 PMCID: PMC10797131 DOI: 10.3389/frai.2023.1278529] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2023] [Accepted: 12/11/2023] [Indexed: 01/23/2024] Open
Abstract
Patients with facial trauma may suffer from injuries such as broken bones, bleeding, swelling, bruising, lacerations, burns, and deformity in the face. Common causes of facial-bone fractures are the results of road accidents, violence, and sports injuries. Surgery is needed if the trauma patient would be deprived of normal functioning or subject to facial deformity based on findings from radiology. Although the image reading by radiologists is useful for evaluating suspected facial fractures, there are certain challenges in human-based diagnostics. Artificial intelligence (AI) is making a quantum leap in radiology, producing significant improvements of reports and workflows. Here, an updated literature review is presented on the impact of AI in facial trauma with a special reference to fracture detection in radiology. The purpose is to gain insights into the current development and demand for future research in facial trauma. This review also discusses limitations to be overcome and current important issues for investigation in order to make AI applications to the trauma more effective and realistic in practical settings. The publications selected for review were based on their clinical significance, journal metrics, and journal indexing.
Collapse
Affiliation(s)
- Tuan D. Pham
- Barts and The London School of Medicine and Dentistry, Queen Mary University of London, London, United Kingdom
| | | | | |
Collapse
|
9
|
Guermazi A, Omoumi P, Tordjman M, Fritz J, Kijowski R, Regnard NE, Carrino J, Kahn CE, Knoll F, Rueckert D, Roemer FW, Hayashi D. How AI May Transform Musculoskeletal Imaging. Radiology 2024; 310:e230764. [PMID: 38165245 PMCID: PMC10831478 DOI: 10.1148/radiol.230764] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2023] [Revised: 06/18/2023] [Accepted: 07/11/2023] [Indexed: 01/03/2024]
Abstract
While musculoskeletal imaging volumes are increasing, there is a relative shortage of subspecialized musculoskeletal radiologists to interpret the studies. Will artificial intelligence (AI) be the solution? For AI to be the solution, the wide implementation of AI-supported data acquisition methods in clinical practice requires establishing trusted and reliable results. This implementation will demand close collaboration between core AI researchers and clinical radiologists. Upon successful clinical implementation, a wide variety of AI-based tools can improve the musculoskeletal radiologist's workflow by triaging imaging examinations, helping with image interpretation, and decreasing the reporting time. Additional AI applications may also be helpful for business, education, and research purposes if successfully integrated into the daily practice of musculoskeletal radiology. The question is not whether AI will replace radiologists, but rather how musculoskeletal radiologists can take advantage of AI to enhance their expert capabilities.
Collapse
Affiliation(s)
- Ali Guermazi
- From the Department of Radiology, Boston University School of
Medicine, Boston, Mass (A.G., F.W.R., D.H.); Department of Radiology, VA Boston
Healthcare System, 1400 VFW Parkway, Suite 1B105, West Roxbury, MA 02132 (A.G.);
Department of Radiology, Lausanne University Hospital and University of
Lausanne, Lausanne, Switzerland (P.O.); Department of Radiology, Hotel Dieu
Hospital and University Paris Cité, Paris, France (M.T.); Department of
Radiology, New York University Grossman School of Medicine, New York, NY (J.F.,
R.K.); Gleamer, Paris, France (N.E.R.); Réseau d’Imagerie Sud
Francilien, Clinique du Mousseau Ramsay Santé, Evry, France (N.E.R.);
Pôle Médical Sénart, Lieusaint, France (N.E.R.); Department
of Radiology and Imaging, Hospital for Special Surgery and Weill Cornell
Medicine, New York, NY (J.C.); Department of Radiology and Institute for
Biomedical Informatics, University of Pennsylvania, Philadelphia, Penn (C.E.K.);
Departments of Artificial Intelligence in Biomedical Engineering (F.K.) and
Radiology (F.W.R.), Universitätsklinikum Erlangen &
Friedrich-Alexander Universität Erlangen-Nürnberg, Erlangen,
Germany (F.K.); School of Medicine & Computation, Information and
Technology Klinikum rechts der Isar, Technical University Munich,
München, Germany (D.R.); Department of Computing, Imperial College
London, London, England (D.R.); and Department of Radiology, Tufts Medical
Center, Tufts University School of Medicine, Boston, Mass (D.H.)
| | - Patrick Omoumi
- From the Department of Radiology, Boston University School of
Medicine, Boston, Mass (A.G., F.W.R., D.H.); Department of Radiology, VA Boston
Healthcare System, 1400 VFW Parkway, Suite 1B105, West Roxbury, MA 02132 (A.G.);
Department of Radiology, Lausanne University Hospital and University of
Lausanne, Lausanne, Switzerland (P.O.); Department of Radiology, Hotel Dieu
Hospital and University Paris Cité, Paris, France (M.T.); Department of
Radiology, New York University Grossman School of Medicine, New York, NY (J.F.,
R.K.); Gleamer, Paris, France (N.E.R.); Réseau d’Imagerie Sud
Francilien, Clinique du Mousseau Ramsay Santé, Evry, France (N.E.R.);
Pôle Médical Sénart, Lieusaint, France (N.E.R.); Department
of Radiology and Imaging, Hospital for Special Surgery and Weill Cornell
Medicine, New York, NY (J.C.); Department of Radiology and Institute for
Biomedical Informatics, University of Pennsylvania, Philadelphia, Penn (C.E.K.);
Departments of Artificial Intelligence in Biomedical Engineering (F.K.) and
Radiology (F.W.R.), Universitätsklinikum Erlangen &
Friedrich-Alexander Universität Erlangen-Nürnberg, Erlangen,
Germany (F.K.); School of Medicine & Computation, Information and
Technology Klinikum rechts der Isar, Technical University Munich,
München, Germany (D.R.); Department of Computing, Imperial College
London, London, England (D.R.); and Department of Radiology, Tufts Medical
Center, Tufts University School of Medicine, Boston, Mass (D.H.)
| | - Mickael Tordjman
- From the Department of Radiology, Boston University School of
Medicine, Boston, Mass (A.G., F.W.R., D.H.); Department of Radiology, VA Boston
Healthcare System, 1400 VFW Parkway, Suite 1B105, West Roxbury, MA 02132 (A.G.);
Department of Radiology, Lausanne University Hospital and University of
Lausanne, Lausanne, Switzerland (P.O.); Department of Radiology, Hotel Dieu
Hospital and University Paris Cité, Paris, France (M.T.); Department of
Radiology, New York University Grossman School of Medicine, New York, NY (J.F.,
R.K.); Gleamer, Paris, France (N.E.R.); Réseau d’Imagerie Sud
Francilien, Clinique du Mousseau Ramsay Santé, Evry, France (N.E.R.);
Pôle Médical Sénart, Lieusaint, France (N.E.R.); Department
of Radiology and Imaging, Hospital for Special Surgery and Weill Cornell
Medicine, New York, NY (J.C.); Department of Radiology and Institute for
Biomedical Informatics, University of Pennsylvania, Philadelphia, Penn (C.E.K.);
Departments of Artificial Intelligence in Biomedical Engineering (F.K.) and
Radiology (F.W.R.), Universitätsklinikum Erlangen &
Friedrich-Alexander Universität Erlangen-Nürnberg, Erlangen,
Germany (F.K.); School of Medicine & Computation, Information and
Technology Klinikum rechts der Isar, Technical University Munich,
München, Germany (D.R.); Department of Computing, Imperial College
London, London, England (D.R.); and Department of Radiology, Tufts Medical
Center, Tufts University School of Medicine, Boston, Mass (D.H.)
| | - Jan Fritz
- From the Department of Radiology, Boston University School of
Medicine, Boston, Mass (A.G., F.W.R., D.H.); Department of Radiology, VA Boston
Healthcare System, 1400 VFW Parkway, Suite 1B105, West Roxbury, MA 02132 (A.G.);
Department of Radiology, Lausanne University Hospital and University of
Lausanne, Lausanne, Switzerland (P.O.); Department of Radiology, Hotel Dieu
Hospital and University Paris Cité, Paris, France (M.T.); Department of
Radiology, New York University Grossman School of Medicine, New York, NY (J.F.,
R.K.); Gleamer, Paris, France (N.E.R.); Réseau d’Imagerie Sud
Francilien, Clinique du Mousseau Ramsay Santé, Evry, France (N.E.R.);
Pôle Médical Sénart, Lieusaint, France (N.E.R.); Department
of Radiology and Imaging, Hospital for Special Surgery and Weill Cornell
Medicine, New York, NY (J.C.); Department of Radiology and Institute for
Biomedical Informatics, University of Pennsylvania, Philadelphia, Penn (C.E.K.);
Departments of Artificial Intelligence in Biomedical Engineering (F.K.) and
Radiology (F.W.R.), Universitätsklinikum Erlangen &
Friedrich-Alexander Universität Erlangen-Nürnberg, Erlangen,
Germany (F.K.); School of Medicine & Computation, Information and
Technology Klinikum rechts der Isar, Technical University Munich,
München, Germany (D.R.); Department of Computing, Imperial College
London, London, England (D.R.); and Department of Radiology, Tufts Medical
Center, Tufts University School of Medicine, Boston, Mass (D.H.)
| | - Richard Kijowski
- From the Department of Radiology, Boston University School of
Medicine, Boston, Mass (A.G., F.W.R., D.H.); Department of Radiology, VA Boston
Healthcare System, 1400 VFW Parkway, Suite 1B105, West Roxbury, MA 02132 (A.G.);
Department of Radiology, Lausanne University Hospital and University of
Lausanne, Lausanne, Switzerland (P.O.); Department of Radiology, Hotel Dieu
Hospital and University Paris Cité, Paris, France (M.T.); Department of
Radiology, New York University Grossman School of Medicine, New York, NY (J.F.,
R.K.); Gleamer, Paris, France (N.E.R.); Réseau d’Imagerie Sud
Francilien, Clinique du Mousseau Ramsay Santé, Evry, France (N.E.R.);
Pôle Médical Sénart, Lieusaint, France (N.E.R.); Department
of Radiology and Imaging, Hospital for Special Surgery and Weill Cornell
Medicine, New York, NY (J.C.); Department of Radiology and Institute for
Biomedical Informatics, University of Pennsylvania, Philadelphia, Penn (C.E.K.);
Departments of Artificial Intelligence in Biomedical Engineering (F.K.) and
Radiology (F.W.R.), Universitätsklinikum Erlangen &
Friedrich-Alexander Universität Erlangen-Nürnberg, Erlangen,
Germany (F.K.); School of Medicine & Computation, Information and
Technology Klinikum rechts der Isar, Technical University Munich,
München, Germany (D.R.); Department of Computing, Imperial College
London, London, England (D.R.); and Department of Radiology, Tufts Medical
Center, Tufts University School of Medicine, Boston, Mass (D.H.)
| | - Nor-Eddine Regnard
- From the Department of Radiology, Boston University School of
Medicine, Boston, Mass (A.G., F.W.R., D.H.); Department of Radiology, VA Boston
Healthcare System, 1400 VFW Parkway, Suite 1B105, West Roxbury, MA 02132 (A.G.);
Department of Radiology, Lausanne University Hospital and University of
Lausanne, Lausanne, Switzerland (P.O.); Department of Radiology, Hotel Dieu
Hospital and University Paris Cité, Paris, France (M.T.); Department of
Radiology, New York University Grossman School of Medicine, New York, NY (J.F.,
R.K.); Gleamer, Paris, France (N.E.R.); Réseau d’Imagerie Sud
Francilien, Clinique du Mousseau Ramsay Santé, Evry, France (N.E.R.);
Pôle Médical Sénart, Lieusaint, France (N.E.R.); Department
of Radiology and Imaging, Hospital for Special Surgery and Weill Cornell
Medicine, New York, NY (J.C.); Department of Radiology and Institute for
Biomedical Informatics, University of Pennsylvania, Philadelphia, Penn (C.E.K.);
Departments of Artificial Intelligence in Biomedical Engineering (F.K.) and
Radiology (F.W.R.), Universitätsklinikum Erlangen &
Friedrich-Alexander Universität Erlangen-Nürnberg, Erlangen,
Germany (F.K.); School of Medicine & Computation, Information and
Technology Klinikum rechts der Isar, Technical University Munich,
München, Germany (D.R.); Department of Computing, Imperial College
London, London, England (D.R.); and Department of Radiology, Tufts Medical
Center, Tufts University School of Medicine, Boston, Mass (D.H.)
| | - John Carrino
- From the Department of Radiology, Boston University School of
Medicine, Boston, Mass (A.G., F.W.R., D.H.); Department of Radiology, VA Boston
Healthcare System, 1400 VFW Parkway, Suite 1B105, West Roxbury, MA 02132 (A.G.);
Department of Radiology, Lausanne University Hospital and University of
Lausanne, Lausanne, Switzerland (P.O.); Department of Radiology, Hotel Dieu
Hospital and University Paris Cité, Paris, France (M.T.); Department of
Radiology, New York University Grossman School of Medicine, New York, NY (J.F.,
R.K.); Gleamer, Paris, France (N.E.R.); Réseau d’Imagerie Sud
Francilien, Clinique du Mousseau Ramsay Santé, Evry, France (N.E.R.);
Pôle Médical Sénart, Lieusaint, France (N.E.R.); Department
of Radiology and Imaging, Hospital for Special Surgery and Weill Cornell
Medicine, New York, NY (J.C.); Department of Radiology and Institute for
Biomedical Informatics, University of Pennsylvania, Philadelphia, Penn (C.E.K.);
Departments of Artificial Intelligence in Biomedical Engineering (F.K.) and
Radiology (F.W.R.), Universitätsklinikum Erlangen &
Friedrich-Alexander Universität Erlangen-Nürnberg, Erlangen,
Germany (F.K.); School of Medicine & Computation, Information and
Technology Klinikum rechts der Isar, Technical University Munich,
München, Germany (D.R.); Department of Computing, Imperial College
London, London, England (D.R.); and Department of Radiology, Tufts Medical
Center, Tufts University School of Medicine, Boston, Mass (D.H.)
| | - Charles E. Kahn
- From the Department of Radiology, Boston University School of
Medicine, Boston, Mass (A.G., F.W.R., D.H.); Department of Radiology, VA Boston
Healthcare System, 1400 VFW Parkway, Suite 1B105, West Roxbury, MA 02132 (A.G.);
Department of Radiology, Lausanne University Hospital and University of
Lausanne, Lausanne, Switzerland (P.O.); Department of Radiology, Hotel Dieu
Hospital and University Paris Cité, Paris, France (M.T.); Department of
Radiology, New York University Grossman School of Medicine, New York, NY (J.F.,
R.K.); Gleamer, Paris, France (N.E.R.); Réseau d’Imagerie Sud
Francilien, Clinique du Mousseau Ramsay Santé, Evry, France (N.E.R.);
Pôle Médical Sénart, Lieusaint, France (N.E.R.); Department
of Radiology and Imaging, Hospital for Special Surgery and Weill Cornell
Medicine, New York, NY (J.C.); Department of Radiology and Institute for
Biomedical Informatics, University of Pennsylvania, Philadelphia, Penn (C.E.K.);
Departments of Artificial Intelligence in Biomedical Engineering (F.K.) and
Radiology (F.W.R.), Universitätsklinikum Erlangen &
Friedrich-Alexander Universität Erlangen-Nürnberg, Erlangen,
Germany (F.K.); School of Medicine & Computation, Information and
Technology Klinikum rechts der Isar, Technical University Munich,
München, Germany (D.R.); Department of Computing, Imperial College
London, London, England (D.R.); and Department of Radiology, Tufts Medical
Center, Tufts University School of Medicine, Boston, Mass (D.H.)
| | - Florian Knoll
- From the Department of Radiology, Boston University School of
Medicine, Boston, Mass (A.G., F.W.R., D.H.); Department of Radiology, VA Boston
Healthcare System, 1400 VFW Parkway, Suite 1B105, West Roxbury, MA 02132 (A.G.);
Department of Radiology, Lausanne University Hospital and University of
Lausanne, Lausanne, Switzerland (P.O.); Department of Radiology, Hotel Dieu
Hospital and University Paris Cité, Paris, France (M.T.); Department of
Radiology, New York University Grossman School of Medicine, New York, NY (J.F.,
R.K.); Gleamer, Paris, France (N.E.R.); Réseau d’Imagerie Sud
Francilien, Clinique du Mousseau Ramsay Santé, Evry, France (N.E.R.);
Pôle Médical Sénart, Lieusaint, France (N.E.R.); Department
of Radiology and Imaging, Hospital for Special Surgery and Weill Cornell
Medicine, New York, NY (J.C.); Department of Radiology and Institute for
Biomedical Informatics, University of Pennsylvania, Philadelphia, Penn (C.E.K.);
Departments of Artificial Intelligence in Biomedical Engineering (F.K.) and
Radiology (F.W.R.), Universitätsklinikum Erlangen &
Friedrich-Alexander Universität Erlangen-Nürnberg, Erlangen,
Germany (F.K.); School of Medicine & Computation, Information and
Technology Klinikum rechts der Isar, Technical University Munich,
München, Germany (D.R.); Department of Computing, Imperial College
London, London, England (D.R.); and Department of Radiology, Tufts Medical
Center, Tufts University School of Medicine, Boston, Mass (D.H.)
| | - Daniel Rueckert
- From the Department of Radiology, Boston University School of
Medicine, Boston, Mass (A.G., F.W.R., D.H.); Department of Radiology, VA Boston
Healthcare System, 1400 VFW Parkway, Suite 1B105, West Roxbury, MA 02132 (A.G.);
Department of Radiology, Lausanne University Hospital and University of
Lausanne, Lausanne, Switzerland (P.O.); Department of Radiology, Hotel Dieu
Hospital and University Paris Cité, Paris, France (M.T.); Department of
Radiology, New York University Grossman School of Medicine, New York, NY (J.F.,
R.K.); Gleamer, Paris, France (N.E.R.); Réseau d’Imagerie Sud
Francilien, Clinique du Mousseau Ramsay Santé, Evry, France (N.E.R.);
Pôle Médical Sénart, Lieusaint, France (N.E.R.); Department
of Radiology and Imaging, Hospital for Special Surgery and Weill Cornell
Medicine, New York, NY (J.C.); Department of Radiology and Institute for
Biomedical Informatics, University of Pennsylvania, Philadelphia, Penn (C.E.K.);
Departments of Artificial Intelligence in Biomedical Engineering (F.K.) and
Radiology (F.W.R.), Universitätsklinikum Erlangen &
Friedrich-Alexander Universität Erlangen-Nürnberg, Erlangen,
Germany (F.K.); School of Medicine & Computation, Information and
Technology Klinikum rechts der Isar, Technical University Munich,
München, Germany (D.R.); Department of Computing, Imperial College
London, London, England (D.R.); and Department of Radiology, Tufts Medical
Center, Tufts University School of Medicine, Boston, Mass (D.H.)
| | - Frank W. Roemer
- From the Department of Radiology, Boston University School of
Medicine, Boston, Mass (A.G., F.W.R., D.H.); Department of Radiology, VA Boston
Healthcare System, 1400 VFW Parkway, Suite 1B105, West Roxbury, MA 02132 (A.G.);
Department of Radiology, Lausanne University Hospital and University of
Lausanne, Lausanne, Switzerland (P.O.); Department of Radiology, Hotel Dieu
Hospital and University Paris Cité, Paris, France (M.T.); Department of
Radiology, New York University Grossman School of Medicine, New York, NY (J.F.,
R.K.); Gleamer, Paris, France (N.E.R.); Réseau d’Imagerie Sud
Francilien, Clinique du Mousseau Ramsay Santé, Evry, France (N.E.R.);
Pôle Médical Sénart, Lieusaint, France (N.E.R.); Department
of Radiology and Imaging, Hospital for Special Surgery and Weill Cornell
Medicine, New York, NY (J.C.); Department of Radiology and Institute for
Biomedical Informatics, University of Pennsylvania, Philadelphia, Penn (C.E.K.);
Departments of Artificial Intelligence in Biomedical Engineering (F.K.) and
Radiology (F.W.R.), Universitätsklinikum Erlangen &
Friedrich-Alexander Universität Erlangen-Nürnberg, Erlangen,
Germany (F.K.); School of Medicine & Computation, Information and
Technology Klinikum rechts der Isar, Technical University Munich,
München, Germany (D.R.); Department of Computing, Imperial College
London, London, England (D.R.); and Department of Radiology, Tufts Medical
Center, Tufts University School of Medicine, Boston, Mass (D.H.)
| | - Daichi Hayashi
- From the Department of Radiology, Boston University School of
Medicine, Boston, Mass (A.G., F.W.R., D.H.); Department of Radiology, VA Boston
Healthcare System, 1400 VFW Parkway, Suite 1B105, West Roxbury, MA 02132 (A.G.);
Department of Radiology, Lausanne University Hospital and University of
Lausanne, Lausanne, Switzerland (P.O.); Department of Radiology, Hotel Dieu
Hospital and University Paris Cité, Paris, France (M.T.); Department of
Radiology, New York University Grossman School of Medicine, New York, NY (J.F.,
R.K.); Gleamer, Paris, France (N.E.R.); Réseau d’Imagerie Sud
Francilien, Clinique du Mousseau Ramsay Santé, Evry, France (N.E.R.);
Pôle Médical Sénart, Lieusaint, France (N.E.R.); Department
of Radiology and Imaging, Hospital for Special Surgery and Weill Cornell
Medicine, New York, NY (J.C.); Department of Radiology and Institute for
Biomedical Informatics, University of Pennsylvania, Philadelphia, Penn (C.E.K.);
Departments of Artificial Intelligence in Biomedical Engineering (F.K.) and
Radiology (F.W.R.), Universitätsklinikum Erlangen &
Friedrich-Alexander Universität Erlangen-Nürnberg, Erlangen,
Germany (F.K.); School of Medicine & Computation, Information and
Technology Klinikum rechts der Isar, Technical University Munich,
München, Germany (D.R.); Department of Computing, Imperial College
London, London, England (D.R.); and Department of Radiology, Tufts Medical
Center, Tufts University School of Medicine, Boston, Mass (D.H.)
| |
Collapse
|
10
|
Pauling C, Kanber B, Arthurs OJ, Shelmerdine SC. Commercially available artificial intelligence tools for fracture detection: the evidence. BJR Open 2024; 6:tzad005. [PMID: 38352182 PMCID: PMC10860511 DOI: 10.1093/bjro/tzad005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Revised: 09/20/2023] [Accepted: 09/30/2023] [Indexed: 02/16/2024] Open
Abstract
Missed fractures are a costly healthcare issue, not only negatively impacting patient lives, leading to potential long-term disability and time off work, but also responsible for high medicolegal disbursements that could otherwise be used to improve other healthcare services. When fractures are overlooked in children, they are particularly concerning as opportunities for safeguarding may be missed. Assistance from artificial intelligence (AI) in interpreting medical images may offer a possible solution for improving patient care, and several commercial AI tools are now available for radiology workflow implementation. However, information regarding their development, evidence for performance and validation as well as the intended target population is not always clear, but vital when evaluating a potential AI solution for implementation. In this article, we review the range of available products utilizing AI for fracture detection (in both adults and children) and summarize the evidence, or lack thereof, behind their performance. This will allow others to make better informed decisions when deciding which product to procure for their specific clinical requirements.
Collapse
Affiliation(s)
- Cato Pauling
- UCL Great Ormond Street Institute of Child Health, University College London, London WC1E 6BT, United Kingdom
| | - Baris Kanber
- Queen Square Multiple Sclerosis Centre, Department of Neuroinflammation, University College London (UCL) Queen Square Institute of Neurology, Faculty of Brain Sciences, University College London, London WC1N 3BG, United Kingdom
- Department of Medical Physics and Biomedical Engineering, Centre for Medical Image Computing, University College London, London WC1E 6BT, United Kingdom
| | - Owen J Arthurs
- UCL Great Ormond Street Institute of Child Health, University College London, London WC1E 6BT, United Kingdom
- Department of Clinical Radiology, Great Ormond Street Hospital for Children NHS Foundation Trust, London WC1N 3JH, United Kingdom
- NIHR Great Ormond Street Hospital Biomedical Research Centre, Bloomsbury, London WC1N 1EH, United Kingdom
| | - Susan C Shelmerdine
- UCL Great Ormond Street Institute of Child Health, University College London, London WC1E 6BT, United Kingdom
- Department of Clinical Radiology, Great Ormond Street Hospital for Children NHS Foundation Trust, London WC1N 3JH, United Kingdom
- NIHR Great Ormond Street Hospital Biomedical Research Centre, Bloomsbury, London WC1N 1EH, United Kingdom
| |
Collapse
|
11
|
Bennani S, Regnard NE, Ventre J, Lassalle L, Nguyen T, Ducarouge A, Dargent L, Guillo E, Gouhier E, Zaimi SH, Canniff E, Malandrin C, Khafagy P, Koulakian H, Revel MP, Chassagnon G. Using AI to Improve Radiologist Performance in Detection of Abnormalities on Chest Radiographs. Radiology 2023; 309:e230860. [PMID: 38085079 DOI: 10.1148/radiol.230860] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
Background Chest radiography remains the most common radiologic examination, and interpretation of its results can be difficult. Purpose To explore the potential benefit of artificial intelligence (AI) assistance in the detection of thoracic abnormalities on chest radiographs by evaluating the performance of radiologists with different levels of expertise, with and without AI assistance. Materials and Methods Patients who underwent both chest radiography and thoracic CT within 72 hours between January 2010 and December 2020 in a French public hospital were screened retrospectively. Radiographs were randomly included until reaching 500 radiographs, with about 50% of radiographs having abnormal findings. A senior thoracic radiologist annotated the radiographs for five abnormalities (pneumothorax, pleural effusion, consolidation, mediastinal and hilar mass, lung nodule) based on the corresponding CT results (ground truth). A total of 12 readers (four thoracic radiologists, four general radiologists, four radiology residents) read half the radiographs without AI and half the radiographs with AI (ChestView; Gleamer). Changes in sensitivity and specificity were measured using paired t tests. Results The study included 500 patients (mean age, 54 years ± 19 [SD]; 261 female, 239 male), with 522 abnormalities visible on 241 radiographs. On average, for all readers, AI use resulted in an absolute increase in sensitivity of 26% (95% CI: 20, 32), 14% (95% CI: 11, 17), 12% (95% CI: 10, 14), 8.5% (95% CI: 6, 11), and 5.9% (95% CI: 4, 8) for pneumothorax, consolidation, nodule, pleural effusion, and mediastinal and hilar mass, respectively (P < .001). Specificity increased with AI assistance (3.9% [95% CI: 3.2, 4.6], 3.7% [95% CI: 3, 4.4], 2.9% [95% CI: 2.3, 3.5], and 2.1% [95% CI: 1.6, 2.6] for pleural effusion, mediastinal and hilar mass, consolidation, and nodule, respectively), except in the diagnosis of pneumothorax (-0.2%; 95% CI: -0.36, -0.04; P = .01). The mean reading time was 81 seconds without AI versus 56 seconds with AI (31% decrease, P < .001). Conclusion AI-assisted chest radiography interpretation resulted in absolute increases in sensitivity for all radiologists of various levels of expertise and reduced the reading times; specificity increased with AI, except in the diagnosis of pneumothorax. © RSNA, 2023 Supplemental material is available for this article.
Collapse
Affiliation(s)
- Souhail Bennani
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Nor-Eddine Regnard
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Jeanne Ventre
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Louis Lassalle
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Toan Nguyen
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Alexis Ducarouge
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Lucas Dargent
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Enora Guillo
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Elodie Gouhier
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Sophie-Hélène Zaimi
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Emma Canniff
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Cécile Malandrin
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Philippe Khafagy
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Hasmik Koulakian
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Marie-Pierre Revel
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| | - Guillaume Chassagnon
- From the Department of Thoracic Imaging, Cochin Hospital, AP-HP, 27 Rue du Faubourg Saint-Jacques, Paris 75014, France (S.B., L.D., E. Guillo, E. Gouhier, S.H.Z., E.C., M.P.R., G.C.); Gleamer, Paris, France (S.B., N.E.R., J.V., L.L., T.N., A.D.); Réseau d'Imagerie Sud Francilien, Lieusant, France (N.E.R., L.L., C.M.); Department of Pediatric Radiology, Armand Trousseau Hospital, AP-HP, Paris, France (T.N.); HFR Fribourg, Fribourg, Switzerland (P.K.); and Centre d'Imagerie Médicale de l'Ouest Parisien, Paris, France (H.K.)
| |
Collapse
|
12
|
Russe MF, Fink A, Ngo H, Tran H, Bamberg F, Reisert M, Rau A. Performance of ChatGPT, human radiologists, and context-aware ChatGPT in identifying AO codes from radiology reports. Sci Rep 2023; 13:14215. [PMID: 37648742 PMCID: PMC10468502 DOI: 10.1038/s41598-023-41512-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2023] [Accepted: 08/28/2023] [Indexed: 09/01/2023] Open
Abstract
While radiologists can describe a fracture's morphology and complexity with ease, the translation into classification systems such as the Arbeitsgemeinschaft Osteosynthesefragen (AO) Fracture and Dislocation Classification Compendium is more challenging. We tested the performance of generic chatbots and chatbots aware of specific knowledge of the AO classification provided by a vector-index and compared it to human readers. In the 100 radiological reports we created based on random AO codes, chatbots provided AO codes significantly faster than humans (mean 3.2 s per case vs. 50 s per case, p < .001) though not reaching human performance (max. chatbot performance of 86% correct full AO codes vs. 95% in human readers). In general, chatbots based on GPT 4 outperformed the ones based on GPT 3.5-Turbo. Further, we found that providing specific knowledge substantially enhances the chatbot's performance and consistency as the context-aware chatbot based on GPT 4 provided 71% consistent correct full AO codes for the compared to the 2% consistent correct full AO codes for the generic ChatGPT 4. This provides evidence, that refining and providing specific context to ChatGPT will be the next essential step in harnessing its power.
Collapse
Affiliation(s)
- Maximilian F Russe
- Department of Diagnostic and Interventional Radiology, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Breisacher Str. 64, 79106, Freiburg, Germany.
| | - Anna Fink
- Department of Diagnostic and Interventional Radiology, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Breisacher Str. 64, 79106, Freiburg, Germany
| | - Helen Ngo
- Department of Diagnostic and Interventional Radiology, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Breisacher Str. 64, 79106, Freiburg, Germany
| | - Hien Tran
- Department of Diagnostic and Interventional Radiology, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Breisacher Str. 64, 79106, Freiburg, Germany
| | - Fabian Bamberg
- Department of Diagnostic and Interventional Radiology, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Breisacher Str. 64, 79106, Freiburg, Germany
| | - Marco Reisert
- Department of Stereotactic and Functional Neurosurgery, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
- Medical Physics, Department of Diagnostic and Interventional Radiology, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Alexander Rau
- Department of Diagnostic and Interventional Radiology, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Breisacher Str. 64, 79106, Freiburg, Germany
- Department of Neuroradiology, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| |
Collapse
|
13
|
Hussain A, Fareed A, Taseen S. Bone fracture detection-Can artificial intelligence replace doctors in orthopedic radiography analysis? Front Artif Intell 2023; 6:1223909. [PMID: 37593091 PMCID: PMC10427856 DOI: 10.3389/frai.2023.1223909] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2023] [Accepted: 07/20/2023] [Indexed: 08/19/2023] Open
Affiliation(s)
- Aariz Hussain
- Karachi Medical and Dental College and Abbasi Shaheed Hospital, Karachi, Sindh, Pakistan
| | | | | |
Collapse
|
14
|
ROZWAG C, VALENTINI F, COTTEN A, DEMONDION X, PREUX P, JACQUES T. Elbow trauma in children: development and evaluation of radiological artificial intelligence models. RESEARCH IN DIAGNOSTIC AND INTERVENTIONAL IMAGING 2023; 6:100029. [PMID: 39077546 PMCID: PMC11265386 DOI: 10.1016/j.redii.2023.100029] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/05/2022] [Accepted: 04/24/2023] [Indexed: 07/31/2024]
Abstract
Rationale and Objectives To develop a model using artificial intelligence (A.I.) able to detect post-traumatic injuries on pediatric elbow X-rays then to evaluate its performances in silico and its impact on radiologists' interpretation in clinical practice. Material and Methods A total of 1956 pediatric elbow radiographs performed following a trauma were retrospectively collected from 935 patients aged between 0 and 18 years. Deep convolutional neural networks were trained on these X-rays. The two best models were selected then evaluated on an external test set involving 120 patients, whose X-rays were performed on a different radiological equipment in another time period. Eight radiologists interpreted this external test set without then with the help of the A.I. models . Results Two models stood out: model 1 had an accuracy of 95.8% and an AUROC of 0.983 and model 2 had an accuracy of 90.5% and an AUROC of 0.975. On the external test set, model 1 kept a good accuracy of 82.5% and AUROC of 0.916 while model 2 had a loss of accuracy down to 69.2% and of AUROC to 0.793. Model 1 significantly improved radiologist's sensitivity (0.82 to 0.88, P = 0.016) and accuracy (0.86 to 0.88, P = 0,047) while model 2 significantly decreased specificity of readers (0.86 to 0.83, P = 0.031). Conclusion End-to-end development of a deep learning model to assess post-traumatic injuries on elbow X-ray in children was feasible and showed that models with close metrics in silico can unpredictably lead radiologists to either improve or lower their performances in clinical settings.
Collapse
Affiliation(s)
- Clémence ROZWAG
- Université de Lille , Lille, France
- Centre hospitalier universitaire de Lille, Lille, France
| | - Franck VALENTINI
- Université de Lille , Lille, France
- Inria Lille – Nord Europe, équipe Scool, Lille, France
- CNRS UMR 9189 – CRIStAL, Lille, France
- École Centrale de Lille, Lille, France
| | - Anne COTTEN
- Université de Lille , Lille, France
- Centre hospitalier universitaire de Lille, Lille, France
| | - Xavier DEMONDION
- Université de Lille , Lille, France
- Centre hospitalier universitaire de Lille, Lille, France
| | - Philippe PREUX
- Université de Lille , Lille, France
- Inria Lille – Nord Europe, équipe Scool, Lille, France
- CNRS UMR 9189 – CRIStAL, Lille, France
- École Centrale de Lille, Lille, France
| | - Thibaut JACQUES
- Université de Lille , Lille, France
- Centre hospitalier universitaire de Lille, Lille, France
| |
Collapse
|
15
|
Tong WJ, Wu SH, Cheng MQ, Huang H, Liang JY, Li CQ, Guo HL, He DN, Liu YH, Xiao H, Hu HT, Ruan SM, Li MD, Lu MD, Wang W. Integration of Artificial Intelligence Decision Aids to Reduce Workload and Enhance Efficiency in Thyroid Nodule Management. JAMA Netw Open 2023; 6:e2313674. [PMID: 37191957 PMCID: PMC10189570 DOI: 10.1001/jamanetworkopen.2023.13674] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/26/2022] [Accepted: 04/01/2023] [Indexed: 05/17/2023] Open
Abstract
Importance To optimize the integration of artificial intelligence (AI) decision aids and reduce workload in thyroid nodule management, it is critical to incorporate personalized AI into the decision-making processes of radiologists with varying levels of expertise. Objective To develop an optimized integration of AI decision aids for reducing radiologists' workload while maintaining diagnostic performance compared with traditional AI-assisted strategy. Design, Setting, and Participants In this diagnostic study, a retrospective set of 1754 ultrasonographic images of 1048 patients with 1754 thyroid nodules from July 1, 2018, to July 31, 2019, was used to build an optimized strategy based on how 16 junior and senior radiologists incorporated AI-assisted diagnosis results with different image features. In the prospective set of this diagnostic study, 300 ultrasonographic images of 268 patients with 300 thyroid nodules from May 1 to December 31, 2021, were used to compare the optimized strategy with the traditional all-AI strategy in terms of diagnostic performance and workload reduction. Data analyses were completed in September 2022. Main Outcomes and Measures The retrospective set of images was used to develop an optimized integration of AI decision aids for junior and senior radiologists based on the selection of AI-assisted significant or nonsignificant features. In the prospective set of images, the diagnostic performance, time-based cost, and assisted diagnosis were compared between the optimized strategy and the traditional all-AI strategy. Results The retrospective set included 1754 ultrasonographic images from 1048 patients (mean [SD] age, 42.1 [13.2] years; 749 women [71.5%]) with 1754 thyroid nodules (mean [SD] size, 16.4 [10.6] mm); 748 nodules (42.6%) were benign, and 1006 (57.4%) were malignant. The prospective set included 300 ultrasonographic images from 268 patients (mean [SD] age, 41.7 [14.1] years; 194 women [72.4%]) with 300 thyroid nodules (mean [SD] size, 17.2 [6.8] mm); 125 nodules (41.7%) were benign, and 175 (58.3%) were malignant. For junior radiologists, the ultrasonographic features that were not improved by AI assistance included cystic or almost completely cystic nodules, anechoic nodules, spongiform nodules, and nodules smaller than 5 mm, whereas for senior radiologists the features that were not improved by AI assistance were cystic or almost completely cystic nodules, anechoic nodules, spongiform nodules, very hypoechoic nodules, nodules taller than wide, lobulated or irregular nodules, and extrathyroidal extension. Compared with the traditional all-AI strategy, the optimized strategy was associated with increased mean task completion times for junior radiologists (reader 11, from 15.2 seconds [95% CI, 13.2-17.2 seconds] to 19.4 seconds [95% CI, 15.6-23.3 seconds]; reader 12, from 12.7 seconds [95% CI, 11.4-13.9 seconds] to 15.6 seconds [95% CI, 13.6-17.7 seconds]), but shorter times for senior radiologists (reader 14, from 19.4 seconds [95% CI, 18.1-20.7 seconds] to 16.8 seconds [95% CI, 15.3-18.3 seconds]; reader 16, from 12.5 seconds [95% CI, 12.1-12.9 seconds] to 10.0 seconds [95% CI, 9.5-10.5 seconds]). There was no significant difference in sensitivity (range, 91%-100%) or specificity (range, 94%-98%) between the 2 strategies for readers 11 to 16. Conclusions and Relevance This diagnostic study suggests that an optimized AI strategy in thyroid nodule management may reduce diagnostic time-based costs without sacrificing diagnostic accuracy for senior radiologists, while the traditional all-AI strategy may still be more beneficial for junior radiologists.
Collapse
Affiliation(s)
- Wen-Juan Tong
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Shao-Hong Wu
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Mei-Qing Cheng
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Hui Huang
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Jin-Yu Liang
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Chao-Qun Li
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Huan-Ling Guo
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Dan-Ni He
- Department of Medical Ultrasonics, The Seventh Affiliated Hospital, Sun Yat-sen University, Shenzhen, China
| | - Yi-Hao Liu
- Clinical Trials Unit, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Han Xiao
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Hang-Tong Hu
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Si-Min Ruan
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Ming-De Li
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Ming-De Lu
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
- Department of Hepatobiliary Surgery, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Wei Wang
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| |
Collapse
|
16
|
Shin HJ, Han K, Ryu L, Kim EK. The impact of artificial intelligence on the reading times of radiologists for chest radiographs. NPJ Digit Med 2023; 6:82. [PMID: 37120423 PMCID: PMC10148851 DOI: 10.1038/s41746-023-00829-4] [Citation(s) in RCA: 18] [Impact Index Per Article: 18.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2023] [Accepted: 04/19/2023] [Indexed: 05/01/2023] Open
Abstract
Whether the utilization of artificial intelligence (AI) during the interpretation of chest radiographs (CXRs) would affect the radiologists' workload is of particular interest. Therefore, this prospective observational study aimed to observe how AI affected the reading times of radiologists in the daily interpretation of CXRs. Radiologists who agreed to have the reading times of their CXR interpretations collected from September to December 2021 were recruited. Reading time was defined as the duration in seconds from opening CXRs to transcribing the image by the same radiologist. As commercial AI software was integrated for all CXRs, the radiologists could refer to AI results for 2 months (AI-aided period). During the other 2 months, the radiologists were automatically blinded to the AI results (AI-unaided period). A total of 11 radiologists participated, and 18,680 CXRs were included. Total reading times were significantly shortened with AI use, compared to no use (13.3 s vs. 14.8 s, p < 0.001). When there was no abnormality detected by AI, reading times were shorter with AI use (mean 10.8 s vs. 13.1 s, p < 0.001). However, if any abnormality was detected by AI, reading times did not differ according to AI use (mean 18.6 s vs. 18.4 s, p = 0.452). Reading times increased as abnormality scores increased, and a more significant increase was observed with AI use (coefficient 0.09 vs. 0.06, p < 0.001). Therefore, the reading times of CXRs among radiologists were influenced by the availability of AI. Overall reading times shortened when radiologists referred to AI; however, abnormalities detected by AI could lengthen reading times.
Collapse
Affiliation(s)
- Hyun Joo Shin
- Department of Radiology, Research Institute of Radiological Science and Center for Clinical Imaging Data Science, Yongin Severance Hospital, Yonsei University College of Medicine, 363, Dongbaekjukjeon-daero, Giheung-gu, Yongin-si, Gyeonggi-do, 16995, South Korea
- Center for Digital Health, Yongin Severance Hospital, Yonsei University College of Medicine, 363, Dongbaekjukjeon-daero, Giheung-gu, Yongin-si, Gyeonggi-do, 16995, South Korea
| | - Kyunghwa Han
- Department of Radiology, Severance Hospital, Research Institute of Radiological Science and Center for Clinical Imaging Data Science, Yonsei University College of Medicine, 50-1 Yonsei-Ro, Seodaemun-Gu, Seoul, 03722, South Korea
| | - Leeha Ryu
- Department of Biostatistics and Computing, Yonsei University Graduate School, 50-1 Yonsei-Ro, Seodaemun-Gu, Seoul, 03722, South Korea
| | - Eun-Kyung Kim
- Department of Radiology, Research Institute of Radiological Science and Center for Clinical Imaging Data Science, Yongin Severance Hospital, Yonsei University College of Medicine, 363, Dongbaekjukjeon-daero, Giheung-gu, Yongin-si, Gyeonggi-do, 16995, South Korea.
- Center for Digital Health, Yongin Severance Hospital, Yonsei University College of Medicine, 363, Dongbaekjukjeon-daero, Giheung-gu, Yongin-si, Gyeonggi-do, 16995, South Korea.
| |
Collapse
|
17
|
Nguyen T, Hermann AL, Ventre J, Ducarouge A, Pourchot A, Marty V, Regnard NE, Guermazi A. High performance for bone age estimation with an artificial intelligence solution. Diagn Interv Imaging 2023:S2211-5684(23)00075-X. [PMID: 37095034 DOI: 10.1016/j.diii.2023.04.003] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2023] [Revised: 04/05/2023] [Accepted: 04/11/2023] [Indexed: 04/26/2023]
Abstract
PURPOSE The purpose of this study was to compare the performance of an artificial intelligence (AI) solution to that of a senior general radiologist for bone age assessment. MATERIAL AND METHODS Anteroposterior hand radiographs of eight boys and eight girls from each age interval between five and 17 year-old from four different radiology departments were retrospectively collected. Two board-certified pediatric radiologists with knowledge of the sex and chronological age of the patients independently estimated the Greulich and Pyle bone age to determine the standard of reference. A senior general radiologist not specialized in pediatric radiology (further referred to as "the reader") then determined the bone age with knowledge of the sex and chronological age. The results of the reader were then compared to those of the AI solution using mean absolute error (MAE) in age estimation. RESULTS The study dataset included a total of 206 patients (102 boys of mean chronological age of 10.9 ± 3.7 [SD] years, 104 girls of mean chronological age of 11 ± 3.7 [SD] years). For both sexes, the AI algorithm showed a significantly lower MAE than the reader (P < 0.007). In boys, the MAE was 0.488 years (95% confidence interval [CI]: 0.28-0.44; r2 = 0.978) for the AI algorithm and 0.771 years (95% CI: 0.64-0.90; r2 = 0.94) for the reader. In girls, the MAE was 0.494 years (95% CI: 0.41-0.56; r2 = 0.973) for the AI algorithm and 0.673 years (95% CI: 0.54-0.81; r2 = 0.934) for the reader. CONCLUSION The AI solution better estimates the Greulich and Pyle bone age than a general radiologist does.
Collapse
Affiliation(s)
- Toan Nguyen
- Department of Pediatric Radiology, Hôpital Armand Trousseau AP-HP, 75012 Paris, France; Gleamer, 75010 Paris, France.
| | - Anne-Laure Hermann
- Department of Pediatric Radiology, Hôpital Armand Trousseau AP-HP, 75012 Paris, France
| | | | | | | | | | - Nor-Eddine Regnard
- Gleamer, 75010 Paris, France; Réseau Imagerie Sud Francilien, 77127 Lieusaint, France
| | - Ali Guermazi
- Department of Radiology, Boston University School of Medicine, VA Boston Healthcare System, 1400 VFW Parkway, Suite 1B105, West Roxbury, MA 02132, United States of America
| |
Collapse
|
18
|
Artificial intelligence in emergency radiology: A review of applications and possibilities. Diagn Interv Imaging 2023; 104:6-10. [PMID: 35933269 DOI: 10.1016/j.diii.2022.07.005] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2022] [Accepted: 07/23/2022] [Indexed: 01/10/2023]
Abstract
Artificial intelligence (AI) applications in radiology have been rising exponentially in the last decade. Although AI has found usage in various areas of healthcare, its utilization in the emergency department (ED) as a tool for emergency radiologists shows great promise towards easing some of the challenges faced daily. There have been numerous reported studies examining the application of AI-based algorithms in identifying common ED conditions to ensure more rapid reporting and in turn quicker patient care. In addition to interpretive applications, AI assists with many of the non-interpretive tasks that are encountered every day by emergency radiologists. These include, but are not limited to, protocolling, image quality control and workflow prioritization. AI continues to face challenges such as physician uptake or costs, but is a long-term investment that shows great potential to relieve many difficulties faced by emergency radiologists and ultimately improve patient outcomes. This review sums up the current advances of AI in emergency radiology, including current diagnostic applications (interpretive) and applications that stretch beyond imaging (non-interpretive), analyzes current drawbacks of AI in emergency radiology and discusses future challenges.
Collapse
|
19
|
Parpaleix A, Parsy C, Cordari M, Mejdoubi M. Assessment of a combined musculoskeletal and chest deep learning-based detection solution in an emergency setting. Eur J Radiol Open 2023; 10:100482. [PMID: 36941993 PMCID: PMC10023863 DOI: 10.1016/j.ejro.2023.100482] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2022] [Revised: 01/31/2023] [Accepted: 03/01/2023] [Indexed: 03/12/2023] Open
Abstract
Rationale and objectives Triage and diagnostic deep learning-based support solutions have started to take hold in everyday emergency radiology practice with the hope of alleviating workflows. Although previous works had proven that artificial intelligence (AI) may increase radiologist and/or emergency physician reading performances, they were restricted to finding, bodypart and/or age subgroups, without evaluating a routine emergency workflow composed of chest and musculoskeletal adult and pediatric cases. We aimed at evaluating a multiple musculoskeletal and chest radiographic findings deep learning-based commercial solution on an adult and pediatric emergency workflow, focusing on discrepancies between emergency and radiology physicians. Material and methods This retrospective, monocentric and observational study included 1772 patients who underwent an emergency radiograph between July and October 2020, excluding spine, skull and plain abdomen procedures. Emergency and radiology reports, obtained without AI as part of the clinical workflow, were collected and discordant cases were reviewed to obtain the radiology reference standard. Case-level AI outputs and emergency reports were compared to the reference standard. DeLong and Wald tests were used to compare ROC-AUC and Sensitivity/Specificity, respectively. Results Results showed an overall AI ROC-AUC of 0.954 with no difference across age or body part subgroups. Real-life emergency physicians' sensitivity was 93.7 %, not significantly different to the AI model (P = 0.105), however in 172/1772 (9.7 %) cases misdiagnosed by emergency physicians. In this subset, AI accuracy was 90.1 %. Conclusion This study highlighted that multiple findings AI solution for emergency radiographs is efficient and complementary to emergency physicians, and could help reduce misdiagnosis in the absence of immediate radiological expertize.
Collapse
Affiliation(s)
- Alexandre Parpaleix
- Department of Radiology, Valenciennes General Hospital, Valenciennes, France
- Correspondence to: Département de radiologie, Centre Hospitalier de Valenciennes, 114 Av. Desandrouin, 59300 Valenciennes, France.
| | - Clémence Parsy
- Department of Radiology, Valenciennes General Hospital, Valenciennes, France
| | | | - Mehdi Mejdoubi
- Department of Radiology, Valenciennes General Hospital, Valenciennes, France
| |
Collapse
|
20
|
Boeken T, Feydy J, Lecler A, Soyer P, Feydy A, Barat M, Duron L. Artificial intelligence in diagnostic and interventional radiology: Where are we now? Diagn Interv Imaging 2023; 104:1-5. [PMID: 36494290 DOI: 10.1016/j.diii.2022.11.004] [Citation(s) in RCA: 51] [Impact Index Per Article: 51.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2022] [Accepted: 11/18/2022] [Indexed: 12/12/2022]
Abstract
The emergence of massively parallel yet affordable computing devices has been a game changer for research in the field of artificial intelligence (AI). In addition, dramatic investment from the web giants has fostered the development of a high-quality software stack. Going forward, the combination of faster computers with dedicated software libraries and the widespread availability of data has opened the door to more flexibility in the design of AI models. Radiomics is a process used to discover new imaging biomarkers that has multiple applications in radiology and can be used in conjunction with AI. AI can be used throughout the various processes of diagnostic imaging, including data acquisition, reconstruction, analysis and reporting. Today, the concept of "AI-augmented" radiologists is preferred to the theory of the replacement of radiologists by AI in many indications. Current evidence bolsters the assumption that AI-assisted radiologists work better and faster. Interventional radiology becomes a data-rich specialty where the entire procedure is fully recorded in a standardized DICOM format and accessible via standard picture archiving and communication systems. No other interventional specialty can bolster such readiness. In this setting, interventional radiology could lead the development of AI-powered applications in the broader interventional community. This article provides an update on the current status of radiomics and AI research, analyzes upcoming challenges and also discusses the main applications in AI in interventional radiology to help radiologists better understand and criticize articles reporting AI in medical imaging.
Collapse
Affiliation(s)
- Tom Boeken
- Université Paris Cité, Faculté de Médecine, Paris 75006, France; Department of Vascular and Oncological Interventional Radiology, Hôpital Européen Georges Pompidou, APHP, Paris 75015, France; HeKA team, INRIA, Paris 75012 , France.
| | | | - Augustin Lecler
- Université Paris Cité, Faculté de Médecine, Paris 75006, France; Department of Radiology, Rothschild Foundation Hospital, Paris 75019, France
| | - Philippe Soyer
- Université Paris Cité, Faculté de Médecine, Paris 75006, France; Department of Radiology, Hôpital Cochin, APHP, Paris 75014, France
| | - Antoine Feydy
- Université Paris Cité, Faculté de Médecine, Paris 75006, France; Department of Radiology, Hôpital Cochin, APHP, Paris 75014, France
| | - Maxime Barat
- Université Paris Cité, Faculté de Médecine, Paris 75006, France; Department of Radiology, Hôpital Cochin, APHP, Paris 75014, France
| | - Loïc Duron
- Université Paris Cité, Faculté de Médecine, Paris 75006, France; Department of Radiology, Rothschild Foundation Hospital, Paris 75019, France
| |
Collapse
|
21
|
Soyer P, Fishman EK, Rowe SP, Patlas MN, Chassagnon G. Does artificial intelligence surpass the radiologist? Diagn Interv Imaging 2022; 103:445-447. [PMID: 35973913 DOI: 10.1016/j.diii.2022.08.001] [Citation(s) in RCA: 35] [Impact Index Per Article: 17.5] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2022] [Accepted: 08/02/2022] [Indexed: 12/30/2022]
Affiliation(s)
- Philippe Soyer
- Department of Radiology, Hôpital Cochin, Assistance Publique-Hopitaux de Paris, 75014 Paris, France; Université Paris Cité, Faculté de Médecine, 75006, Paris, France.
| | - Elliot K Fishman
- The Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins University School of Medicine, Baltimore, MD, 21287, USA
| | - Steven P Rowe
- The Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins University School of Medicine, Baltimore, MD, 21287, USA
| | - Michael N Patlas
- Department of Radiology, Hamilton General Hospital, McMaster University Hamilton, ON, Canada L8L 2X2
| | - Guillaume Chassagnon
- Department of Radiology, Hôpital Cochin, Assistance Publique-Hopitaux de Paris, 75014 Paris, France; Université Paris Cité, Faculté de Médecine, 75006, Paris, France
| |
Collapse
|