Ankle fracture classification using deep learning: automating detailed AO Foundation/Orthopedic Trauma Association (AO/OTA) 2018 malleolar fracture identification reaches a high degree of correct classification

Artificial intelligence diagnostic accuracy in fracture detection from plain radiographs and comparing it with clinicians: a systematic review and meta-analysis

PURPOSE

Fracture detection is one of the most commonly used and studied aspects of artificial intelligence (AI) in medicine. In this systematic review and meta-analysis, we aimed to summarize available literature and data regarding AI performance in fracture detection on plain radiographs and various factors affecting it.

METHODS

We systematically reviewed studies evaluating AI algorithms in detecting bone fractures in plain radiographs, combined their performance using meta-analysis (a bivariate regression approach), and compared it with that of clinicians. We also analyzed the factors potentially affecting algorithm performance using meta-regression.

RESULTS

Our analysis included 100 studies. In 83 studies with confusion matrices, AI algorithms showed a sensitivity of 91.43% and a specificity of 92.12% (Area under the summary receiver operator curve = 0.968). After adjustment and false discovery rate correction, tibia/fibula (excluding ankle) fractures were associated with higher (7.0%, p=0.004) AI sensitivity, while more recent publications (5.5%, p=0.003) and Xception architecture (6.6%, p<0.001) were associated with higher specificity. Clinicians and AI showed similar specificity in fracture identification, although AI leaned to higher sensitivity (7.6%, p=0.07). Radiologists, on the other hand, were more specific than AI overall and in several subgroups, and more sensitive to hip fractures before FDR correction.

CONCLUSIONS

Currently available AI aids could result in a significant improvement in care where radiologists are not readily available. Moreover, identifying factors affecting algorithm performance could guide AI development teams in their process of optimizing their products.

[Translated article] Fractures of the peroneal malleolus: Current concepts

Ankle fractures represent up to 9% of all fractures, with an increased incidence in the elderly population. Among these fractures, isolated fractures of the lateral malleolus are the most common, representing 65-70% of all cases. The therapeutic decision-making primarily relies on the stability of the ankle ring, considering it stable if affected at one point and unstable if two or more points are affected. Surgical treatment focuses on restoring the length of the fibula, joint reconstruction, stabilising the syndesmosis, and providing a stable fixation. It is crucial to rule out associated injuries that may influence therapeutic management. This article reviews the evaluation and management of lateral malleolus fractures, proposes a decision-making algorithm, and examines several fibular fixation options.

Fractures of the peroneal malleolus: Current concepts

Ankle fractures represent up to 9% of all fractures, with an increased incidence in the elderly population. Among these fractures, isolated fractures of the lateral malleolus are the most common, representing 65-70% of all cases. The therapeutic decision-making primarily relies on the stability of the ankle ring, considering it stable if affected at one point and unstable if two or more points are affected. Surgical treatment focuses on restoring the length of the fibula, joint reconstruction, stabilizing the syndesmosis, and providing a stable fixation. It is crucial to rule out associated injuries that may influence therapeutic management. This article reviews the evaluation and management of lateral malleolus fractures, proposes a decision-making algorithm, and examines several fibular fixation options.

Development and evaluation of a deep learning-based model for simultaneous detection and localization of rib and clavicle fractures in trauma patients' chest radiographs

Purpose

To develop a rib and clavicle fracture detection model for chest radiographs in trauma patients using a deep learning (DL) algorithm.

Materials and methods

We retrospectively collected 56 145 chest X-rays (CXRs) from trauma patients in a trauma center between August 2008 and December 2016. A rib/clavicle fracture detection DL algorithm was trained using this data set with 991 (1.8%) images labeled by experts with fracture site locations. The algorithm was tested on independently collected 300 CXRs in 2017. An external test set was also collected from hospitalized trauma patients in a regional hospital for evaluation. The receiver operating characteristic curve with area under the curve (AUC), accuracy, sensitivity, specificity, precision, and negative predictive value of the model on each test set was evaluated. The prediction probability on the images was visualized as heatmaps.

Results

The trained DL model achieved an AUC of 0.912 (95% CI 87.8 to 94.7) on the independent test set. The accuracy, sensitivity, and specificity on the given cut-off value are 83.7, 86.8, and 80.4, respectively. On the external test set, the model had a sensitivity of 88.0 and an accuracy of 72.5. While the model exhibited a slight decrease in accuracy on the external test set, it maintained its sensitivity in detecting fractures.

Conclusion

The algorithm detects rib and clavicle fractures concomitantly in the CXR of trauma patients with high accuracy in locating lesions through heatmap visualization.

AI applications in musculoskeletal imaging: a narrative review

This narrative review focuses on clinical applications of artificial intelligence (AI) in musculoskeletal imaging. A range of musculoskeletal disorders are discussed using a clinical-based approach, including trauma, bone age estimation, osteoarthritis, bone and soft-tissue tumors, and orthopedic implant-related pathology. Several AI algorithms have been applied to fracture detection and classification, which are potentially helpful tools for radiologists and clinicians. In bone age assessment, AI methods have been applied to assist radiologists by automatizing workflow, thus reducing workload and inter-observer variability. AI may potentially aid radiologists in identifying and grading abnormal findings of osteoarthritis as well as predicting the onset or progression of this disease. Either alone or combined with radiomics, AI algorithms may potentially improve diagnosis and outcome prediction of bone and soft-tissue tumors. Finally, information regarding appropriate positioning of orthopedic implants and related complications may be obtained using AI algorithms. In conclusion, rather than replacing radiologists, the use of AI should instead help them to optimize workflow, augment diagnostic performance, and keep up with ever-increasing workload.Relevance statement This narrative review provides an overview of AI applications in musculoskeletal imaging. As the number of AI technologies continues to increase, it will be crucial for radiologists to play a role in their selection and application as well as to fully understand their potential value in clinical practice. Key points • AI may potentially assist musculoskeletal radiologists in several interpretative tasks.• AI applications to trauma, age estimation, osteoarthritis, tumors, and orthopedic implants are discussed.• AI should help radiologists to optimize workflow and augment diagnostic performance.

Artificial intelligence in foot and ankle surgery: current concepts

The twenty-first century has proven that data are the new gold. Artificial intelligence (AI) driven technologies might potentially change the clinical practice in all medical specialities, including orthopedic surgery. AI has a broad spectrum of subcomponents, including machine learning, which consists of a subdivision called deep learning. AI has the potential to increase healthcare delivery, improve indications and interventions, and minimize errors. In orthopedic surgery. AI supports the surgeon in the evaluation of radiological images, training of surgical residents, and excellent performance of machine-assisted surgery. The AI algorithms improve the administrative and management processes of hospitals and clinics, electronic healthcare databases, monitoring the outcomes, and safety controls. AI models are being developed in nearly all orthopedic subspecialties, including arthroscopy, arthroplasty, tumor, spinal and pediatric surgery. The present study discusses current applications, limitations, and future prospective of AI in foot and ankle surgery.

[Ligamentous injuries of the ankle joint : Importance of radiological imaging procedures]

Injuries of the ankle joint show lesions of the lateral ligamentous apparatus with an incidence of 1 in 1000 cases. With a range of 0.3-0.7 in 1000 cases, injuries of the medial ligamentous apparatus occur more frequently than previously thought and are often overlooked. Correct imaging diagnostics of the ankle ligaments are mandatory for the prognosis and treatment planning. In cases of clinically strong suspicion of ligamentous injury or osteochondral lesions of the ankle joint, after primary radiographic evaluation magnetic resonance imaging (MRI) is primarily recommended for a more precise assessment. Additionally, the sensitivity for assessment of the ligaments with ultrasonography is good (91% vs. 97% with MRI) but its specificity is inferior compared to MRI (63% vs. 93% with MRI). Supination trauma is the most common ankle injury and attention should be paid to the anterior tibiofibular ligament and calcaneofibular ligament. In approximately 50%, injuries to the superficial layer of the medial collateral ligament complex are associated with lesions of the posterior tibiotalar ligament.

Performance of ChatGPT, human radiologists, and context-aware ChatGPT in identifying AO codes from radiology reports

While radiologists can describe a fracture's morphology and complexity with ease, the translation into classification systems such as the Arbeitsgemeinschaft Osteosynthesefragen (AO) Fracture and Dislocation Classification Compendium is more challenging. We tested the performance of generic chatbots and chatbots aware of specific knowledge of the AO classification provided by a vector-index and compared it to human readers. In the 100 radiological reports we created based on random AO codes, chatbots provided AO codes significantly faster than humans (mean 3.2 s per case vs. 50 s per case, p < .001) though not reaching human performance (max. chatbot performance of 86% correct full AO codes vs. 95% in human readers). In general, chatbots based on GPT 4 outperformed the ones based on GPT 3.5-Turbo. Further, we found that providing specific knowledge substantially enhances the chatbot's performance and consistency as the context-aware chatbot based on GPT 4 provided 71% consistent correct full AO codes for the compared to the 2% consistent correct full AO codes for the generic ChatGPT 4. This provides evidence, that refining and providing specific context to ChatGPT will be the next essential step in harnessing its power.

Deep learning classification of shoulder fractures on plain radiographs of the humerus, scapula and clavicle

In this study, we present a deep learning model for fracture classification on shoulder radiographs using a convolutional neural network (CNN). The primary aim was to evaluate the classification performance of the CNN for proximal humeral fractures (PHF) based on the AO/OTA classification system. Secondary objectives included evaluating the model's performance for diaphyseal humerus, clavicle, and scapula fractures. The training dataset consisted of 6,172 examinations, including 2-7 radiographs per examination. The overall area under the curve (AUC) for fracture classification was 0.89, indicating good performance. For PHF classification, 12 out of 16 classes achieved an AUC of 0.90 or greater. Additionally, the CNN model had excellent overall AUC for diaphyseal humerus fractures (0.97), clavicle fractures (0.96), and good AUC for scapula fractures (0.87). Despite the limitations of the study, such as the reliance on ground truth labels provided by students with limited radiographic assessment experience, our findings are in concordance with previous studies, further consolidating CNN as potent fracture classifiers in plain radiographs. The inclusion of multiple radiographs with different views from each examination, as well as the generally unselected nature of the sample, contributed to the overall generalizability of the study. This is the fifth study published by our group on AI in orthopaedic radiographs, which has consistently shown promising results. The next challenge for the orthopaedic research community will be to transfer these results from the research setting into clinical practice. External validation of the CNN model should be conducted in the future before it is considered for use in a clinical setting.

[Ligamentous injuries of the ankle joint : Importance of radiological imaging procedures]

Injuries of the ankle joint show lesions of the lateral ligamentous apparatus with an incidence of 1 in 1000 cases. With a range of 0.3-0.7 in 1000 cases, injuries of the medial ligamentous apparatus occur more frequently than previously thought and are often overlooked. Correct imaging diagnostics of the ankle ligaments are mandatory for the prognosis and treatment planning. In cases of clinically strong suspicion of ligamentous injury or osteochondral lesions of the ankle joint, after primary radiographic evaluation magnetic resonance imaging (MRI) is primarily recommended for a more precise assessment. Additionally, the sensitivity for assessment of the ligaments with ultrasonography is good (91% vs. 97% with MRI) but its specificity is inferior compared to MRI (63% vs. 93% with MRI). Supination trauma is the most common ankle injury and attention should be paid to the anterior tibiofibular ligament and calcaneofibular ligament. In approximately 50%, injuries to the superficial layer of the medial collateral ligament complex are associated with lesions of the posterior tibiotalar ligament.

The Application of Design Thinking in Developing a Deep Learning Algorithm for Hip Fracture Detection

(1) Background: Design thinking is a problem-solving approach that has been applied in various sectors, including healthcare and medical education. While deep learning (DL) algorithms can assist in clinical practice, integrating them into clinical scenarios can be challenging. This study aimed to use design thinking steps to develop a DL algorithm that accelerates deployment in clinical practice and improves its performance to meet clinical requirements. (2) Methods: We applied the design thinking process to interview clinical doctors and gain insights to develop and modify the DL algorithm to meet clinical scenarios. We also compared the DL performance of the algorithm before and after the integration of design thinking. (3) Results: After empathizing with clinical doctors and defining their needs, we identified the unmet need of five trauma surgeons as "how to reduce the misdiagnosis of femoral fracture by pelvic plain film (PXR) at initial emergency visiting". We collected 4235 PXRs from our hospital, of which 2146 had a hip fracture (51%) from 2008 to 2016. We developed hip fracture DL detection models based on the Xception convolutional neural network by using these images. By incorporating design thinking, we improved the diagnostic accuracy from 0.91 (0.84-0.96) to 0.95 (0.93-0.97), the sensitivity from 0.97 (0.89-1.00) to 0.97 (0.94-0.99), and the specificity from 0.84 (0.71-0.93) to 0.93(0.990-0.97). (4) Conclusions: In summary, this study demonstrates that design thinking can ensure that DL solutions developed for trauma care are user-centered and meet the needs of patients and healthcare providers.

Artificial intelligence CAD tools in trauma imaging: a scoping review from the American Society of Emergency Radiology (ASER) AI/ML Expert Panel

BACKGROUND

AI/ML CAD tools can potentially improve outcomes in the high-stakes, high-volume model of trauma radiology. No prior scoping review has been undertaken to comprehensively assess tools in this subspecialty.

PURPOSE

To map the evolution and current state of trauma radiology CAD tools along key dimensions of technology readiness.

METHODS

Following a search of databases, abstract screening, and full-text document review, CAD tool maturity was charted using elements of data curation, performance validation, outcomes research, explainability, user acceptance, and funding patterns. Descriptive statistics were used to illustrate key trends.

RESULTS

A total of 4052 records were screened, and 233 full-text articles were selected for content analysis. Twenty-one papers described FDA-approved commercial tools, and 212 reported algorithm prototypes. Works ranged from foundational research to multi-reader multi-case trials with heterogeneous external data. Scalable convolutional neural network-based implementations increased steeply after 2016 and were used in all commercial products; however, options for explainability were narrow. Of FDA-approved tools, 9/10 performed detection tasks. Dataset sizes ranged from < 100 to > 500,000 patients, and commercialization coincided with public dataset availability. Cross-sectional torso datasets were uniformly small. Data curation methods with ground truth labeling by independent readers were uncommon. No papers assessed user acceptance, and no method included human-computer interaction. The USA and China had the highest research output and frequency of research funding.

CONCLUSIONS

Trauma imaging CAD tools are likely to improve patient care but are currently in an early stage of maturity, with few FDA-approved products for a limited number of uses. The scarcity of high-quality annotated data remains a major barrier.

Development of convolutional neural network model for diagnosing osteochondral lesions of the talus using anteroposterior ankle radiographs

Deep learning is an advanced machine learning technique that is used in several medical fields to diagnose diseases and predict therapeutic outcomes. In this study, using anteroposterior ankle radiographs, we developed a convolutional neural network (CNN) model to diagnose osteochondral lesions of the talus (OLTs) using ankle radiographs as input data. We evaluated whether a CNN model trained on anteroposterior ankle radiographs could help diagnose the presence of OLT. We retrospectively collected 379 cases (OLT cases = 133, non-OLT cases = 246) of anteroposterior ankle radiographs taken at a university hospital between January 2010 and December 2020. The OLT was diagnosed using ankle magnetic resonance images of each patient. Among the 379 cases, 70% of the included data were randomly selected as the training set, 10% as the validation set, and the remaining 20% were assigned to the test set to evaluate the model performance. To accurately classify OLT and non-OLT, we cropped the area of the ankle on anteroposterior ankle radiographs, resized the image to 224 × 224, and used it as the input data. We then used the Visual Geometry Group Network model to determine whether the input image was OLT or non-OLT. The performance of the CNN model for the area under the curve, accuracy, positive predictive value, and negative predictive value on the test data were 0.774 (95% confidence interval [CI], 0.673-0.875), 81.58% (95% CI, 0.729-0.903), 80.95% (95% CI, 0.773-0.846), and 81.82% (95% CI, 0.804-0.832), respectively. A CNN model trained on anteroposterior ankle radiographs achieved meaningful accuracy in diagnosing OLT and demonstrated that it could help diagnose OLT.

Artificial Intelligence in Musculoskeletal Conditions

Artificial intelligence (AI) refers to computer capabilities that resemble human intelligence. AI implies the ability to learn and perform tasks that have not been specifically programmed. Moreover, it is an iterative process involving the ability of computerized systems to capture information, transform it into knowledge, and process it to produce adaptive changes in the environment. A large labeled database is needed to train the AI system and generate a robust algorithm. Otherwise, the algorithm cannot be applied in a generalized way. AI can facilitate the interpretation and acquisition of radiological images. In addition, it can facilitate the detection of trauma injuries and assist in orthopedic and rehabilitative processes. The applications of AI in musculoskeletal conditions are promising and are likely to have a significant impact on the future management of these patients.

Evaluation of ensemble strategy on the development of multiple view ankle fracture detection algorithm

OBJECTIVE

To identify the feasibility and efficiency of deep convolutional neural networks (DCNNs) in the detection of ankle fractures and to explore ensemble strategies that applied multiple projections of radiographs.Ankle radiographs (AXRs) are the primary tool used to diagnose ankle fractures. Applying DCNN algorithms on AXRs can potentially improve the diagnostic accuracy and efficiency of detecting ankle fractures.

METHODS

A DCNN was trained using a trauma image registry, including 3102 AXRs. We separately trained the DCNN on anteroposterior (AP) and lateral (Lat) AXRs. Different ensemble methods, such as "sum-up," "severance-OR," and "severance-Both," were evaluated to incorporate the results of the model using different projections of view.

RESULTS

The AP/Lat model's individual sensitivity, specificity, positive-predictive value, accuracy, and F1 score were 79%/84%, 90%/86%, 88%/86%, 83%/85%, and 0.816/0.850, respectively. Furthermore, the area under the receiver operating characteristic curve (AUROC) of the AP/Lat model was 0.890/0.894 (95% CI: 0.826-0.954/0.831-0.953). The sum-up method generated balanced results by applying both models and obtained an AUROC of 0.917 (95% CI: 0.863-0.972) with 87% accuracy. The severance-OR method resulted in a better sensitivity of 90%, and the severance-Both method obtained a high specificity of 94%.

CONCLUSION

Ankle fracture in the AXR could be identified by the trained DCNN algorithm. The selection of ensemble methods can depend on the clinical situation which might help clinicians detect ankle fractures efficiently without interrupting the current clinical pathway.

ADVANCES IN KNOWLEDGE

This study demonstrated different ensemble strategies of AI algorithms on multiple view AXRs to optimize the performance in various clinical needs.

Advancements in Artificial Intelligence for Foot and Ankle Surgery: A Systematic Review

Background

There has been a rapid increase in research applying artificial intelligence (AI) to various subspecialties of orthopaedic surgery, including foot and ankle surgery. The purpose of this systematic review is to (1) characterize the topics and objectives of studies using AI in foot and ankle surgery, (2) evaluate the performance of their models, and (3) evaluate their validity (internal or external validation).

Methods

A systematic literature review was conducted using PubMed/MEDLINE and Embase databases in December 2022. All studies that used AI or its subsets machine learning (ML) and deep learning (DL) in the setting of foot and ankle surgery relevant to orthopaedic surgeons were included. Studies were evaluated for their demographics, subject area, outcomes of interest, model(s) tested, model(s)' performance, and validity (internal or external).

Results

A total of 31 studies met inclusion criteria: 14 studies investigated AI for image interpretation, 13 studies investigated AI for clinical predictions, and 4 studies were grouped as "other." Studies commonly explored AI for ankle fractures, calcaneus fractures, hallux valgus, Achilles tendon pathologies, plantar fasciitis, and sports injuries. For studies reporting the area under the receiver operating characteristic curve (AUC), AUCs ranged from 0.64 (poor) to 0.99 (excellent). Two studies (6.45%) reported external validation.

Conclusion

Applications of AI in the field of foot and ankle surgery are expanding, particularly for image interpretation and clinical predictions. Current model performances range from poor to excellent, and most studies lack external validation, demonstrating a need for further research prior to deploying AI-based clinical applications.

Level of Evidence

Level III, retrospective cohort study.

Robotic Technology in Foot and Ankle Surgery: A Comprehensive Review

Recent developments in robotic technologies in the field of orthopaedic surgery have largely been focused on higher volume arthroplasty procedures, with a paucity of attention paid to robotic potential for foot and ankle surgery. The aim of this paper is to summarize past and present developments foot and ankle robotics and describe outcomes associated with these interventions, with specific emphasis on the following topics: translational and preclinical utilization of robotics, deep learning and artificial intelligence modeling in foot and ankle, current applications for robotics in foot and ankle surgery, and therapeutic and orthotic-related utilizations of robotics related to the foot and ankle. Herein, we describe numerous recent robotic advancements across foot and ankle surgery, geared towards optimizing intra-operative performance, improving detection of foot and ankle pathology, understanding ankle kinematics, and rehabilitating post-surgically. Future research should work to incorporate robotics specifically into surgical procedures as other specialties within orthopaedics have done, and to further individualize machinery to patients, with the ultimate goal to improve perioperative and post-operative outcomes.

Deep Learning Applications in Surgery: Current Uses and Future Directions

Deep learning (DL) is a subset of machine learning that is rapidly gaining traction in surgical fields. Its tremendous capacity for powerful data-driven problem-solving has generated computational breakthroughs in many realms, with the fields of medicine and surgery becoming increasingly prominent avenues. Through its multi-layer architecture of interconnected neural networks, DL enables feature extraction and pattern recognition of highly complex and large-volume data. Across various surgical specialties, DL is being applied to optimize both preoperative planning and intraoperative performance in new and innovative ways. Surgeons are now able to integrate deep learning tools into their practice to improve patient safety and outcomes. Through this review, we explore the applications of deep learning in surgery and related subspecialties with an aim to shed light on the practical utilization of this technology in the present and near future.

Development and external validation of automated detection, classification, and localization of ankle fractures: inside the black box of a convolutional neural network (CNN)

Purpose

Convolutional neural networks (CNNs) are increasingly being developed for automated fracture detection in orthopaedic trauma surgery. Studies to date, however, are limited to providing classification based on the entire image—and only produce heatmaps for approximate fracture localization instead of delineating exact fracture morphology. Therefore, we aimed to answer (1) what is the performance of a CNN that detects, classifies, localizes, and segments an ankle fracture, and (2) would this be externally valid?

Methods

The training set included 326 isolated fibula fractures and 423 non-fracture radiographs. The Detectron2 implementation of the Mask R-CNN was trained with labelled and annotated radiographs. The internal validation (or ‘test set’) and external validation sets consisted of 300 and 334 radiographs, respectively. Consensus agreement between three experienced fellowship-trained trauma surgeons was defined as the ground truth label. Diagnostic accuracy and area under the receiver operator characteristic curve (AUC) were used to assess classification performance. The Intersection over Union (IoU) was used to quantify accuracy of the segmentation predictions by the CNN, where a value of 0.5 is generally considered an adequate segmentation.

Results

The final CNN was able to classify fibula fractures according to four classes (Danis-Weber A, B, C and No Fracture) with AUC values ranging from 0.93 to 0.99. Diagnostic accuracy was 89% on the test set with average sensitivity of 89% and specificity of 96%. External validity was 89–90% accurate on a set of radiographs from a different hospital. Accuracies/AUCs observed were 100/0.99 for the ‘No Fracture’ class, 92/0.99 for ‘Weber B’, 88/0.93 for ‘Weber C’, and 76/0.97 for ‘Weber A’. For the fracture bounding box prediction by the CNN, a mean IoU of 0.65 (SD ± 0.16) was observed. The fracture segmentation predictions by the CNN resulted in a mean IoU of 0.47 (SD ± 0.17).

Conclusions

This study presents a look into the ‘black box’ of CNNs and represents the first automated delineation (segmentation) of fracture lines on (ankle) radiographs. The AUC values presented in this paper indicate good discriminatory capability of the CNN and substantiate further study of CNNs in detecting and classifying ankle fractures.

Level of evidence

II, Diagnostic imaging study.

[Artificial intelligence and novel approaches for treatment of non-union in bone : From established standard methods in medicine up to novel fields of research]

Methods of artificial intelligence (AI) have found applications in many fields of medicine within the last few years. Some disciplines already use these methods regularly within their clinical routine. However, the fields of application are wide and there are still many opportunities to apply these new AI concepts. This review article gives an insight into the history of AI and defines the special terms and fields, such as machine learning (ML), neural networks and deep learning. The classical steps in developing AI models are demonstrated here, as well as the iteration of data rectification and preparation, the training of a model and subsequent validation before transfer into a clinical setting are explained. Currently, musculoskeletal disciplines implement methods of ML and also neural networks, e.g. for identification of fractures or for classifications. Also, predictive models based on risk factor analysis for prevention of complications are being initiated. As non-union in bone is a rare but very complex disease with dramatic socioeconomic impact for the healthcare system, many open questions arise which could be better understood by using methods of AI in the future. New fields of research applying AI models range from predictive models and cost analysis to personalized treatment strategies.

Artificial intelligence and computer vision in orthopaedic trauma : the why, what, and how

Artificial intelligence (AI) is, in essence, the concept of 'computer thinking', encompassing methods that train computers to perform and learn from executing certain tasks, called machine learning, and methods to build intricate computer models that both learn and adapt, called complex neural networks. Computer vision is a function of AI by which machine learning and complex neural networks can be applied to enable computers to capture, analyze, and interpret information from clinical images and visual inputs. This annotation summarizes key considerations and future perspectives concerning computer vision, questioning the need for this technology (the 'why'), the current applications (the 'what'), and the approach to unlocking its full potential (the 'how'). Cite this article: Bone Joint J 2022;104-B(8):911-914.

Deep Learning Algorithms Improve the Detection of Subtle Lisfranc Malalignments on Weightbearing Radiographs

BACKGROUND

Detection of Lisfranc malalignment leading to the instability of the joint, particularly in subtle cases, has been a concern for foot and ankle care providers. X-ray radiographs are the mainstay in the diagnosis of these injuries; thus, improving the performance of clinicians in interpreting radiographs can noticeably affect the quality of health care in these patients. Here we assessed the performance of deep learning algorithms on weightbearing radiographs for detection of Lisfranc joint malalignment in patients with Lisfranc instability.

METHODS

In a retrospective study, 640 patients with Lisfranc malalignment leading to instability were recruited plus 640 individuals with uninjured feet and healthy Lisfranc joint as the control group. All radiographs were screened by orthopaedic surgeons. Two deep learning models were trained, validated, and tested (in a ratio 80:10:10) using a single-view (anteroposterior) and 3-view (anteroposterior, lateral, oblique) radiographs. The performances of the models were reported as sensitivity, specificity, positive and negative predictive values, accuracy, F score, and area under the curve (AUC).

RESULTS

No significant differences were observed between the patients and the controls regarding age, gender, race, and body mass index. The best deep learning algorithm outperformed our human interpreters (<1% vs ~10% misdiagnosis), 94.8% sensitivity, 96.9% specificity, 98.6% accuracy, 95.8% F score, and 99.4% AUC.

CONCLUSION

Deep learning methods have shown promising potential in acting as an assistant interpreter of radiographic images in patients with Lisfranc malalignment. Developing these algorithms can hasten and improve the accuracy of diagnosis and reduce further costs and burdens on the patients and health care system.

LEVEL OF EVIDENCE

Level III, case-control Machine Learning study.

Research hotspots and emerging trends of deep learning applications in orthopedics: A bibliometric and visualized study

Background

As a research hotspot, deep learning has been continuously combined with various research fields in medicine. Recently, there is a growing amount of deep learning-based researches in orthopedics. This bibliometric analysis aimed to identify the hotspots of deep learning applications in orthopedics in recent years and infer future research trends.

Methods

We screened global publication on deep learning applications in orthopedics by accessing the Web of Science Core Collection. The articles and reviews were collected without language and time restrictions. Citespace was applied to conduct the bibliometric analysis of the publications.

Results

A total of 822 articles and reviews were finally retrieved. The analysis showed that the application of deep learning in orthopedics has great prospects for development based on the annual publications. The most prolific country is the USA, followed by China. University of California San Francisco, and Skeletal Radiology are the most prolific institution and journal, respectively. LeCun Y is the most frequently cited author, and Nature has the highest impact factor in the cited journals. The current hot keywords are convolutional neural network, classification, segmentation, diagnosis, image, fracture, and osteoarthritis. The burst keywords are risk factor, identification, localization, and surgery. The timeline viewer showed two recent research directions for bone tumors and osteoporosis.

Conclusion

Publications on deep learning applications in orthopedics have increased in recent years, with the USA being the most prolific. The current research mainly focused on classifying, diagnosing and risk predicting in osteoarthritis and fractures from medical images. Future research directions may put emphasis on reducing intraoperative risk, predicting the occurrence of postoperative complications, screening for osteoporosis, and identification and classification of bone tumors from conventional imaging.

Automatic Detection of Medial and Lateral Compartments from Histological Sections of Mouse Knee Joints Using the Single-Shot Multibox Detector Algorithm

OBJECTIVE

Although mouse osteoarthritis (OA) models are widely used, their histological analysis may be susceptible to arbitrariness and inter-examiner variability in conventional methods. Therefore, a method for the unbiased scoring of OA histology is needed. In this study, as the first step for establishing this system, we developed a computer-vision algorithm that automatically detects the medial and lateral compartments of mouse knee sections in a rigorous and unbiased manner.

DESIGN

A total of 706 images of coronal sections of mouse knee joints stained by hematoxylin and eosin, safranin O, or toluidine blue were randomly divided into training and validation images at a ratio of 80:20. A model to detect both compartments automatically was built by machine learning using a single-shot multibox detector (SSD) algorithm with training images. The model was tested to determine whether it could accurately detect both compartments by analyzing the validation images and 52 images of sections stained with Picrosirius red, a method not used for the training images.

RESULTS

The trained model accurately detected both medial and lateral compartments of all 140 validation images regardless of the staining method employed, severity of articular cartilage defects, and the anatomical positions and conditions of the sections. Our model also correctly detected both compartments of 50 of 52 Picrosirius red-stained images.

CONCLUSIONS

By applying deep learning based on the SSD algorithm, we successfully developed a model that detects the locations of the medial and lateral compartments of tissue sections of mouse knee joints with high accuracy.

Detection and classification of mandibular fracture on CT scan using deep convolutional neural network

OBJECTIVES

This study aimed to evaluate the accuracy and reliability of convolutional neural networks (CNNs) for the detection and classification of mandibular fracture on spiral computed tomography (CT).

MATERIALS AND METHODS

Between January 2013 and July 2020, 686 patients with mandibular fractures who underwent CT scan were classified and annotated by three experienced maxillofacial surgeons serving as the ground truth. An algorithm including two convolutional neural networks (U-Net and ResNet) was trained, validated, and tested using 222, 56, and 408 CT scans, respectively. The diagnostic performance of the algorithm was compared with the ground truth and evaluated by DICE, accuracy, sensitivity, specificity, and area under the ROC curve (AUC).

RESULTS

One thousand five hundred six mandibular fractures in nine subregions of 686 patients were diagnosed. The DICE of mandible segmentation using U-Net was 0.943. The accuracies of nine subregions were all above 90%, with a mean AUC of 0.956.

CONCLUSIONS

CNNs showed comparable reliability and accuracy in detecting and classifying mandibular fractures on CT.

CLINICAL RELEVANCE

The algorithm for automatic detection and classification of mandibular fractures will help improve diagnostic efficiency and provide expertise to areas with lower medical levels.

CNN Attention Guidance for Improved Orthopedics Radiographic Fracture Classification

Convolutional neural networks (CNNs) have gained significant popularity in orthopedic imaging in recent years due to their ability to solve fracture classification problems. A common criticism of CNNs is their opaque learning and reasoning process, making it difficult to trust machine diagnosis and the subsequent adoption of such algorithms in clinical setting. This is especially true when the CNN is trained with limited amount of medical data, which is a common issue as curating sufficiently large amount of annotated medical image data is a long and costly process. While interest has been devoted to explaining CNN learnt knowledge by visualizing network attention, the utilization of the visualized attention to improve network learning has been rarely investigated. This paper explores the effectiveness of regularizing CNN network with human-provided attention guidance on where in the image the network should look for answering clues. On two orthopedics radiographic fracture classification datasets, through extensive experiments we demonstrate that explicit human-guided attention indeed can direct correct network attention and consequently significantly improve classification performance. The development code for the proposed attention guidance is publicly available on GitHub.

Musculoskeletal trauma and artificial intelligence: current trends and projections

Musculoskeletal trauma accounts for a significant fraction of emergency department visits and patients seeking urgent care, with a high financial cost to society. Diagnostic imaging is indispensable in the workup and management of trauma patients. However, diagnostic imaging represents a complex multifaceted system, with many aspects of its workflow prone to inefficiencies or human error. Recent technological innovations in artificial intelligence and machine learning have shown promise to revolutionize our systems for providing medical care to patients. This review will provide a general overview of the current state of artificial intelligence and machine learning applications in different aspects of trauma imaging and provide a vision for how such applications could be leveraged to enhance our diagnostic imaging systems and optimize patient outcomes.

Real-world analysis of artificial intelligence in musculoskeletal trauma

Musculoskeletal trauma accounts for a large percentage of emergency room visits and is amongst the top causes of unscheduled patient visits to the emergency room. Musculoskeletal trauma results in expenditure of billions of dollars and protracted losses of quality-adjusted life years. New and innovative methods are needed to minimise the impact by ensuring quick and accurate assessment. However, each of the currently utilised radiological procedures, such as radiography, ultrasonography, computed tomography, and magnetic resonance imaging, has resulted in implosion of medical imaging data. Deep learning, a recent advancement in artificial intelligence, has demonstrated the potential to analyse medical images with sensitivity and specificity at par with experts. In this review article, we intend to summarise and showcase the various developments which have occurred in the dynamic field of artificial intelligence and machine learning and how their applicability to different aspects of imaging in trauma can be explored to improvise our existing reporting systems and improvise on patient outcomes.

Automating classification of osteoarthritis according to Kellgren-Lawrence in the knee using deep learning in an unfiltered adult population

Background

Prevalence for knee osteoarthritis is rising in both Sweden and globally due to increased age and obesity in the population. This has subsequently led to an increasing demand for knee arthroplasties. Correct diagnosis and classification of a knee osteoarthritis (OA) are therefore of a great interest in following-up and planning for either conservative or operative management. Most orthopedic surgeons rely on standard weight bearing radiographs of the knee. Improving the reliability and reproducibility of these interpretations could thus be hugely beneficial. Recently, deep learning which is a form of artificial intelligence (AI), has been showing promising results in interpreting radiographic images. In this study, we aim to evaluate how well an AI can classify the severity of knee OA, using entire image series and not excluding common visual disturbances such as an implant, cast and non-degenerative pathologies.

Methods

We selected 6103 radiographic exams of the knee taken at Danderyd University Hospital between the years 2002-2016 and manually categorized them according to the Kellgren & Lawrence grading scale (KL). We then trained a convolutional neural network (CNN) of ResNet architecture using PyTorch. We evaluated the results against a test set of 300 exams that had been reviewed independently by two senior orthopedic surgeons who settled eventual interobserver disagreements through consensus sessions.

Results

The CNN yielded an overall AUC of more than 0.87 for all KL grades except KL grade 2, which yielded an AUC of 0.8 and a mean AUC of 0.92. When merging adjacent KL grades, all but one group showed near perfect results with AUC > 0.95 indicating excellent performance.

Conclusion

We have found that we could teach a CNN to correctly diagnose and classify the severity of knee OA using the KL grading system without cleaning the input data from major visual disturbances such as implants and other pathologies.

Presenting artificial intelligence, deep learning, and machine learning studies to clinicians and healthcare stakeholders: an introductory reference with a guideline and a Clinical AI Research (CAIR) checklist proposal

Background and purpose - Artificial intelligence (AI), deep learning (DL), and machine learning (ML) have become common research fields in orthopedics and medicine in general. Engineers perform much of the work. While they gear the results towards healthcare professionals, the difference in competencies and goals creates challenges for collaboration and knowledge exchange. We aim to provide clinicians with a context and understanding of AI research by facilitating communication between creators, researchers, clinicians, and readers of medical AI and ML research.Methods and results - We present the common tasks, considerations, and pitfalls (both methodological and ethical) that clinicians will encounter in AI research. We discuss the following topics: labeling, missing data, training, testing, and overfitting. Common performance and outcome measures for various AI and ML tasks are presented, including accuracy, precision, recall, F1 score, Dice score, the area under the curve, and ROC curves. We also discuss ethical considerations in terms of privacy, fairness, autonomy, safety, responsibility, and liability regarding data collecting or sharing.Interpretation - We have developed guidelines for reporting medical AI research to clinicians in the run-up to a broader consensus process. The proposed guidelines consist of a Clinical Artificial Intelligence Research (CAIR) checklist and specific performance metrics guidelines to present and evaluate research using AI components. Researchers, engineers, clinicians, and other stakeholders can use these proposal guidelines and the CAIR checklist to read, present, and evaluate AI research geared towards a healthcare setting.

Artificial intelligence for the classification of fractures around the knee in adults according to the 2018 AO/OTA classification system

BACKGROUND

Fractures around the knee joint are inherently complex in terms of treatment; complication rates are high, and they are difficult to diagnose on a plain radiograph. An automated way of classifying radiographic images could improve diagnostic accuracy and would enable production of uniformly classified records of fractures to be used in researching treatment strategies for different fracture types. Recently deep learning, a form of artificial intelligence (AI), has shown promising results for interpreting radiographs. In this study, we aim to evaluate how well an AI can classify knee fractures according to the detailed 2018 AO-OTA fracture classification system.

METHODS

We selected 6003 radiograph exams taken at Danderyd University Hospital between the years 2002-2016, and manually categorized them according to the AO/OTA classification system and by custom classifiers. We then trained a ResNet-based neural network on this data. We evaluated the performance against a test set of 600 exams. Two senior orthopedic surgeons had reviewed these exams independently where we settled exams with disagreement through a consensus session.

RESULTS

We captured a total of 49 nested fracture classes. Weighted mean AUC was 0.87 for proximal tibia fractures, 0.89 for patella fractures and 0.89 for distal femur fractures. Almost ¾ of AUC estimates were above 0.8, out of which more than half reached an AUC of 0.9 or above indicating excellent performance.

CONCLUSION

Our study shows that neural networks can be used not only for fracture identification but also for more detailed classification of fractures around the knee joint.

Artificial Intelligence in Fractured Dental Implant Detection and Classification: Evaluation Using Dataset from Two Dental Hospitals

Fracture of a dental implant (DI) is a rare mechanical complication that is a critical cause of DI failure and explantation. The purpose of this study was to evaluate the reliability and validity of a three different deep convolutional neural network (DCNN) architectures (VGGNet-19, GoogLeNet Inception-v3, and automated DCNN) for the detection and classification of fractured DI using panoramic and periapical radiographic images. A total of 21,398 DIs were reviewed at two dental hospitals, and 251 intact and 194 fractured DI radiographic images were identified and included as the dataset in this study. All three DCNN architectures achieved a fractured DI detection and classification accuracy of over 0.80 AUC. In particular, automated DCNN architecture using periapical images showed the highest and most reliable detection (AUC = 0.984, 95% CI = 0.900–1.000) and classification (AUC = 0.869, 95% CI = 0.778–0.929) accuracy performance compared to fine-tuned and pre-trained VGGNet-19 and GoogLeNet Inception-v3 architectures. The three DCNN architectures showed acceptable accuracy in the detection and classification of fractured DIs, with the best accuracy performance achieved by the automated DCNN architecture using only periapical images.