Ceiling Effect of the Combined Norwegian and Danish Knee Ligament Registers Limits Anterior Cruciate Ligament Reconstruction Outcome Prediction

AKIRA: Deep learning tool for image standardization, implant detection and arthritis grading to establish a radiographic registry in patients with anterior cruciate ligament injuries

PURPOSE

Developing large-scale, standardized radiographic registries for anterior cruciate ligament (ACL) injuries with artificial intelligence (AI) tools can enhance personalized orthopaedics. We propose deploying Artificial Intelligence for Knee Imaging Registration and Analysis (AKIRA), a trio of deep learning (DL) algorithms, to automatically classify and annotate radiographs. We hypothesize that algorithms can efficiently organize radiographs based on laterality, projection, identify implants and classify osteoarthritis (OA) grade.

METHODS

A collection of 20,836 knee radiographs from all time points of treatment (mean orthopaedic follow-up 70.7 months; interquartile range [IQR]: 6.8-172 months) were aggregated from 1628 ACL-injured patients (median age 26 years [IQR: 19-42], 57% male). Three DL algorithms (EfficientNet, YOLO [You Only Look Once] and Residual Network) were employed. Radiograph laterality and projection (anterior-posterior [AP], lateral, sunrise, posterior-anterior, hip-knee-ankle and Camp-Coventry intercondylar [notch]) were labelled by a DL model. Manually provided labels of metal fixation implants were used to develop a DL object detection algorithm. The degree of OA, both as measured by specific Kellgren-Lawrence (KL) grades, as well as based on a binarized label of OA (defined as KL Grade ≥2), on standing AP radiographs were classified using a DL algorithm. Individual model performances were evaluated on a subset of images prior to the deployment of AKIRA to registry construction using all ACL radiographs.

RESULTS

The classification algorithms showed excellent performance in classifying radiographic laterality (F1 score: 0.962-0.975) and projection (F1 score: 0.941-1.0). The object detection algorithm achieved high precision-recall (area under the precision-recall curve: 0.695-0.992) for identifying various metal fixations. The KL classifier reached concordances of 0.39-0.40, improving to 0.81-0.82 for binary OA labels. Sequential deployment of AKIRA following internal validation processed and labelled all 20,836 images with the appropriate views, implants, and the presence of OA within 88 min.

CONCLUSION

AKIRA effectively automated the classification and object detection in a large radiograph cohort of ACL injuries, creating an AI-enabled radiographic registry with comprehensive details on laterality, projection, implants and OA.

STUDY DESIGN

Cross-sectional study.

LEVEL OF EVIDENCE

Level IV.

Machine learning models predicting risk of revision or secondary knee injury after anterior cruciate ligament reconstruction demonstrate variable discriminatory and accuracy performance: a systematic review

BACKGROUND

To summarize the statistical performance of machine learning in predicting revision, secondary knee injury, or reoperations following anterior cruciate ligament reconstruction (ACLR), and to provide a general overview of the statistical performance of these models.

METHODS

Three online databases (PubMed, MEDLINE, EMBASE) were searched from database inception to February 6, 2024, to identify literature on the use of machine learning to predict revision, secondary knee injury (e.g. anterior cruciate ligament (ACL) or meniscus), or reoperation in ACLR. The authors adhered to the PRISMA and R-AMSTAR guidelines as well as the Cochrane Handbook for Systematic Reviews of Interventions. Demographic data and machine learning specifics were recorded. Model performance was recorded using discrimination, area under the curve (AUC), concordance, calibration, and Brier score. Factors deemed predictive for revision, secondary injury or reoperation were also extracted. The MINORS criteria were used for methodological quality assessment.

RESULTS

Nine studies comprising 125,427 patients with a mean follow-up of 5.82 (0.08-12.3) years were included in this review. Two of nine (22.2%) studies served as external validation analyses. Five (55.6%) studies reported on mean AUC (strongest model range 0.77-0.997). Four (44.4%) studies reported mean concordance (strongest model range: 0.67-0.713). Two studies reported on Brier score, calibration intercept, and calibration slope, with values ranging from 0.10 to 0.18, 0.0051-0.006, and 0.96-0.97 amongst highest performing models, respectively. Four studies reported calibration error, with all four studies demonstrating significant miscalibration at either two or five-year follow-ups amongst 10 of 14 models assessed.

CONCLUSION

Machine learning models designed to predict the risk of revision or secondary knee injury demonstrate variable discriminatory performance when evaluated with AUC or concordance metrics. Furthermore, there is variable calibration, with several models demonstrating evidence of miscalibration at two or five-year marks. The lack of external validation of existing models limits the generalizability of these findings. Future research should focus on validating current models in addition to developing new multimodal neural networks to improve accuracy and reliability.

A practical guide to the implementation of AI in orthopaedic research, Part 6: How to evaluate the performance of AI research?

Artificial intelligence's (AI) accelerating progress demands rigorous evaluation standards to ensure safe, effective integration into healthcare's high-stakes decisions. As AI increasingly enables prediction, analysis and judgement capabilities relevant to medicine, proper evaluation and interpretation are indispensable. Erroneous AI could endanger patients; thus, developing, validating and deploying medical AI demands adhering to strict, transparent standards centred on safety, ethics and responsible oversight. Core considerations include assessing performance on diverse real-world data, collaborating with domain experts, confirming model reliability and limitations, and advancing interpretability. Thoughtful selection of evaluation metrics suited to the clinical context along with testing on diverse data sets representing different populations improves generalisability. Partnering software engineers, data scientists and medical practitioners ground assessment in real needs. Journals must uphold reporting standards matching AI's societal impacts. With rigorous, holistic evaluation frameworks, AI can progress towards expanding healthcare access and quality.

Level of Evidence

Level V.

Return to the Pre-Injury Level of Sport after Anterior Cruciate Ligament Reconstruction: A Practical Review with Medical Recommendations

Returning to sport after anterior cruciate ligament reconstruction (ACLR) can be a challenging and complex process for the athlete, with the rate of return to the pre-injury level of sport observed to be less than athlete expectations. Of the athletes that do return to sport (RTS), knee re-injury rates remain high, and multiple studies have observed impaired athletic performance upon RTS after ACLR as well as reduced playing time, productivity, and career lengths. To mitigate re-injury and improve RTS outcomes, multiple RTS after ACLR consensus statements/clinical practice guidelines have recommended objective RTS testing criteria to be met prior to medical clearance for unrestricted sports participation. While the achievement of RTS testing criteria can improve RTS rates after ACLR, current criteria do not appear valid for predicting safe RTS. Therefore, there is a need to review the various factors related to the successful return to the pre-injury level of sport after ACLR, clarify the utility of objective performance testing and RTS criteria, further discuss safe RTS decision-making as well as present strategies to reduce the risk of ACL injury/re-injury upon RTS. This article provides a practical review of the current RTS after ACLR literature, as well as makes medical recommendations for rehabilitation and RTS decision-making after ACLR.

Unsupervised Machine Learning of the Combined Danish and Norwegian Knee Ligament Registers: Identification of 5 Distinct Patient Groups With Differing ACL Revision Rates

BACKGROUND

Most clinical machine learning applications use a supervised learning approach using labeled variables. In contrast, unsupervised learning enables pattern detection without a prespecified outcome.

PURPOSE/HYPOTHESIS

The purpose of this study was to apply unsupervised learning to the combined Danish and Norwegian knee ligament register (KLR) with the goal of detecting distinct subgroups. It was hypothesized that resulting groups would have differing rates of subsequent anterior cruciate ligament reconstruction (ACLR) revision.

STUDY DESIGN

Cohort study; Level of evidence, 3.

METHODS

K-prototypes clustering was performed on the complete case KLR data. After performing the unsupervised learning analysis, the authors defined clinically relevant characteristics of each cluster using variable summaries, surgeons' domain knowledge, and Shapley Additive exPlanations analysis.

RESULTS

Five clusters were identified. Cluster 1 (revision rate, 9.9%) patients were young (mean age, 22 years; SD, 6 years), received hamstring tendon (HT) autograft (91%), and had lower baseline Knee injury and Osteoarthritis Outcome Score (KOOS) Sport and Recreation (Sports) scores (mean, 25.0; SD, 15.6). Cluster 2 (revision rate, 6.9%) patients received HT autograft (89%) and had higher baseline KOOS Sports scores (mean, 67.2; SD, 16.5). Cluster 3 (revision rate, 4.7%) patients received bone-patellar tendon-bone (BPTB) or quadriceps tendon (QT) autograft (94%) and had higher baseline KOOS Sports scores (mean, 65.8; SD, 16.4). Cluster 4 (revision rate, 4.1%) patients received BPTB or QT autograft (88%) and had low baseline KOOS Sports scores (mean, 20.5; SD, 14.0). Cluster 5 (revision rate, 3.1%) patients were older (mean age, 42 years; SD, 7 years), received HT autograft (89%), and had low baseline KOOS Sports scores (mean, 23.4; SD, 17.6).

CONCLUSION

Unsupervised learning identified 5 distinct KLR patient subgroups and each grouping was associated with a unique ACLR revision rate. Patients can be approximately classified into 1 of the 5 clusters based on only 3 variables: age, graft choice (HT, BPTB, or QT autograft), and preoperative KOOS Sports subscale score. If externally validated, the resulting groupings may enable quick risk stratification for future patients undergoing ACLR in the clinical setting. Patients in cluster 1 are considered high risk (9.9%), cluster 2 patients medium risk (6.9%), and patients in clusters 3 to 5 low risk (3.1%-4.7%) for revision ACLR.

The Role of Artificial Intelligence in Anterior Cruciate Ligament Injuries: Current Concepts and Future Perspectives

The remarkable progress in data aggregation and deep learning algorithms has positioned artificial intelligence (AI) and machine learning (ML) to revolutionize the field of medicine. AI is becoming more and more prevalent in the healthcare sector, and its impact on orthopedic surgery is already evident in several fields. This review aims to examine the literature that explores the comprehensive clinical relevance of AI-based tools utilized before, during, and after anterior cruciate ligament (ACL) reconstruction. The review focuses on current clinical applications and future prospects in preoperative management, encompassing risk prediction and diagnostics; intraoperative tools, specifically navigation, identifying complex anatomic landmarks during surgery; and postoperative applications in terms of postoperative care and rehabilitation. Additionally, AI tools in educational and training settings are presented. Orthopedic surgeons are showing a growing interest in AI, as evidenced by the applications discussed in this review, particularly those related to ACL injury. The exponential increase in studies on AI tools applicable to the management of ACL tears promises a significant future impact in its clinical application, with growing attention from orthopedic surgeons.

Application of Machine Learning Algorithms for Prognostic Assessment in Rotator Cuff Pathologies: A Clinical Data-Based Approach

AIM

The overall aim of this proposal is to ameliorate the care of rotator cuff (RC) tear patients by applying an innovative machine learning approach for outcome prediction after arthroscopic repair.

MATERIALS AND METHODS

We applied state-of-the-art machine learning algorithms to evaluate the best predictors of the outcome, and 100 RC patients were evaluated at baseline (T0), after 1 month (T1), 3 months (T2), 6 months (T3), and 1 year (T4) from surgical intervention. The outcome measure was the Costant-Murley Shoulder Score, whereas age, sex, BMI, the 36-Item Short-Form Survey, the Simple Shoulder Test, the Hospital Anxiety and Depression Scale, the American Shoulder and Elbow Surgeons Score, the Oxford Shoulder Score, and the Shoulder Pain and Disability Index were considered as predictive factors. Support vector machine (SVM), k-nearest neighbors (k-NN), naïve Bayes (NB), and random forest (RF) algorithms were employed.

RESULTS

Across all sessions, the classifiers demonstrated suboptimal performance when using both the complete and shrunken sets of features. Specifically, the logistic regression (LR) classifier achieved a mean accuracy of 46.5% ± 6%, while the random forest (RF) classifier achieved 51.25% ± 4%. For the shrunken set of features, LR obtained a mean accuracy of 48.5% ± 6%, and RF achieved 45.5% ± 4.5%. No statistical differences were found when comparing the performance metrics of ML algorithms.

CONCLUSIONS

This study underlines the importance of extending the application of AI methods to new predictors, such as neuroimaging and kinematic data, in order to better record significant shifts in RC patients' prognosis.

LIMITATIONS

The data quality within the cohort could represent a limitation, since certain variables, such as smoking, diabetes, and work injury, are known to have an impact on the outcome.