Machine Learning on Medicare Claims Poorly Predicts the Individual Risk of 30-Day Unplanned Readmission After Total Joint Arthroplasty, Yet Uncovers Interesting Population-level Associations With Annual Procedure Volumes

Utility of Machine Learning, Natural Language Processing, and Artificial Intelligence in Predicting Hospital Readmissions After Orthopaedic Surgery: A Systematic Review and Meta-Analysis

BACKGROUND

Numerous applications and strategies have been utilized to help assess the trends and patterns of readmissions after orthopaedic surgery in an attempt to extrapolate possible risk factors and causative agents. The aim of this work is to systematically summarize the available literature on the extent to which natural language processing, machine learning, and artificial intelligence (AI) can help improve the predictability of hospital readmissions after orthopaedic and spine surgeries.

METHODS

This is a systematic review and meta-analysis. PubMed, Embase and Google Scholar were searched, up until August 30, 2023, for studies that explore the use of AI, natural language processing, and machine learning tools for the prediction of readmission rates after orthopedic procedures. Data regarding surgery type, patient population, readmission outcomes, advanced models utilized, comparison methods, predictor sets, the inclusion of perioperative predictors, validation method, size of training and testing sample, accuracy, and receiver operating characteristics (C-statistic), among other factors, were extracted and assessed.

RESULTS

A total of 26 studies were included in our final dataset. The overall summary C-statistic showed a mean of 0.71 across all models, indicating a reasonable level of predictiveness. A total of 15 articles (57%) were attributed to the spine, making it the most commonly explored orthopaedic field in our study. When comparing accuracy of prediction models between different fields, models predicting readmissions after hip/knee arthroplasty procedures had a higher prediction accuracy (mean C-statistic = 0.79) than spine (mean C-statistic = 0.7) and shoulder (mean C-statistic = 0.67). In addition, models that used single institution data, and those that included intraoperative and/or postoperative outcomes, had a higher mean C-statistic than those utilizing other data sources, and that include only preoperative predictors. According to the Prediction model Risk of Bias Assessment Tool, the majority of the articles in our study had a high risk of bias.

CONCLUSION

AI tools perform reasonably well in predicting readmissions after orthopaedic procedures. Future work should focus on standardizing study methodologies and designs, and improving the data analysis process, in an attempt to produce more reliable and tangible results.

LEVEL OF EVIDENCE

Level III. See Instructions for Authors for a complete description of levels of evidence.

Postoperative Communication Volume Following Total Joint Arthroplasty Can Be a Precursor for Emergency Department Visits

Background

Unplanned calls, messages, and visits to the clinic can occur at a higher rate as newer technologies allow patients more accessibility and connectivity to clinicians. By reviewing postoperative patient phone calls and electronic portal messages, we compared the methods and frequency of communications between conventional and robotic joint arthroplasty cases.

Methods

A retrospective review of total hip, total knee, and unicompartmental knee arthroplasty procedures by fellowship-trained adult reconstruction surgeons at our hospitals between 2017 and 2022 was performed. Any unplanned postoperative communication within 30 days of the postoperative period and unplanned emergency department visits were collected.

Results

There were 12,300 robotic and manual consecutive primary total hip, total knee, and unicompartmental knee arthroplasty procedures performed on 10,908 patients over the study period. A total of 905 (40.4%) patients and 2012 (23.2%) patients sent an electronic text message (ETM) in the robotic and manual arthroplasty cohorts (P < .0001), respectively. Overall, 1942 (86.6%) patients in the robotic arthroplasty group and 6417 (74%) patients in the manual arthroplasty group had at least one phone call within the first month after their joint arthroplasty.

Conclusions

Robotic arthroplasty patients place an increased demand on the orthopaedic surgery department in terms of unplanned patient contacts. Robotic arthroplasty patients had a significantly increased rate of unplanned postoperative ETMs and phone calls when compared to manual arthroplasty patients. An increased number of postoperative phone calls, but not ETMs, can also be indicative of an emergency department visit. These findings can be used in the perioperative setting to counsel and educate patients about expectations.

Medical Imaging Applications Developed Using Artificial Intelligence Demonstrate High Internal Validity Yet Are Limited in Scope and Lack External Validation

PURPOSE

To (1) review definitions and concepts necessary to interpret applications of deep learning (DL; a domain of artificial intelligence that leverages neural networks to make predictions on media inputs such as images) and (2) identify knowledge and translational gaps in the literature to provide insight into specific areas for improvement as adoption of this technology continues.

METHODS

A comprehensive search of the literature was performed in December 2023 for articles regarding the use of DL in sports medicine. For each study, information regarding the joint of focus, specific anatomic structure/pathology to which DL was applied, imaging modality utilized, source of images used for model training and testing, data set size, model performance, and whether the DL model was externally validated was recorded. A numerical scale was used to rate each DL model's clinical impact, with 1 corresponding to proof-of-concept studies with little to no direct clinical impact and 5 corresponding to practice-changing clinical impact and ready for clinical deployment.

RESULTS

Fifty-five studies were identified, all of which were published within the past 5 years, while 82% were published within the past 3 years. Of the DL models identified, 84% were developed for classification tasks, 9% for automated measurements, and 7% for segmentation. A total of 62% of studies utilized magnetic resonance imaging as the imaging modality, 25% radiographs, and 7% ultrasound, while 1 study each used computed tomography, arthroscopic images, or arthroscopic video. Sixty-five percent of studies focused on the detection of tears (anterior cruciate ligament [ACL], rotator cuff [RC], and meniscus). The diagnostic performance of ACL tears, as determined by the area under the receiver operator curve (AUROC), ranged from 0.81 to 0.99 for ACL tears (excellent to near perfect), 0.83 to 0.94 for RC tears (excellent), and from 0.75 to 0.96 for meniscus tears (acceptable to excellent). In addition, 3 studies focused on detection of cartilage lesions had AUROC ranging from 0.90 to 0.92 (excellent performance). However, only 4 (7%) studies externally validated their models, suggesting that they may not be generalizable or may not perform well when applied to populations other than that used to develop the model. Finally, the mean clinical impact score was 2 (range, 1-3) on scale of 1 to 5, corresponding to limited clinical applicability.

CONCLUSIONS

DL models in orthopaedic sports medicine show generally excellent performance (high internal validity) but require external validation to facilitate clinical deployment. In addition, current models have low clinical applicability and fail to advance the field due to a focus on routine tasks and a narrow conceptual framework.

LEVEL OF EVIDENCE

Level IV, scoping review of Level I to IV studies.