Efficacy and Applications of Artificial Intelligence and Machine Learning Analyses in Total Joint Arthroplasty: A Call for Improved Reporting

Machine learning model identifies patient gait speed throughout the episode of care, generating notifications for clinician evaluation

INTRODUCTION

The advent of digital and mobile health innovations, especially use of wearables for passive data collection, allows remote monitoring and creates an abundance of data. For this information to be interpretable, machine learning (ML) processes are necessary.

RESEARCH QUESTION

Can a machine learning model successfully identify patients expected to have low gait speed in the early recovery period following joint replacement surgery?

METHODS

A commercial database from a smartphone-based care management platform passively collecting mobility data pre- and post-lower limb arthroplasty was used. We sought to create a ML model to predict gait speed recovery curves and identify patients at risk of poor gait speed outcome, a measure associated with range of motion and patient-reported outcomes. Model performance including sensitivity, specificity, precision, and accuracy were determined. Receiver operator curve (ROC) analysis was used to compare true and false positive rates. To benchmark our model, we compared threshold-based notifications based on the patient's current gait speed.

RESULTS

The performance of the predictive model was significantly improved compared to baseline of threshold-based exceptions using current gait speed. The ML model currently provides 53 % precision, 88 % accuracy, 36 % sensitivity, and 95 % specificity on the held-out test set. The ROC analysis suggests good clinical performance (AUC=0.81).

SIGNIFICANCE

Utilization of ML to predict gait recovery following total joint replacement is feasible and provides results with excellent specificity. This model will allow inclusion of additional data for retraining as patient populations evolve. Clinician feedback regarding notifications, including resulting actions and outcomes, can be used to further inform the model and improve clinical utility.

Assessing Ability for ChatGPT to Answer Total Knee Arthroplasty-Related Questions

BACKGROUND

Artificial intelligence in the field of orthopaedics has been a topic of increasing interest and opportunity in recent years. Its applications are widespread both for physicians and patients, including use in clinical decision-making, in the operating room, and in research. In this study, we aimed to assess the quality of ChatGPT answers when asked questions related to total knee arthroplasty.

METHODS

ChatGPT prompts were created by turning 15 of the American Academy of Orthopaedic Surgeons Clinical Practice Guidelines into questions. An online survey was created, which included screenshots of each prompt and answers to the 15 questions. Surgeons were asked to grade ChatGPT answers from 1 to 5 based on their characteristics: (1) relevance, (2) accuracy, (3) clarity, (4) completeness, (5) evidence-based, and (6) consistency. There were 11 Adult Joint Reconstruction fellowship-trained surgeons who completed the survey. Questions were subclassified based on the subject of the prompt: (1) risk factors, (2) implant/intraoperative, and (3) pain/functional outcomes. The average and standard deviation for all answers, as well as for each subgroup, were calculated. Inter-rater reliability (IRR) was also calculated.

RESULTS

All answer characteristics were graded as being above average (ie, a score > 3). Relevance demonstrated the highest scores (4.43 ± 0.77) by surgeons surveyed, and consistency demonstrated the lowest scores (3.54 ± 1.10). ChatGPT prompts in the Risk Factors group demonstrated the best responses, while those in the Pain/Functional Outcome group demonstrated the lowest. The overall IRR was found to be 0.33 (poor reliability), with the highest IRR for relevance (0.43) and the lowest for evidence-based (0.28).

CONCLUSIONS

ChatGPT can answer questions regarding well-established clinical guidelines in total knee arthroplasty with above-average accuracy but demonstrates variable reliability. This investigation is the first step in understanding large language model artificial intelligence like ChatGPT and how well they perform in the field of arthroplasty.

Artificial Intelligence Machine Learning Algorithms Versus Standard Linear Demographic Analysis in Predicting Implant Size of Anatomic and Reverse Total Shoulder Arthroplasty

BACKGROUND

Accurate and precise templating is paramount for anatomic total shoulder arthroplasty (TSA) and reverse total shoulder arthroplasty (RSA) to enhance preoperative planning, streamline surgery, and improve implant positioning. We aimed to evaluate the predictive potential of readily available patient demographic data in TSA and RSA implant sizing, independent of implant design.

METHODS

A total of 578 consecutive, primary, noncemented shoulder arthroplasty cases were retrospectively reviewed. Demographic variables and implant characteristics were recorded. Multivariate linear regressions were conducted to predict implant sizes using patient demographic variables.

RESULTS

Linear models accurately predict TSA implant sizes within 2 millimeters of humerus stem sizes 75.3% of the time, head diameter 82.1%, head height 82.1%, and RSA glenosphere diameter 77.6% of the time. Linear models predict glenoid implant sizes accurately 68.2% and polyethylene thickness 76.6% of the time and within one size 100% and 95.7% of the time, respectively.

CONCLUSION

Linear models accurately predict shoulder arthroplasty implant sizes from demographic data. No significant statistical differences were observed between linear models and machine learning algorithms, although the analysis was underpowered. Future sufficiently powered studies are required for more robust assessment of machine learning models in predicting primary shoulder arthroplasty implant sizes based on patient demographics.

Enabling Personalized Medicine in Orthopaedic Surgery Through Artificial Intelligence: A Critical Analysis Review

» The application of artificial intelligence (AI) in the field of orthopaedic surgery holds potential for revolutionizing health care delivery across 3 crucial domains: (I) personalized prediction of clinical outcomes and adverse events, which may optimize patient selection, surgical planning, and enhance patient safety and outcomes; (II) diagnostic automated and semiautomated imaging analyses, which may reduce time burden and facilitate precise and timely diagnoses; and (III) forecasting of resource utilization, which may reduce health care costs and increase value for patients and institutions.» Computer vision is one of the most highly studied areas of AI within orthopaedics, with applications pertaining to fracture classification, identification of the manufacturer and model of prosthetic implants, and surveillance of prosthesis loosening and failure.» Prognostic applications of AI within orthopaedics include identifying patients who will likely benefit from a specified treatment, predicting prosthetic implant size, postoperative length of stay, discharge disposition, and surgical complications. Not only may these applications be beneficial to patients but also to institutions and payors because they may inform potential cost expenditure, improve overall hospital efficiency, and help anticipate resource utilization.» AI infrastructure development requires institutional financial commitment and a team of clinicians and data scientists with expertise in AI that can complement skill sets and knowledge. Once a team is established and a goal is determined, teams (1) obtain, curate, and label data; (2) establish a reference standard; (3) develop an AI model; (4) evaluate the performance of the AI model; (5) externally validate the model, and (6) reinforce, improve, and evaluate the model's performance until clinical implementation is possible.» Understanding the implications of AI in orthopaedics may eventually lead to wide-ranging improvements in patient care. However, AI, while holding tremendous promise, is not without methodological and ethical limitations that are essential to address. First, it is important to ensure external validity of programs before their use in a clinical setting. Investigators should maintain high quality data records and registry surveillance, exercise caution when evaluating others' reported AI applications, and increase transparency of the methodological conduct of current models to improve external validity and avoid propagating bias. By addressing these challenges and responsibly embracing the potential of AI, the medical field may eventually be able to harness its power to improve patient care and outcomes.

Using Unsupervised Machine Learning to Predict Quality of Life After Total Knee Arthroplasty

BACKGROUND

Patient-reported outcome measures (PROMs) are an important metric to assess total knee arthroplasty (TKA) patients. The purpose of this study was to use a machine learning (ML) algorithm to identify patient features that impact PROMs after TKA.

METHODS

Data from 636 TKA patients enrolled in our patient database between 2018 and 2022, were retrospectively reviewed. Their mean age was 68 years (range, 39 to 92), 56.7% women, and mean body mass index of 31.17 (range, 16 to 58). Patient demographics and the Functional Comorbidity Index were collected alongside Patient-Reported Outcome Measures Information System Global Health v1.2 (PROMIS GH-P) physical component scores preoperatively, at 3 months, and 1 year after TKA. An unsupervised ML algorithm (spectral clustering) was used to identify patient features impacting PROMIS GH-P scores at the various time points.

RESULTS

The algorithm identified 5 patient clusters that varied by demographics, comorbidities, and pain scores. Each cluster was associated with predictable trends in PROMIS GH-P scores across the time points. Notably, patients who had the worst preoperative PROMIS GH-P scores (cluster 5) had the most improvement after TKA, whereas patients who had higher global health rating preoperatively had more modest improvement (clusters 1, 2, and 3). Two out of Five patient clusters (cluster 4 and 5) showed improvement in PROMIS GH-P scores that met a minimally clinically important difference at 1-year postoperative.

CONCLUSIONS

The unsupervised ML algorithm identified patient clusters that had predictable changes in PROMs after TKA. It is a positive step toward providing precision medical care for each of our arthroplasty patients.

The Use of Artificial Intelligence for the Prediction of Periprosthetic Joint Infection Following Aseptic Revision Total Knee Arthroplasty

Periprosthetic joint infection (PJI) following revision total knee arthroplasty (TKA) for aseptic failure is associated with poor outcomes, patient morbidity, and high health care expenditures. The aim of this study was to develop novel machine learning algorithms for the prediction of PJI following revision TKA for patients with aseptic indications for revision surgery. A single-institution database consisting of 1,432 consecutive revision TKA patients with aseptic etiologies was retrospectively identified. The patient cohort included 208 patients (14.5%) who underwent re-revision surgery for PJI. Three machine learning algorithms (artificial neural networks, support vector machines, k-nearest neighbors) were developed to predict this outcome and these models were assessed by discrimination, calibration, and decision curve analysis. This is a retrospective study. Among the three machine learning models, the neural network model achieved the best performance across discrimination (area under the receiver operating characteristic curve = 0.78), calibration, and decision curve analysis. The strongest predictors for PJI following revision TKA for aseptic reasons were prior open procedure prior to revision surgery, drug abuse, obesity, and diabetes. This study utilized machine learning as a tool for the prediction of PJI following revision TKA for aseptic failure with excellent performance. The validated machine learning models can aid surgeons in patient-specific risk stratifying to assist in preoperative counseling and clinical decision making for patients undergoing aseptic revision TKA.

The Impact of Machine Learning on Total Joint Arthroplasty Patient Outcomes: A Systemic Review

BACKGROUND

Supervised machine learning techniques have been increasingly applied to predict patient outcomes after hip and knee arthroplasty procedures. The purpose of this study was to systematically review the applications of supervised machine learning techniques to predict patient outcomes after primary total hip and knee arthroplasty.

METHODS

A comprehensive literature search using the electronic databases MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials, and Cochrane Database of Systematic Reviews was conducted in July of 2021. The inclusion criteria were studies that utilized supervised machine learning techniques to predict patient outcomes after primary total hip or knee arthroplasty.

RESULTS

Search criteria yielded n = 30 relevant studies. Topics of study included patient complications (n = 6), readmissions (n = 1), revision (n = 2), patient-reported outcome measures (n = 4), patient satisfaction (n = 4), inpatient status and length of stay (LOS) (n = 9), opioid usage (n = 3), and patient function (n = 1). Studies involved TKA (n = 12), THA (n = 11), or a combination (n = 7). Less than 35% of predictive outcomes had an area under the receiver operating characteristic curve (AUC) in the excellent or outstanding range. Additionally, only 9 of the studies found improvement over logistic regression, and only 9 studies were externally validated.

CONCLUSION

Supervised machine learning algorithms are powerful tools that have been increasingly applied to predict patient outcomes after total hip and knee arthroplasty. However, these algorithms should be evaluated in the context of prognostic accuracy, comparison to traditional statistical techniques for outcome prediction, and application to populations outside the training set. While machine learning algorithms have been received with considerable interest, they should be critically assessed and validated prior to clinical adoption.

Identifying Trends and Quantifying Growth for Technological Innovation in Knee Arthroplasty: An Analysis of a Patent Database (1990 to 2020)

Technological innovation is the key for surgical progress in knee arthroplasty and improvement in patient outcomes. Exploring patented technologies can help elucidate trends and growth for numerous innovative technologies. However, patent databases, which contain millions of patents, remain underused in arthroplasty research. Therefore, the present study aimed to: (1) quantify patent activity; (2) group patents related to similar technologies into well-defined clusters; and (3) compare growth between technologies in the field of knee arthroplasty over a 30-year period. An open-source international patent database was queried from January 1990 to January 2020 for all patents related to knee arthroplasty A search strategy identified 70,154 patents, of which 24,425 were unique and included analysis. Patents were grouped into 14 independent technology clusters using Cooperative Patent Classification (CPC) codes. Patent activity was normalized via a validated formula adjusting for exponential growth. Compound annual growth rates (CAGR) were calculated (5-year, 10-year, and 30-year CAGR) and compared for each cluster. Overall yearly patent activity increased by 2,023%, from 104 patents in 1990 to 2,208 patents in 2020. The largest technology clusters were "drugs" (n = 5,347; 23.8%), "components" (n = 4,343; 19.0%), "instruments" (n = 3,130; 13.7%), and "materials" (n = 2,378; 10.4%). The fastest growing technologies with their 5-year CAGR were: "user interfaces for surgical systems" (58.1%); "robotics" (28.6%); "modularity" (21.1%); "navigation" (15.7%); and "computer modeling" (12.5%). Since 1990, overall patent growth rate has been greatest for "computer modeling" (8.4%), "robotics" (8.0%), "navigation" (7.9%), and "patient-specific instrumentation" (6.4%). Most patents in knee arthroplasty for the last 30 years have focused on drugs, components, instruments, and materials. Recent exponential growth was mainly observed for user interfaces for surgical systems, robotics, modularity, navigation, and computer-assisted technologies. Innovation theory would suggest that these rapidly growing technologies are experiencing high innovation output, increased resource investments, growing adoption by providers, and significant clinical impact. Periodic monitoring of technological innovation via patent databases can be useful to establish trends and future directions in the field of knee arthroplasty.

Artificial intelligence for automated identification of total shoulder arthroplasty implants

BACKGROUND

Accurate and rapid identification of implant manufacturer and model is critical in the evaluation and management of patients requiring revision total shoulder arthroplasty (TSA). Failure to correctly identify implant designs in these circumstances may lead to delay in care, unexpected intraoperative challenges, increased morbidity, and excess health care costs. Deep learning (DL) permits automated image processing and holds the potential to mitigate such challenges while improving the value of care rendered. The purpose of this study was to develop an automated DL algorithm to identify shoulder arthroplasty implants from plain radiographs.

METHODS

A total of 3060 postoperative images from patients who underwent TSA between 2011 and 2021 performed by 26 fellowship-trained surgeons at 2 independent tertiary academic hospitals in the Pacific Northwest and Mid-Atlantic Northeast were included. A DL algorithm was trained using transfer learning and data augmentation to classify 22 different reverse TSA and anatomic TSA prostheses from 8 implant manufacturers. Images were split into training and testing cohorts (2448 training and 612 testing images). Optimized model performance was assessed using standardized metrics including the multiclass area under the receiver operating characteristic curve (AUROC) and compared with a reference standard of implant data from operative reports.

RESULTS

The algorithm classified implants at a mean speed of 0.079 seconds (±0.002 seconds) per image. The optimized model discriminated between 8 manufacturers (22 unique implants) with AUROCs of 0.994-1.000, accuracy of 97.1%, and sensitivities between 0.80 and 1.00 on the independent testing set. In the subset of single-institution implant predictions, a DL model identified 6 specific implants with AUROCs of 0.999-1.000, accuracy of 99.4%, and sensitivity >0.97 for all implants. Saliency maps revealed key differentiating features across implant manufacturers and designs recognized by the algorithm for classification.

CONCLUSION

A DL model demonstrated excellent accuracy in identifying 22 unique TSA implants from 8 manufacturers. This algorithm may provide a clinically meaningful adjunct in assisting with preoperative planning for the failed TSA and allows for scalable expansion with additional radiographic data and validation efforts.

Concerns surrounding application of artificial intelligence in hip and knee arthroplasty

Literature surrounding artificial intelligence (AI)-related applications for hip and knee arthroplasty has proliferated. However, meaningful advances that fundamentally transform the practice and delivery of joint arthroplasty are yet to be realized, despite the broad range of applications as we continue to search for meaningful and appropriate use of AI. AI literature in hip and knee arthroplasty between 2018 and 2021 regarding image-based analyses, value-based care, remote patient monitoring, and augmented reality was reviewed. Concerns surrounding meaningful use and appropriate methodological approaches of AI in joint arthroplasty research are summarized. Of the 233 AI-related orthopaedics articles published, 178 (76%) constituted original research, while the rest consisted of editorials or reviews. A total of 52% of original AI-related research concerns hip and knee arthroplasty (n = 92), and a narrative review is described. Three studies were externally validated. Pitfalls surrounding present-day research include conflating vernacular (“AI/machine learning”), repackaging limited registry data, prematurely releasing internally validated prediction models, appraising model architecture instead of inputted data, withholding code, and evaluating studies using antiquated regression-based guidelines. While AI has been applied to a variety of hip and knee arthroplasty applications with limited clinical impact, the future remains promising if the question is meaningful, the methodology is rigorous and transparent, the data are rich, and the model is externally validated. Simple checkpoints for meaningful AI adoption include ensuring applications focus on: administrative support over clinical evaluation and management; necessity of the advanced model; and the novelty of the question being answered.

Cite this article: Bone Joint J 2022;104-B(12):1292–1303.

Meaningless Applications and Misguided Methodologies in Artificial Intelligence-Related Orthopaedic Research Propagates Hype Over Hope

There exists great hope and hype in the literature surrounding applications of artificial intelligence (AI) to orthopaedic surgery. Between 2018 and 2021, a total of 178 AI-related articles were published in orthopaedics. However, for every 2 original research papers that apply AI to orthopaedics, a commentary or review is published (30.3%). AI-related research in orthopaedics frequently fails to provide use cases that offer the uninitiated an opportunity to appraise the importance of AI by studying meaningful questions, evaluating unknown hypotheses, or analyzing quality data. The hype perpetuates a feed-forward cycle that relegates AI to a meaningless buzzword by rewarding those with nascent understanding and rudimentary technical knowhow into committing several basic errors: (1) inappropriately conflating vernacular ("AI/machine learning"), (2) repackaging registry data, (3) prematurely releasing internally validated algorithms, (4) overstating the "black box phenomenon" by failing to provide weighted analysis, (5) claiming to evaluate AI rather than the data itself, and (6) withholding full model architecture code. Relevant AI-specific guidelines are forthcoming, but forced application of the original Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis guidelines designed for regression analyses is irrelevant and misleading. To safeguard meaningful use, AI-related research efforts in orthopaedics should be (1) directed toward administrative support over clinical evaluation and management, (2) require the use of the advanced model, and (3) answer a question that was previously unknown, unanswered, or unquantifiable.