Machine Learning and Primary Total Knee Arthroplasty: Patient Forecasting for a Patient-Specific Payment Model

Machine learning for predicting duration of surgery and length of stay: A literature review on joint arthroplasty

INTRODUCTION

In recent years, different factors such as population aging have caused escalating demand for hip and knee arthroplasty straining already limited hospitals' resources. To address this challenge, focus is put on medical and operational efficiency improvements. This includes an increased use of machine learning (ML) to predict duration of surgery (DOS) and length of stay (LOS) for total knee and total hip arthroplasty, which can be utilized for optimizing resource allocation to satisfy medical and operational limitations. This paper explores the development and performance of ML models in predicting DOS and LOS.

METHODS

A systematic search of publications between 2010-2023 was conducted following PRISMA guidelines. Considering the inclusion and exclusion criteria, 28 out of 722 gathered papers from PubMed, Web of Science, and manual search were included in the study. Descriptive statistics was used to analyze the extracted data regarding data preprocessing, model development, and model performance assessment.

RESULTS

Most of the papers work on LOS as a binary variable. Patient's age was identified as the most frequently used and reported as important variable for predicting DOS and LOS. Investigations also illustrated that within the resulting 28 papers, more than 71% of models reached good to perfect performance based on the area under the receiver operating characteristic curve (AUC), where artificial neural networks and ensemble learning models had the biggest share among the best-performing models.

CONCLUSION

The utilization of ML models is increasing in the literature. The current performance level indicates that ML can potentially turn to powerful tools in predicting DOS and LOS for different purposes. Meanwhile, the literature is not matured yet in reporting real-life application. Future studies can focus on model specification and validation by considering empirical application.

Artificial intelligence-based analysis of associations between learning curve and clinical outcomes in endoscopic and microsurgical lumbar decompression surgery

PURPOSE

A common spine surgery procedure involves decompression of the lumbar spine. The impact of the surgeon's learning curve on relevant clinical outcomes is currently not well examined in the literature. A variety of machine learning algorithms have been investigated in this study to determine how a surgeon's learning curve and other clinical parameters will influence prolonged lengths of stay (LOS), extended operating times (OT), and complications, as well as whether these clinical parameters can be reliably predicted.

METHODS

A retrospective monocentric cohort study of patients with lumbar spinal stenosis treated with microsurgical (MSD) and full-endoscopic (FED) decompression was conducted. The study included 206 patients with lumbar spinal stenosis who underwent FED (63; 30.6%) and MSD (118; 57.3%). Prolonged LOS and OT were defined as those exceeding the 75th percentile of the cohort. Furthermore, complications were assessed as a dependent variable. Using unsupervised learning, clusters were identified in the data, which helped distinguish between the early learning curve (ELC) and the late learning curve (LLC). From 15 algorithms, the top five algorithms that best fit the data were selected for each prediction task. We calculated the accuracy of prediction (Acc) and the area under the curve (AUC). The most significant predictors were determined using a feature importance analysis.

RESULTS

For the FED group, the median number of surgeries with case surgery type at the time of surgery was 72 in the ELC group and 274 in the LLC group. FED patients did not significantly differ in outcome variables (LOS, OT, complication rate) between the ELC and LLC group. The random forest model demonstrated the highest mean accuracy and AUC across all folds for each classification task. For OT, it achieved an accuracy of 76.08% and an AUC of 0.89. For LOS, the model reached an accuracy of 83.83% and an AUC of 0.91. Lastly, in predicting complications, the random forest model attained the highest accuracy of 89.90% and an AUC of 0.94. Feature importance analysis indicated that LOS, OT, and complications were more significantly affected by patient characteristics than the surgical technique (FED versus MSD) or the surgeon's learning curve.

CONCLUSIONS

A median of 72 cases of FED surgeries led to comparable clinical outcomes in the early learning curve phase compared to experienced surgeons. These outcomes seem to be more significantly affected by patient characteristics than the learning curve or the surgical technique. Several study variables, including the learning curve, can be used to predict whether lumbar decompression surgery will result in an increased LOS, OT, or complications. To introduce the provided prediction tools into clinics, the algorithms need to be implemented into open-source software and externally validated through large-scale randomized controlled trials.

Machine learning model identifies patient gait speed throughout the episode of care, generating notifications for clinician evaluation

INTRODUCTION

The advent of digital and mobile health innovations, especially use of wearables for passive data collection, allows remote monitoring and creates an abundance of data. For this information to be interpretable, machine learning (ML) processes are necessary.

RESEARCH QUESTION

Can a machine learning model successfully identify patients expected to have low gait speed in the early recovery period following joint replacement surgery?

METHODS

A commercial database from a smartphone-based care management platform passively collecting mobility data pre- and post-lower limb arthroplasty was used. We sought to create a ML model to predict gait speed recovery curves and identify patients at risk of poor gait speed outcome, a measure associated with range of motion and patient-reported outcomes. Model performance including sensitivity, specificity, precision, and accuracy were determined. Receiver operator curve (ROC) analysis was used to compare true and false positive rates. To benchmark our model, we compared threshold-based notifications based on the patient's current gait speed.

RESULTS

The performance of the predictive model was significantly improved compared to baseline of threshold-based exceptions using current gait speed. The ML model currently provides 53 % precision, 88 % accuracy, 36 % sensitivity, and 95 % specificity on the held-out test set. The ROC analysis suggests good clinical performance (AUC=0.81).

SIGNIFICANCE

Utilization of ML to predict gait recovery following total joint replacement is feasible and provides results with excellent specificity. This model will allow inclusion of additional data for retraining as patient populations evolve. Clinician feedback regarding notifications, including resulting actions and outcomes, can be used to further inform the model and improve clinical utility.

Machine Learning Prediction Model to Predict Length of Stay of Patients Undergoing Hip or Knee Arthroplasties: Results from a High-Volume Single-Center Multivariate Analysis

Background: The growth of arthroplasty procedures requires innovative strategies to reduce inpatients' hospital length of stay (LOS). This study aims to develop a machine learning prediction model that may aid in predicting LOS after hip or knee arthroplasties. Methods: A collection of all the clinical notes of patients who underwent elective primary or revision arthroplasty from 1 January 2019 to 31 December 2019 was performed. The hospitalization was classified as "short LOS" if it was less than or equal to 6 days and "long LOS" if it was greater than 7 days. Clinical data from pre-operative laboratory analysis, vital parameters, and demographic characteristics of patients were screened. Final data were used to train a logistic regression model with the aim of predicting short or long LOS. Results: The final dataset was composed of 1517 patients (795 "long LOS", 722 "short LOS", p = 0.3196) with a total of 1541 hospital admissions (729 "long LOS", 812 "short LOS", p < 0.001). The complete model had a prediction efficacy of 78.99% (AUC 0.7899). Conclusions: Machine learning may facilitate day-by-day clinical practice determination of which patients are suitable for a shorter LOS and which for a longer LOS, in which a cautious approach could be recommended.

Optimizing Total Hip and Knee Arthroplasty Among an Underserved Population: Lessons Learned From a Quality-Improvement Initiative

Background

Under-represented minorities and those with noncommercial insurance have higher medical comorbidities and complications following elective total joint arthroplasty (TJA). In an effort to bridge this gap, our center implemented a preoperative optimization protocol for TJA in a Medicaid Clinic (Clinic). The purpose of this study is to assess the effectiveness of that protocol and highlight challenges associated with caring for this patient population.

Methods

This retrospective analysis included 117 patients undergoing TJA between January 2015 and January 2020. In 2015, the protocol was implemented as a mandatory practice prior to TJA. A contemporary control cohort from the private office was also analyzed. Patient demographics, American Society of Anesthesiologists score, and postoperative complications were collected.

Results

Within the clinic group, 52.5% (62) patients identified as Hispanic with 46.6% (55) Spanish-speaking as primary language, compared to 9.3% (11) and 8.5% (10) in the office group (P = .0001), respectively. Clinic group patients were significantly more likely to experience a complication compared to office patients (20 vs 7, respectively). There was no difference in complication or reoperation rate between clinic patients who underwent the optimization protocol and those who did not.

Conclusions

The findings from this study highlight the demographic and comorbidities profile of an underserved population, and report on results of a quality improvement initiative among that population, which failed to improve postoperative outcomes. These results underscore the need for further study in this population to improve outcomes and health equity.

Robotics in Arthroplasty: Historical Progression, Contemporary Applications, and Future Horizons With Artificial Intelligence (AI) Integration

Robotic technology is increasingly utilized in surgical procedures to enhance precision, particularly in tasks demanding delicate maneuvers beyond human capabilities. Robotic orthopedic surgery emerges as a dynamic and compelling technology reshaping the landscape of surgical practice. This aids surgeons in achieving enhanced accuracy and reproducibility, ultimately aiming for improved patient outcomes. As of now, the majority of these systems are in a developed stage and are gradually gaining broader adoption. These systems have to show that they are user-friendly, are successful in clinical settings, and have a good cost-effectiveness ratio before they can be widely adopted in the field of surgery. In this review, we examine the evolution of robotics in orthopedic surgery, assess its current applications, and provide insights into the future trajectory of this technology, particularly in light of advances in artificial intelligence (AI) and machine learning (ML).

Artificial intelligence in total and unicompartmental knee arthroplasty

The application of Artificial intelligence (AI) and machine learning (ML) tools in total (TKA) and unicompartmental knee arthroplasty (UKA) emerges with the potential to improve patient-centered decision-making and outcome prediction in orthopedics, as ML algorithms can generate patient-specific risk models. This review aims to evaluate the potential of the application of AI/ML models in the prediction of TKA outcomes and the identification of populations at risk.An extensive search in the following databases: MEDLINE, Scopus, Cinahl, Google Scholar, and EMBASE was conducted using the PIOS approach to formulate the research question. The PRISMA guideline was used for reporting the evidence of the data extracted. A modified eight-item MINORS checklist was employed for the quality assessment. The databases were screened from the inception to June 2022.Forty-four out of the 542 initially selected articles were eligible for the data analysis; 5 further articles were identified and added to the review from the PUBMED database, for a total of 49 articles included. A total of 2,595,780 patients were identified, with an overall average age of the patients of 70.2 years ± 7.9 years old. The five most common AI/ML models identified in the selected articles were: RF, in 38.77% of studies; GBM, in 36.73% of studies; ANN in 34.7% of articles; LR, in 32.65%; SVM in 26.53% of articles.This systematic review evaluated the possible uses of AI/ML models in TKA, highlighting their potential to lead to more accurate predictions, less time-consuming data processing, and improved decision-making, all while minimizing user input bias to provide risk-based patient-specific care.

The classification algorithms to support the management of the patient with femur fracture

Effectiveness in health care is a specific characteristic of each intervention and outcome evaluated. Especially with regard to surgical interventions, organization, structure and processes play a key role in determining this parameter. In addition, health care services by definition operate in a context of limited resources, so rationalization of service organization becomes the primary goal for health care management. This aspect becomes even more relevant for those surgical services for which there are high volumes. Therefore, in order to support and optimize the management of patients undergoing surgical procedures, the data analysis could play a significant role. To this end, in this study used different classification algorithms for characterizing the process of patients undergoing surgery for a femoral neck fracture. The models showed significant accuracy with values of 81%, and parameters such as Anaemia and Gender proved to be determined risk factors for the patient's length of stay. The predictive power of the implemented model is assessed and discussed in view of its capability to support the management and optimisation of the hospitalisation process for femoral neck fracture, and is compared with different model in order to identify the most promising algorithms. In the end, the support of artificial intelligence algorithms laying the basis for building more accurate decision-support tools for healthcare practitioners.

Supervised machine learning for the prediction of post-operative clinical outcomes of hip and knee replacements: a review

Prediction models are being increasingly used in the medical field to identify risk factors and possible outcomes. Some of these are presently being used to develop guidelines for improving clinical practice. The application of machine learning (ML), comprising a powerful set of computational tools for analysing data, has been clearly expanding in the role of predictive modelling. This paper reviews the latest developments of supervised ML techniques that have been used to analyse data related to post-operative total hip and knee replacements. The aim was to review the most recent findings of relevant published studies by outlining the methodologies employed (most-widely used supervised ML techniques), data sources, domains, limitations of predictive analytics and the quality of predictions.

Accuracy of machine learning to predict the outcomes of shoulder arthroplasty: a systematic review

BACKGROUND

Artificial intelligence (AI) uses computer systems to simulate cognitive capacities to accomplish goals like problem-solving and decision-making. Machine learning (ML), a branch of AI, makes algorithms find connections between preset variables, thereby producing prediction models. ML can aid shoulder surgeons in determining which patients may be susceptible to worse outcomes and complications following shoulder arthroplasty (SA) and align patient expectations following SA. However, limited literature is available on ML utilization in total shoulder arthroplasty (TSA) and reverse TSA.

METHODS

A systematic literature review in accordance with PRISMA guidelines was performed to identify primary research articles evaluating ML's ability to predict SA outcomes. With duplicates removed, the initial query yielded 327 articles, and after applying inclusion and exclusion criteria, 12 articles that had at least 1 month follow-up time were included.

RESULTS

ML can predict 30-day postoperative complications with a 90% accuracy, postoperative range of motion with a higher-than-85% accuracy, and clinical improvement in patient-reported outcome measures above minimal clinically important differences with a 93%-99% accuracy. ML can predict length of stay, operative time, discharge disposition, and hospitalization costs.

CONCLUSION

ML can accurately predict outcomes and complications following SA and healthcare utilization. Outcomes are highly dependent on the type of algorithms used, data input, and features selected for the model.

LEVEL OF EVIDENCE

III.

Bibliometric analysis of machine learning trends and hotspots in arthroplasty literature over 31 years

Background

Artificial intelligence has demonstrated utility in orthopedic research. Algorithmic models derived from machine learning have demonstrated adaptive learning with predictive application towards outcomes, leading to increased traction in the literature. This study aims to identify machine learning arthroplasty research trends and anticipate emerging key terms.

Methods

Published literature focused on machine learning in arthroplasty from 1992 to 2023 was selected through the Web of Science Core Collection of Clarivate Analytics. Following that, bibliometric indicators were attained and brought in to perform an additional examination using Bibliometrix and VOSviewer to identify historical and present patterns within the literature.

Results

A total of 235 documents were obtained through bibliometric sourcing based on machine learning applications within the arthroplasty literature. Thirty-four countries published articles on the topic, and the United States was demonstrated to be the largest global contributor. Four hundred-five institutions internationally contributed articles, with Harvard Medical School and the University of California system as the most relevant institutes, with 75 and 44 articles produced, respectively. Kwon YM was the most productive author, while Haeberle HS and Ramkumar PN were the most impactful based on h-index. The Thematic map and Co-occurrence visualization helped identify both major and niche themes present in the scientific databases.

Conclusions

Machine learning in arthroplasty research continues to gain traction with a growing annual production rate and contributions from international authors and institutions. Institutions and authors based in the United States are the leading contributors to machine learning applications within arthroplasty research. This research discerns trends that have occurred, are presently ongoing, and are emerging within this field, aiming to inform future hotspot development.

Enabling Personalized Medicine in Orthopaedic Surgery Through Artificial Intelligence: A Critical Analysis Review

» The application of artificial intelligence (AI) in the field of orthopaedic surgery holds potential for revolutionizing health care delivery across 3 crucial domains: (I) personalized prediction of clinical outcomes and adverse events, which may optimize patient selection, surgical planning, and enhance patient safety and outcomes; (II) diagnostic automated and semiautomated imaging analyses, which may reduce time burden and facilitate precise and timely diagnoses; and (III) forecasting of resource utilization, which may reduce health care costs and increase value for patients and institutions.» Computer vision is one of the most highly studied areas of AI within orthopaedics, with applications pertaining to fracture classification, identification of the manufacturer and model of prosthetic implants, and surveillance of prosthesis loosening and failure.» Prognostic applications of AI within orthopaedics include identifying patients who will likely benefit from a specified treatment, predicting prosthetic implant size, postoperative length of stay, discharge disposition, and surgical complications. Not only may these applications be beneficial to patients but also to institutions and payors because they may inform potential cost expenditure, improve overall hospital efficiency, and help anticipate resource utilization.» AI infrastructure development requires institutional financial commitment and a team of clinicians and data scientists with expertise in AI that can complement skill sets and knowledge. Once a team is established and a goal is determined, teams (1) obtain, curate, and label data; (2) establish a reference standard; (3) develop an AI model; (4) evaluate the performance of the AI model; (5) externally validate the model, and (6) reinforce, improve, and evaluate the model's performance until clinical implementation is possible.» Understanding the implications of AI in orthopaedics may eventually lead to wide-ranging improvements in patient care. However, AI, while holding tremendous promise, is not without methodological and ethical limitations that are essential to address. First, it is important to ensure external validity of programs before their use in a clinical setting. Investigators should maintain high quality data records and registry surveillance, exercise caution when evaluating others' reported AI applications, and increase transparency of the methodological conduct of current models to improve external validity and avoid propagating bias. By addressing these challenges and responsibly embracing the potential of AI, the medical field may eventually be able to harness its power to improve patient care and outcomes.

Using Unsupervised Machine Learning to Predict Quality of Life After Total Knee Arthroplasty

BACKGROUND

Patient-reported outcome measures (PROMs) are an important metric to assess total knee arthroplasty (TKA) patients. The purpose of this study was to use a machine learning (ML) algorithm to identify patient features that impact PROMs after TKA.

METHODS

Data from 636 TKA patients enrolled in our patient database between 2018 and 2022, were retrospectively reviewed. Their mean age was 68 years (range, 39 to 92), 56.7% women, and mean body mass index of 31.17 (range, 16 to 58). Patient demographics and the Functional Comorbidity Index were collected alongside Patient-Reported Outcome Measures Information System Global Health v1.2 (PROMIS GH-P) physical component scores preoperatively, at 3 months, and 1 year after TKA. An unsupervised ML algorithm (spectral clustering) was used to identify patient features impacting PROMIS GH-P scores at the various time points.

RESULTS

The algorithm identified 5 patient clusters that varied by demographics, comorbidities, and pain scores. Each cluster was associated with predictable trends in PROMIS GH-P scores across the time points. Notably, patients who had the worst preoperative PROMIS GH-P scores (cluster 5) had the most improvement after TKA, whereas patients who had higher global health rating preoperatively had more modest improvement (clusters 1, 2, and 3). Two out of Five patient clusters (cluster 4 and 5) showed improvement in PROMIS GH-P scores that met a minimally clinically important difference at 1-year postoperative.

CONCLUSIONS

The unsupervised ML algorithm identified patient clusters that had predictable changes in PROMs after TKA. It is a positive step toward providing precision medical care for each of our arthroplasty patients.

Can machine learning models predict prolonged length of hospital stay following primary total knee arthroplasty based on a national patient cohort data?

INTRODUCTION

The total length of stay (LOS) is one of the biggest determinators of overall care costs associated with total knee arthroplasty (TKA). An accurate prediction of LOS could aid in optimizing discharge strategy for patients in need and diminishing healthcare expenditure. The aim of this study was to predict LOS following TKA using machine learning models developed on a national-scale patient cohort.

METHODS

The ACS-NSQIP database was queried to acquire 267,966 TKA cases from 2013 to 2020. Four machine learning models-artificial neural network (ANN), random forest, histogram-based gradient boosting, and k-nearest neighbor were trained and tested on the dataset for the prediction of prolonged LOS (LOS exceeded the 75th of all values in the cohort). The model performance was assessed by discrimination (area under the receiver operating characteristic curve [AUC]), calibration, and clinical utility.

RESULTS

ANN delivered the best performance among the four models. ANN distinguished prolonged LOS in the study cohort with an AUC of 0.71 and accurately predicted the probability of prolonged LOS for individual patients (calibration slope: 0.82; calibration intercept: 0.03; Brier score: 0.089). All models demonstrated clinical utility by generating positive net benefits in decision curve analyses. Operation time, pre-operative transfusion, pre-operative laboratory tests (hematocrit, platelet count, and white blood cell count), and BMI were the strongest predictors of prolonged LOS.

CONCLUSION

ANN demonstrated modest discrimination capacity and excellent performance in calibration and clinical utility for the prediction of prolonged LOS following TKA. Clinical application of the machine learning models has the potential to improve care coordination and discharge planning for patients at high risk of extended hospitalization after surgery. Incorporating more relevant patient factors may further increase the models' prediction strength.

Comparable performance of machine learning algorithms in predicting readmission and complications following total joint arthroplasty with external validation

BACKGROUND

The purpose of the study was to use Machine Learning (ML) to construct a risk calculator for patients who undergo Total Joint Arthroplasty (TJA) on the basis of New York State Statewide Planning and Research Cooperative System (SPARCS) data and externally validate the calculator on a single TJA center.

METHODS

Seven ML algorithms, i.e., logistic regression, adaptive boosting, gradient boosting (Xg Boost), random forest (RF) classifier, support vector machine, and single and a five-layered neural network were trained on the derivation cohort. Models were trained on 68% of data, validated on 15%, tested on 15%, and externally validated on 2% of the data from a single arthroplasty center.

RESULTS

Validation of the models showed that the RF classifier performed best in terms of 30-d mortality AUROC (Area Under the Receiver Operating Characteristic) 0.78, 30-d readmission (AUROC 0.61) and 90-d composite complications (AUROC 0.73) amongst the test set. Additionally, Xg Boost was found to be the best predicting model for 90-d readmission and 90-d composite complications (AUC 0.73). External validation demonstrated that models achieved similar AUROCs to the test set although variation occurred in top model performance for 90-d composite complications and readmissions between our test and external validation set.

CONCLUSION

This was the first study to investigate the use of ML to create a predictive risk calculator from state-wide data and then externally validate it with data from a single arthroplasty center. Discrimination between best performing ML models and between the test set and the external validation set are comparable.

LEVEL OF EVIDENCE

III.

Current clinical applications of artificial intelligence in shoulder surgery: what the busy shoulder surgeon needs to know and what's coming next

Background

Artificial intelligence (AI) is a continuously expanding field with the potential to transform a variety of industries-including health care-by providing automation, efficiency, precision, accuracy, and decision-making support for simple and complex tasks. Basic knowledge of the key features as well as limitations of AI is paramount to understand current developments in this field and to successfully apply them to shoulder surgery. The purpose of the present review is to provide an overview of AI within orthopedics and shoulder surgery exploring current and forthcoming AI applications.

Methods

PubMed and Scopus databases were searched to provide a narrative review of the most relevant literature on AI applications in shoulder surgery.

Results

Despite the enormous clinical and research potential of AI, orthopedic surgery has been a relatively late adopter of AI technologies. Image evaluation, surgical planning, aiding decision-making, and facilitating patient evaluations over time are some of the current areas of development with enormous opportunities to improve surgical practice, research, and education. Furthermore, the advancement of AI-driven strategies has the potential to create a more efficient medical system that may reduce the overall cost of delivering and implementing quality health care for patients with shoulder pathology.

Conclusion

AI is an expanding field with the potential for broad clinical and research applications in orthopedic surgery. Many challenges still need to be addressed to fully leverage the potential of AI to clinical practice and research such as privacy issues, data ownership, and external validation of the proposed models.

The Impact of Machine Learning on Total Joint Arthroplasty Patient Outcomes: A Systemic Review

BACKGROUND

Supervised machine learning techniques have been increasingly applied to predict patient outcomes after hip and knee arthroplasty procedures. The purpose of this study was to systematically review the applications of supervised machine learning techniques to predict patient outcomes after primary total hip and knee arthroplasty.

METHODS

A comprehensive literature search using the electronic databases MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials, and Cochrane Database of Systematic Reviews was conducted in July of 2021. The inclusion criteria were studies that utilized supervised machine learning techniques to predict patient outcomes after primary total hip or knee arthroplasty.

RESULTS

Search criteria yielded n = 30 relevant studies. Topics of study included patient complications (n = 6), readmissions (n = 1), revision (n = 2), patient-reported outcome measures (n = 4), patient satisfaction (n = 4), inpatient status and length of stay (LOS) (n = 9), opioid usage (n = 3), and patient function (n = 1). Studies involved TKA (n = 12), THA (n = 11), or a combination (n = 7). Less than 35% of predictive outcomes had an area under the receiver operating characteristic curve (AUC) in the excellent or outstanding range. Additionally, only 9 of the studies found improvement over logistic regression, and only 9 studies were externally validated.

CONCLUSION

Supervised machine learning algorithms are powerful tools that have been increasingly applied to predict patient outcomes after total hip and knee arthroplasty. However, these algorithms should be evaluated in the context of prognostic accuracy, comparison to traditional statistical techniques for outcome prediction, and application to populations outside the training set. While machine learning algorithms have been received with considerable interest, they should be critically assessed and validated prior to clinical adoption.

Artificial Intelligence for Automated Implant Identification in Knee Arthroplasty: A Multicenter External Validation Study Exceeding 3.5 Million Plain Radiographs

BACKGROUND

Surgical management of complications following knee arthroplasty demands accurate and timely identification of implant manufacturer and model. Automated image processing using deep machine learning has been previously developed and internally validated; however, external validation is essential prior to scaling clinical implementation for generalizability.

METHODS

We trained, validated, and externally tested a deep learning system to classify knee arthroplasty systems as one of the 9 models from 4 manufacturers derived from 4,724 original, retrospectively collected anteroposterior plain knee radiographs across 3 academic referral centers. From these radiographs, 3,568 were used for training, 412 for validation, and 744 for external testing. Augmentation was applied to the training set (n = 3,568,000) to increase model robustness. Performance was determined by the area under the receiver operating characteristic curve, sensitivity, specificity, and accuracy. Implant identification processing speed was calculated. The training and testing sets were drawn from statistically different populations of implants (P < .001).

RESULTS

After 1,000 training epochs by the deep learning system, the system discriminated 9 implant models with a mean area under the receiver operating characteristic curve of 0.989, accuracy of 97.4%, sensitivity of 89.2%, and specificity of 99.0% in the external testing dataset of 744 anteroposterior radiographs. The software classified implants at a mean speed of 0.02 seconds per image.

CONCLUSION

An artificial intelligence-based software for identifying knee arthroplasty implants demonstrated excellent internal and external validation. Although continued surveillance is necessary with implant library expansion, this software represents a responsible and meaningful clinical application of artificial intelligence with immediate potential to globally scale and assist in preoperative planning prior to revision knee arthroplasty.

Artificial Intelligence for Automated Implant Identification in Total Hip Arthroplasty: A Multicenter External Validation Study Exceeding Two Million Plain Radiographs

BACKGROUND

The surgical management of complications after total hip arthroplasty (THA) necessitates accurate identification of the femoral implant manufacturer and model. Automated image processing using deep learning has been previously developed and internally validated; however, external validation is necessary prior to responsible application of artificial intelligence (AI)-based technologies.

METHODS

We trained, validated, and externally tested a deep learning system to classify femoral-sided THA implants as one of the 8 models from 2 manufacturers derived from 2,954 original, deidentified, retrospectively collected anteroposterior plain radiographs across 3 academic referral centers and 13 surgeons. From these radiographs, 2,117 were used for training, 249 for validation, and 588 for external testing. Augmentation was applied to the training set (n = 2,117,000) to increase model robustness. Performance was evaluated by area under the receiver operating characteristic curve, sensitivity, specificity, and accuracy. Implant identification processing speed was calculated.

RESULTS

The training and testing sets were drawn from statistically different populations of implants (P < .001). After 1,000 training epochs by the deep learning system, the system discriminated 8 implant models with a mean area under the receiver operating characteristic curve of 0.991, accuracy of 97.9%, sensitivity of 88.6%, and specificity of 98.9% in the external testing dataset of 588 anteroposterior radiographs. The software classified implants at a mean speed of 0.02 seconds per image.

CONCLUSION

An AI-based software demonstrated excellent internal and external validation. Although continued surveillance is necessary with implant library expansion, this software represents responsible and meaningful clinical application of AI with immediate potential to globally scale and assist in preoperative planning prior to revision THA.

The Viability of an Artificial Intelligence/Machine Learning Prediction Model to Determine Candidates for Knee Arthroplasty

BACKGROUND

The purpose of this study is to assess the viability of a knee arthroplasty prediction model using 3-view X-rays that helps determine if patients with knee pain are candidates for total knee arthroplasty (TKA), unicompartmental knee arthroplasty (UKA), or are not arthroplasty candidates.

METHODS

Analysis was performed using radiographic and surgical data from a high-volume joint replacement practice. The dataset included 3 different X-ray views (anterior-posterior, lateral, and sunrise) for 2,767 patients along with information of whether that patient underwent an arthroplasty surgery (UKA or TKA) or not. This resulted in a dataset including 8,301 images from 2,707 patients. This dataset was then split into a training set (70%) and holdout test set (30%). A computer vision model was trained using a transfer learning approach. The performance of the computer vision model was evaluated on the holdout test set. Accuracy and multiclass receiver operating characteristic area under curve was used to evaluate the performance of the model.

RESULTS

The artificial intelligence model achieved an accuracy of 87.8% on the holdout test set and a quadratic Cohen's kappa score of 0.811. The multiclass receiver operating characteristic area under curve score for TKA was calculated to be 0.97; for UKA a score of 0.96 and for No Surgery a score of 0.98 was achieved. An accuracy of 93.8% was achieved for predicting Surgery versus No Surgery and 88% for TKA versus not TKA was achieved.

CONCLUSION

The artificial intelligence/machine learning model demonstrated viability for predicting which patients are candidates for a UKA, TKA, or no surgical intervention.

Artificial intelligence and ChatGPT in Orthopaedics and sports medicine

Artificial intelligence (AI) is looked upon nowadays as the potential major catalyst for the fourth industrial revolution. In the last decade, AI use in Orthopaedics increased approximately tenfold. Artificial intelligence helps with tracking activities, evaluating diagnostic images, predicting injury risk, and several other uses. Chat Generated Pre-trained Transformer (ChatGPT), which is an AI-chatbot, represents an extremely controversial topic in the academic community. The aim of this review article is to simplify the concept of AI and study the extent of AI use in Orthopaedics and sports medicine literature. Additionally, the article will also evaluate the role of ChatGPT in scientific research and publications.Level of evidence: Level V, letter to review.

A role for artificial intelligence applications inside and outside of the operating theatre: a review of contemporary use associated with total knee arthroplasty

BACKGROUND

Artificial intelligence (AI) has become involved in many aspects of everyday life, from voice-activated virtual assistants built into smartphones to global online search engines. Similarly, many areas of modern medicine have found ways to incorporate such technologies into mainstream practice. Despite the enthusiasm, robust evidence to support the utility of AI in contemporary total knee arthroplasty (TKA) remains limited. The purpose of this review was to provide an up-to-date summary of the use of AI in TKA and to explore its current and future value.

METHODS

Initially, a structured systematic review of the literature was carried out, following PRISMA search principles, with the aim of summarising the understanding of the field and identifying clinical and knowledge gaps.

RESULTS

A limited body of published work exists in this area. Much of the available literature is of poor methodological quality and many published studies could be best described as "demonstration of concepts" rather than "proof of concepts". There exists almost no independent validation of reported findings away from designer/host sites, and the extrapolation of key results to general orthopaedic sites is limited.

CONCLUSION

While AI has certainly shown value in a small number of specific TKA-associated applications, the majority to date have focused on risk, cost and outcome prediction, rather than surgical care, per se. Extensive future work is needed to demonstrate external validity and reliability in non-designer settings. Well-performed studies are warranted to ensure that the scientific evidence base supporting the use of AI in knee arthroplasty matches the global hype.

Improved performance of machine learning models in predicting length of stay, discharge disposition, and inpatient mortality after total knee arthroplasty using patient-specific variables

BACKGROUND

This study aimed to compare the performance of ten predictive models using different machine learning (ML) algorithms and compare the performance of models developed using patient-specific vs. situational variables in predicting select outcomes after primary TKA.

METHODS

Data from 2016 to 2017 from the National Inpatient Sample were used to identify 305,577 discharges undergoing primary TKA, which were included in the training, testing, and validation of 10 ML models. 15 predictive variables consisting of 8 patient-specific and 7 situational variables were utilized to predict length of stay (LOS), discharge disposition, and mortality. Using the best performing algorithms, models trained using either 8 patient-specific and 7 situational variables were then developed and compared.

RESULTS

For models developed using all 15 variables, Linear Support Vector Machine (LSVM) was the most responsive model for predicting LOS. LSVM and XGT Boost Tree were equivalently most responsive for predicting discharge disposition. LSVM and XGT Boost Linear were equivalently most responsive for predicting mortality. Decision List, CHAID, and LSVM were the most reliable models for predicting LOS and discharge disposition, while XGT Boost Tree, Decision List, LSVM, and CHAID were most reliable for mortality. Models developed using the 8 patient-specific variables outperformed those developed using the 7 situational variables, with few exceptions.

CONCLUSION

This study revealed that performance of different models varied, ranging from poor to excellent, and demonstrated that models developed using patient-specific variables were typically better predictive of quality metrics after TKA than those developed employing situational variables.

LEVEL OF EVIDENCE

III.

Assessing the predictive capacity of machine learning models using patient-specific variables in determining in-hospital outcomes after THA

Background

Machine learning is a subset of artificial intelligence using algorithmic modeling to progressively learn and create predictive models. Clinical application of machine learning can aid physicians through identification of risk factors and implications of predicted patient outcomes.

Aims

The aim of this study was to compare patient-specific and situation perioperative variables through optimized machine learning models to predict postoperative outcomes.

Methods

Data from 2016 to 2017 from the National Inpatient Sample was used to identify 177,442 discharges undergoing primary total hip arthroplasty, which were included in the training, testing, and validation of 10 machine learning models. 15 predictive variables consisting of 8 patient-specific and 7 situational specific variables were utilized to predict 3 outcome variables: length of stay, discharge, and mortality. The machine learning models were assessed in responsiveness via area under the curve and reliability.

Results

For all outcomes, Linear Support Vector Machine had the highest responsiveness among all models when using all variables. When utilizing patient-specific variables only, responsiveness of the top 3 models ranged between 0.639 and 0.717 for length of stay, 0.703-0.786 for discharge disposition, and 0.887-0.952 for mortality. The top 3 models utilizing situational variables only produced responsiveness of 0.552-0.589 for length of stay, 0.543-0.574 for discharge disposition, and 0.469-0.536 for mortality.

Conclusions

Linear Support Vector Machine was the most responsive machine learning model of the 10 algorithms trained, while decision list was most reliable. Responsiveness was observed to be consistently higher with patient-specific variables than situational variables, emphasizing the predictive capacity and value of patient-specific variables. The current practice in machine learning literature generally deploys a single model, it is suboptimal to develop optimized models for application into clinical practice. The limitation of other algorithms may prohibit potential more reliable and responsive models.Level of Evidence III.

Detection of Prosthetic Loosening in Hip and Knee Arthroplasty Using Machine Learning: A Systematic Review and Meta-Analysis

Background: prosthetic loosening after hip and knee arthroplasty is one of the most common causes of joint arthroplasty failure and revision surgery. Diagnosis of prosthetic loosening is a difficult problem and, in many cases, loosening is not clearly diagnosed until accurately confirmed during surgery. The purpose of this study is to conduct a systematic review and meta-analysis to demonstrate the analysis and performance of machine learning in diagnosing prosthetic loosening after total hip arthroplasty (THA) and total knee arthroplasty (TKA). Materials and Methods: three comprehensive databases, including MEDLINE, EMBASE, and the Cochrane Library, were searched for studies that evaluated the detection accuracy of loosening around arthroplasty implants using machine learning. Data extraction, risk of bias assessment, and meta-analysis were performed. Results: five studies were included in the meta-analysis. All studies were retrospective studies. In total, data from 2013 patients with 3236 images were assessed; these data involved 2442 cases (75.5%) with THAs and 794 cases (24.5%) with TKAs. The most common and best-performing machine learning algorithm was DenseNet. In one study, a novel stacking approach using a random forest showed similar performance to DenseNet. The pooled sensitivity across studies was 0.92 (95% CI 0.84-0.97), the pooled specificity was 0.95 (95% CI 0.93-0.96), and the pooled diagnostic odds ratio was 194.09 (95% CI 61.60-611.57). The I2 statistics for sensitivity and specificity were 96% and 62%, respectively, showing that there was significant heterogeneity. The summary receiver operating characteristics curve indicated the sensitivity and specificity, as did the prediction regions, with an AUC of 0.9853. Conclusions: the performance of machine learning using plain radiography showed promising results with good accuracy, sensitivity, and specificity in the detection of loosening around THAs and TKAs. Machine learning can be incorporated into prosthetic loosening screening programs.

Improving Resource Utilization for Arthroplasty Care by Leveraging Machine Learning and Optimization: A Systematic Review

Background

There is a growing demand for total joint arthroplasty (TJA) surgery. The applications of machine learning (ML), mathematical optimization, and computer simulation have the potential to improve efficiency of TJA care delivery through outcome prediction and surgical scheduling optimization, easing the burden on health-care systems. The purpose of this study was to evaluate strategies using advances in analytics and computational modeling that may improve planning and the overall efficiency of TJA care.

Methods

A systematic review including MEDLINE, Embase, and IEEE Xplore databases was completed from inception to October 3, 2022, for identification of studies generating ML models for TJA length of stay, duration of surgery, and hospital readmission prediction. A scoping review of optimization strategies in elective surgical scheduling was also conducted.

Results

Twenty studies were included for evaluating ML predictions and 17 in the scoping review of scheduling optimization. Among studies generating linear or logistic control models alongside ML models, only 1 found a control model to outperform its ML counterpart. Furthermore, neural networks performed superior to or at the same level as conventional ML models in all but 1 study. Implementation of mathematical and simulation strategies improved the optimization efficiency when compared to traditional scheduling methods at the operational level.

Conclusions

High-performing predictive ML-based models have been developed for TJA, as have mathematical strategies for elective surgical scheduling optimization. By leveraging artificial intelligence for outcome prediction and surgical optimization, there exist greater opportunities for improved resource utilization and cost-savings in TJA than when using traditional modeling and scheduling methods.

Artificial Intelligence in Musculoskeletal Conditions

Artificial intelligence (AI) refers to computer capabilities that resemble human intelligence. AI implies the ability to learn and perform tasks that have not been specifically programmed. Moreover, it is an iterative process involving the ability of computerized systems to capture information, transform it into knowledge, and process it to produce adaptive changes in the environment. A large labeled database is needed to train the AI system and generate a robust algorithm. Otherwise, the algorithm cannot be applied in a generalized way. AI can facilitate the interpretation and acquisition of radiological images. In addition, it can facilitate the detection of trauma injuries and assist in orthopedic and rehabilitative processes. The applications of AI in musculoskeletal conditions are promising and are likely to have a significant impact on the future management of these patients.

Machine Learning Can Accurately Predict Overnight Stay, Readmission, and 30-Day Complications Following Anterior Cruciate Ligament Reconstruction

PURPOSE

This study aimed to develop machine learning (ML) models to predict hospital admission (overnight stay) as well as short-term complications and readmission rates following anterior cruciate ligament reconstruction (ACLR). Furthermore, we sought to compare the ML models with logistic regression models in predicting ACLR outcomes.

METHODS

The American College of Surgeons National Surgical Quality Improvement Program database was queried for patients who underwent elective ACLR from 2012 to 2018. Artificial neural network ML and logistic regression models were developed to predict overnight stay, 30-day postoperative complications, and ACL-related readmission, and model performance was compared using the area under the receiver operating characteristic curve. Regression analyses were used to identify variables that were significantly associated with the predicted outcomes.

RESULTS

A total of 21,636 elective ACLR cases met inclusion criteria. Variables associated with hospital admission included White race, obesity, hypertension, and American Society of Anesthesiologists classification 3 and greater, anesthesia other than general, prolonged operative time, and inpatient setting. The incidence of hospital admission (overnight stay) was 10.2%, 30-day complications was 1.3%, and 30-day readmission for ACLR-related causes was 0.9%. Compared with logistic regression models, artificial neural network models reported superior area under the receiver operating characteristic curve values in predicting overnight stay (0.835 vs 0.589), 30-day complications (0.742 vs 0.590), reoperation (0.842 vs 0.601), ACLR-related readmission (0.872 vs 0.606), deep-vein thrombosis (0.804 vs 0.608), and surgical-site infection (0.818 vs 0.596).

CONCLUSIONS

The ML models developed in this study demonstrate an application of ML in which data from a national surgical patient registry was used to predict hospital admission and 30-day postoperative complications after elective ACLR. ML models developed performed well, outperforming regression models in predicting hospital admission and short-term complications following elective ACLR. ML models performed best when predicting ACLR-related readmissions and reoperations, followed by overnight stay.

LEVEL OF EVIDENCE

IV, retrospective comparative prognostic trial.

Hospital length of stay prediction for general surgery and total knee arthroplasty admissions: Systematic review and meta-analysis of published prediction models

Objective

Systematic review of length of stay (LOS) prediction models to assess the study methods (including prediction variables), study quality, and performance of predictive models (using area under receiver operating curve (AUROC)) for general surgery populations and total knee arthroplasty (TKA).

Method

LOS prediction models published since 2010 were identified in five major research databases. The main outcomes were model performance metrics including AUROC, prediction variables, and level of validation. Risk of bias was assessed using the PROBAST checklist.

Results

Five general surgery studies (15 models) and 10 TKA studies (24 models) were identified. All general surgery and 20 TKA models used statistical approaches; 4 TKA models used machine learning approaches. Risk scores, diagnosis, and procedure types were predominant predictors used. Risk of bias was ranked as moderate in 3/15 and high in 12/15 studies. Discrimination measures were reported in 14/15 and calibration measures in 3/15 studies, with only 4/39 externally validated models (3 general surgery and 1 TKA). Meta-analysis of externally validated models (3 general surgery) suggested the AUROC 95% prediction interval is excellent and ranges between 0.803 and 0.970.

Conclusion

This is the first systematic review assessing quality of risk prediction models for prolonged LOS in general surgery and TKA groups. We showed that these risk prediction models were infrequently externally validated with poor study quality, typically related to poor reporting. Both machine learning and statistical modelling methods, plus the meta-analysis, showed acceptable to good predictive performance, which are encouraging. Moving forward, a focus on quality methods and external validation is needed before clinical application.

Applications of artificial intelligence in orthopaedic surgery

The practice of medicine is rapidly transforming as a result of technological breakthroughs. Artificial intelligence (AI) systems are becoming more and more relevant in medicine and orthopaedic surgery as a result of the nearly exponential growth in computer processing power, cloud based computing, and development, and refining of medical-task specific software algorithms. Because of the extensive role of technologies such as medical imaging that bring high sensitivity, specificity, and positive/negative prognostic value to management of orthopaedic disorders, the field is particularly ripe for the application of machine-based integration of imaging studies, among other applications. Through this review, we seek to promote awareness in the orthopaedics community of the current accomplishments and projected uses of AI and ML as described in the literature. We summarize the current state of the art in the use of ML and AI in five key orthopaedic disciplines: joint reconstruction, spine, orthopaedic oncology, trauma, and sports medicine.

Machine Learning Models for Predicting Postoperative Outcomes following Skull Base Meningioma Surgery

Objective  While predictive analytic techniques have been used to analyze meningioma postoperative outcomes, to our knowledge, there have been no studies that have investigated the utility of machine learning (ML) models in prognosticating outcomes among skull base meningioma patients. The present study aimed to develop models for predicting postoperative outcomes among skull base meningioma patients, specifically prolonged hospital length of stay (LOS), nonroutine discharge disposition, and high hospital charges. We also validated the predictive performance of our models on out-of-sample testing data. Methods  Patients who underwent skull base meningioma surgery between 2016 and 2019 at an academic institution were included in our study. Prolonged hospital LOS and high hospital charges were defined as >4 days and >$47,887, respectively. Elastic net logistic regression algorithms were trained to predict postoperative outcomes using 70% of available data, and their predictive performance was evaluated on the remaining 30%. Results  A total of 265 patients were included in our final analysis. Our cohort was majority female (77.7%) and Caucasian (63.4%). Elastic net logistic regression algorithms predicting prolonged LOS, nonroutine discharge, and high hospital charges achieved areas under the receiver operating characteristic curve of 0.798, 0.752, and 0.592, respectively. Further, all models were adequately calibrated as determined by the Spiegelhalter Z -test ( p >0.05). Conclusion  Our study developed models predicting prolonged hospital LOS, nonroutine discharge disposition, and high hospital charges among skull base meningioma patients. Our models highlight the utility of ML as a tool to aid skull base surgeons in providing high-value health care and optimizing clinical workflows.

A Machine Learning Framework for Assessing the Risk of Venous Thromboembolism in Patients Undergoing Hip or Knee Replacement

Venous thromboembolism (VTE) is a well-recognized complication that is prevalent in patients undergoing major orthopedic surgery (e.g., total hip arthroplasty and total knee arthroplasty). For years, to identify patients at high risk of developing VTE, physicians have relied on traditional risk scoring systems, which are too simplistic to capture the risk level accurately. In this paper, we propose a data-driven machine learning framework to identify such high-risk patients before they undergo a major hip or knee surgery. Using electronic health records of more than 392,000 patients who undergone a major orthopedic surgery, and following a guided feature selection using the genetic algorithm, we trained a fully connected deep neural network model to predict high-risk patients for developing VTE. We identified several risk factors for VTE that were not previously recognized. The best FCDNN model trained using the selected features yielded an area under the ROC curve (AUC) of 0.873, which was remarkably higher than the best AUC obtained by including only risk factors previously known in the medical literature. Our findings suggest several interesting and important insights. The traditional risk scoring tables that are being widely used by physicians to identify high-risk patients are not considering a comprehensive set of risk factors, nor are they as powerful as cutting-edge machine learning methods in distinguishing low- from high-risk patients.

Prediction of total healthcare cost following total shoulder arthroplasty utilizing machine learning

BACKGROUND

Given the increase in demand in treatment of glenohumeral arthritis with anatomic total (aTSA) and reverse shoulder arthroplasty (RTSA), it is imperative to improve quality of patient care while controlling costs as private and federal insurers continue its gradual transition toward bundled payment models. Big data analytics with machine learning shows promise in predicting health care costs. This is significant as cost prediction may help control cost by enabling health care systems to appropriately allocate resources that help mitigate the cause of increased cost.

METHODS

The Nationwide Readmissions Database (NRD) was accessed in 2018. The database was queried for all primary aTSA and RTSA by International Classification of Diseases, Tenth Revision (ICD-10) procedure codes: 0RRJ0JZ and 0RRK0JZ for aTSA and 0RRK00Z and 0RRJ00Z for RTSA. Procedures were categorized by diagnoses: osteoarthritis (OA), rheumatoid arthritis (RA), avascular necrosis (AVN), fracture, and rotator cuff arthropathy (RCA). Costs were calculated by utilizing the total hospital charge and each hospital's cost-to-charge ratio. Hospital characteristics were included, such as volume of procedures performed by the respective hospital for the calendar year and wage index, which represents the relative average hospital wage for the respective geographic area. Unplanned readmissions within 90 days were calculated using unique patient identifiers, and cost of readmissions was added to the total admission cost to represent the short-term perioperative health care cost. Machine learning algorithms were used to predict patients with immediate postoperative admission costs greater than 1 standard deviation from the mean, and readmissions.

RESULTS

A total of 49,354 patients were isolated for analysis, with an average patient age of 69.9 ± 9.6 years. The average perioperative cost of care was $18,843 ± $10,165. In total, there were 4279 all-cause readmissions, resulting in an average cost of $13,871.00 ± $14,301.06 per readmission. Wage index, hospital volume, patient age, readmissions, and diagnosis-related group severity were the factors most correlated with the total cost of care. The logistic regression and random forest algorithms were equivalent in predicting the total cost of care (area under the receiver operating characteristic curve = 0.83).

CONCLUSION

After shoulder arthroplasty, there is significant variability in cumulative hospital costs, and this is largely affected by readmissions. Hospital characteristics, such as geographic area and volume, are key determinants of overall health care cost. When accounting for this, machine learning algorithms may predict cases with high likelihood of increased resource utilization and/or readmission.

Concerns surrounding application of artificial intelligence in hip and knee arthroplasty

Literature surrounding artificial intelligence (AI)-related applications for hip and knee arthroplasty has proliferated. However, meaningful advances that fundamentally transform the practice and delivery of joint arthroplasty are yet to be realized, despite the broad range of applications as we continue to search for meaningful and appropriate use of AI. AI literature in hip and knee arthroplasty between 2018 and 2021 regarding image-based analyses, value-based care, remote patient monitoring, and augmented reality was reviewed. Concerns surrounding meaningful use and appropriate methodological approaches of AI in joint arthroplasty research are summarized. Of the 233 AI-related orthopaedics articles published, 178 (76%) constituted original research, while the rest consisted of editorials or reviews. A total of 52% of original AI-related research concerns hip and knee arthroplasty (n = 92), and a narrative review is described. Three studies were externally validated. Pitfalls surrounding present-day research include conflating vernacular (“AI/machine learning”), repackaging limited registry data, prematurely releasing internally validated prediction models, appraising model architecture instead of inputted data, withholding code, and evaluating studies using antiquated regression-based guidelines. While AI has been applied to a variety of hip and knee arthroplasty applications with limited clinical impact, the future remains promising if the question is meaningful, the methodology is rigorous and transparent, the data are rich, and the model is externally validated. Simple checkpoints for meaningful AI adoption include ensuring applications focus on: administrative support over clinical evaluation and management; necessity of the advanced model; and the novelty of the question being answered.

Cite this article: Bone Joint J 2022;104-B(12):1292–1303.

Prediction of Admission Costs Following Anterior Cervical Discectomy and Fusion Utilizing Machine Learning

STUDY DESIGN

Retrospective case series.

OBJECTIVE

Predict cost following anterior cervical discectomy and fusion (ACDF) within the 90-day global period using machine learning models.

BACKGROUND

The incidence of ACDF has been increasing with a disproportionate decrease in reimbursement. As bundled payment models become common, it is imperative to identify factors that impact the cost of care.

MATERIALS AND METHODS

The Nationwide Readmissions Database (NRD) was accessed in 2018 for all primary ACDFs by the International Classification of Diseases 10th Revision (ICD-10) procedure codes. Costs were calculated by utilizing the total hospital charge and each hospital's cost-to-charge ratio. Hospital characteristics, such as volume of procedures performed and wage index, were also queried. Readmissions within 90 days were identified, and cost of readmissions was added to the total admission cost to represent the 90-day healthcare cost. Machine learning algorithms were used to predict patients with 90-day admission costs >1 SD from the mean.

RESULTS

There were 42,485 procedures included in this investigation with an average age of 57.7±12.3 years with 50.6% males. The average cost of the operative admission was $24,874±25,610, the average cost of readmission was $25,371±11,476, and the average total cost was $26,977±28,947 including readmissions costs. There were 10,624 patients who were categorized as high cost. Wage index, hospital volume, age, and diagnosis-related group severity were most correlated with the total cost of care. Gradient boosting trees algorithm was most predictive of the total cost of care (area under the curve=0.86).

CONCLUSIONS

Bundled payment models utilize wage index and diagnosis-related groups to determine reimbursement of ACDF. However, machine learning algorithms identified additional variables, such as hospital volume, readmission, and patient age, that are also important for determining the cost of care. Machine learning can improve cost-effectiveness and reduce the financial burden placed upon physicians and hospitals by implementing patient-specific reimbursement.

Artificial intelligence accurately identifies total hip arthroplasty implants: a tool for revision surgery

BACKGROUND

A critical part in preoperative planning for revision arthroplasty surgery involves the identification of the failed implant. Using a predictive artificial neural network (ANN) model, the objectives of this study were: (1) to develop a machine-learning algorithm using operative big data to identify an implant from a radiograph; and (2) to compare algorithms that optimise accuracy in a timely fashion.

METHODS

Using 2116 postoperative anteroposterior (AP) hip radiographs of total hip arthroplasties from 2002 to 2019, 10 artificial neural networks were modeled and trained to classify the radiograph according to the femoral stem implanted. Stem brand and model was confirmed with 1594 operative reports. Model performance was determined by classification accuracy toward a random 706 AP hip radiographs, and again on a consecutive series of 324 radiographs prospectively collected over 2019.

RESULTS

The Dense-Net 201 architecture outperformed all others with 100.00% accuracy in training data, 95.15% accuracy on validation data, and 91.16% accuracy in the unique prospective series of patients. This outperformed all other models on the validation (p < 0.0001) and novel series (p < 0.0001). The convolutional neural network also displayed the probability (confidence) of the femoral stem classification for any input radiograph. This neural network averaged a runtime of 0.96 (SD 0.02) seconds for an iPhone 6 to calculate from a given radiograph when converted to an application.

CONCLUSIONS

Neural networks offer a useful adjunct to the surgeon in preoperative identification of the prior implant.

Comparative Analysis of the Ability of Machine Learning Models in Predicting In-hospital Postoperative Outcomes After Total Hip Arthroplasty

BACKGROUND

Machine learning (ML) methods have shown promise in a wide range of applications including the development of patient-specific predictive models before surgical interventions. The purpose of this study was to develop, test, and compare four distinct ML models to predict postoperative parameters after primary total hip arthroplasty.

METHODS

Data from the Nationwide Inpatient Sample were used to identify patients undergoing total hip arthroplasty from 2016 to 2017. Linear support vector machine (LSVM), random forest (RF), neural network (NN), and extreme gradient boost trees (XGBoost) predictive of mortality, length of stay, and discharge disposition were developed and validated using 15 predictive patient-specific and hospital-specific factors. Area under the curve of the receiver operating characteristic (AUCROC) curve and accuracy were used as validity metrics, and the strongest predictive variables under each model were assessed.

RESULTS

A total of 177,442 patients were included in this analysis. For mortality, the XGBoost, NN, and LSVM models all had excellent responsiveness during validation while RF had fair responsiveness. LSVM had the highest responsiveness with an AUCROC of 0.973 during validation. For the length of stay, the LSVM and NN models had fair responsiveness while the XGBoost and random forest models had poor responsiveness. LSVM had the highest responsiveness with an AUCROC of 0.744 during validation. For the discharge disposition outcome, LSVM had good responsiveness while the XGBoost, NN, and RF models all had fair responsiveness. LSVM had the highest responsiveness with an AUCROC of 0.801.

DISCUSSION

The ML methods tested demonstrated a range of poor-to-excellent responsiveness and accuracy in the prediction of the assessed metrics, with LSVM being the best performer. Such models should be further developed, with eventual integration into clinical practice to inform patient discussions and management decision making, with the potential for integration into tiered bundled payment models.

Meaningless Applications and Misguided Methodologies in Artificial Intelligence-Related Orthopaedic Research Propagates Hype Over Hope

There exists great hope and hype in the literature surrounding applications of artificial intelligence (AI) to orthopaedic surgery. Between 2018 and 2021, a total of 178 AI-related articles were published in orthopaedics. However, for every 2 original research papers that apply AI to orthopaedics, a commentary or review is published (30.3%). AI-related research in orthopaedics frequently fails to provide use cases that offer the uninitiated an opportunity to appraise the importance of AI by studying meaningful questions, evaluating unknown hypotheses, or analyzing quality data. The hype perpetuates a feed-forward cycle that relegates AI to a meaningless buzzword by rewarding those with nascent understanding and rudimentary technical knowhow into committing several basic errors: (1) inappropriately conflating vernacular ("AI/machine learning"), (2) repackaging registry data, (3) prematurely releasing internally validated algorithms, (4) overstating the "black box phenomenon" by failing to provide weighted analysis, (5) claiming to evaluate AI rather than the data itself, and (6) withholding full model architecture code. Relevant AI-specific guidelines are forthcoming, but forced application of the original Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis guidelines designed for regression analyses is irrelevant and misleading. To safeguard meaningful use, AI-related research efforts in orthopaedics should be (1) directed toward administrative support over clinical evaluation and management, (2) require the use of the advanced model, and (3) answer a question that was previously unknown, unanswered, or unquantifiable.

Artificial Learning and Machine Learning Applications in Spine Surgery: A Systematic Review

OBJECTIVES

This current systematic review sought to identify and evaluate all current research-based spine surgery applications of AI/ML in optimizing preoperative patient selection, as well as predicting and managing postoperative outcomes and complications.

METHODS

A comprehensive search of publications was conducted through the EMBASE, Medline, and PubMed databases using relevant keywords to maximize the sensitivity of the search. No limits were placed on level of evidence or timing of the study. Findings were reported according to the PRISMA guidelines.

RESULTS

After application of inclusion and exclusion criteria, 41 studies were included in this review. Bayesian networks had the highest average AUC (.80), and neural networks had the best accuracy (83.0%), sensitivity (81.5%), and specificity (71.8%). Preoperative planning/cost prediction models (.89,82.2%) and discharge/length of stay models (.80,78.0%) each reported significantly higher average AUC and accuracy compared to readmissions/reoperation prediction models (.67,70.2%) (P < .001, P = .005, respectively). Model performance also significantly varied across postoperative management applications for average AUC and accuracy values (P < .001, P < .027, respectively).

CONCLUSIONS

Generally, authors of the reviewed studies concluded that AI/ML offers a potentially beneficial tool for providers to optimize patient care and improve cost-efficiency. More specifically, AI/ML models performed best, on average, when optimizing preoperative patient selection and planning and predicting costs, hospital discharge, and length of stay. However, models were not as accurate in predicting postoperative complications, adverse events, and readmissions and reoperations. An understanding of AI/ML-based applications is becoming increasingly important, particularly in spine surgery, as the volume of reported literature, technology accessibility, and clinical applications continue to rapidly expand.

[Artificial intelligence and novel approaches for treatment of non-union in bone : From established standard methods in medicine up to novel fields of research]

Methods of artificial intelligence (AI) have found applications in many fields of medicine within the last few years. Some disciplines already use these methods regularly within their clinical routine. However, the fields of application are wide and there are still many opportunities to apply these new AI concepts. This review article gives an insight into the history of AI and defines the special terms and fields, such as machine learning (ML), neural networks and deep learning. The classical steps in developing AI models are demonstrated here, as well as the iteration of data rectification and preparation, the training of a model and subsequent validation before transfer into a clinical setting are explained. Currently, musculoskeletal disciplines implement methods of ML and also neural networks, e.g. for identification of fractures or for classifications. Also, predictive models based on risk factor analysis for prevention of complications are being initiated. As non-union in bone is a rare but very complex disease with dramatic socioeconomic impact for the healthcare system, many open questions arise which could be better understood by using methods of AI in the future. New fields of research applying AI models range from predictive models and cost analysis to personalized treatment strategies.

Machine learning algorithms predict extended postoperative opioid use in primary total knee arthroplasty

PURPOSE

Adequate postoperative pain control following total knee arthroplasty (TKA) is required to achieve optimal patient recovery. However, the postoperative recovery may lead to an unnaturally extended opioid use, which has been associated with adverse outcomes. This study hypothesizes that machine learning models can accurately predict extended opioid use following primary TKA.

METHODS

A total of 8873 consecutive patients that underwent primary TKA were evaluated, including 643 patients (7.2%) with extended postoperative opioid use (> 90 days). Electronic patient records were manually reviewed to identify patient demographics and surgical variables associated with prolonged postoperative opioid use. Five machine learning algorithms were developed, encompassing the breadth of state-of-the-art machine learning algorithms available in the literature, to predict extended opioid use following primary TKA, and these models were assessed by discrimination, calibration, and decision curve analysis.

RESULTS

The strongest predictors for prolonged opioid prescription following primary TKA were preoperative opioid duration (100% importance; p < 0.01), drug abuse (54% importance; p < 0.01), and depression (47% importance; p < 0.01). The five machine learning models all achieved excellent performance across discrimination (AUC > 0.83), calibration, and decision curve analysis. Higher net benefits for all machine learning models were demonstrated, when compared to the default strategies of changing management for all patients or no patients.

CONCLUSION

The study findings show excellent model performance for the prediction of extended postoperative opioid use following primary total knee arthroplasty, highlighting the potential of these models to assist in preoperatively identifying at risk patients, and allowing the implementation of individualized peri-operative counselling and pain management strategies to mitigate complications associated with prolonged opioid use.

LEVEL OF EVIDENCE

IV.

Machine Learning for Predicting Lower Extremity Muscle Strain in National Basketball Association Athletes

Background

In professional sports, injuries resulting in loss of playing time have serious implications for both the athlete and the organization. Efforts to quantify injury probability utilizing machine learning have been met with renewed interest, and the development of effective models has the potential to supplement the decision-making process of team physicians.

Purpose/Hypothesis

The purpose of this study was to (1) characterize the epidemiology of time-loss lower extremity muscle strains (LEMSs) in the National Basketball Association (NBA) from 1999 to 2019 and (2) determine the validity of a machine-learning model in predicting injury risk. It was hypothesized that time-loss LEMSs would be infrequent in this cohort and that a machine-learning model would outperform conventional methods in the prediction of injury risk.

Study Design

Case-control study; Level of evidence, 3.

Methods

Performance data and rates of the 4 major muscle strain injury types (hamstring, quadriceps, calf, and groin) were compiled from the 1999 to 2019 NBA seasons. Injuries included all publicly reported injuries that resulted in lost playing time. Models to predict the occurrence of a LEMS were generated using random forest, extreme gradient boosting (XGBoost), neural network, support vector machines, elastic net penalized logistic regression, and generalized logistic regression. Performance was compared utilizing discrimination, calibration, decision curve analysis, and the Brier score.

Results

A total of 736 LEMSs resulting in lost playing time occurred among 2103 athletes. Important variables for predicting LEMS included previous number of lower extremity injuries; age; recent history of injuries to the ankle, hamstring, or groin; and recent history of concussion as well as 3-point attempt rate and free throw attempt rate. The XGBoost machine achieved the best performance based on discrimination assessed via internal validation (area under the receiver operating characteristic curve, 0.840), calibration, and decision curve analysis.

Conclusion

Machine learning algorithms such as XGBoost outperformed logistic regression in the prediction of a LEMS that will result in lost time. Several variables increased the risk of LEMS, including a history of various lower extremity injuries, recent concussion, and total number of previous injuries.

Machine Learning With Electronic Health Record Data Outperforms a Risk Assessment Prediction Tool in Predicting Discharge Disposition After Total Joint Arthroplasty

The Risk Assessment Prediction Tool (RAPT) predicts discharge disposition after total joint arthroplasty with only 75% accuracy. The goal of this study was to evaluate whether higher accuracy can be achieved with basic electronic health record (EHR) data combined with machine learning (ML) algorithms. Three ML analysis models were developed: model 1 (M1) evaluated the accuracy of predicted discharge disposition in concordance with the RAPT; model 2 (M2) used the RAPT questionnaire to develop an ML algorithm to predict the likelihood of discharge to home vs facility; and model 3 (M3) was developed with non-RAPT data (age, surgeon, and discharge preference) with the same ML training process as M2. Evaluation metrics included accuracy for home discharge (HD), positive predictive value for HD (PPV-HD), negative predictive value for HD (NPV-HD), sensitivity, specificity, and area under the receiver operating curve (AUROC). A total of 1405 patients were included. With M1, the overall accuracy for HD was 83.5%, PPVHD was 92.1%, NPV-HD was 45%, sensitivity was 0.88, and specificity was 0.56. With M2, the overall accuracy for HD decreased to 82.8%, PPV-HD was 91.7%, NPV-HD was 43.1%, sensitivity was 0.87, specificity was 0.53, and mean AUROC was 0.87±0.03. With M3, overall accuracy for HD increased to 90.3%, PPV-HD was 95.2%, NPV-HD was 68.6%, sensitivity was 0.93, specificity was 0.76, and AUROC was 0.91±0.02. The use of basic EHR data combined with ML can exceed the accuracy of the RAPT. Applying big data on an individual level for this purpose may allow for safer and more appropriate discharge planning. [Orthopedics. 2022;45(4):e211-e215.].

Performance of Artificial Intelligence-Based Algorithms to Predict Prolonged Length of Stay after Lumbar Decompression Surgery

Background: Decompression of the lumbar spine is one of the most common procedures performed in spine surgery. Hospital length of stay (LOS) is a clinically relevant metric used to assess surgical success, patient outcomes, and socioeconomic impact. This study aimed to investigate a variety of machine learning and deep learning algorithms to reliably predict whether a patient undergoing decompression of lumbar spinal stenosis will experience a prolonged LOS. Methods: Patients undergoing treatment for lumbar spinal stenosis with microsurgical and full-endoscopic decompression were selected within this retrospective monocentric cohort study. Prolonged LOS was defined as an LOS greater than or equal to the 75th percentile of the cohort (normal versus prolonged stay; binary classification task). Unsupervised learning with K-means clustering was used to find clusters in the data. Hospital stay classes were predicted with logistic regression, RandomForest classifier, stochastic gradient descent (SGD) classifier, K-nearest neighbors, Decision Tree classifier, Gaussian Naive Bayes (GaussianNB), support vector machines (SVM), a custom-made convolutional neural network (CNN), multilayer perceptron artificial neural network (MLP), and radial basis function neural network (RBNN) in Python. Prediction accuracy and area under the curve (AUC) were calculated. Feature importance analysis was utilized to find the most important predictors. Further, we developed a decision tree based on the Chi-square automatic interaction detection (CHAID) algorithm to investigate cut-offs of predictors for clinical decision-making. Results: 236 patients and 14 feature variables were included. K-means clustering separated data into two clusters distinguishing the data into two patient risk characteristic groups. The algorithms reached AUCs between 67.5% and 87.3% for the classification of LOS classes. Feature importance analysis of deep learning algorithms indicated that operation time was the most important feature in predicting LOS. A decision tree based on CHAID could predict 84.7% of the cases. Conclusions: Machine learning and deep learning algorithms can predict whether patients will experience an increased LOS following lumbar decompression surgery. Therefore, medical resources can be more appropriately allocated to patients who are at risk of prolonged LOS.

The current role of the virtual elements of artificial intelligence in total knee arthroplasty

The current applications of the virtual elements of artificial intelligence (AI), machine learning (ML), and deep learning (DL) in total knee arthroplasty (TKA) are diverse.

ML can predict the length of stay (LOS) and costs before primary TKA, the risk of transfusion after primary TKA, postoperative dissatisfaction after TKA, the size of TKA components, and poorest outcomes. The prediction of distinct results with ML models applying specific data is already possible; nevertheless, the prediction of more complex results is still imprecise. Remote patient monitoring systems offer the ability to more completely assess the individuals experiencing TKA in terms of mobility and rehabilitation compliance.

DL can accurately identify the presence of TKA, distinguish between specific arthroplasty designs, and identify and classify knee osteoarthritis as accurately as an orthopedic surgeon. DL allows for the detection of prosthetic loosening from radiographs.

Regarding the architectures associated with DL, artificial neural networks (ANNs) and convolutional neural networks (CNNs), ANNs can predict LOS, inpatient charges, and discharge disposition prior to primary TKA and CNNs allow for differentiation between different implant types with near-perfect accuracy.

The Utility of Machine Learning Algorithms for the Prediction of Early Revision Surgery After Primary Total Hip Arthroplasty

BACKGROUND

Revision total hip arthroplasty (THA) is associated with increased morbidity, mortality, and healthcare costs due to a technically more demanding surgical procedure when compared with primary THA. Therefore, a better understanding of risk factors for early revision THA is essential to develop strategies for mitigating the risk of patients undergoing early revision. This study aimed to develop and validate novel machine learning (ML) models for the prediction of early revision after primary THA.

METHODS

A total of 7,397 consecutive patients who underwent primary THA were evaluated, including 566 patients (6.6%) with confirmed early revision THA (<2 years from index THA). Electronic patient records were manually reviewed to identify patient demographics, implant characteristics, and surgical variables that may be associated with early revision THA. Six ML algorithms were developed to predict early revision THA, and these models were assessed by discrimination, calibration, and decision curve analysis.

RESULTS

The strongest predictors for early revision after primary THA were Charlson Comorbidity Index, body mass index >35 kg/m2, and depression. The six ML models all achieved excellent performance across discrimination (area under the curve >0.80), calibration, and decision curve analysis.

CONCLUSION

This study developed ML models for the prediction of early revision surgery for patients after primary THA. The study findings show excellent performance on discrimination, calibration, and decision curve analysis for all six candidate models, highlighting the potential of these models to assist in clinical practice patient-specific preoperative quantification of increased risk of early revision THA.

Efficacy and Applications of Artificial Intelligence and Machine Learning Analyses in Total Joint Arthroplasty: A Call for Improved Reporting

BACKGROUND

There has been a considerable increase in total joint arthroplasty (TJA) research using machine learning (ML). Therefore, the purposes of this study were to synthesize the applications and efficacies of ML reported in the TJA literature, and to assess the methodological quality of these studies.

METHODS

PubMed, OVID/MEDLINE, and Cochrane libraries were queried in January 2021 for articles regarding the use of ML in TJA. Study demographics, topic, primary and secondary outcomes, ML model development and testing, and model presentation and validation were recorded. The TRIPOD (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis) guidelines were used to assess the methodological quality.

RESULTS

Fifty-five studies were identified: 31 investigated clinical outcomes and resource utilization; 11, activity and motion surveillance; 10, imaging detection; and 3, natural language processing. For studies reporting the area under the receiver operating characteristic curve (AUC), the median AUC (and range) was 0.80 (0.60 to 0.97) among 26 clinical outcome studies, 0.99 (0.83 to 1.00) among 6 imaging-based studies, and 0.88 (0.76 to 0.98) among 3 activity and motion surveillance studies. Twelve studies compared ML to logistic regression, with 9 (75%) reporting that ML was superior. The average number of TRIPOD guidelines met was 11.5 (range: 5 to 18), with 38 (69%) meeting greater than half of the criteria. Presentation and explanation of the full model for individual predictions and assessments of model calibration were poorly reported (<30%).

CONCLUSIONS

The performance of ML models was good to excellent when applied to a wide variety of clinically relevant outcomes in TJA. However, reporting of certain key methodological and model presentation criteria was inadequate. Despite the recent surge in TJA literature utilizing ML, the lack of consistent adherence to reporting guidelines needs to be addressed to bridge the gap between model development and clinical implementation.

Can machine learning models predict failure of revision total hip arthroplasty?

INTRODUCTION

Revision total hip arthroplasty (THA) represents a technically demanding surgical procedure which is associated with significant morbidity and mortality. Understanding risk factors for failure of revision THA is of clinical importance to identify at-risk patients. This study aimed to develop and validate novel machine learning algorithms for the prediction of re-revision surgery for patients following revision total hip arthroplasty.

METHODS

A total of 2588 consecutive patients that underwent revision THA was evaluated, including 408 patients (15.7%) with confirmed re-revision THA. Electronic patient records were manually reviewed to identify patient demographics, implant characteristics and surgical variables that may be associated with re-revision THA. Machine learning algorithms were developed to predict re-revision THA and these models were assessed by discrimination, calibration and decision curve analysis.

RESULTS

The strongest predictors for re-revision THA as predicted by the four validated machine learning models were the American Society of Anaesthesiology score, obesity (> 35 kg/m²) and indication for revision THA. The four machine learning models all achieved excellent performance across discrimination (AUC > 0.80), calibration and decision curve analysis. Higher net benefits for all machine learning models were demonstrated, when compared to the default strategies of changing management for all patients or no patients.

CONCLUSION

This study developed four machine learning models for the prediction of re-revision surgery for patients following revision total hip arthroplasty. The study findings show excellent model performance, highlighting the potential of these computational models to assist in preoperative patient optimization and counselling to improve revision THA patient outcomes.

LEVEL OF EVIDENCE

Level III, case-control retrospective analysis.

Artificial intelligence in knee arthroplasty: current concept of the available clinical applications

Background

Artificial intelligence (AI) is defined as the study of algorithms that allow machines to reason and perform cognitive functions such as problem-solving, objects, images, word recognition, and decision-making. This study aimed to review the published articles and the comprehensive clinical relevance of AI-based tools used before, during, and after knee arthroplasty.

Methods

The search was conducted through PubMed, EMBASE, and MEDLINE databases from 2000 to 2021 using the 2009 Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocol (PRISMA).

Results

A total of 731 potential articles were reviewed, and 132 were included based on the inclusion criteria and exclusion criteria. Some steps of the knee arthroplasty procedure were assisted and improved by using AI-based tools. Before surgery, machine learning was used to aid surgeons in optimizing decision-making. During surgery, the robotic-assisted systems improved the accuracy of knee alignment, implant positioning, and ligamentous balance. After surgery, remote patient monitoring platforms helped to capture patients’ functional data.

Conclusion

In knee arthroplasty, the AI-based tools improve the decision-making process, surgical planning, accuracy, and repeatability of surgical procedures.

Predictive capacity of four machine learning models for in-hospital postoperative outcomes following total knee arthroplasty

Background

Machine learning (ML) methods have shown promise in the development of patient-specific predictive models prior to surgical interventions. The purpose of this study was to develop, test, and compare four distinct ML models to predict postoperative parameters following primary total knee arthroplasty (TKA).

Methods

Data from the Nationwide Inpatient Sample was used to identify patients undergoing TKA during 2016-2017. Four distinct ML models predictive of mortality, length of stay (LOS), and discharge disposition were developed and validated using 15 predictive patient and hospital-specific factors. Area under the curve of the receiver operating characteristic curve (AUCROC) and accuracy were used as validity metrics, and the strongest predictive variables under each model were assessed.

Results

A total of 305,577 patients were included. For mortality, the XGBoost, neural network (NN), and LSVM models all had excellent responsiveness during validation, while random forest (RF) had fair responsiveness. For predicting LOS, all four models had poor responsiveness. For the discharge disposition outcome, the LSVM, NN, and XGBoost models had good responsiveness, while the RF model had poor responsiveness. LSVM and XGBoost had the highest responsiveness for predicting discharge disposition with an AUCROC of 0.747.

Discussion

The ML models tested demonstrated a range of poor to excellent responsiveness and accuracy in the prediction of the assessed metrics, with considerable variability noted in the predictive precision between the models. The continued development of ML models should be encouraged, with eventual integration into clinical practice in order to inform patient discussions, management decision making, and health policy.