1
|
van Spanning SH, Verweij LPE, Hendrickx LAM, Allaart LJH, Athwal GS, Lafosse T, Lafosse L, Doornberg JN, Oosterhoff JHF, van den Bekerom MPJ, Alexander Buijze G. Methodology and development of a machine learning probability calculator: Data heterogeneity limits ability to predict recurrence after arthroscopic Bankart repair. Knee Surg Sports Traumatol Arthrosc 2024. [PMID: 39324357 DOI: 10.1002/ksa.12443] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/13/2024] [Revised: 08/02/2024] [Accepted: 08/02/2024] [Indexed: 09/27/2024]
Abstract
PURPOSE The aim of this study was to develop and train a machine learning (ML) algorithm to create a clinical decision support tool (i.e., ML-driven probability calculator) to be used in clinical practice to estimate recurrence rates following an arthroscopic Bankart repair (ABR). METHODS Data from 14 previously published studies were collected. Inclusion criteria were (1) patients treated with ABR without remplissage for traumatic anterior shoulder instability and (2) a minimum of 2 years follow-up. Risk factors associated with recurrence were identified using bivariate logistic regression analysis. Subsequently, four ML algorithms were developed and internally validated. The predictive performance was assessed using discrimination, calibration and the Brier score. RESULTS In total, 5591 patients underwent ABR with a recurrence rate of 15.4% (n = 862). Age <35 years, participation in contact and collision sports, bony Bankart lesions and full-thickness rotator cuff tears increased the risk of recurrence (all p < 0.05). A single shoulder dislocation (compared to multiple dislocations) lowered the risk of recurrence (p < 0.05). Due to the unavailability of certain variables in some patients, a portion of the patient data had to be excluded before pooling the data set to create the algorithm. A total of 797 patients were included providing information on risk factors associated with recurrence. The discrimination (area under the receiver operating curve) ranged between 0.54 and 0.57 for prediction of recurrence. CONCLUSION ML was not able to predict the recurrence following ABR with the current available predictors. Despite a global coordinated effort, the heterogeneity of clinical data limited the predictive capabilities of the algorithm, emphasizing the need for standardized data collection methods in future studies. LEVEL OF EVIDENCE Level IV, retrospective cohort study.
Collapse
Affiliation(s)
- Sanne H van Spanning
- Alps Surgery Institute, Hand, Upper Limb, Peripheral Nerve, Brachial Plexus and Microsurgery Unit, Clinique Générale, Annecy, France
- Amsterdam Shoulder and Elbow Centre of Expertise (ASECE), Amsterdam, the Netherlands
- Department of Human Movement Sciences, Faculty of Behavioural and Movement Sciences, Vrije Universiteit Amsterdam, Amsterdam Movement Sciences, Amsterdam, the Netherlands
- Department of Orthopedic Surgery, OLVG, Shoulder and Elbow Unit, Amsterdam, the Netherlands
| | - Lukas P E Verweij
- Department of Human Movement Sciences, Faculty of Behavioural and Movement Sciences, Vrije Universiteit Amsterdam, Amsterdam Movement Sciences, Amsterdam, the Netherlands
- Amsterdam Movement Sciences, Musculoskeletal Health Program, Amsterdam, the Netherlands
- Department of Amsterdam UMC, Department of Orthopedic Surgery and Sports Medicine, Location AMC, University of Amsterdam, Amsterdam, the Netherlands
| | - Laurent A M Hendrickx
- Department of Amsterdam UMC, Department of Orthopedic Surgery and Sports Medicine, Location AMC, University of Amsterdam, Amsterdam, the Netherlands
- Department of Orthopaedic & Trauma Surgery, Flinders Medical Centre, Flinders University, Adelaide, South Australia, Australia
| | - Laurens J H Allaart
- Alps Surgery Institute, Hand, Upper Limb, Peripheral Nerve, Brachial Plexus and Microsurgery Unit, Clinique Générale, Annecy, France
- Amsterdam Shoulder and Elbow Centre of Expertise (ASECE), Amsterdam, the Netherlands
- Department of Human Movement Sciences, Faculty of Behavioural and Movement Sciences, Vrije Universiteit Amsterdam, Amsterdam Movement Sciences, Amsterdam, the Netherlands
| | - George S Athwal
- Roth McFarlane Hand and Upper Limb Centre, Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada
| | - Thibault Lafosse
- Alps Surgery Institute, Hand, Upper Limb, Peripheral Nerve, Brachial Plexus and Microsurgery Unit, Clinique Générale, Annecy, France
| | - Laurent Lafosse
- Alps Surgery Institute, Hand, Upper Limb, Peripheral Nerve, Brachial Plexus and Microsurgery Unit, Clinique Générale, Annecy, France
| | - Job N Doornberg
- Department of Orthopaedic & Trauma Surgery, Flinders Medical Centre, Flinders University, Adelaide, South Australia, Australia
- Department of Orthopaedic and Trauma Surgery, University Medical Center Groningen, University of Groningen, Groningen, the Netherlands
| | - Jacobien H F Oosterhoff
- Department of Engineering Systems and Services, Faculty Technology Policy and Management, Delft University of Technology, Delft, the Netherlands
| | - Michel P J van den Bekerom
- Department of Human Movement Sciences, Faculty of Behavioural and Movement Sciences, Vrije Universiteit Amsterdam, Amsterdam Movement Sciences, Amsterdam, the Netherlands
- Department of Orthopedic Surgery, OLVG, Shoulder and Elbow Unit, Amsterdam, the Netherlands
- Amsterdam Movement Sciences, Musculoskeletal Health Program, Amsterdam, the Netherlands
| | - Geert Alexander Buijze
- Alps Surgery Institute, Hand, Upper Limb, Peripheral Nerve, Brachial Plexus and Microsurgery Unit, Clinique Générale, Annecy, France
- Department of Amsterdam UMC, Department of Orthopedic Surgery and Sports Medicine, Location AMC, University of Amsterdam, Amsterdam, the Netherlands
- Department of Orthopedic Surgery, Montpellier University Medical Centre, Lapeyronie Hospital, University of Montpellier, Montpellier, France
| |
Collapse
|
2
|
Oosterhoff JHF, de Hond AAH, Peters RM, van Steenbergen LN, Sorel JC, Zijlstra WP, Poolman RW, Ring D, Jutte PC, Kerkhoffs GMMJ, Putter H, Steyerberg EW, Doornberg JN. Machine Learning Did Not Outperform Conventional Competing Risk Modeling to Predict Revision Arthroplasty. Clin Orthop Relat Res 2024; 482:1472-1482. [PMID: 38470976 PMCID: PMC11272341 DOI: 10.1097/corr.0000000000003018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/11/2023] [Accepted: 02/01/2024] [Indexed: 03/14/2024]
Abstract
BACKGROUND Estimating the risk of revision after arthroplasty could inform patient and surgeon decision-making. However, there is a lack of well-performing prediction models assisting in this task, which may be due to current conventional modeling approaches such as traditional survivorship estimators (such as Kaplan-Meier) or competing risk estimators. Recent advances in machine learning survival analysis might improve decision support tools in this setting. Therefore, this study aimed to assess the performance of machine learning compared with that of conventional modeling to predict revision after arthroplasty. QUESTION/PURPOSE Does machine learning perform better than traditional regression models for estimating the risk of revision for patients undergoing hip or knee arthroplasty? METHODS Eleven datasets from published studies from the Dutch Arthroplasty Register reporting on factors associated with revision or survival after partial or total knee and hip arthroplasty between 2018 and 2022 were included in our study. The 11 datasets were observational registry studies, with a sample size ranging from 3038 to 218,214 procedures. We developed a set of time-to-event models for each dataset, leading to 11 comparisons. A set of predictors (factors associated with revision surgery) was identified based on the variables that were selected in the included studies. We assessed the predictive performance of two state-of-the-art statistical time-to-event models for 1-, 2-, and 3-year follow-up: a Fine and Gray model (which models the cumulative incidence of revision) and a cause-specific Cox model (which models the hazard of revision). These were compared with a machine-learning approach (a random survival forest model, which is a decision tree-based machine-learning algorithm for time-to-event analysis). Performance was assessed according to discriminative ability (time-dependent area under the receiver operating curve), calibration (slope and intercept), and overall prediction error (scaled Brier score). Discrimination, known as the area under the receiver operating characteristic curve, measures the model's ability to distinguish patients who achieved the outcomes from those who did not and ranges from 0.5 to 1.0, with 1.0 indicating the highest discrimination score and 0.50 the lowest. Calibration plots the predicted versus the observed probabilities; a perfect plot has an intercept of 0 and a slope of 1. The Brier score calculates a composite of discrimination and calibration, with 0 indicating perfect prediction and 1 the poorest. A scaled version of the Brier score, 1 - (model Brier score/null model Brier score), can be interpreted as the amount of overall prediction error. RESULTS Using machine learning survivorship analysis, we found no differences between the competing risks estimator and traditional regression models for patients undergoing arthroplasty in terms of discriminative ability (patients who received a revision compared with those who did not). We found no consistent differences between the validated performance (time-dependent area under the receiver operating characteristic curve) of different modeling approaches because these values ranged between -0.04 and 0.03 across the 11 datasets (the time-dependent area under the receiver operating characteristic curve of the models across 11 datasets ranged between 0.52 to 0.68). In addition, the calibration metrics and scaled Brier scores produced comparable estimates, showing no advantage of machine learning over traditional regression models. CONCLUSION Machine learning did not outperform traditional regression models. CLINICAL RELEVANCE Neither machine learning modeling nor traditional regression methods were sufficiently accurate in order to offer prognostic information when predicting revision arthroplasty. The benefit of these modeling approaches may be limited in this context.
Collapse
Affiliation(s)
- Jacobien H. F. Oosterhoff
- Amsterdam UMC, University of Amsterdam, Department of Orthopedic Surgery and Sports Medicine, Amsterdam, the Netherlands
- Department of Engineering Systems and Services, Faculty of Technology Policy and Management, Delft University of Technology, Delft, the Netherlands
| | - Anne A. H. de Hond
- Clinical AI Implementation and Research Lab, Leiden University Medical Center, Leiden, the Netherlands
- Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, the Netherlands
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
| | - Rinne M. Peters
- Department of Orthopaedic Surgery, Medical Center Leeuwarden, Leeuwarden, the Netherlands
| | | | - Juliette C. Sorel
- Department of Orthopaedic Surgery, Leiden University Medical Centre, Leiden, the Netherlands
| | - Wierd P. Zijlstra
- Department of Orthopaedic Surgery, Medical Center Leeuwarden, Leeuwarden, the Netherlands
| | - Rudolf W. Poolman
- Department of Orthopaedic Surgery, Leiden University Medical Centre, Leiden, the Netherlands
| | - David Ring
- Department of Surgery and Perioperative Care, Dell Medical School, University of Texas, Austin, TX, USA
| | - Paul C. Jutte
- Department of Orthopaedic and Trauma Surgery, University Medical Center Groningen, University of Groningen, the Netherlands
| | - Gino M. M. J. Kerkhoffs
- Amsterdam UMC, University of Amsterdam, Department of Orthopedic Surgery and Sports Medicine, Amsterdam, the Netherlands
| | - Hein Putter
- Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, the Netherlands
| | - Ewout W. Steyerberg
- Clinical AI Implementation and Research Lab, Leiden University Medical Center, Leiden, the Netherlands
- Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, the Netherlands
| | - Job N. Doornberg
- Department of Orthopaedic and Trauma Surgery, University Medical Center Groningen, University of Groningen, the Netherlands
| | | |
Collapse
|
3
|
Gutierrez-Naranjo JM, Moreira A, Valero-Moreno E, Bullock TS, Ogden LA, Zelle BA. -A machine learning model to predict surgical site infection after surgery of lower extremity fractures. INTERNATIONAL ORTHOPAEDICS 2024; 48:1887-1896. [PMID: 38700699 DOI: 10.1007/s00264-024-06194-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/23/2023] [Accepted: 04/22/2024] [Indexed: 06/14/2024]
Abstract
PURPOSE This study aimed to develop machine learning algorithms for identifying predictive factors associated with the risk of postoperative surgical site infection in patients with lower extremity fractures. METHODS A machine learning analysis was conducted on a dataset comprising 1,579 patients who underwent surgical fixation for lower extremity fractures to create a predictive model for risk stratification of postoperative surgical site infection. We evaluated different clinical and demographic variables to train four machine learning models (neural networks, boosted generalised linear model, naïve bayes, and penalised discriminant analysis). Performance was measured by the area under the curve score, Youdon's index and Brier score. A multivariate adaptive regression splines (MARS) was used to optimise predictor selection. RESULTS The final model consisted of five predictors. (1) Operating room time, (2) ankle region, (3) open injury, (4) body mass index, and (5) age. The best-performing machine learning algorithm demonstrated a promising predictive performance, with an area under the ROC curve, Youdon's index, and Brier score of 77.8%, 62.5%, and 5.1%-5.6%, respectively. CONCLUSION The proposed predictive model not only assists surgeons in determining high-risk factors for surgical site infections but also empowers patients to closely monitor these factors and take proactive measures to prevent complications. Furthermore, by considering the identified predictors, this model can serve as a reference for implementing preventive measures and reducing postoperative complications, ultimately enhancing patient outcomes. However, further investigations involving larger datasets and external validations are required to confirm the reliability and applicability of our model.
Collapse
Affiliation(s)
| | - Alvaro Moreira
- Department of Pediatrics, UT Health San Antonio, San Antonio, TX, USA.
| | | | - Travis S Bullock
- Department of Orthopaedics, UT Health San Antonio, San Antonio, TX, 78229-3900, USA
| | - Liliana A Ogden
- Department of Orthopaedics, UT Health San Antonio, San Antonio, TX, 78229-3900, USA
| | - Boris A Zelle
- Department of Orthopaedics, UT Health San Antonio, San Antonio, TX, 78229-3900, USA.
| |
Collapse
|
4
|
Kelly M. CORR Insights®: Can a Psychological Profile Predict Successful Return to Full Duty After a Musculoskeletal Injury? Clin Orthop Relat Res 2024; 482:630-632. [PMID: 38363558 PMCID: PMC10936998 DOI: 10.1097/corr.0000000000003008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/08/2023] [Accepted: 01/23/2024] [Indexed: 02/17/2024]
Affiliation(s)
- Michael Kelly
- Professor Of Clinical Orthopaedic Surgery, University of California San Diego, Department of Orthopaedic Surgery, San Diego, CA, USA
- Director of Scoliosis and Spinal Deformities, Division of Orthopedics & Scoliosis at Rady Children's Hospital-San Diego, San Diego, CA, USA
| |
Collapse
|
5
|
Dijkstra H, van de Kuit A, de Groot T, Canta O, Groot OQ, Oosterhoff JH, Doornberg JN. Systematic review of machine-learning models in orthopaedic trauma. Bone Jt Open 2024; 5:9-19. [PMID: 38226447 PMCID: PMC10790183 DOI: 10.1302/2633-1462.51.bjo-2023-0095.r1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/17/2024] Open
Abstract
Aims Machine-learning (ML) prediction models in orthopaedic trauma hold great promise in assisting clinicians in various tasks, such as personalized risk stratification. However, an overview of current applications and critical appraisal to peer-reviewed guidelines is lacking. The objectives of this study are to 1) provide an overview of current ML prediction models in orthopaedic trauma; 2) evaluate the completeness of reporting following the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement; and 3) assess the risk of bias following the Prediction model Risk Of Bias Assessment Tool (PROBAST) tool. Methods A systematic search screening 3,252 studies identified 45 ML-based prediction models in orthopaedic trauma up to January 2023. The TRIPOD statement assessed transparent reporting and the PROBAST tool the risk of bias. Results A total of 40 studies reported on training and internal validation; four studies performed both development and external validation, and one study performed only external validation. The most commonly reported outcomes were mortality (33%, 15/45) and length of hospital stay (9%, 4/45), and the majority of prediction models were developed in the hip fracture population (60%, 27/45). The overall median completeness for the TRIPOD statement was 62% (interquartile range 30 to 81%). The overall risk of bias in the PROBAST tool was low in 24% (11/45), high in 69% (31/45), and unclear in 7% (3/45) of the studies. High risk of bias was mainly due to analysis domain concerns including small datasets with low number of outcomes, complete-case analysis in case of missing data, and no reporting of performance measures. Conclusion The results of this study showed that despite a myriad of potential clinically useful applications, a substantial part of ML studies in orthopaedic trauma lack transparent reporting, and are at high risk of bias. These problems must be resolved by following established guidelines to instil confidence in ML models among patients and clinicians. Otherwise, there will remain a sizeable gap between the development of ML prediction models and their clinical application in our day-to-day orthopaedic trauma practice.
Collapse
Affiliation(s)
- Hidde Dijkstra
- Department of Orthopaedic Surgery, University Medical Centre Groningen, Groningen, Netherlands
- University Center for Geriatric Medicine, University of Groningen, University Medical Center Groningen, Groningen, Netherlands
| | - Anouk van de Kuit
- Department of Orthopaedic Surgery, University Medical Centre Groningen, Groningen, Netherlands
| | - Tom de Groot
- Department of Orthopaedic Surgery, University Medical Centre Groningen, Groningen, Netherlands
- Department of Orthopaedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA
| | - Olga Canta
- Department of Orthopaedic Surgery, University Medical Centre Groningen, Groningen, Netherlands
| | - Olivier Q. Groot
- Department of Orthopaedic Surgery, University Medical Centre Utrecht, University of Utrecht, Utrecht, Netherlands
| | - Jacobien H. Oosterhoff
- Department of Engineering Systems & Services, Faculty Technology Policy and Management, Delft University of Technology, Delft, Netherlands
| | - Job N. Doornberg
- Department of Orthopaedic Surgery, University Medical Centre Groningen, Groningen, Netherlands
- Department of Orthopaedic Trauma Surgery, Flinders Medical Center, Flinders University, Adelaide, Australia
| | | |
Collapse
|
6
|
Warren E, Hurley ET, Park CN, Crook BS, Lorentz S, Levin JM, Anakwenze O, MacDonald PB, Klifto CS. Evaluation of information from artificial intelligence on rotator cuff repair surgery. JSES Int 2024; 8:53-57. [PMID: 38312282 PMCID: PMC10837709 DOI: 10.1016/j.jseint.2023.09.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/06/2024] Open
Abstract
Purpose The purpose of this study was to analyze the quality and readability of information regarding rotator cuff repair surgery available using an online AI software. Methods An open AI model (ChatGPT) was used to answer 24 commonly asked questions from patients on rotator cuff repair. Questions were stratified into one of three categories based on the Rothwell classification system: fact, policy, or value. The answers for each category were evaluated for reliability, quality and readability using The Journal of the American Medical Association Benchmark criteria, DISCERN score, Flesch-Kincaid Reading Ease Score and Grade Level. Results The Journal of the American Medical Association Benchmark criteria score for all three categories was 0, which is the lowest score indicating no reliable resources cited. The DISCERN score was 51 for fact, 53 for policy, and 55 for value questions, all of which are considered good scores. Across question categories, the reliability portion of the DISCERN score was low, due to a lack of resources. The Flesch-Kincaid Reading Ease Score (and Flesch-Kincaid Grade Level) was 48.3 (10.3) for the fact class, 42.0 (10.9) for the policy class, and 38.4 (11.6) for the value class. Conclusion The quality of information provided by the open AI chat system was generally high across all question types but had significant shortcomings in reliability due to the absence of source material citations. The DISCERN scores of the AI generated responses matched or exceeded previously published results of studies evaluating the quality of online information about rotator cuff repairs. The responses were U.S. 10th grade or higher reading level which is above the AMA and NIH recommendation of 6th grade reading level for patient materials. The AI software commonly referred the user to seek advice from orthopedic surgeons to improve their chances of a successful outcome.
Collapse
Affiliation(s)
- Eric Warren
- Duke University School of Medicine, Duke University, Durham, NC, USA
| | - Eoghan T. Hurley
- Department of Orthopaedic Surgery, Duke University, Durham, NC, USA
| | - Caroline N. Park
- Department of Orthopaedic Surgery, Duke University, Durham, NC, USA
| | - Bryan S. Crook
- Department of Orthopaedic Surgery, Duke University, Durham, NC, USA
| | - Samuel Lorentz
- Department of Orthopaedic Surgery, Duke University, Durham, NC, USA
| | - Jay M. Levin
- Department of Orthopaedic Surgery, Duke University, Durham, NC, USA
| | - Oke Anakwenze
- Department of Orthopaedic Surgery, Duke University, Durham, NC, USA
| | - Peter B. MacDonald
- Section of Orthopaedic Surgery & The Pan Am Clinic, University of Manitoba, Winnipeg, MB, Canada
| | | |
Collapse
|
7
|
de Groot TM, Ramsey D, Groot OQ, Fourman M, Karhade AV, Twining PK, Berner EA, Fenn BP, Collins AK, Raskin K, Lozano S, Newman E, Ferrone M, Doornberg JN, Schwab JH. Does the SORG Machine-learning Algorithm for Extremity Metastases Generalize to a Contemporary Cohort of Patients? Temporal Validation From 2016 to 2020. Clin Orthop Relat Res 2023; 481:2419-2430. [PMID: 37229565 PMCID: PMC10642892 DOI: 10.1097/corr.0000000000002698] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Revised: 03/15/2023] [Accepted: 04/21/2023] [Indexed: 05/27/2023]
Abstract
BACKGROUND The ability to predict survival accurately in patients with osseous metastatic disease of the extremities is vital for patient counseling and guiding surgical intervention. We, the Skeletal Oncology Research Group (SORG), previously developed a machine-learning algorithm (MLA) based on data from 1999 to 2016 to predict 90-day and 1-year survival of surgically treated patients with extremity bone metastasis. As treatment regimens for oncology patients continue to evolve, this SORG MLA-driven probability calculator requires temporal reassessment of its accuracy. QUESTION/PURPOSE Does the SORG-MLA accurately predict 90-day and 1-year survival in patients who receive surgical treatment for a metastatic long-bone lesion in a more recent cohort of patients treated between 2016 and 2020? METHODS Between 2017 and 2021, we identified 674 patients 18 years and older through the ICD codes for secondary malignant neoplasm of bone and bone marrow and CPT codes for completed pathologic fractures or prophylactic treatment of an impending fracture. We excluded 40% (268 of 674) of patients, including 18% (118) who did not receive surgery; 11% (72) who had metastases in places other than the long bones of the extremities; 3% (23) who received treatment other than intramedullary nailing, endoprosthetic reconstruction, or dynamic hip screw; 3% (23) who underwent revision surgery, 3% (17) in whom there was no tumor, and 2% (15) who were lost to follow-up within 1 year. Temporal validation was performed using data on 406 patients treated surgically for bony metastatic disease of the extremities from 2016 to 2020 at the same two institutions where the MLA was developed. Variables used to predict survival in the SORG algorithm included perioperative laboratory values, tumor characteristics, and general demographics. To assess the models' discrimination, we computed the c-statistic, commonly referred to as the area under the receiver operating characteristic (AUC) curve for binary classification. This value ranged from 0.5 (representing chance-level performance) to 1.0 (indicating excellent discrimination) Generally, an AUC of 0.75 is considered high enough for use in clinical practice. To evaluate the agreement between predicted and observed outcomes, a calibration plot was used, and the calibration slope and intercept were calculated. Perfect calibration would result in a slope of 1 and intercept of 0. For overall performance, the Brier score and null-model Brier score were determined. The Brier score can range from 0 (representing perfect prediction) to 1 (indicating the poorest prediction). Proper interpretation of the Brier score necessitates a comparison with the null-model Brier score, which represents the score for an algorithm that predicts a probability equal to the population prevalence of the outcome for each patient. Finally, a decision curve analysis was conducted to compare the potential net benefit of the algorithm with other decision-support methods, such as treating all or none of the patients. Overall, 90-day and 1-year mortality were lower in the temporal validation cohort than in the development cohort (90 day: 23% versus 28%; p < 0.001, and 1 year: 51% versus 59%; p<0.001). RESULTS Overall survival of the patients in the validation cohort improved from 28% mortality at the 90-day timepoint in the cohort on which the model was trained to 23%, and 59% mortality at the 1-year timepoint to 51%. The AUC was 0.78 (95% CI 0.72 to 0.82) for 90-day survival and 0.75 (95% CI 0.70 to 0.79) for 1-year survival, indicating the model could distinguish the two outcomes reasonably. For the 90-day model, the calibration slope was 0.71 (95% CI 0.53 to 0.89), and the intercept was -0.66 (95% CI -0.94 to -0.39), suggesting the predicted risks were overly extreme, and that in general, the risk of the observed outcome was overestimated. For the 1-year model, the calibration slope was 0.73 (95% CI 0.56 to 0.91) and the intercept was -0.67 (95% CI -0.90 to -0.43). With respect to overall performance, the model's Brier scores for the 90-day and 1-year models were 0.16 and 0.22. These scores were higher than the Brier scores of internal validation of the development study (0.13 and 0.14) models, indicating the models' performance has declined over time. CONCLUSION The SORG MLA to predict survival after surgical treatment of extremity metastatic disease showed decreased performance on temporal validation. Moreover, in patients undergoing innovative immunotherapy, the possibility of mortality risk was overestimated in varying severity. Clinicians should be aware of this overestimation and discount the prediction of the SORG MLA according to their own experience with this patient population. Generally, these results show that temporal reassessment of these MLA-driven probability calculators is of paramount importance because the predictive performance may decline over time as treatment regimens evolve. The SORG-MLA is available as a freely accessible internet application at https://sorg-apps.shinyapps.io/extremitymetssurvival/ .Level of Evidence Level III, prognostic study.
Collapse
Affiliation(s)
- Tom M. de Groot
- Massachusetts General Hospital, Boston, MA, USA
- University Medical Center Groningen, Groningen, the Netherlands
| | - Duncan Ramsey
- University of Texas RGV School of Medicine, Edinburg, TX, USA
| | | | | | | | | | | | | | | | | | | | - Eric Newman
- Massachusetts General Hospital, Boston, MA, USA
| | | | | | | |
Collapse
|
8
|
Vaish A, Migliorini F, Vaishya R. Artificial intelligence in foot and ankle surgery: current concepts. ORTHOPADIE (HEIDELBERG, GERMANY) 2023; 52:1011-1016. [PMID: 37626240 PMCID: PMC10692015 DOI: 10.1007/s00132-023-04426-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 07/13/2023] [Indexed: 08/27/2023]
Abstract
The twenty-first century has proven that data are the new gold. Artificial intelligence (AI) driven technologies might potentially change the clinical practice in all medical specialities, including orthopedic surgery. AI has a broad spectrum of subcomponents, including machine learning, which consists of a subdivision called deep learning. AI has the potential to increase healthcare delivery, improve indications and interventions, and minimize errors. In orthopedic surgery. AI supports the surgeon in the evaluation of radiological images, training of surgical residents, and excellent performance of machine-assisted surgery. The AI algorithms improve the administrative and management processes of hospitals and clinics, electronic healthcare databases, monitoring the outcomes, and safety controls. AI models are being developed in nearly all orthopedic subspecialties, including arthroscopy, arthroplasty, tumor, spinal and pediatric surgery. The present study discusses current applications, limitations, and future prospective of AI in foot and ankle surgery.
Collapse
Affiliation(s)
- Abhishek Vaish
- Department of Orthopaedics and Joint Replacement Surgery, Indraprastha Apollo Hospital, Sarita Vihar, 110076, New Delhi, India
| | - Filippo Migliorini
- Department of Orthopaedic, Trauma, and Reconstructive Surgery, RWTH University Medical Centre of Aachen, Pauwelsstraße 30, 52064, Aachen, Germany.
- Department of Orthopaedic and Trauma Surgery, Academic Hospital of Bolzano (SABES-ASDAA), 39100 Bolzano, Italy.
| | - Raju Vaishya
- Department of Orthopaedics and Joint Replacement Surgery, Indraprastha Apollo Hospital, Sarita Vihar, 110076, New Delhi, India
| |
Collapse
|
9
|
Yan Z, Liu M, Wang X, Wang J, Wang Z, Liu J, Wu S, Luan X. Construction and Validation of Machine Learning Algorithms to Predict Chronic Post-Surgical Pain Among Patients Undergoing Total Knee Arthroplasty. Pain Manag Nurs 2023; 24:627-633. [PMID: 37156678 DOI: 10.1016/j.pmn.2023.04.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2023] [Revised: 04/10/2023] [Accepted: 04/12/2023] [Indexed: 05/10/2023]
Abstract
BACKGROUND Chronic post-surgical pain (CPSP) is a common but undertreated condition with a high prevalence among patients undergoing total knee arthroplasty (TKA). An effective model for CPSP prediction has not been established yet. AIMS To construct and validate machine learning models for the early prediction of CPSP among patients undergoing TKA. DESIGN A prospective cohort study. PARTICIPANTS/SUBJECTS A total of 320 patients in the modeling group and 150 patients in the validation group were recruited from two independent hospitals between December 2021 and July 2022. They were followed up for 6 months to determine the outcomes of CPSP through telephone interviews. METHODS Four machine learning algorithms were developed through 10-fold cross-validation for five times. In the validation group, the discrimination and calibration of the machine learning algorithms were compared by the logistic regression model. The importance of the variables in the best model identified was ranked. RESULTS The incidence of CPSP in the modeling group was 25.3%, and that in the validation group was 27.6%. Compared with other models, the random forest model achieved the best performance with the highest C-statistic of 0.897 and the lowest Brier score of 0.119 in the validation group. The top three important factors for predicting CPSP were knee joint function, fear of movement, and pain at rest in the baseline. CONCLUSIONS The random forest model demonstrated good discrimination and calibration capacity for identifying patients undergoing TKA at high risk for CPSP. Clinical nurses would screen out high-risk patients for CPSP by using the risk factors identified in the random forest model, and efficiently distribute preventive strategy.
Collapse
Affiliation(s)
- Zeping Yan
- School of Nursing and Rehabilitation, Cheeloo College of Medicine, Shandong University, Jinan, China; University of Health and Rehabilitation Sciences, Qingdao, China; Beijing Bo'ai Hospital, China Rehabilitation Research Center, Beijing, China
| | - Mengqi Liu
- School of Nursing and Rehabilitation, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Xiaoli Wang
- School of Nursing and Rehabilitation, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Jiurui Wang
- School of Nursing and Rehabilitation, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Zhiwei Wang
- School of Nursing and Rehabilitation, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Jian Liu
- School of Nursing and Rehabilitation, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Shicai Wu
- Beijing Bo'ai Hospital, China Rehabilitation Research Center, Beijing, China
| | - Xiaorong Luan
- School of Nursing and Rehabilitation, Qilu Hospital, Shandong University, Jinan, China.
| |
Collapse
|
10
|
Verweij LP, van der Linde JA, van Deurzen DF, van den Bekerom MP. High variability in what is considered important to report following instability surgery: a Delphi study among Dutch shoulder specialists. JSES Int 2023; 7:2316-2320. [PMID: 37969493 PMCID: PMC10638571 DOI: 10.1016/j.jseint.2023.06.020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2023] Open
Abstract
Background Standardized reporting leads to high-quality data and can reduce administration time. The aim of this study was to (1) get an insight into the variability of what is considered important to report in the surgical report following shoulder instability surgery and (2) determine which elements should be included in the surgical report following shoulder instability surgery according to Dutch surgeons using a Delphi method. Methods Dutch orthopedic shoulder surgeons were included in a panel for a Delphi study consisting of 3 rounds. Importance of the elements was rated on a 9-point Likert scale. High variability was defined as an element that received at least 1 score between 1 and 3 and 1 score between 7 and 9 in round 3. Consensus was defined as ≥80% of the panel giving a score of 7 or more. Results Seventeen shoulder specialists completed all 3 rounds and identified a total of 82 elements for the arthroscopic Bankart repair and 60 for the open Latarjet. High variability was observed in 57 (70%) and 52 (87%) of the elements, respectively. After round 3, the panel reached consensus on 27 and 11 elements that should be mentioned in the surgical report following arthroscopic Bankart repair and open Latarjet. Conclusion There is high variability in what shoulder specialists regard essential to report. Consensus was reached on 27 and 11 elements to be reported following arthroscopic Bankart repair and open Latarjet, respectively. Future studies on an international scale can further improve data collection and communication between specialists.
Collapse
Affiliation(s)
- Lukas P.E. Verweij
- Amsterdam UMC, Location AMC, University of Amsterdam, Department of Orthopedic Surgery and Sports Medicine, Amsterdam, the Netherlands
- Amsterdam Movement Sciences, Musculoskeletal Health Program, Amsterdam, the Netherlands
- Amsterdam Shoulder and Elbow Center of Expertise (ASECE), Amsterdam, the Netherlands
| | - Just A. van der Linde
- Amsterdam Shoulder and Elbow Center of Expertise (ASECE), Amsterdam, the Netherlands
- Reiner Haga Orthopaedic Centre, Zoetermeer, the Netherlands
| | - Derek F.P. van Deurzen
- Amsterdam Shoulder and Elbow Center of Expertise (ASECE), Amsterdam, the Netherlands
- Shoulder and Elbow Unit, Joint Research, Department of Orthopedic Surgery, OLVG, Amsterdam, the Netherlands
| | - Michel P.J. van den Bekerom
- Amsterdam Shoulder and Elbow Center of Expertise (ASECE), Amsterdam, the Netherlands
- Shoulder and Elbow Unit, Joint Research, Department of Orthopedic Surgery, OLVG, Amsterdam, the Netherlands
- Department of Orthopedic Surgery, Medical Center Jan van Goyen, Amsterdam, the Netherlands
- Faculty of Behavioural and Movement Sciences, Department of Human Movement Sciences, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
| |
Collapse
|
11
|
Macken AA, Macken LC, Oosterhoff JHF, Boileau P, Athwal GS, Doornberg JN, Lafosse L, Lafosse T, van den Bekerom MPJ, Buijze GA. Developing a machine learning algorithm to predict the probability of aseptic loosening of the glenoid component after anatomical total shoulder arthroplasty: protocol for a retrospective, multicentre study. BMJ Open 2023; 13:e074700. [PMID: 37852772 PMCID: PMC10603397 DOI: 10.1136/bmjopen-2023-074700] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/14/2023] [Accepted: 09/25/2023] [Indexed: 10/20/2023] Open
Abstract
INTRODUCTION Despite technological advancements in recent years, glenoid component loosening remains a common complication after anatomical total shoulder arthroplasty (ATSA) and is one of the main causes of revision surgery. Increasing emphasis is placed on the prevention of glenoid component failure. Previous studies have successfully predicted range of motion, patient-reported outcomes and short-term complications after ATSA using machine learning methods, but an accurate predictive model for (glenoid component) revision is currently lacking. This study aims to use a large international database to accurately predict aseptic loosening of the glenoid component after ATSA using machine learning algorithms. METHODS AND ANALYSIS For this multicentre, retrospective study, individual patient data will be compiled from previously published studies reporting revision of ATSA. A systematic literature search will be performed in Medline (PubMed) identifying all studies reporting outcomes of ATSA. Authors will be contacted and invited to participate in the Machine Learning Consortium by sharing their anonymised databases. All databases reporting revisions after ATSA will be included, and individual patients with a follow-up less than 2 years or a fracture as the indication for ATSA will be excluded. First, features (predictive variables) will be identified using a random forest feature selection. The resulting features from the compiled database will be used to train various machine learning algorithms (stochastic gradient boosting, random forest, support vector machine, neural network and elastic-net penalised logistic regression). The developed and validated algorithms will be evaluated across discrimination (c-statistic), calibration, the Brier score and the decision curve analysis. The best-performing algorithm will be used to create an open-access online prediction tool. ETHICS AND DISSEMINATION Data will be collected adhering to the WHO regulation on data sharing. An Institutional Review Board review is not applicable. The study results will be published in a peer-reviewed journal.
Collapse
Affiliation(s)
- Arno Alexander Macken
- Department of Orthopaedics and Sports Medicine, Erasmus Medical Center, Rotterdam, The Netherlands
- Alps Surgery Institute, Clinique Generale Annecy, Annecy, France
| | - Loïc C Macken
- Faculty of Science, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Jacobien H F Oosterhoff
- Department of Engineering Systems and Services, Delft University of Technology, Delft, The Netherlands
| | - Pascal Boileau
- Institut de Chirurgie Réparatrice, Locomoteur & Sport, Centre Hospitalier Universitaire de Nice, Nice, France
| | - George S Athwal
- Roth McFarlane Hand and Upper Limb Center, Schulich School of Medicine and Dentistry, London, Ontario, Canada
| | - Job N Doornberg
- Orthopaedic Surgery, University Medical Centre Groningen, Groningen, The Netherlands
| | - Laurent Lafosse
- Alps Surgery Institute, Clinique Generale Annecy, Annecy, France
| | - Thibault Lafosse
- Alps Surgery Institute, Clinique Generale Annecy, Annecy, France
| | - Michel P J van den Bekerom
- Department of Human Movement Sciences, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
- Department of Orthopaedic Surgery, OLVG, Amsterdam, The Netherlands
| | - Geert Alexander Buijze
- Alps Surgery Institute, Clinique Generale Annecy, Annecy, France
- Department of Orthopedic Surgery, Hôpital Lapeyronie, Montpellier, France
| |
Collapse
|
12
|
MacLean CH, Antao VC, Chin AS, McLawhorn AS. Population-Based Applications and Analytics Using Patient-Reported Outcome Measures. J Am Acad Orthop Surg 2023; 31:1078-1087. [PMID: 37276464 PMCID: PMC10519290 DOI: 10.5435/jaaos-d-23-00133] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Revised: 04/19/2023] [Accepted: 04/20/2023] [Indexed: 06/07/2023] Open
Abstract
The intersection of big data and artificial intelligence (AI) has resulted in advances in numerous areas, including machine learning, computer vision, and natural language processing. Although there are many potentially transformative applications of AI in health care, including precision medicine, this industry has been slow to adopt these technologies. At the same time, the operations of health care have historically been system-directed and physician-directed rather than patient-centered. The application of AI to patient-reported outcome measures (PROMs), which provide insight into patient-centered health outcomes, could steer research and healthcare delivery toward decisions that optimize outcomes important to patients. Historically, PROMs have only been collected within research registries. However, the increasing availability of PROMs within electronic health records has led to their inclusion in big data ecosystems, where they can inform or be informed by other data elements. The use of big data to analyze PROMs can help establish norms, evaluate data distribution, and determine proportions of patients achieving change or threshold standards. This information can be used for benchmarking, risk adjustment, predictive modeling, and ultimately improving the health of individuals and populations.
Collapse
Affiliation(s)
- Catherine H. MacLean
- From the Center for the Advancement of Value in Musculoskeletal Care (Dr. MacLean, Dr. Antao, Ms. Chin), Hospital for Special Surgery, New York, NY (MacLean, Antao, and Chin), and the Adult Reconstruction and Joint Replacement Service, Hospital for Special Surgery, New York, NY (McLawhorn)
| | - Vinicius C. Antao
- From the Center for the Advancement of Value in Musculoskeletal Care (Dr. MacLean, Dr. Antao, Ms. Chin), Hospital for Special Surgery, New York, NY (MacLean, Antao, and Chin), and the Adult Reconstruction and Joint Replacement Service, Hospital for Special Surgery, New York, NY (McLawhorn)
| | - Amy S. Chin
- From the Center for the Advancement of Value in Musculoskeletal Care (Dr. MacLean, Dr. Antao, Ms. Chin), Hospital for Special Surgery, New York, NY (MacLean, Antao, and Chin), and the Adult Reconstruction and Joint Replacement Service, Hospital for Special Surgery, New York, NY (McLawhorn)
| | - Alexander S. McLawhorn
- From the Center for the Advancement of Value in Musculoskeletal Care (Dr. MacLean, Dr. Antao, Ms. Chin), Hospital for Special Surgery, New York, NY (MacLean, Antao, and Chin), and the Adult Reconstruction and Joint Replacement Service, Hospital for Special Surgery, New York, NY (McLawhorn)
| |
Collapse
|
13
|
Salimi M, Parry JA, Shahrokhi R, Mosalamiaghili S. Application of artificial intelligence in trauma orthopedics: Limitation and prospects. World J Clin Cases 2023; 11:4231-4240. [PMID: 37449222 PMCID: PMC10337008 DOI: 10.12998/wjcc.v11.i18.4231] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/26/2023] [Revised: 04/23/2023] [Accepted: 05/08/2023] [Indexed: 06/26/2023] Open
Abstract
The varieties and capabilities of artificial intelligence and machine learning in orthopedic surgery are extensively expanding. One promising method is neural networks, emphasizing big data and computer-based learning systems to develop a statistical fracture-detecting model. It derives patterns and rules from outstanding amounts of data to analyze the probabilities of different outcomes using new sets of similar data. The sensitivity and specificity of machine learning in detecting fractures vary from previous studies. AI may be most promising in the diagnosis of less-obvious fractures that are more commonly missed. Future studies are necessary to develop more accurate and effective detection models that can be used clinically.
Collapse
Affiliation(s)
- Maryam Salimi
- Department of Orthopaedic Surgery, Denver Health Medical Center, Denver, CO 80215, United States
| | - Joshua A Parry
- Department of Orthopaedic Surgery, Denver Health Medical Center, Denver, CO 80215, United States
| | - Raha Shahrokhi
- Student Research Committee, Shiraz University of Medical Sciences, Shiraz 7138433608, Iran
| | | |
Collapse
|
14
|
Gould DJ, Bailey JA, Spelman T, Bunzli S, Dowsey MM, Choong PFM. Predicting 30-day readmission following total knee arthroplasty using machine learning and clinical expertise applied to clinical administrative and research registry data in an Australian cohort. ARTHROPLASTY 2023; 5:30. [PMID: 37259173 DOI: 10.1186/s42836-023-00186-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2022] [Accepted: 04/10/2023] [Indexed: 06/02/2023] Open
Abstract
BACKGROUND Thirty-day readmission is an increasingly important problem for total knee arthroplasty (TKA) patients. The aim of this study was to develop a risk prediction model using machine learning and clinical insight for 30-day readmission in primary TKA patients. METHOD Data used to train and internally validate a multivariable predictive model were obtained from a single tertiary referral centre for TKA located in Victoria, Australia. Hospital administrative data and clinical registry data were utilised, and predictors were selected through systematic review and subsequent consultation with clinicians caring for TKA patients. Logistic regression and random forest models were compared to one another. Calibration was evaluated by visual inspection of calibration curves and calculation of the integrated calibration index (ICI). Discriminative performance was evaluated using the area under the receiver operating characteristic curve (AUC-ROC). RESULTS The models developed in this study demonstrated adequate calibration for use in the clinical setting, despite having poor discriminative performance. The best-calibrated readmission prediction model was a logistic regression model trained on administrative data using risk factors identified from systematic review and meta-analysis, which are available at the initial consultation (ICI = 0.012, AUC-ROC = 0.589). Models developed to predict complications associated with readmission also had reasonable calibration (ICI = 0.012, AUC-ROC = 0.658). CONCLUSION Discriminative performance of the prediction models was poor, although machine learning provided a slight improvement. The models were reasonably well calibrated, meaning they provide accurate patient-specific probabilities of these outcomes. This information can be used in shared clinical decision-making for discharge planning and post-discharge follow up.
Collapse
Affiliation(s)
- Daniel J Gould
- Department of Surgery, St Vincent's Hospital Melbourne, University of Melbourne, Level 2 Clinical Sciences Building, 29 Regent Street, Fitzroy, VIC, 3065, Australia.
| | - James A Bailey
- School of Computing and Information Systems, University of Melbourne, Doug McDonell Building, Parkville, VIC, 3052, Australia
| | - Tim Spelman
- Department of Surgery, St Vincent's Hospital Melbourne, University of Melbourne, Level 2 Clinical Sciences Building, 29 Regent Street, Fitzroy, VIC, 3065, Australia
| | - Samantha Bunzli
- School of Health Sciences and Social Work, Griffith University, Nathan Campus, Nathan, QLD, 4111, Australia
| | - Michelle M Dowsey
- Department of Surgery, St Vincent's Hospital Melbourne, University of Melbourne, Level 2 Clinical Sciences Building, 29 Regent Street, Fitzroy, VIC, 3065, Australia
- Department of Orthopaedics, St. Vincent's Hospital Melbourne, Level 3/35 Victoria Parade, Fitzroy, VIC, 3065, Australia
| | - Peter F M Choong
- Department of Surgery, St Vincent's Hospital Melbourne, University of Melbourne, Level 2 Clinical Sciences Building, 29 Regent Street, Fitzroy, VIC, 3065, Australia
- Department of Orthopaedics, St. Vincent's Hospital Melbourne, Level 3/35 Victoria Parade, Fitzroy, VIC, 3065, Australia
| |
Collapse
|
15
|
Mulugeta G, Zewotir T, Tegegne AS, Juhar LH, Muleta MB. Classification of imbalanced data using machine learning algorithms to predict the risk of renal graft failures in Ethiopia. BMC Med Inform Decis Mak 2023; 23:98. [PMID: 37217892 DOI: 10.1186/s12911-023-02185-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2022] [Accepted: 04/25/2023] [Indexed: 05/24/2023] Open
Abstract
INTRODUCTION The prevalence of end-stage renal disease has raised the need for renal replacement therapy over recent decades. Even though a kidney transplant offers an improved quality of life and lower cost of care than dialysis, graft failure is possible after transplantation. Hence, this study aimed to predict the risk of graft failure among post-transplant recipients in Ethiopia using the selected machine learning prediction models. METHODOLOGY The data was extracted from the retrospective cohort of kidney transplant recipients at the Ethiopian National Kidney Transplantation Center from September 2015 to February 2022. In response to the imbalanced nature of the data, we performed hyperparameter tuning, probability threshold moving, tree-based ensemble learning, stacking ensemble learning, and probability calibrations to improve the prediction results. Merit-based selected probabilistic (logistic regression, naive Bayes, and artificial neural network) and tree-based ensemble (random forest, bagged tree, and stochastic gradient boosting) models were applied. Model comparison was performed in terms of discrimination and calibration performance. The best-performing model was then used to predict the risk of graft failure. RESULTS A total of 278 completed cases were analyzed, with 21 graft failures and 3 events per predictor. Of these, 74.8% are male, and 25.2% are female, with a median age of 37. From the comparison of models at the individual level, the bagged tree and random forest have top and equal discrimination performance (AUC-ROC = 0.84). In contrast, the random forest has the best calibration performance (brier score = 0.045). Under testing the individual model as a meta-learner for stacking ensemble learning, the result of stochastic gradient boosting as a meta-learner has the top discrimination (AUC-ROC = 0.88) and calibration (brier score = 0.048) performance. Regarding feature importance, chronic rejection, blood urea nitrogen, number of post-transplant admissions, phosphorus level, acute rejection, and urological complications are the top predictors of graft failure. CONCLUSIONS Bagging, boosting, and stacking, with probability calibration, are good choices for clinical risk predictions working on imbalanced data. The data-driven probability threshold is more beneficial than the natural threshold of 0.5 to improve the prediction result from imbalanced data. Integrating various techniques in a systematic framework is a smart strategy to improve prediction results from imbalanced data. It is recommended for clinical experts in kidney transplantation to use the final calibrated model as a decision support system to predict the risk of graft failure for individual patients.
Collapse
Affiliation(s)
- Getahun Mulugeta
- Department of Statistics, Bahir Dar University, Bahir Dar, Ethiopia.
| | - Temesgen Zewotir
- School of Mathematics, Statistics, and Computer Science, KwaZulu-Natal University, Durban, South Africa
| | | | - Leja Hamza Juhar
- St. Paul's Hospital Millennium Medical College, Addis Ababa, Ethiopia
| | | |
Collapse
|
16
|
Affiliation(s)
- Andrew S Bi
- NYU Langone Orthopedic Hospital, New York, NY
| |
Collapse
|
17
|
Gupta P, Kingston KA, O’Malley M, Williams RJ, Ramkumar PN. Advancements in Artificial Intelligence for Foot and Ankle Surgery: A Systematic Review. FOOT & ANKLE ORTHOPAEDICS 2023; 8:24730114221151079. [PMID: 36817020 PMCID: PMC9929923 DOI: 10.1177/24730114221151079] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/16/2023] Open
Abstract
Background There has been a rapid increase in research applying artificial intelligence (AI) to various subspecialties of orthopaedic surgery, including foot and ankle surgery. The purpose of this systematic review is to (1) characterize the topics and objectives of studies using AI in foot and ankle surgery, (2) evaluate the performance of their models, and (3) evaluate their validity (internal or external validation). Methods A systematic literature review was conducted using PubMed/MEDLINE and Embase databases in December 2022. All studies that used AI or its subsets machine learning (ML) and deep learning (DL) in the setting of foot and ankle surgery relevant to orthopaedic surgeons were included. Studies were evaluated for their demographics, subject area, outcomes of interest, model(s) tested, model(s)' performance, and validity (internal or external). Results A total of 31 studies met inclusion criteria: 14 studies investigated AI for image interpretation, 13 studies investigated AI for clinical predictions, and 4 studies were grouped as "other." Studies commonly explored AI for ankle fractures, calcaneus fractures, hallux valgus, Achilles tendon pathologies, plantar fasciitis, and sports injuries. For studies reporting the area under the receiver operating characteristic curve (AUC), AUCs ranged from 0.64 (poor) to 0.99 (excellent). Two studies (6.45%) reported external validation. Conclusion Applications of AI in the field of foot and ankle surgery are expanding, particularly for image interpretation and clinical predictions. Current model performances range from poor to excellent, and most studies lack external validation, demonstrating a need for further research prior to deploying AI-based clinical applications. Level of Evidence Level III, retrospective cohort study.
Collapse
Affiliation(s)
- Puneet Gupta
- Department of Orthopaedic Surgery, George Washington University School of Medicine and Health Sciences, Washington, DC, USA
| | | | - Martin O’Malley
- Hospital for Special Surgery, New York, NY, USA,Brooklyn Nets, National Basketball Association (NBA), Brooklyn, NY, USA
| | - Riley J. Williams
- Hospital for Special Surgery, New York, NY, USA,Brooklyn Nets, National Basketball Association (NBA), Brooklyn, NY, USA
| | - Prem N. Ramkumar
- Hospital for Special Surgery, New York, NY, USA,Brooklyn Nets, National Basketball Association (NBA), Brooklyn, NY, USA,Prem N. Ramkumar, MD, MBA, Hospital for Special Surgery, 535 E 70th St, New York, NY 10021-4898, USA.
| |
Collapse
|
18
|
Jimenez AE, Porras JL, Azad TD, Shah PP, Jackson CM, Gallia G, Bettegowda C, Weingart J, Mukherjee D. Machine Learning Models for Predicting Postoperative Outcomes following Skull Base Meningioma Surgery. J Neurol Surg B Skull Base 2022; 83:635-645. [PMID: 36393884 PMCID: PMC9653296 DOI: 10.1055/a-1885-1447] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2021] [Accepted: 06/20/2022] [Indexed: 10/17/2022] Open
Abstract
Objective While predictive analytic techniques have been used to analyze meningioma postoperative outcomes, to our knowledge, there have been no studies that have investigated the utility of machine learning (ML) models in prognosticating outcomes among skull base meningioma patients. The present study aimed to develop models for predicting postoperative outcomes among skull base meningioma patients, specifically prolonged hospital length of stay (LOS), nonroutine discharge disposition, and high hospital charges. We also validated the predictive performance of our models on out-of-sample testing data. Methods Patients who underwent skull base meningioma surgery between 2016 and 2019 at an academic institution were included in our study. Prolonged hospital LOS and high hospital charges were defined as >4 days and >$47,887, respectively. Elastic net logistic regression algorithms were trained to predict postoperative outcomes using 70% of available data, and their predictive performance was evaluated on the remaining 30%. Results A total of 265 patients were included in our final analysis. Our cohort was majority female (77.7%) and Caucasian (63.4%). Elastic net logistic regression algorithms predicting prolonged LOS, nonroutine discharge, and high hospital charges achieved areas under the receiver operating characteristic curve of 0.798, 0.752, and 0.592, respectively. Further, all models were adequately calibrated as determined by the Spiegelhalter Z -test ( p >0.05). Conclusion Our study developed models predicting prolonged hospital LOS, nonroutine discharge disposition, and high hospital charges among skull base meningioma patients. Our models highlight the utility of ML as a tool to aid skull base surgeons in providing high-value health care and optimizing clinical workflows.
Collapse
Affiliation(s)
- Adrian E. Jimenez
- Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
| | - Jose L. Porras
- Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
| | - Tej D. Azad
- Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
| | - Pavan P. Shah
- Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
| | - Christopher M. Jackson
- Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
| | - Gary Gallia
- Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
| | - Chetan Bettegowda
- Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
| | - Jon Weingart
- Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
| | - Debraj Mukherjee
- Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
| |
Collapse
|
19
|
van de Kuit A, Oosterhoff JHF, Dijkstra H, Sprague S, Bzovsky S, Bhandari M, Swiontkowski M, Schemitsch EH, IJpma FFA, Poolman RW, Doornberg JN, Hendrickx LAM. Patients With Femoral Neck Fractures Are at Risk for Conversion to Arthroplasty After Internal Fixation: A Machine-learning Algorithm. Clin Orthop Relat Res 2022; 480:2350-2360. [PMID: 35767811 PMCID: PMC9653184 DOI: 10.1097/corr.0000000000002283] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/01/2022] [Accepted: 05/31/2022] [Indexed: 01/31/2023]
Abstract
BACKGROUND Femoral neck fractures are common and are frequently treated with internal fixation. A major disadvantage of internal fixation is the substantially high number of conversions to arthroplasty because of nonunion, malunion, avascular necrosis, or implant failure. A clinical prediction model identifying patients at high risk of conversion to arthroplasty may help clinicians in selecting patients who could have benefited from arthroplasty initially. QUESTION/PURPOSE What is the predictive performance of a machine-learning (ML) algorithm to predict conversion to arthroplasty within 24 months after internal fixation in patients with femoral neck fractures? METHODS We included 875 patients from the Fixation using Alternative Implants for the Treatment of Hip fractures (FAITH) trial. The FAITH trial consisted of patients with low-energy femoral neck fractures who were randomly assigned to receive a sliding hip screw or cancellous screws for internal fixation. Of these patients, 18% (155 of 875) underwent conversion to THA or hemiarthroplasty within the first 24 months. All patients were randomly divided into a training set (80%) and test set (20%). First, we identified 27 potential patient and fracture characteristics that may have been associated with our primary outcome, based on biomechanical rationale and previous studies. Then, random forest algorithms (an ML learning, decision tree-based algorithm that selects variables) identified 10 predictors of conversion: BMI, cardiac disease, Garden classification, use of cardiac medication, use of pulmonary medication, age, lung disease, osteoarthritis, sex, and the level of the fracture line. Based on these variables, five different ML algorithms were trained to identify patterns related to conversion. The predictive performance of these trained ML algorithms was assessed on the training and test sets based on the following performance measures: (1) discrimination (the model's ability to distinguish patients who had conversion from those who did not; expressed with the area under the receiver operating characteristic curve [AUC]), (2) calibration (the plotted estimated versus the observed probabilities; expressed with the calibration curve intercept and slope), and (3) the overall model performance (Brier score: a composite of discrimination and calibration). RESULTS None of the five ML algorithms performed well in predicting conversion to arthroplasty in the training set and the test set; AUCs of the algorithms in the training set ranged from 0.57 to 0.64, slopes of calibration plots ranged from 0.53 to 0.82, calibration intercepts ranged from -0.04 to 0.05, and Brier scores ranged from 0.14 to 0.15. The algorithms were further evaluated in the test set; AUCs ranged from 0.49 to 0.73, calibration slopes ranged from 0.17 to 1.29, calibration intercepts ranged from -1.28 to 0.34, and Brier scores ranged from 0.13 to 0.15. CONCLUSION The predictive performance of the trained algorithms was poor, despite the use of one of the best datasets available worldwide on this subject. If the current dataset consisted of different variables or more patients, the performance may have been better. Also, various reasons for conversion to arthroplasty were pooled in this study, but the separate prediction of underlying pathology (such as, avascular necrosis or nonunion) may be more precise. Finally, it may be possible that it is inherently difficult to predict conversion to arthroplasty based on preoperative variables alone. Therefore, future studies should aim to include more variables and to differentiate between the various reasons for arthroplasty. LEVEL OF EVIDENCE Level III, prognostic study.
Collapse
Affiliation(s)
- Anouk van de Kuit
- Department of Orthopaedic Surgery, University Medical Center Groningen, University of Groningen, Groningen, the Netherlands
| | - Jacobien H. F. Oosterhoff
- Department of Orthopaedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Department of Orthopaedic Surgery, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
| | - Hidde Dijkstra
- Department of Trauma Surgery, University Medical Centre Groningen, University of Groningen, Groningen, the Netherlands
| | - Sheila Sprague
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada
- Division of Orthopaedic Surgery, Department of Surgery, McMaster University, Hamilton, ON, Canada
| | - Sofia Bzovsky
- Division of Orthopaedic Surgery, Department of Surgery, McMaster University, Hamilton, ON, Canada
| | - Mohit Bhandari
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada
- Division of Orthopaedic Surgery, Department of Surgery, McMaster University, Hamilton, ON, Canada
| | - Marc Swiontkowski
- Department of Orthopaedic Surgery, University of Minnesota, Minneapolis, MN, USA
| | | | - Frank F. A. IJpma
- Department of Trauma Surgery, University Medical Centre Groningen, University of Groningen, Groningen, the Netherlands
| | - Rudolf W. Poolman
- Department of Orthopaedic Surgery, University Medical Center Leiden, Leiden University, Leiden, the Netherlands
| | - Job N. Doornberg
- Department of Orthopaedic Surgery, University Medical Center Groningen, University of Groningen, Groningen, the Netherlands
- Department of Trauma Surgery, University Medical Centre Groningen, University of Groningen, Groningen, the Netherlands
- Department of Orthopaedic and Trauma Surgery, Flinders Medical Centre, Flinders University, Adelaide, Australia
| | | |
Collapse
|
20
|
Oosterhoff JHF, Oberai T, Karhade AV, Doornberg JN, Kerkhoffs GM, Jaarsma RL, Schwab JH, Heng M. Does the SORG Orthopaedic Research Group Hip Fracture Delirium Algorithm Perform Well on an Independent Intercontinental Cohort of Patients With Hip Fractures Who Are 60 Years or Older? Clin Orthop Relat Res 2022; 480:2205-2213. [PMID: 35561268 PMCID: PMC10476833 DOI: 10.1097/corr.0000000000002246] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/20/2021] [Accepted: 04/22/2022] [Indexed: 01/31/2023]
Abstract
BACKGROUND Postoperative delirium in patients aged 60 years or older with hip fractures adversely affects clinical and functional outcomes. The economic cost of delirium is estimated to be as high as USD 25,000 per patient, with a total budgetary impact between USD 6.6 to USD 82.4 billion annually in the United States alone. Forty percent of delirium episodes are preventable, and accurate risk stratification can decrease the incidence and improve clinical outcomes in patients. A previously developed clinical prediction model (the SORG Orthopaedic Research Group hip fracture delirium machine-learning algorithm) is highly accurate on internal validation (in 28,207 patients with hip fractures aged 60 years or older in a US cohort) in identifying at-risk patients, and it can facilitate the best use of preventive interventions; however, it has not been tested in an independent population. For an algorithm to be useful in real life, it must be valid externally, meaning that it must perform well in a patient cohort different from the cohort used to "train" it. With many promising machine-learning prediction models and many promising delirium models, only few have also been externally validated, and even fewer are international validation studies. QUESTION/PURPOSE Does the SORG hip fracture delirium algorithm, initially trained on a database from the United States, perform well on external validation in patients aged 60 years or older in Australia and New Zealand? METHODS We previously developed a model in 2021 for assessing risk of delirium in hip fracture patients using records of 28,207 patients obtained from the American College of Surgeons National Surgical Quality Improvement Program. Variables included in the original model included age, American Society of Anesthesiologists (ASA) class, functional status (independent or partially or totally dependent for any activities of daily living), preoperative dementia, preoperative delirium, and preoperative need for a mobility aid. To assess whether this model could be applied elsewhere, we used records from an international hip fracture registry. Between June 2017 and December 2018, 6672 patients older than 60 years of age in Australia and New Zealand were treated surgically for a femoral neck, intertrochanteric hip, or subtrochanteric hip fracture and entered into the Australian & New Zealand Hip Fracture Registry. Patients were excluded if they had a pathological hip fracture or septic shock. Of all patients, 6% (402 of 6672) did not meet the inclusion criteria, leaving 94% (6270 of 6672) of patients available for inclusion in this retrospective analysis. Seventy-one percent (4249 of 5986) of patients were aged 80 years or older, after accounting for 5% (284 of 6270) of missing values; 68% (4292 of 6266) were female, after accounting for 0.06% (4 of 6270) of missing values, and 83% (4690 of 5661) of patients were classified as ASA III/IV, after accounting for 10% (609 of 6270) of missing values. Missing data were imputed using the missForest methodology. In total, 39% (2467 of 6270) of patients developed postoperative delirium. The performance of the SORG hip fracture delirium algorithm on the validation cohort was assessed by discrimination, calibration, Brier score, and a decision curve analysis. Discrimination, known as the area under the receiver operating characteristic curves (c-statistic), measures the model's ability to distinguish patients who achieved the outcomes from those who did not and ranges from 0.5 to 1.0, with 1.0 indicating the highest discrimination score and 0.50 the lowest. Calibration plots the predicted versus the observed probabilities, a perfect plot has an intercept of 0 and a slope of 1. The Brier score calculates a composite of discrimination and calibration, with 0 indicating perfect prediction and 1 the poorest. RESULTS The SORG hip fracture algorithm, when applied to an external patient cohort, distinguished between patients at low risk and patients at moderate to high risk of developing postoperative delirium. The SORG hip fracture algorithm performed with a c-statistic of 0.74 (95% confidence interval 0.73 to 0.76). The calibration plot showed high accuracy in the lower predicted probabilities (intercept -0.28, slope 0.52) and a Brier score of 0.22 (the null model Brier score was 0.24). The decision curve analysis showed that the model can be beneficial compared with no model or compared with characterizing all patients as at risk for developing delirium. CONCLUSION Algorithms developed with machine learning are a potential tool for refining treatment of at-risk patients. If high-risk patients can be reliably identified, resources can be appropriately directed toward their care. Although the current iteration of SORG should not be relied on for patient care, it suggests potential utility in assessing risk. Further assessment in different populations, made easier by international collaborations and standardization of registries, would be useful in the development of universally valid prediction models. The model can be freely accessed at: https://sorg-apps.shinyapps.io/hipfxdelirium/ . LEVEL OF EVIDENCE Level III, therapeutic study.
Collapse
Affiliation(s)
- Jacobien H. F. Oosterhoff
- Department of Orthopaedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Amsterdam University Medical Centers, University of Amsterdam, Department of Orthopaedic Surgery, Amsterdam Movement Sciences, the Netherlands
- Department of Orthopaedic and Trauma Surgery, Flinders Medical Centre, Flinders University, Adelaide, SA, Australia
| | - Tarandeep Oberai
- Department of Orthopaedic and Trauma Surgery, Flinders Medical Centre, Flinders University, Adelaide, SA, Australia
| | - Aditya V. Karhade
- Department of Orthopaedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Job N. Doornberg
- Department of Orthopaedic Surgery, University Medical Centre Groningen, University of Groningen, the Netherlands
| | - Gino M.M.J. Kerkhoffs
- Amsterdam University Medical Centers, University of Amsterdam, Department of Orthopaedic Surgery, Amsterdam Movement Sciences, the Netherlands
| | - Ruurd L. Jaarsma
- Department of Orthopaedic and Trauma Surgery, Flinders Medical Centre, Flinders University, Adelaide, SA, Australia
| | - Joseph H. Schwab
- Department of Orthopaedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Marilyn Heng
- Harvard Medical School Orthopedic Trauma Initiative, Massachusetts General Hospital, Boston, MA, USA
| |
Collapse
|
21
|
Song Y, Yang X, Luo Y, Ouyang C, Yu Y, Ma Y, Li H, Lou J, Liu Y, Chen Y, Cao J, Mi W. Comparison of logistic regression and machine learning methods for predicting postoperative delirium in elderly patients: A retrospective study. CNS Neurosci Ther 2022; 29:158-167. [PMID: 36217732 PMCID: PMC9804041 DOI: 10.1111/cns.13991] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2022] [Revised: 09/16/2022] [Accepted: 09/23/2022] [Indexed: 02/06/2023] Open
Abstract
AIMS To compare the performance of logistic regression and machine learning methods in predicting postoperative delirium (POD) in elderly patients. METHOD This was a retrospective study of perioperative medical data from patients undergoing non-cardiac and non-neurology surgery over 65 years old from January 2014 to August 2019. Forty-six perioperative variables were used to predict POD. A traditional logistic regression and five machine learning models (Random Forest, GBM, AdaBoost, XGBoost, and a stacking ensemble model) were compared by the area under the receiver operating characteristic curve (AUC-ROC), sensitivity, specificity, and precision. RESULTS In total, 29,756 patients were enrolled, and the incidence of POD was 3.22% after variable screening. AUCs were 0.783 (0.765-0.8) for the logistic regression method, 0.78 for random forest, 0.76 for GBM, 0.74 for AdaBoost, 0.73 for XGBoost, and 0.77 for the stacking ensemble model. The respective sensitivities for the 6 aforementioned models were 74.2%, 72.2%, 76.8%, 63.6%, 71.6%, and 67.4%. The respective specificities for the 6 aforementioned models were 70.7%, 99.8%, 96.5%, 98.8%, 96.5%, and 96.1%. The respective precision values for the 6 aforementioned models were 7.8%, 52.3%, 55.6%, 57%, 54.5%, and 56.4%. CONCLUSIONS The optimal application of the logistic regression model could provide quick and convenient POD risk identification to help improve the perioperative management of surgical patients because of its better sensitivity, fewer variables, and easier interpretability than the machine learning model.
Collapse
Affiliation(s)
- Yu‐xiang Song
- Department of AnesthesiologyThe First Medical Center of Chinese PLA General HospitalBeijingChina,Medical School of Chinese People's Liberation ArmyBeijingChina
| | - Xiao‐dong Yang
- Institute of Computing TechnologyChinese Academy of SciencesBeijingChina
| | - Yun‐gen Luo
- Department of AnesthesiologyThe First Medical Center of Chinese PLA General HospitalBeijingChina,Medical School of Chinese People's Liberation ArmyBeijingChina
| | - Chun‐lei Ouyang
- Department of AnesthesiologyThe First Medical Center of Chinese PLA General HospitalBeijingChina
| | - Yao Yu
- Department of AnesthesiologyThe First Medical Center of Chinese PLA General HospitalBeijingChina
| | - Yu‐long Ma
- Department of AnesthesiologyThe First Medical Center of Chinese PLA General HospitalBeijingChina
| | - Hao Li
- Department of AnesthesiologyThe First Medical Center of Chinese PLA General HospitalBeijingChina
| | - Jing‐sheng Lou
- Department of AnesthesiologyThe First Medical Center of Chinese PLA General HospitalBeijingChina
| | - Yan‐hong Liu
- Department of AnesthesiologyThe First Medical Center of Chinese PLA General HospitalBeijingChina
| | - Yi‐qiang Chen
- Institute of Computing TechnologyChinese Academy of SciencesBeijingChina
| | - Jiang‐bei Cao
- Department of AnesthesiologyThe First Medical Center of Chinese PLA General HospitalBeijingChina
| | - Wei‐dong Mi
- Department of AnesthesiologyThe First Medical Center of Chinese PLA General HospitalBeijingChina
| |
Collapse
|
22
|
van Spanning SH, Verweij LPE, Allaart LJH, Hendrickx LAM, Doornberg JN, Athwal GS, Lafosse T, Lafosse L, van den Bekerom MPJ, Buijze GA. Development and training of a machine learning algorithm to identify patients at risk for recurrence following an arthroscopic Bankart repair (CLEARER): protocol for a retrospective, multicentre, cohort study. BMJ Open 2022; 12:e055346. [PMID: 36508223 PMCID: PMC9462090 DOI: 10.1136/bmjopen-2021-055346] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/15/2022] Open
Abstract
INTRODUCTION Shoulder instability is a common injury, with a reported incidence of 23.9 per 100 000 person-years. There is still an ongoing debate on the most effective treatment strategy. Non-operative treatment has recurrence rates of up to 60%, whereas operative treatments such as the Bankart repair and bone block procedures show lower recurrence rates (16% and 2%, respectively) but higher complication rates (<2% and up to 30%, respectively). Methods to determine risk of recurrence have been developed; however, patient-specific decision-making tools are still lacking. Artificial intelligence and machine learning algorithms use self-learning complex models that can be used to make patient-specific decision-making tools. The aim of the current study is to develop and train a machine learning algorithm to create a prediction model to be used in clinical practice-as an online prediction tool-to estimate recurrence rates following a Bankart repair. METHODS AND ANALYSIS This is a multicentre retrospective cohort study. Patients with traumatic anterior shoulder dislocations that were treated with an arthroscopic Bankart repair without remplissage will be included. This study includes two parts. Part 1, collecting all potential factors influencing the recurrence rate following an arthroscopic Bankart repair in patients using multicentre data, aiming to include data from >1000 patients worldwide. Part 2, the multicentre data will be re-evaluated (and where applicable complemented) using machine learning algorithms to predict outcomes. Recurrence will be the primary outcome measure. ETHICS AND DISSEMINATION For safe multicentre data exchange and analysis, our Machine Learning Consortium adhered to the WHO regulation 'Policy on Use and Sharing of Data Collected by WHO in Member States Outside the Context of Public Health Emergencies'. The study results will be disseminated through publication in a peer-reviewed journal. No Institutional Review Board is required for this study.
Collapse
Affiliation(s)
- Sanne H van Spanning
- Orthopaedic Surgery, OLVG, Amsterdam, Noord-Holland, The Netherlands
- Hand, Upper Limb, Peripheral Nerve, Brachial Plexus and Microsurgery Unit, Alps Surgery Institute, Annecy, France
- Department of Human Movement Sciences, Faculty of Behavioural and Movement Sciences, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Lukas P E Verweij
- Orthopedic Surgery, Amsterdam Movement Sciences, Amsterdam UMC Locatie AMC, Amsterdam, North Holland, The Netherlands
- Academic Center for Evidence-based Sports Medicine (ACES), Amsterdam UMC Locatie AMC, Amsterdam, North Holland, The Netherlands
- Amsterdam Collaboration for Health and Safety in Sports (ACHSS), International Olympic Committee (IOC) Research Centre, Amsterdam UMC, Amsterdam, Netherlands
| | - Laurens J H Allaart
- Hand, Upper Limb, Peripheral Nerve, Brachial Plexus and Microsurgery Unit, Alps Surgery Institute, Annecy, France
- Department of Human Movement Sciences, Faculty of Behavioural and Movement Sciences, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Laurent A M Hendrickx
- Orthopedic Surgery, Amsterdam Movement Sciences, Amsterdam UMC Locatie AMC, Amsterdam, North Holland, The Netherlands
- Academic Center for Evidence-based Sports Medicine (ACES), Amsterdam UMC Locatie AMC, Amsterdam, North Holland, The Netherlands
- Department of Orthopaedic & Trauma Surgery, Flinders Medical Centre, Flinders University, Adelaide, South Australia, Australia
| | - Job N Doornberg
- Department of Orthopaedic & Trauma Surgery, Flinders Medical Centre, Flinders University, Adelaide, South Australia, Australia
| | - George S Athwal
- Roth McFarlane Hand and Upper Limb Center, Schulich School of Medicine and Dentistry, London, Ontario, Canada
| | - Thibault Lafosse
- Hand, Upper Limb, Peripheral Nerve, Brachial Plexus and Microsurgery Unit, Alps Surgery Institute, Annecy, France
| | - Laurent Lafosse
- Hand, Upper Limb, Peripheral Nerve, Brachial Plexus and Microsurgery Unit, Alps Surgery Institute, Annecy, France
| | - Michel P J van den Bekerom
- Orthopaedic Surgery, OLVG, Amsterdam, Noord-Holland, The Netherlands
- Department of Human Movement Sciences, Faculty of Behavioural and Movement Sciences, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Geert Alexander Buijze
- Hand, Upper Limb, Peripheral Nerve, Brachial Plexus and Microsurgery Unit, Alps Surgery Institute, Annecy, France
- Orthopedic Surgery, Amsterdam Movement Sciences, Amsterdam UMC Locatie AMC, Amsterdam, North Holland, The Netherlands
- Department of Orthopaedic Surgery, Montpellier University Medical Center, Montpellier, Languedoc-Roussillon, France
| |
Collapse
|