Development and Internal Validation of Machine Learning Algorithms for Preoperative Survival Prediction of Extremity Metastatic Disease

Are Current Survival Prediction Tools Useful When Treating Subsequent Skeletal-related Events From Bone Metastases?

BACKGROUND

Bone metastasis in advanced cancer is challenging because of pain, functional issues, and reduced life expectancy. Treatment planning is complex, with consideration of factors such as location, symptoms, and prognosis. Prognostic models help guide treatment choices, with Skeletal Oncology Research Group machine-learning algorithms (SORG-MLAs) showing promise in predicting survival for initial spinal metastases and extremity metastases treated with surgery or radiotherapy. Improved therapies extend patient lifespans, increasing the risk of subsequent skeletal-related events (SREs). Patients experiencing subsequent SREs often suffer from disease progression, indicating a deteriorating condition. For these patients, a thorough evaluation, including accurate survival prediction, is essential to determine the most appropriate treatment and avoid aggressive surgical treatment for patients with a poor survival likelihood. Patients experiencing subsequent SREs often suffer from disease progression, indicating a deteriorating condition. However, some variables in the SORG prediction model, such as tumor histology, visceral metastasis, and previous systemic therapies, might remain consistent between initial and subsequent SREs. Given the prognostic difference between patients with and without a subsequent SRE, the efficacy of established prognostic models-originally designed for individuals with an initial SRE-in addressing a subsequent SRE remains uncertain. Therefore, it is crucial to verify the model's utility for subsequent SREs.

QUESTION/PURPOSE

We aimed to evaluate the reliability of the SORG-MLAs for survival prediction in patients undergoing surgery or radiotherapy for a subsequent SRE for whom both the initial and subsequent SREs occurred in the spine or extremities.

METHODS

We retrospectively included 738 patients who were 20 years or older who received surgery or radiotherapy for initial and subsequent SREs at a tertiary referral center and local hospital in Taiwan between 2010 and 2019. We excluded 74 patients whose initial SRE was in the spine and in whom the subsequent SRE occurred in the extremities and 37 patients whose initial SRE was in the extremities and the subsequent SRE was in the spine. The rationale was that different SORG-MLAs were exclusively designed for patients who had an initial spine metastasis and those who had an initial extremity metastasis, irrespective of whether they experienced metastatic events in other areas (for example, a patient experiencing an extremity SRE before his or her spinal SRE would also be regarded as a candidate for an initial spinal SRE). Because these patients were already validated in previous studies, we excluded them in case we overestimated our result. Five patients with malignant primary bone tumors and 38 patients in whom the metastasis's origin could not be identified were excluded, leaving 584 patients for analysis. The 584 included patients were categorized into two subgroups based on the location of initial and subsequent SREs: the spine group (68% [399]) and extremity group (32% [185]). No patients were lost to follow-up. Patient data at the time they presented with a subsequent SRE were collected, and survival predictions at this timepoint were calculated using the SORG-MLAs. Multiple imputation with the Missforest technique was conducted five times to impute the missing proportions of each predictor. The effectiveness of SORG-MLAs was gauged through several statistical measures, including discrimination (measured by the area under the receiver operating characteristic curve [AUC]), calibration, overall performance (Brier score), and decision curve analysis. Discrimination refers to the model's ability to differentiate between those with the event and those without the event. An AUC ranges from 0.5 to 1.0, with 0.5 indicating the worst discrimination and 1.0 indicating perfect discrimination. An AUC of 0.7 is considered clinically acceptable discrimination. Calibration is the comparison between the frequency of observed events and the predicted probabilities. In an ideal calibration, the observed and predicted survival rates should be congruent. The logarithm of observed-to-expected survival ratio [log(O:E)] offers insight into the model's overall calibration by considering the total number of observed (O) and expected (E) events. The Brier score measures the mean squared difference between the predicted probability of possible outcomes for each individual and the observed outcomes, ranging from 0 to 1, with 0 indicating perfect overall performance and 1 indicating the worst performance. Moreover, the prevalence of the outcome should be considered, so a null-model Brier score was also calculated by assigning a probability equal to the prevalence of the outcome (in this case, the actual survival rate) to each patient. The benefit of the prediction model is determined by comparing its Brier score with that of the null model. If a prediction model's Brier score is lower than the null model's Brier score, the prediction model is deemed as having good performance. A decision curve analysis was performed for models to evaluate the "net benefit," which weighs the true positive rate over the false positive rate against the "threshold probabilities," the ratio of risk over benefit after an intervention was derived based on a comprehensive clinical evaluation and a well-discussed shared-decision process. A good predictive model should yield a higher net benefit than default strategies (treating all patients and treating no patients) across a range of threshold probabilities.

RESULTS

For the spine group, the algorithms displayed acceptable AUC results (median AUCs of 0.69 to 0.72) for 42-day, 90-day, and 1-year survival predictions after treatment for a subsequent SRE. In contrast, the extremity group showed median AUCs ranging from 0.65 to 0.73 for the corresponding survival periods. All Brier scores were lower than those of their null model, indicating the SORG-MLAs' good overall performances for both cohorts. The SORG-MLAs yielded a net benefit for both cohorts; however, they overestimated 1-year survival probabilities in patients with a subsequent SRE in the spine, with a median log(O:E) of -0.60 (95% confidence interval -0.77 to -0.42).

CONCLUSION

The SORG-MLAs maintain satisfactory discriminatory capacity and offer considerable net benefits through decision curve analysis, indicating their continued viability as prediction tools in this clinical context. However, the algorithms overestimate 1-year survival rates for patients with a subsequent SRE of the spine, warranting consideration of specific patient groups. Clinicians and surgeons should exercise caution when using the SORG-MLAs for survival prediction in these patients and remain aware of potential mispredictions when tailoring treatment plans, with a preference for less invasive treatments. Ultimately, this study emphasizes the importance of enhancing prognostic algorithms and developing innovative tools for patients with subsequent SREs as the life expectancy in patients with bone metastases continues to improve and healthcare providers will encounter these patients more often in daily practice.

LEVEL OF EVIDENCE

Level III, prognostic study.

Artificial Intelligence in Detection, Management, and Prognosis of Bone Metastasis: A Systematic Review

BACKGROUND

Metastasis commonly occur in the bone tissue. Artificial intelligence (AI) has become increasingly prevalent in the medical sector as support in decision-making, diagnosis, and treatment processes. The objective of this systematic review was to assess the reliability of AI systems in clinical, radiological, and pathological aspects of bone metastases.

METHODS

We included studies that evaluated the use of AI applications in patients affected by bone metastases. Two reviewers performed a digital search on 31 December 2023 on PubMed, Scopus, and Cochrane library and extracted authors, AI method, interest area, main modalities used, and main objectives from the included studies.

RESULTS

We included 59 studies that analyzed the contribution of computational intelligence in diagnosing or forecasting outcomes in patients with bone metastasis. Six studies were specific for spine metastasis. The study involved nuclear medicine (44.1%), clinical research (28.8%), radiology (20.4%), or molecular biology (6.8%). When a primary tumor was reported, prostate cancer was the most common, followed by lung, breast, and kidney.

CONCLUSIONS

Appropriately trained AI models may be very useful in merging information to achieve an overall improved diagnostic accuracy and treatment for metastasis in the bone. Nevertheless, there are still concerns with the use of AI systems in medical settings. Ethical considerations and legal issues must be addressed to facilitate the safe and regulated adoption of AI technologies. The limitations of the study comprise a stronger emphasis on early detection rather than tumor management and prognosis as well as a high heterogeneity for type of tumor, AI technology and radiological techniques, pathology, or laboratory samples involved.

Comparison of Classically and Machine Learning Generated Survival Prediction Models for Patients With Spinal Metastasis - A meta-Analysis of Two Recently Developed Algorithms

STUDY DESIGN

A systemic review and a meta-analysis. We also provided a retrospective cohort for validation in this study.

OBJECTIVE

(1) Using a meta-analysis to determine the pooled discriminatory ability of The Skeletal Oncology Research Group (SORG) classical algorithm (CA) and machine learning algorithms (MLA); and (2) test the hypothesis that SORG-CA has less variability in performance than SORG-MLA in non-American validation cohorts as SORG-CA does not incorporates regional-specific variables such as body mass index as input.

METHODS

After data extraction from the included studies, logit-transformation was applied for extracted AUCs for further analysis. The discriminatory abilities of both algorithms were directly compared by their logit (AUC)s. Further subgroup analysis by region (America vs non-America) was also conducted by comparing the corresponding logit (AUC).

RESULTS

The pooled logit (AUC)s of 90-day SORG-CA was .82 (95% confidence interval [CI], .53-.11), 1-year SORG-CA was 1.11 (95% CI, .74-1.48), 90-day SORG-MLA was 1.36 (95% CI, 1.09-1.63), and 1-year SORG-MLA was 1.57 (95% CI, 1.17-1.98). All the algorithms performed better in United States than in Taiwan (P < .001). The performance of SORG-CA was less influenced by a non-American cohort than SORG-MLA.

CONCLUSION

These observations might highlight the importance of incorporating region-specific variables into existing models to make them generalizable to racially or geographically distinct regions.

Comparison of Two Modern Survival Prediction Tools, SORG-MLA and METSSS, in Patients With Symptomatic Long-bone Metastases Who Underwent Local Treatment With Surgery Followed by Radiotherapy and With Radiotherapy Alone

BACKGROUND

Survival estimation for patients with symptomatic skeletal metastases ideally should be made before a type of local treatment has already been determined. Currently available survival prediction tools, however, were generated using data from patients treated either operatively or with local radiation alone, raising concerns about whether they would generalize well to all patients presenting for assessment. The Skeletal Oncology Research Group machine-learning algorithm (SORG-MLA), trained with institution-based data of surgically treated patients, and the Metastases location, Elderly, Tumor primary, Sex, Sickness/comorbidity, and Site of radiotherapy model (METSSS), trained with registry-based data of patients treated with radiotherapy alone, are two of the most recently developed survival prediction models, but they have not been tested on patients whose local treatment strategy is not yet decided.

QUESTIONS/PURPOSES

(1) Which of these two survival prediction models performed better in a mixed cohort made up both of patients who received local treatment with surgery followed by radiotherapy and who had radiation alone for symptomatic bone metastases? (2) Which model performed better among patients whose local treatment consisted of only palliative radiotherapy? (3) Are laboratory values used by SORG-MLA, which are not included in METSSS, independently associated with survival after controlling for predictions made by METSSS?

METHODS

Between 2010 and 2018, we provided local treatment for 2113 adult patients with skeletal metastases in the extremities at an urban tertiary referral academic medical center using one of two strategies: (1) surgery followed by postoperative radiotherapy or (2) palliative radiotherapy alone. Every patient's survivorship status was ascertained either by their medical records or the national death registry from the Taiwanese National Health Insurance Administration. After applying a priori designated exclusion criteria, 91% (1920) were analyzed here. Among them, 48% (920) of the patients were female, and the median (IQR) age was 62 years (53 to 70 years). Lung was the most common primary tumor site (41% [782]), and 59% (1128) of patients had other skeletal metastases in addition to the treated lesion(s). In general, the indications for surgery were the presence of a complete pathologic fracture or an impending pathologic fracture, defined as having a Mirels score of ≥ 9, in patients with an American Society of Anesthesiologists (ASA) classification of less than or equal to IV and who were considered fit for surgery. The indications for radiotherapy were relief of pain, local tumor control, prevention of skeletal-related events, and any combination of the above. In all, 84% (1610) of the patients received palliative radiotherapy alone as local treatment for the target lesion(s), and 16% (310) underwent surgery followed by postoperative radiotherapy. Neither METSSS nor SORG-MLA was used at the point of care to aid clinical decision-making during the treatment period. Survival was retrospectively estimated by these two models to test their potential for providing survival probabilities. We first compared SORG to METSSS in the entire population. Then, we repeated the comparison in patients who received local treatment with palliative radiation alone. We assessed model performance by area under the receiver operating characteristic curve (AUROC), calibration analysis, Brier score, and decision curve analysis (DCA). The AUROC measures discrimination, which is the ability to distinguish patients with the event of interest (such as death at a particular time point) from those without. AUROC typically ranges from 0.5 to 1.0, with 0.5 indicating random guessing and 1.0 a perfect prediction, and in general, an AUROC of ≥ 0.7 indicates adequate discrimination for clinical use. Calibration refers to the agreement between the predicted outcomes (in this case, survival probabilities) and the actual outcomes, with a perfect calibration curve having an intercept of 0 and a slope of 1. A positive intercept indicates that the actual survival is generally underestimated by the prediction model, and a negative intercept suggests the opposite (overestimation). When comparing models, an intercept closer to 0 typically indicates better calibration. Calibration can also be summarized as log(O:E), the logarithm scale of the ratio of observed (O) to expected (E) survivors. A log(O:E) > 0 signals an underestimation (the observed survival is greater than the predicted survival); and a log(O:E) < 0 indicates the opposite (the observed survival is lower than the predicted survival). A model with a log(O:E) closer to 0 is generally considered better calibrated. The Brier score is the mean squared difference between the model predictions and the observed outcomes, and it ranges from 0 (best prediction) to 1 (worst prediction). The Brier score captures both discrimination and calibration, and it is considered a measure of overall model performance. In Brier score analysis, the "null model" assigns a predicted probability equal to the prevalence of the outcome and represents a model that adds no new information. A prediction model should achieve a Brier score at least lower than the null-model Brier score to be considered as useful. The DCA was developed as a method to determine whether using a model to inform treatment decisions would do more good than harm. It plots the net benefit of making decisions based on the model's predictions across all possible risk thresholds (or cost-to-benefit ratios) in relation to the two default strategies of treating all or no patients. The care provider can decide on an acceptable risk threshold for the proposed treatment in an individual and assess the corresponding net benefit to determine whether consulting with the model is superior to adopting the default strategies. Finally, we examined whether laboratory data, which were not included in the METSSS model, would have been independently associated with survival after controlling for the METSSS model's predictions by using the multivariable logistic and Cox proportional hazards regression analyses.

RESULTS

Between the two models, only SORG-MLA achieved adequate discrimination (an AUROC of > 0.7) in the entire cohort (of patients treated operatively or with radiation alone) and in the subgroup of patients treated with palliative radiotherapy alone. SORG-MLA outperformed METSSS by a wide margin on discrimination, calibration, and Brier score analyses in not only the entire cohort but also the subgroup of patients whose local treatment consisted of radiotherapy alone. In both the entire cohort and the subgroup, DCA demonstrated that SORG-MLA provided more net benefit compared with the two default strategies (of treating all or no patients) and compared with METSSS when risk thresholds ranged from 0.2 to 0.9 at both 90 days and 1 year, indicating that using SORG-MLA as a decision-making aid was beneficial when a patient's individualized risk threshold for opting for treatment was 0.2 to 0.9. Higher albumin, lower alkaline phosphatase, lower calcium, higher hemoglobin, lower international normalized ratio, higher lymphocytes, lower neutrophils, lower neutrophil-to-lymphocyte ratio, lower platelet-to-lymphocyte ratio, higher sodium, and lower white blood cells were independently associated with better 1-year and overall survival after adjusting for the predictions made by METSSS.

CONCLUSION

Based on these discoveries, clinicians might choose to consult SORG-MLA instead of METSSS for survival estimation in patients with long-bone metastases presenting for evaluation of local treatment. Basing a treatment decision on the predictions of SORG-MLA could be beneficial when a patient's individualized risk threshold for opting to undergo a particular treatment strategy ranged from 0.2 to 0.9. Future studies might investigate relevant laboratory items when constructing or refining a survival estimation model because these data demonstrated prognostic value independent of the predictions of the METSSS model, and future studies might also seek to keep these models up to date using data from diverse, contemporary patients undergoing both modern operative and nonoperative treatments.

LEVEL OF EVIDENCE

Level III, diagnostic study.

Comparison of reconstructive techniques for nonprimary malignancies in the proximal humerus

BACKGROUND

Endoprostheses (EPC) are often utilized for reconstruction of the proximal humerus with either hemiarthroplasty (HA) or reverse arthroplasty (RA) constructs. RA constructs have improved outcomes in patients with primary lesions, but no studies have compared techniques in metastatic disease. The aim of this study is to compare functional outcomes and complications between HA and RA constructs in patients undergoing endoprosthetic reconstruction for proximal humerus metastases.

METHODS

We retrospectively reviewed our institutional arthroplasty database to identify 66 (56% male; 38 HA and 28 RA) patients with a proximal humerus reconstruction for a non-primary malignancy. The majority (88%) presented with pathologic fracture, and the most common diagnosis was renal cell carcinoma (48%).

RESULTSS

Patients with RA reconstructions had better postoperative forward elevation (74° vs. 32°, p < 0.01) and higher functional outcome scores. HA patients had more complications (odds ratio 13, p < 0.01), with instability being the most common complication.

CONCLUSIONS

Patients with nonprimary malignancies of the proximal humerus had improved functional outcomes and fewer complications after undergoing reconstruction with a reverse EPC compared to a HA EPC. Preference for reverse EPC should be given in patients with good prognosis and ability to complete postoperative rehabilitation.

Machine Learning-Assisted Decision Making in Orthopaedic Oncology

» Artificial intelligence is an umbrella term for computational calculations that are designed to mimic human intelligence and problem-solving capabilities, although in the future, this may become an incomplete definition. Machine learning (ML) encompasses the development of algorithms or predictive models that generate outputs without explicit instructions, assisting in clinical predictions based on large data sets. Deep learning is a subset of ML that utilizes layers of networks that use various inter-relational connections to define and generalize data.» ML algorithms can enhance radiomics techniques for improved image evaluation and diagnosis. While ML shows promise with the advent of radiomics, there are still obstacles to overcome.» Several calculators leveraging ML algorithms have been developed to predict survival in primary sarcomas and metastatic bone disease utilizing patient-specific data. While these models often report exceptionally accurate performance, it is crucial to evaluate their robustness using standardized guidelines.» While increased computing power suggests continuous improvement of ML algorithms, these advancements must be balanced against challenges such as diversifying data, addressing ethical concerns, and enhancing model interpretability.

Comparing long and intermediate-length plates for metastatic bone disease of the proximal humerus: A retrospective analysis

AIMS

Managing proximal humerus pathologic fractures requires strategic planning to ensure optimal patient outcomes. Traditionally, fixation of the humerus using long devices has been considered the standard of care, but emerging evidence has challenged this approach. This study aimed to compare long plates (LPs) and intermediate-length plates (IPs) in this clinical context.

METHODS

Forty-four patients with proximal humerus metastatic bone disease were retrospectively studied from 2013 to 2019, with 11 (25%) receiving long plates (LPs) and 33 (75%) intermediate-length plates (IPs). Outcomes included tumor progression, reoperation rates, postoperative anemia, blood loss, operation time, and hospitalization duration. Tumor progression was classified into three categories, with Type III progression (new metastatic lesions in the distal humerus) theoretically benefiting most from whole bone stabilization.

RESULTS

Tumor progression occurred in three patients (7%), all of them was in IPs. No revision surgery was needed to address these tumor progressions, including one type III progression which occurred 34 months postoperatively after IP surgery. IP were associated with a reduced operation time compared with LP (median, 1.5 h [IQR, 1.2-1.9] vs. 2.4 [IQR, 1.7-2.5]; p = 0.004). No differences were found for the other perioperative outcomes.

CONCLUSIONS

Our findings reveal a low incidence of tumor progression and low reoperation rates in both groups. The shortened operative time associated with IP use suggests its particular suitability for patients with limited life expectancy. Further research is needed to elucidate the ideal prosthesis length that best balances the risks and benefits when addressing proximal humerus metastatic disease.

Preoperative Albumin and Postoperative CRP/Albumin Ratio (CARS) are Independent Predictive Factors in Estimating 1-Year Mortality in Patients Operated for Proximal Femoral Metastasis with Endoprosthesis

Background

Proximal femur resection and prosthetic reconstruction are preferred in patients with extensive bone destruction, pathological fractures, tumours resistant to radiation therapy, and patients with more proximal metastatic lesions. There is increasing evidence that the CRP/albumin ratio (CAR) is an independent marker of inflammation in various primary organ cancers and maybe a more accurate prognostic factor. We aimed to evaluate whether preoperative and postoperative CAR values could be a factor in predicting mortality in these patients. We hypothesized that CAR could predict these patients' postoperative 90-day and 1-year mortality.

Methods

The patient's age and gender, primary tumour, number of bone metastases, and presence of visceral metastases were recorded using imaging techniques such as computed tomography and bone scan or positron emission tomography. The following laboratory data were analyzed before and after surgery.

Results

The mean age of the patients was 62.67 ± 14.8; 56.9% were female (n:29), and 43.1% were male (n:22). When the results of the ROC analysis of the parameters in predicting 1-year mortality were examined, and the cut-off value for preoperative albumin was taken as ≤ 3.75, the AUC value was found to be statistically significant as 0.745 (p:0.003). When the cut-off value for postoperative CAR was taken as ≥ 87.32, the AUC value was found to be 0.7 statistically significant (p:0.015).

Conclusion

Length of stay, preoperative albumin and postoperative CAR values can be used as independent predictive values in predicting 1-year mortality in patients undergoing endoprosthesis due to proximal femur metastasis.

Opportunistic CT for Prediction of Adverse Postoperative Events in Patients with Spinal Metastases

The purpose of this study was to assess the value of body composition measures obtained from opportunistic abdominal computed tomography (CT) in order to predict hospital length of stay (LOS), 30-day postoperative complications, and reoperations in patients undergoing surgery for spinal metastases. 196 patients underwent CT of the abdomen within three months of surgery for spinal metastases. Automated body composition segmentation and quantifications of the cross-sectional areas (CSA) of abdominal visceral and subcutaneous adipose tissue and abdominal skeletal muscle was performed. From this, 31% (61) of patients had postoperative complications within 30 days, and 16% (31) of patients underwent reoperation. Lower muscle CSA was associated with increased postoperative complications within 30 days (OR [95% CI] = 0.99 [0.98-0.99], p = 0.03). Through multivariate analysis, it was found that lower muscle CSA was also associated with an increased postoperative complication rate after controlling for the albumin, ASIA score, previous systemic therapy, and thoracic metastases (OR [95% CI] = 0.99 [0.98-0.99], p = 0.047). LOS and reoperations were not associated with any body composition measures. Low muscle mass may serve as a biomarker for the prediction of complications in patients with spinal metastases. The routine assessment of muscle mass on opportunistic CTs may help to predict outcomes in these patients.

Psoas muscle area is an independent survival prognosticator in patients undergoing surgery for long-bone metastases

BACKGROUND

Predictive analytics is gaining popularity as an aid to treatment planning for patients with bone metastases, whose expected survival should be considered. Decreased psoas muscle area (PMA), a morphometric indicator of suboptimal nutritional status, has been associated with mortality in various cancers, but never been integrated into current survival prediction algorithms (SPA) for patients with skeletal metastases. This study investigates whether decreased PMA predicts worse survival in patients with extremity metastases and whether incorporating PMA into three modern SPAs (PATHFx, SORG-NG, and SORG-MLA) improves their performance.

METHODS

One hundred eighty-five patients surgically treated for long-bone metastases between 2014 and 2019 were divided into three PMA tertiles (small, medium, and large) based on their psoas size on CT. Kaplan-Meier, multivariable regression, and Cox proportional hazards analyses were employed to compare survival between tertiles and examine factors associated with mortality. Logistic regression analysis was used to assess whether incorporating adjusted PMA values enhanced the three SPAs' discriminatory abilities. The clinical utility of incorporating PMA into these SPAs was evaluated by decision curve analysis (DCA).

RESULTS

Patients with small PMA had worse 90-day and 1-year survival after surgery (log-rank test p < 0.001). Patients in the large PMA group had a higher chance of surviving 90 days (odds ratio, OR, 3.72, p = 0.02) and 1 year than those in the small PMA group (OR 3.28, p = 0.004). All three SPAs had increased AUC after incorporation of adjusted PMA. DCA indicated increased net benefits at threshold probabilities >0.5 after the addition of adjusted PMA to these SPAs.

CONCLUSIONS

Decreased PMA on CT is associated with worse survival in surgically treated patients with extremity metastases, even after controlling for three contemporary SPAs. Physicians should consider the additional prognostic value of PMA on survival in patients undergoing consideration for operative management due to extremity metastases.

Outcomes of Long Bones Treated With Carbon-Fiber Nails for Oncologic Indications: International Multi-institutional Study

BACKGROUND

Intramedullary nail fixation is commonly used for prophylactic stabilization of impending and fixation of complete pathological fractures of the long bones. However, metallic artifacts complicate imaging evaluation for bone healing or tumor progression and postoperative radiation planning. Carbon-fiber implants have gained popularity as an alternative, given their radiolucency and superior axial bending. This study evaluates incidences of mechanical and nonmechanical complications.

METHODS

Adult patients (age 18 years and older) treated with carbon-fiber nails for impending/complete pathological long bone fractures secondary to metastases from 2013 to 2020 were analyzed for incidences and risk factors of mechanical and nonmechanical complications. Mechanical complications included aseptic screw loosening and structural failures of host bone and carbon-fiber implants. Deep infection and tumor progression were considered nonmechanical. Other complications/adverse events were also reported.

RESULTS

A total of 239 patients were included; 47% were male, and 53% were female, with a median age of 68 (IQR, 59 to 75) years. Most common secondary metastases were related to breast cancer (19%), lung cancer (19%), multiple myeloma (18%), and sarcoma (13%). In total, 17 of 30 patients with metastatic sarcoma received palliative intramedullary nail fixation for impending/complete pathological fractures, and 13 of 30 received prophylactic nail stabilization of bone radiated preoperatively to manage juxta-osseous soft-tissue sarcomas, where partial resection of the periosteum or bone was necessary for negative margin resection. 33 (14%) patients had complications. Mechanical failures included 4 (1.7%) structural host bone failures, 7 (2.9%) implant structural failures, and 1 (0.4%) aseptic loosening of distal locking screws. Nonmechanical failures included 8 (3.3%) peri-implant infections and 15 (6.3%) tumor progressions with implant contamination. The 90-day and 1-year mortalities were 28% (61/239) and 53% (53/102), respectively. The literature reported comparable failure and mortality rates with conventional titanium treatment.

CONCLUSIONS

Carbon-fiber implants might be an alternative for treating impending and sustained pathological fractures secondary to metastatic bone disease. The seemingly comparable complication profile warrants further cohort studies comparing carbon-fiber and titanium nail complications.

Prognostic factors and real-life applicability of prognostic models for patients with bone metastases of carcinoma

OBJECTIVE

This study aimed to investigate the factors affecting the survival of patients with bone carcinoma metastases and assess the clinical applicability of existing prognostic models.

METHODS

We retrospectively evaluated 247 patients who presented to our hospital between 2011 and 2021 diagnosed with bone carcinoma metastasis. Demographic data, general health status, primary diagnoses, laboratory and radiological findings, pathological fracture status, treatment methods, and survival times of the patients were recorded, and the effects of these variables on survival time were evaluated. Previously developed Katagiri, Janssen, 2013-Spring, PathFX, and SORG prognostic models were applied, and the predictive performances of these models were evaluated by comparing the predicted survival time with the actual survival time of our patients.

RESULTS

After the multivariate analysis, the following factors were shown to be significantly associated with the survival time of patients: blood hemoglobin and leukocyte levels, lactate dehydrogenase concentration, prognostic nutritional index, body mass index, performance status, medium and fast-growing groups of primary tumors, presence of extraspinal and visceral or brain metastases, and pathological fractures. According to receiver operating characteristics and Brier scores, SORG had the overall highest performance scores, while the Janssen nomogram had the lowest.

CONCLUSION

Our report showed that all prognostic models were clinically applicable, but their performances varied. Among them, the SORG predictive model had the best performance scores overall and is the model the authors suggested for survival prediction among patients with carcinoma bone metastases.

LEVEL OF EVIDENCE

Level IV, Prognostic Study.

Does the SORG Machine-learning Algorithm for Extremity Metastases Generalize to a Contemporary Cohort of Patients? Temporal Validation From 2016 to 2020

BACKGROUND

The ability to predict survival accurately in patients with osseous metastatic disease of the extremities is vital for patient counseling and guiding surgical intervention. We, the Skeletal Oncology Research Group (SORG), previously developed a machine-learning algorithm (MLA) based on data from 1999 to 2016 to predict 90-day and 1-year survival of surgically treated patients with extremity bone metastasis. As treatment regimens for oncology patients continue to evolve, this SORG MLA-driven probability calculator requires temporal reassessment of its accuracy.

QUESTION/PURPOSE

Does the SORG-MLA accurately predict 90-day and 1-year survival in patients who receive surgical treatment for a metastatic long-bone lesion in a more recent cohort of patients treated between 2016 and 2020?

METHODS

Between 2017 and 2021, we identified 674 patients 18 years and older through the ICD codes for secondary malignant neoplasm of bone and bone marrow and CPT codes for completed pathologic fractures or prophylactic treatment of an impending fracture. We excluded 40% (268 of 674) of patients, including 18% (118) who did not receive surgery; 11% (72) who had metastases in places other than the long bones of the extremities; 3% (23) who received treatment other than intramedullary nailing, endoprosthetic reconstruction, or dynamic hip screw; 3% (23) who underwent revision surgery, 3% (17) in whom there was no tumor, and 2% (15) who were lost to follow-up within 1 year. Temporal validation was performed using data on 406 patients treated surgically for bony metastatic disease of the extremities from 2016 to 2020 at the same two institutions where the MLA was developed. Variables used to predict survival in the SORG algorithm included perioperative laboratory values, tumor characteristics, and general demographics. To assess the models' discrimination, we computed the c-statistic, commonly referred to as the area under the receiver operating characteristic (AUC) curve for binary classification. This value ranged from 0.5 (representing chance-level performance) to 1.0 (indicating excellent discrimination) Generally, an AUC of 0.75 is considered high enough for use in clinical practice. To evaluate the agreement between predicted and observed outcomes, a calibration plot was used, and the calibration slope and intercept were calculated. Perfect calibration would result in a slope of 1 and intercept of 0. For overall performance, the Brier score and null-model Brier score were determined. The Brier score can range from 0 (representing perfect prediction) to 1 (indicating the poorest prediction). Proper interpretation of the Brier score necessitates a comparison with the null-model Brier score, which represents the score for an algorithm that predicts a probability equal to the population prevalence of the outcome for each patient. Finally, a decision curve analysis was conducted to compare the potential net benefit of the algorithm with other decision-support methods, such as treating all or none of the patients. Overall, 90-day and 1-year mortality were lower in the temporal validation cohort than in the development cohort (90 day: 23% versus 28%; p < 0.001, and 1 year: 51% versus 59%; p<0.001).

RESULTS

Overall survival of the patients in the validation cohort improved from 28% mortality at the 90-day timepoint in the cohort on which the model was trained to 23%, and 59% mortality at the 1-year timepoint to 51%. The AUC was 0.78 (95% CI 0.72 to 0.82) for 90-day survival and 0.75 (95% CI 0.70 to 0.79) for 1-year survival, indicating the model could distinguish the two outcomes reasonably. For the 90-day model, the calibration slope was 0.71 (95% CI 0.53 to 0.89), and the intercept was -0.66 (95% CI -0.94 to -0.39), suggesting the predicted risks were overly extreme, and that in general, the risk of the observed outcome was overestimated. For the 1-year model, the calibration slope was 0.73 (95% CI 0.56 to 0.91) and the intercept was -0.67 (95% CI -0.90 to -0.43). With respect to overall performance, the model's Brier scores for the 90-day and 1-year models were 0.16 and 0.22. These scores were higher than the Brier scores of internal validation of the development study (0.13 and 0.14) models, indicating the models' performance has declined over time.

CONCLUSION

The SORG MLA to predict survival after surgical treatment of extremity metastatic disease showed decreased performance on temporal validation. Moreover, in patients undergoing innovative immunotherapy, the possibility of mortality risk was overestimated in varying severity. Clinicians should be aware of this overestimation and discount the prediction of the SORG MLA according to their own experience with this patient population. Generally, these results show that temporal reassessment of these MLA-driven probability calculators is of paramount importance because the predictive performance may decline over time as treatment regimens evolve. The SORG-MLA is available as a freely accessible internet application at https://sorg-apps.shinyapps.io/extremitymetssurvival/ .Level of Evidence Level III, prognostic study.

Development of Machine-Learning Models to Predict Ambulation Outcomes Following Spinal Metastasis Surgery

STUDY DESIGN

Retrospective cohort study.

PURPOSE

This study aimed to develop machine-learning algorithms to predict ambulation outcomes following surgery for spinal metastasis.

OVERVIEW OF LITERATURE

Postoperative ambulation status following spinal metastasis surgery is currently difficult to predict. The improved ability to predict this important postoperative outcome would facilitate management decision-making and help in determining realistic treatment goals.

METHODS

This retrospective study included patients who underwent spinal metastasis at a university-based medical center in Thailand between January 2009 and November 2021. Collected data included preoperative parameters and ambulatory status 90 and 180 days following surgery. Thirteen machine-learning algorithms, namely, artificial neural network, logistic regression, CatBoost classifier, linear discriminant analysis, extreme gradient boosting, extra trees classifier, random forest classifier, gradient boosting classifier, light gradient boosting machine, naïve Bayes, K-neighbor classifier, Ada boost classifier, and decision tree classifier were developed to predict ambulatory status 90 and 180 days following surgery. Model performance was evaluated using the area under the receiver operating characteristic curve (AUC) and F1-score.

RESULTS

In total, 167 patients were enrolled. The number of patients classified as ambulatory 90 and 180 days following surgery was 140 (81.9%) and 137 (82.0%), respectively. The extreme gradient boosting algorithm was found to most accurately predict 180-day ambulatory outcome (AUC, 0.85; F1-score, 0.90), and the decision tree algorithm most accurately predicted 90-day ambulatory outcome (AUC, 0.94; F1-score, 0.88).

CONCLUSIONS

Machine-learning algorithms were effective in predicting ambulatory status following surgery for spinal metastasis. Based on our data, the extreme gradient boosting and decision tree best predicted postoperative ambulatory status 180 and 90 days after spinal metastasis surgery, respectively.

Predicting diagnosis and survival of bone metastasis in breast cancer using machine learning

This study aimed at establishing more accurate predictive models based on novel machine learning algorithms, with the overarching goal of providing clinicians with effective decision-making assistance. We retrospectively analyzed the breast cancer patients recorded in the Surveillance, Epidemiology, and End Results (SEER) database from 2010 to 2016. Multivariable logistic regression analyses were used to identify risk factors for bone metastases in breast cancer, whereas Cox proportional hazards regression analyses were used to identify prognostic factors for breast cancer with bone metastasis (BCBM). Based on the identified risk and prognostic factors, we developed diagnostic and prognostic models that incorporate six machine learning classifiers. We then used the area under the receiver operating characteristic (ROC) curve (AUC), learning curve, precision curve, calibration plot, and decision curve analysis to evaluate performance of the machine learning models. Univariable and multivariable logistic regression analyses showed that bone metastases were significantly associated with age, race, sex, grade, T stage, N stage, surgery, radiotherapy, chemotherapy, tumor size, brain metastasis, liver metastasis, lung metastasis, breast subtype, and PR. Univariate and multivariate Cox regression analyses revealed that age, race, marital status, grade, surgery, radiotherapy, chemotherapy, brain metastasis, liver metastasis, lung metastasis, breast subtype, ER, and PR were closely associated with the prognosis of BCBM. Among the six machine learning models, the XGBoost algorithm predicted the most accurate results (Diagnostic model AUC = 0.98; Prognostic model AUC = 0.88). According to the Shapley additive explanations (SHAP), the most critical feature of the diagnostic model was surgery, followed by N stage. Interestingly, surgery was also the most critical feature of prognostic model, followed by liver metastasis. Based on the XGBoost algorithm, we could effectively predict the diagnosis and survival of bone metastasis in breast cancer and provide targeted references for the treatment of BCBM patients.

A Machine Learning Algorithm for Predicting 6-Week Survival in Spinal Metastasis: An External Validation Study Using 2,768 Taiwanese Patients

INTRODUCTION

There are predictive algorithms for predicting 3-month and 1-year survival in patients with spinal metastasis. However, advance in surgical technique, immunotherapy, and advanced radiation therapy has enabled shortening of postoperative recovery, which returns dividends to the overall quality-adjusted life-year. As such, the Skeletal Oncology Research Group machine learning algorithm (SORG-MLA) was proposed to predict 6-week survival in patients with spinal metastasis, whereas its utility for patients treated with nonsurgical treatment was untested externally. This study aims to validate the survival prediction of the 6-week SORG-MLA for patients with spinal metastasis and provide the measurement of model consistency (MC).

METHODS

Discrimination using area under the receiver operating characteristic curve, calibration, Brier score, and decision curve analysis were conducted to assess the model's performance in the Taiwanese-based cohort. MC was also applied to detect the proportion of paradoxical predictions among 6-week, 3-month, and 1-year survival predictions. The long-term prognosis should not be better than the shorter-term prognosis in that of an individual.

RESULTS

The 6-week survival rate was 84.2%. The SORG-MLA retained good discrimination with an area under the receiver operating characteristic curve of 0.78 (95% confidence interval, 0.75 to 0.80) and good prediction accuracy with a Brier score of 0.11 (null model Brier score 0.13). There is an underestimation of the 6-week survival rate when the predicted survival rate is less than 50%. Decision curve analysis showed that the model was suitable for use over all threshold probabilities. MC showed suboptimal consistency between 6-week and 90-day survival prediction (78%).

CONCLUSIONS

The results of this study supported the utility of the algorithm. The online tool ( https://sorg-apps.shinyapps.io/spinemetssurvival/ ) can be used by both clinicians and patients in informative decision-making discussion before management of spinal metastasis.

Social determinants of health in prognostic machine learning models for orthopaedic outcomes: A systematic review

RATIONAL

Social determinants of health (SDOH) are being considered more frequently when providing orthopaedic care due to their impact on treatment outcomes. Simultaneously, prognostic machine learning (ML) models that facilitate clinical decision making have become popular tools in the field of orthopaedic surgery. When ML-driven tools are developed, it is important that the perpetuation of potential disparities is minimized. One approach is to consider SDOH during model development. To date, it remains unclear whether and how existing prognostic ML models for orthopaedic outcomes consider SDOH variables.

OBJECTIVE

To investigate whether prognostic ML models for orthopaedic surgery outcomes account for SDOH, and to what extent SDOH variables are included in the final models.

METHODS

A systematic search was conducted in PubMed, Embase and Cochrane for studies published up to 17 November 2020. Two reviewers independently extracted SDOH features using the PROGRESS+ framework (place of residence, race/ethnicity, Occupation, gender/sex, religion, education, social capital, socioeconomic status, 'Plus+' age, disability, and sexual orientation).

RESULTS

The search yielded 7138 studies, of which 59 met the inclusion criteria. Across all studies, 96% (57/59) considered at least one PROGRESS+ factor during development. The most common factors were age (95%; 56/59) and gender/sex (96%; 57/59). Differential effect analyses, such as subgroup analysis, covariate adjustment, and baseline comparison, were rarely reported (10%; 6/59). The majority of models included age (92%; 54/59) and gender/sex (69%; 41/59) as final input variables. However, factors such as insurance status (7%; 4/59), marital status (7%; 4/59) and income (3%; 2/59) were seldom included.

CONCLUSION

The current level of reporting and consideration of SDOH during the development of prognostic ML models for orthopaedic outcomes is limited. Healthcare providers should be critical of the models they consider using and knowledgeable regarding the quality of model development, such as adherence to recognized methodological standards. Future efforts should aim to avoid bias and disparities when developing ML-driven applications for orthopaedics.

Evaluation of different scoring systems for spinal metastases based on a Chinese cohort

INTRODUCTIONS

The spine is one of the most common sites of metastasis for malignancies. This study aimed to compare the predictive performance of seven commonly used prognostic scoring systems for surgically treated spine metastases. It is expected to assist surgeons in selecting appropriate scoring systems to support clinical decision-making and better inform patients.

METHODS

We performed a retrospective study involving 268 surgically treated patients with spine metastases between 2017 and 2020 at a single regional oncology center in China. The revised Tokuhashi, Tomita, modified Bauer, revised Katagiri, van der Linden, Skeletal Oncology Research Group (SORG) nomogram, and SORG machine-learning (ML) scoring systems were externally validated. The area under the curve (AUC) of the receiver operating characteristic curve was used to evaluate sensitivity and specificity at different postoperative time points. The actual survival time was compared with the reference survival time provided in the original publication.

RESULTS

In the present study, the median survival was 16.6 months. The SORG ML scoring system demonstrated the highest accuracy in predicting 90-day (AUC: 0.743) and 1-year survival (AUC: 0.787). The revised Katagiri demonstrated the highest accuracy (AUC: 0.761) in predicting 180-day survival. The revised Katagiri demonstrated the highest accuracy (AUC: 0.779) in predicting 2-year survival. Based on this series, the actual life expectancy was underestimated compared with the original reference survival time.

CONCLUSIONS

None of the scoring systems can perform optimally at all time points and for all pathology types, and the reference survival times provided in the original study need to be updated. A cautious awareness of the underestimation by these models is of paramount importance in relation to current patients.

A machine learning-Based model to predict early death among bone metastatic breast cancer patients: A large cohort of 16,189 patients

Purpose: This study aims to develop a prediction model to categorize the risk of early death among breast cancer patients with bone metastases using machine learning models. Methods: This study examined 16,189 bone metastatic breast cancer patients between 2010 and 2019 from a large oncological database in the United States. The patients were divided into two groups at random in a 90:10 ratio. The majority of patients (n = 14,582, 90%) were served as the training group to train and optimize prediction models, whereas patients in the validation group (n = 1,607, 10%) were utilized to validate the prediction models. Four models were introduced in the study: the logistic regression model, gradient boosting tree model, decision tree model, and random forest model. Results: Early death accounted for 17.4% of all included patients. Multivariate analysis demonstrated that older age; a separated, divorced, or widowed marital status; nonmetropolitan counties; brain metastasis; liver metastasis; lung metastasis; and histologic type of unspecified neoplasms were significantly associated with more early death, whereas a lower grade, a positive estrogen receptor (ER) status, cancer-directed surgery, radiation, and chemotherapy were significantly the protective factors. For the purpose of developing prediction models, the 12 variables were used. Among all the four models, the gradient boosting tree had the greatest AUC [0.829, 95% confident interval (CI): 0.802-0.856], and the random forest (0.828, 95% CI: 0.801-0.855) and logistic regression (0.819, 95% CI: 0.791-0.847) models came in second and third, respectively. The discrimination slopes for the three models were 0.258, 0.223, and 0.240, respectively, and the corresponding accuracy rates were 0.801, 0.770, and 0.762, respectively. The Brier score of gradient boosting tree was the lowest (0.109), followed by the random forest (0.111) and logistic regression (0.112) models. Risk stratification showed that patients in the high-risk group (46.31%) had a greater six-fold chance of early death than those in the low-risk group (7.50%). Conclusion: The gradient boosting tree model demonstrates promising performance with favorable discrimination and calibration in the study, and this model can stratify the risk probability of early death among bone metastatic breast cancer patients.

Does the SORG Orthopaedic Research Group Hip Fracture Delirium Algorithm Perform Well on an Independent Intercontinental Cohort of Patients With Hip Fractures Who Are 60 Years or Older?

BACKGROUND

Postoperative delirium in patients aged 60 years or older with hip fractures adversely affects clinical and functional outcomes. The economic cost of delirium is estimated to be as high as USD 25,000 per patient, with a total budgetary impact between USD 6.6 to USD 82.4 billion annually in the United States alone. Forty percent of delirium episodes are preventable, and accurate risk stratification can decrease the incidence and improve clinical outcomes in patients. A previously developed clinical prediction model (the SORG Orthopaedic Research Group hip fracture delirium machine-learning algorithm) is highly accurate on internal validation (in 28,207 patients with hip fractures aged 60 years or older in a US cohort) in identifying at-risk patients, and it can facilitate the best use of preventive interventions; however, it has not been tested in an independent population. For an algorithm to be useful in real life, it must be valid externally, meaning that it must perform well in a patient cohort different from the cohort used to "train" it. With many promising machine-learning prediction models and many promising delirium models, only few have also been externally validated, and even fewer are international validation studies.

QUESTION/PURPOSE

Does the SORG hip fracture delirium algorithm, initially trained on a database from the United States, perform well on external validation in patients aged 60 years or older in Australia and New Zealand?

METHODS

We previously developed a model in 2021 for assessing risk of delirium in hip fracture patients using records of 28,207 patients obtained from the American College of Surgeons National Surgical Quality Improvement Program. Variables included in the original model included age, American Society of Anesthesiologists (ASA) class, functional status (independent or partially or totally dependent for any activities of daily living), preoperative dementia, preoperative delirium, and preoperative need for a mobility aid. To assess whether this model could be applied elsewhere, we used records from an international hip fracture registry. Between June 2017 and December 2018, 6672 patients older than 60 years of age in Australia and New Zealand were treated surgically for a femoral neck, intertrochanteric hip, or subtrochanteric hip fracture and entered into the Australian & New Zealand Hip Fracture Registry. Patients were excluded if they had a pathological hip fracture or septic shock. Of all patients, 6% (402 of 6672) did not meet the inclusion criteria, leaving 94% (6270 of 6672) of patients available for inclusion in this retrospective analysis. Seventy-one percent (4249 of 5986) of patients were aged 80 years or older, after accounting for 5% (284 of 6270) of missing values; 68% (4292 of 6266) were female, after accounting for 0.06% (4 of 6270) of missing values, and 83% (4690 of 5661) of patients were classified as ASA III/IV, after accounting for 10% (609 of 6270) of missing values. Missing data were imputed using the missForest methodology. In total, 39% (2467 of 6270) of patients developed postoperative delirium. The performance of the SORG hip fracture delirium algorithm on the validation cohort was assessed by discrimination, calibration, Brier score, and a decision curve analysis. Discrimination, known as the area under the receiver operating characteristic curves (c-statistic), measures the model's ability to distinguish patients who achieved the outcomes from those who did not and ranges from 0.5 to 1.0, with 1.0 indicating the highest discrimination score and 0.50 the lowest. Calibration plots the predicted versus the observed probabilities, a perfect plot has an intercept of 0 and a slope of 1. The Brier score calculates a composite of discrimination and calibration, with 0 indicating perfect prediction and 1 the poorest.

RESULTS

The SORG hip fracture algorithm, when applied to an external patient cohort, distinguished between patients at low risk and patients at moderate to high risk of developing postoperative delirium. The SORG hip fracture algorithm performed with a c-statistic of 0.74 (95% confidence interval 0.73 to 0.76). The calibration plot showed high accuracy in the lower predicted probabilities (intercept -0.28, slope 0.52) and a Brier score of 0.22 (the null model Brier score was 0.24). The decision curve analysis showed that the model can be beneficial compared with no model or compared with characterizing all patients as at risk for developing delirium.

CONCLUSION

Algorithms developed with machine learning are a potential tool for refining treatment of at-risk patients. If high-risk patients can be reliably identified, resources can be appropriately directed toward their care. Although the current iteration of SORG should not be relied on for patient care, it suggests potential utility in assessing risk. Further assessment in different populations, made easier by international collaborations and standardization of registries, would be useful in the development of universally valid prediction models. The model can be freely accessed at: https://sorg-apps.shinyapps.io/hipfxdelirium/ .

LEVEL OF EVIDENCE

Level III, therapeutic study.

Prognostic significance of lab data and performance comparison by validating survival prediction models for patients with spinal metastases after radiotherapy

BACKGROUND AND PURPOSE

Well-performing survival prediction models (SPMs) help patients and healthcare professionals to choose treatment aligning with prognosis. This retrospective study aims to investigate the prognostic impacts of laboratory data and to compare the performances of Metastases location, Elderly, Tumor primary, Sex, Sickness/comorbidity, and Site of radiotherapy (METSSS) model, New England Spinal Metastasis Score (NESMS), and Skeletal Oncology Research Group machine learning algorithm (SORG-MLA) for spinal metastases (SM).

MATERIALS AND METHODS

From 2010 to 2018, patients who received radiotherapy (RT) for SM at a tertiary center were enrolled and the data were retrospectively collected. Multivariate logistic and Cox-proportional-hazard regression analyses were used to assess the association between laboratory values and survival. The area under receiver-operating characteristics curve (AUROC), calibration analysis, Brier score, and decision curve analysis were used to evaluate the performance of SPMs.

RESULTS

A total of 2786 patients were included for analysis. The 90-day and 1-year survival rates after RT were 70.4% and 35.7%, respectively. Higher albumin, hemoglobin, or lymphocyte count were associated with better survival, while higher alkaline phosphatase, white blood cell count, neutrophil count, neutrophil-to-lymphocyte ratio, platelet-to-lymphocyte ratio, or international normalized ratio were associated with poor prognosis. SORG-MLA has the best discrimination (AUROC 90-day, 0.78; 1-year 0.76), best calibrations, and the lowest Brier score (90-day 0.16; 1-year 0.18). The decision curve of SORG-MLA is above the other two competing models with threshold probabilities from 0.1 to 0.8.

CONCLUSION

Laboratory data are of prognostic significance in survival prediction after RT for SM. Machine learning-based model SORG-MLA outperforms statistical regression-based model METSSS model and NESMS in survival predictions.

The Prediction of Survival after Surgical Management of Bone Metastases of the Extremities—A Comparison of Prognostic Models

Individualized survival prognostic models for symptomatic patients with appendicular metastatic bone disease are key to guiding clinical decision-making for the orthopedic surgeon. Several prognostic models have been developed in recent years; however, most orthopedic surgeons have not incorporated these models into routine practice. This is possibly due to uncertainty concerning their accuracy and the lack of comparison publications and recommendations. Our aim was to conduct a review and quality assessment of these models. A computerized literature search in MEDLINE, EMBASE and PubMed up to February 2022 was done, using keywords: “Bone metastasis”, “survival”, “extremity” and “prognosis”. We evaluated each model’s performance, assessing the estimated discriminative power and calibration accuracy for the analyzed patients. We included 11 studies out of the 1779 citations initially retrieved. The 11 studies included seven different models for estimating survival. Among externally validated survival prediction scores, PATHFx 3.0, 2013-SPRING and potentially Optimodel were found to be the best models in terms of performance. Currently, it is still a challenge to recommend any of the models as the standard for predicting survival for these patients. However, some models show better performance status and other quality characteristics. We recommend future, large, multicenter, prospective studies to compare between PATHfx 3.0, SPRING 2013 and OptiModel using the same external validation dataset.

A machine learning algorithm for predicting prolonged postoperative opioid prescription after lumbar disc herniation surgery. An external validation study using 1,316 patients from a Taiwanese cohort

BACKGROUND CONTEXT

Preoperative prediction of prolonged postoperative opioid prescription helps identify patients for increased surveillance after surgery. The SORG machine learning model has been developed and successfully tested using 5,413 patients from the United States (US) to predict the risk of prolonged opioid prescription after surgery for lumbar disc herniation. However, external validation is an often-overlooked element in the process of incorporating prediction models in current clinical practice. This cannot be stressed enough in prediction models where medicolegal and cultural differences may play a major role.

PURPOSE

The authors aimed to investigate the generalizability of the US citizens prediction model SORG to a Taiwanese patient cohort.

STUDY DESIGN

Retrospective study at a large academic medical center in Taiwan.

PATIENT SAMPLE

Of 1,316 patients who were 20 years or older undergoing initial operative management for lumbar disc herniation between 2010 and 2018.

OUTCOME MEASURES

The primary outcome of interest was prolonged opioid prescription defined as continuing opioid prescription to at least 90 to 180 days after the first surgery for lumbar disc herniation at our institution.

METHODS

Baseline characteristics were compared between the external validation cohort and the original developmental cohorts. Discrimination (area under the receiver operating characteristic curve and the area under the precision-recall curve), calibration, overall performance (Brier score), and decision curve analysis were used to assess the performance of the SORG ML algorithm in the validation cohort. This study had no funding source or conflict of interests.

RESULTS

Overall, 1,316 patients were identified with sustained postoperative opioid prescription in 41 (3.1%) patients. The validation cohort differed from the development cohort on several variables including 93% of Taiwanese patients receiving NSAIDS preoperatively compared with 22% of US citizens patients, while 30% of Taiwanese patients received opioids versus 25% in the US. Despite these differences, the SORG prediction model retained good discrimination (area under the receiver operating characteristic curve of 0.76 and the area under the precision-recall curve of 0.33) and good overall performance (Brier score of 0.028 compared with null model Brier score of 0.030) while somewhat overestimating the chance of prolonged opioid use (calibration slope of 1.07 and calibration intercept of -0.87). Decision-curve analysis showed the SORG model was suitable for clinical use.

CONCLUSIONS

Despite differences at baseline and a very strict opioid policy, the SORG algorithm for prolonged opioid use after surgery for lumbar disc herniation has good discriminative abilities and good overall performance in a Han Chinese patient group in Taiwan. This freely available digital application can be used to identify high-risk patients and tailor prevention policies for these patients that may mitigate the long-term adverse consequence of opioid dependence: https://sorg-apps.shinyapps.io/lumbardiscopioid/.

Characteristics, Prognosis, and Competing Risk Nomograms of Cutaneous Malignant Melanoma: Evidence for Pigmentary Disorders

Purpose

Cutaneous malignant melanoma (CMM) always presents as a complex disease process with poor prognosis. The objective of the present study was to explore the influence of solitary or multiple cancers on the prognosis of patients with CMM to better understand the landscape of CMM.

Methods

We reviewed the records of CMM patients between 2004 and 2015 from the Surveillance, Epidemiology, and End Results Program. The cumulative incidence function was used to represent the probabilities of death. A novel causal inference method was leveraged to explore the risk difference to death between different types of CMM, and nomograms were built based on competing risk models.

Results

The analysis cohort contained 165,043 patients with CMM as the first primary malignancy. Patients with recurrent CMM and multiple primary tumors had similar overall survival status (p = 0.064), while their demographics and cause-specific death demonstrated different characteristics than those of patients with solitary CMM (p < 0.001), whose mean survival times are 75.4 and 77.3 months and 66.2 months, respectively. Causal inference was further applied to unveil the risk difference of solitary and multiple tumors in subgroups, which was significantly different from the total population (p < 0.05), and vulnerable groups with high risk of death were identified. The established competing risk nomograms had a concordance index >0.6 on predicting the probabilities of death of CMM or other cancers individually across types of CMM.

Conclusion

Patients with different types of CMM had different prognostic characteristics and different risk of cause-specific death. The results of this study are of great significance in identifying the high risk of cause-specific death, enabling targeted intervention in the early period at both the population and individual levels.

Decreased psoas muscle area is a prognosticator for 90-day and 1-year survival in patients undergoing surgical treatment for spinal metastasis

BACKGROUND AND AIMS

Survival estimation for patients with spinal metastasis is crucial to treatment decisions. Psoas muscle area (PMA), a surrogate for total muscle mass, has been proposed as a useful survival prognosticator. However, few studies have validated the predictive value of decreased PMA in an Asian cohort or its predictive value after controlling for existing preoperative scoring systems (PSSs). In this study, we aim to answer: (1) Is PMA associated with survival in Han Chinese patients with spinal metastasis? (2) Is PMA a good prognosticator according to concordance index (c-index) and decision curve analysis (DCA) after controlling for six existing and commonly used PSSs?

METHODS

This study included 180 adult (≥18 years old) Taiwanese patients with a mean age of 58.3 years (range: 22-85) undergoing surgical treatment for spinal metastasis. A patient's PMA was classified into decreased, medium, and large if it fell into the lower (0-33%), middle (33-67%), and upper (67-100%) 1/3 in the study cohort, respectively. We used logistic and cox proportional-hazard regressions to assess whether PMA was associated with 90-day, 1-year, and overall survival. The model performance before and after addition of PMA to six commonly used PSSs, including Tomita score, original Tokuhashi score, revised Tokuhashi score, modified Bauer score, New England Spinal Metastasis Score, and Skeletal Oncology Research Group machine learning algorithms (SORG-MLAs), was compared by c-index and DCA to determine if PMA was a useful survival prognosticator.

RESULTS

Patients with a larger PMA is associated with better 90-day, but not 1-year, survival. The model performance of 90-day survival prediction improved after PMA was incorporated into all PSSs except SORG-MLAs. PMA barely improved the discriminatory ability (c-index, 0.74; 95% confidence interval [CI], 0.67-0.82 vs. c-index, 0.74; 95% CI, 0.66-0.81) and provided little gain of clinical net benefit on DCA for SORG-MLAs' 90-day survival prediction.

CONCLUSIONS

PMA is a prognosticator for 90-day survival and improves the discriminatory ability of earlier-proposed PSSs in our Asian cohort. However, incorporating PMA into more modern PSSs such as SORG-MLAs did not significantly improve its prediction performance.

Body Composition Predictors of Adverse Postoperative Events in Patients Undergoing Surgery for Long Bone Metastases

Body composition assessed using opportunistic CT has been recently identified as a predictor of outcome in patients with cancer. The purpose of this study was to determine whether the cross-sectional area (CSA) and the attenuation of abdominal subcutaneous adipose tissue, visceral adipose tissue (VAT), and paraspinous and abdominal muscles are the predictors of length of hospital stay, 30-day postoperative complications, and revision surgery in patients treated for long bone metastases.

Neuro-Particle Swarm Optimization Based In-Situ Prediction Model for Heavy Metals Concentration in Groundwater and Surface Water

Limited monitoring activities to assess data on heavy metal (HM) concentration contribute to worldwide concern for the environmental quality and the degree of toxicants in areas where there are elevated metals concentrations. Hence, this study used in-situ physicochemical parameters to the limited data on HM concentration in SW and GW. The site of the study was Marinduque Island Province in the Philippines, which experienced two mining disasters. Prediction model results showed that the SW models during the dry and wet seasons recorded a mean squared error (MSE) ranging from 6 × 10-7 to 0.070276. The GW models recorded a range from 5 × 10-8 to 0.045373, all of which were approaching the ideal MSE value of 0. Kling-Gupta efficiency values of developed models were all greater than 0.95. The developed neural network-particle swarm optimization (NN-PSO) models for SW and GW were compared to linear and support vector machine (SVM) models and previously published deterministic and artificial intelligence (AI) models. The findings indicated that the developed NN-PSO models are superior to the developed linear and SVM models, up to 1.60 and 1.40 times greater than the best model observed created by linear and SVM models for SW and GW, respectively. The developed models were also on par with previously published deterministic and AI-based models considering their prediction capability. Sensitivity analysis using Olden's connection weights approach showed that pH influenced the concentration of HM significantly. Established on the research findings, it can be stated that the NN-PSO is an effective and practical approach in the prediction of HM concentration in water resources that contributes a solution to the limited HM concentration monitored data.

International Validation of the SORG Machine-learning Algorithm for Predicting the Survival of Patients with Extremity Metastases Undergoing Surgical Treatment

BACKGROUND

The Skeletal Oncology Research Group machine-learning algorithms (SORG-MLAs) estimate 90-day and 1-year survival in patients with long-bone metastases undergoing surgical treatment and have demonstrated good discriminatory ability on internal validation. However, the performance of a prediction model could potentially vary by race or region, and the SORG-MLA must be externally validated in an Asian cohort. Furthermore, the authors of the original developmental study did not consider the Eastern Cooperative Oncology Group (ECOG) performance status, a survival prognosticator repeatedly validated in other studies, in their algorithms because of missing data.

QUESTIONS/PURPOSES

(1) Is the SORG-MLA generalizable to Taiwanese patients for predicting 90-day and 1-year mortality? (2) Is the ECOG score an independent factor associated with 90-day and 1-year mortality while controlling for SORG-MLA predictions?

METHODS

All 356 patients who underwent surgery for long-bone metastases between 2014 and 2019 at one tertiary care center in Taiwan were included. Ninety-eight percent (349 of 356) of patients were of Han Chinese descent. The median (range) patient age was 61 years (25 to 95), 52% (184 of 356) were women, and the median BMI was 23 kg/m2 (13 to 39 kg/m2). The most common primary tumors were lung cancer (33% [116 of 356]) and breast cancer (16% [58 of 356]). Fifty-five percent (195 of 356) of patients presented with a complete pathologic fracture. Intramedullary nailing was the most commonly performed type of surgery (59% [210 of 356]), followed by plate screw fixation (23% [81 of 356]) and endoprosthetic reconstruction (18% [65 of 356]). Six patients were lost to follow-up within 90 days; 30 were lost to follow-up within 1 year. Eighty-five percent (301 of 356) of patients were followed until death or for at least 2 years. Survival was 82% (287 of 350) at 90 days and 49% (159 of 326) at 1 year. The model's performance metrics included discrimination (concordance index [c-index]), calibration (intercept and slope), and Brier score. In general, a c-index of 0.5 indicates random guess and a c-index of 0.8 denotes excellent discrimination. Calibration refers to the agreement between the predicted outcomes and the actual outcomes, with a perfect calibration having an intercept of 0 and a slope of 1. The Brier score of a prediction model must be compared with and ideally should be smaller than the score of the null model. A decision curve analysis was then performed for the 90-day and 1-year prediction models to evaluate their net benefit across a range of different threshold probabilities. A multivariate logistic regression analysis was used to evaluate whether the ECOG score was an independent prognosticator while controlling for the SORG-MLA's predictions. We did not perform retraining/recalibration because we were not trying to update the SORG-MLA algorithm in this study.

RESULTS

The SORG-MLA had good discriminatory ability at both timepoints, with a c-index of 0.80 (95% confidence interval 0.74 to 0.86) for 90-day survival prediction and a c-index of 0.84 (95% CI 0.80 to 0.89) for 1-year survival prediction. However, the calibration analysis showed that the SORG-MLAs tended to underestimate Taiwanese patients' survival (90-day survival prediction: calibration intercept 0.78 [95% CI 0.46 to 1.10], calibration slope 0.74 [95% CI 0.53 to 0.96]; 1-year survival prediction: calibration intercept 0.75 [95% CI 0.49 to 1.00], calibration slope 1.22 [95% CI 0.95 to 1.49]). The Brier score of the 90-day and 1-year SORG-MLA prediction models was lower than their respective null model (0.12 versus 0.16 for 90-day prediction; 0.16 versus 0.25 for 1-year prediction), indicating good overall performance of SORG-MLAs at these two timepoints. Decision curve analysis showed SORG-MLAs provided net benefits when threshold probabilities ranged from 0.40 to 0.95 for 90-day survival prediction and from 0.15 to 1.0 for 1-year prediction. The ECOG score was an independent factor associated with 90-day mortality (odds ratio 1.94 [95% CI 1.01 to 3.73]) but not 1-year mortality (OR 1.07 [95% CI 0.53 to 2.17]) after controlling for SORG-MLA predictions for 90-day and 1-year survival, respectively.

CONCLUSION

SORG-MLAs retained good discriminatory ability in Taiwanese patients with long-bone metastases, although their actual survival time was slightly underestimated. More international validation and incremental value studies that address factors such as the ECOG score are warranted to refine the algorithms, which can be freely accessed online at https://sorg-apps.shinyapps.io/extremitymetssurvival/.

LEVEL OF EVIDENCE

Level III, therapeutic study.

Artificial intelligence applied to musculoskeletal oncology: a systematic review

Developments in artificial intelligence have the potential to improve the care of patients with musculoskeletal tumors. We performed a systematic review of the published scientific literature to identify the current state of the art of artificial intelligence applied to musculoskeletal oncology, including both primary and metastatic tumors, and across the radiology, nuclear medicine, pathology, clinical research, and molecular biology literature. Through this search, we identified 252 primary research articles, of which 58 used deep learning and 194 used other machine learning techniques. Articles involving deep learning have mostly involved bone scintigraphy, histopathology, and radiologic imaging. Articles involving other machine learning techniques have mostly involved transcriptomic analyses, radiomics, and clinical outcome prediction models using medical records. These articles predominantly present proof-of-concept work, other than the automated bone scan index for bone metastasis quantification, which has translated to clinical workflows in some regions. We systematically review and discuss this literature, highlight opportunities for multidisciplinary collaboration, and identify potentially clinically useful topics with a relative paucity of research attention. Musculoskeletal oncology is an inherently multidisciplinary field, and future research will need to integrate and synthesize noisy siloed data from across clinical, imaging, and molecular datasets. Building the data infrastructure for collaboration will help to accelerate progress towards making artificial intelligence truly useful in musculoskeletal oncology.

Body composition predictors of mortality in patients undergoing surgery for long bone metastases

Background and Objectives

Body composition measurements using computed tomography (CT) may serve as imaging biomarkers of survival in patients with and without cancer. This study assesses whether body composition measurements obtained on abdominal CTs are independently associated with 90‐day and 1‐year mortality in patients with long‐bone metastases undergoing surgery.

Methods

This single institutional retrospective study included 212 patients who had undergone surgery for long‐bone metastases and had a CT of the abdomen within 90 days before surgery. Quantification of cross‐sectional areas (CSA) and CT attenuation of abdominal subcutaneous adipose tissue, visceral adipose tissue, and paraspinous and abdominal muscles were performed at L4. Multivariate Cox proportional‐hazards analyses were performed.

Results

Sarcopenia was independently associated with 90‐day mortality (hazard ratio [HR] = 1.87; 95% confidence interval [CI] = 1.11–3.16; p = 0.019) and 1‐year mortality (HR = 1.50; 95% CI = 1.02–2.19; p = 0.038) in multivariate analysis while controlling for clinical variables such as primary tumors, comorbidities, and chemotherapy. Abdominal fat CSAs and muscle attenuation were not associated with mortality.

Conclusions

The presence of sarcopenia assessed by CT is predictive of 90‐day and 1‐year mortality in patients undergoing surgery for long‐bone metastases. This body composition measurement can be used as novel imaging biomarker supplementing existing prognostic tools to optimize patient selection for surgery and improve shared decision making.

Comparison between different prognostic models to be used for metastatic bone disease on appendicular skeleton in a Chilean population

PURPOSE

Several preoperation prognosis models used on the treatment of metastatic bone disease on appendicular skeleton have been devised. The purpose of this study was to compare the performance of different survival prognostic models on patients with metastatic bone disease in long bones in a Chilean population.

METHODS

This is a multicentric retrospective study. We retrospectively reviewed the medical records of 136 patients who were confirmed with metastatic bone disease of the appendicular skeleton and who were treated surgically from 2016 to 2019. The minimum follow-up time was 12 months. All patients were assessed using four appendicular metastatic bone disease scoring systems. A preoperative predicted survival time for all 136 patients was retrospectively calculated making use of the revised Katagiri, PathFx, Optimodel and IOR score model.

RESULTS

The PathFx model demonstrated an accuracy at predicting 3 (area under the curve [AUC] = 0.61) and 6-month (AUC = 0.65) survival time after surgical management. IOR score model demonstrated an accuracy at predicting 12-month survival time (AUC = 0.64). The survival rate reached the 44% in a year. The median survival time to death or last follow-up time was 14.9 months (SD ± 15).

CONCLUSION

PathFx score model demonstrated the highest accuracy at predicting a survival time of 3 and 6 months. IOR score model was the most accurate measure at predicting a survival time of 12-months. To our knowledge, this is the first study reporting a comparative analysis of metastatic bone disease with predicting models in a country located in Latin America. PathFx's and IOR score models are the ones to be used in the Chilean population as the predictive models in metastatic bone disease of the appendicular skeleton.

What's new in the management of metastatic bone disease

Metastatic bone disease is a common complication of malignant tumours. As cancer treatment improves the overall survival of patients, the number of patients with bone metastases is expected to increase. The treatments for bone metastases include surgery, radiotherapy, and bone-modifying agents, with patients with a short expected prognosis requiring less invasive treatment. Patients with metastatic bone disease show greatly varying primary tumour histology, metastases sites and numbers, and comorbidities. Therefore, randomised clinical trials are indispensable to compare treatments for these patients. This editorial reviews recent findings on the diagnosis and prognosis prediction and discusses the current treatment of patients with metastatic bone disease.

The preoperative machine learning algorithm for extremity metastatic disease can predict 90-day and 1-year survival: An external validation study

BACKGROUND

The prediction of survival is valuable to optimize treatment of metastatic long-bone disease. The Skeletal Oncology Research Group (SORG) machine-learning (ML) algorithm has been previously developed and internally validated. The purpose of this study was to determine if the SORG ML algorithm accurately predicts 90-day and 1-year survival in an external metastatic long-bone disease patient cohort.

METHODS

A retrospective review of 264 patients who underwent surgery for long-bone metastases between 2003 and 2019 was performed. Variables used in the stochastic gradient boosting SORG algorithm were age, sex, primary tumor type, visceral/brain metastases, systemic therapy, and 10 preoperative laboratory values. Model performance was calculated by discrimination, calibration, and overall performance.

RESULTS

The SORG ML algorithms retained good discriminative ability (area under the cure [AUC]: 0.83; 95% confidence interval [CI]: 0.76-0.88 for 90-day mortality and AUC: 0.84; 95% CI: 0.79-0.88 for 1-year mortality), calibration, overall performance, and decision curve analysis.

CONCLUSION

The previously developed ML algorithms demonstrated good performance in the current study, thereby providing external validation. The models were incorporated into an accessible application (https://sorg-apps.shinyapps.io/extremitymetssurvival/) that may be freely utilized by clinicians in helping predict survival for individual patients and assist in informative decision-making discussion before operative management of long bone metastatic lesions.

Prognostic Factors Influencing Survival and a Treatment Pattern Analysis of Conventional Palliative Radiotherapy for Patients with Bone Metastases

BACKGROUND

Treatment indication for bone metastases is influenced by patient prognosis. Single-fraction radiotherapy (SFRT) was proven equally effective as multiple fractionation regimens (MFRT) but continues to be underused.

OBJECTIVE

Primary objectives: (a) to identify prognostic factors for overall survival and (b) to analyze treatment patterns of palliative radiotherapy (proportion of SFRT indication and predictive factors of radiotherapy regimen) for bone metastases.

METHODS

582 patients with bone metastases who underwent conventional radiotherapy between January 1st 2014-31 December 2017 were analyzed. The Cox proportional hazard model was used to identify predictors of overall survival. For the treatment pattern analysis, 677 radiotherapy courses were evaluated. The logistic regression model was used to identify potential predictors of radiotherapy regimen.

RESULTS

The 3-year overall survival was 15%. Prognostic factors associated with poor overall survival were multiple bone metastases [hazard ratio (HR = 5.4)], poor performance status (HR = 1.5) and brain metastases (HR = 1.37). SFRT prescription increased from 41% in 2017 to 51% in 2017. Predictors of SFRT prescription were a poor performance status [odds ratio (OR = 0.55)], lung (OR = 0.49) and urologic primaries (OR = 0.33) and the half-body lower site of irradiation (OR = 0.59). Spinal metastases were more likely to receive MFRT (OR = 2.09).

CONCLUSIONS

Based on the prognostic factors we identified, a selection protocol for patients candidates for palliative radiotherapy to bone metastases could be established, in order to further increase SFRT prescription in our institution.

The Surgical Management of Proximal Femoral Metastases: A Narrative Review

The proximal femur is a common location for the development of bony metastatic disease. Metastatic bone disease in this location can cause debilitating pain, pathologic fractures, reduced quality of life, anemia or hypercalcemia. A thorough history, physical examination and preoperative investigations are required to ensure accurate diagnosis and prognosis. The goals of surgical management is to provide pain relief and return to function with a construct that provides stability to allow for immediate weightbearing. Current surgical treatment options include intramedullary nailing, hemiarthroplasty or total hip arthroplasty and endoprosthetic reconstructions. Oligometastatic renal cell carcinoma must be given special consideration as tumor resection and reconstruction has survival benefit. Both tumor and patient characteristics must be taken into account before deciding on the appropriate surgical intervention.

Current Overview of Treatment for Metastatic Bone Disease

The number of patients with bone metastasis increases as medical management and surgery improve the overall survival of patients with cancer. Bone metastasis can cause skeletal complications, including bone pain, pathological fractures, spinal cord or nerve root compression, and hypercalcemia. Before initiation of treatment for bone metastasis, it is important to exclude primary bone malignancy, which would require a completely different therapeutic approach. It is essential to select surgical methods considering the patient’s prognosis, quality of life, postoperative function, and risk of postoperative complications. Therefore, bone metastasis treatment requires a multidisciplinary team approach, including radiologists, oncologists, and orthopedic surgeons. Recently, many novel palliative treatment options have emerged for bone metastases, such as stereotactic body radiation therapy, radiopharmaceuticals, vertebroplasty, minimally invasive spine stabilization with percutaneous pedicle screws, acetabuloplasty, embolization, thermal ablation techniques, electrochemotherapy, and high-intensity focused ultrasound. These techniques are beneficial for patients who may not benefit from surgery or radiotherapy.

What's new in musculoskeletal oncology

We reviewed the recent literature related to primary musculoskeletal tumors and metastatic bone tumors. With regard to primary bone tumors, computer navigation systems and three-dimensional-printed prostheses seem to be new treatment options, especially in challenging anatomical locations, such as the sacrum and pelvis. Regarding the treatment of giant cell tumor of bone, recent studies have suggested that denosumab administration is related to a higher local recurrence rate following curettage, but a lower local recurrence rate following en bloc resection. In addition, there was no difference in the local recurrence rate at five years after surgery between short-term and long-term denosumab therapy. With regard to soft tissue tumors, percutaneous cryoablation appears to be a new treatment option for extra-abdominal desmoid tumors, with encouraging results. Regarding soft tissue sarcomas, a negative surgical margin of < 1 mm is sufficient to control local recurrence. Pexidartinib seems to be a promising systemic therapy for the treatment of tenosynovial giant cell tumors for which surgery is not expected to improve the function of the affected limb. Finally, the life expectancy of patients is the most important factor in determining the optimal surgical procedure for patients with impending or pathological fractures of the long bone due to metastatic bone tumors. Elevated C-reactive protein level was found to be an independent poor prognostic factor at 1 year after surgery for long bone metastases.

Improving Risk Identification of Adverse Outcomes in Chronic Heart Failure Using SMOTE+ENN and Machine Learning

PURPOSE

This study sought to develop models with good identification for adverse outcomes in patients with heart failure (HF) and find strong factors that affect prognosis.

PATIENTS AND METHODS

A total of 5004 qualifying cases were selected, among which 498 cases had adverse outcomes and 4506 cases were discharged after improvement. The study subjects were hospitalized patients diagnosed with HF from a regional cardiovascular hospital and the cardiology department of a medical university hospital in Shanxi Province of China between January 2014 and June 2019. Synthesizing minority oversampling technology combined with edited nearest neighbors (SMOTE+ENN) was used to pre-process unbalanced data. Traditional logistic regression (LR), k-nearest neighbor (KNN), support vector machine (SVM), random forest (RF), and extreme gradient boosting (XGBoost) were used to build risk identification models, and each model was repeated 100 times. Model discrimination and calibration were estimated using F1-score, the area under the receiver-operating characteristic curve (AUROC), and Brier score. The best performing of the five models was used to identify the risk of adverse outcomes and evaluate the influencing factors.

RESULTS

The SME-XGBoost was the best performing model with means of F1-score (0.3673, 95% confidence interval [CI]: 0.3633-0.3712), AUC (0.8010, CI: 0.7974-0.8046), and Brier score (0.1769, CI: 0.1748-0.1789). Age, N-terminal pronatriuretic peptide, pulmonary disease, etc. were the most significant factors of adverse outcomes in patients with HF.

CONCLUSION

The combination of SMOTE+ENN and advanced machine learning methods effectively improved the discrimination efficacy of adverse outcomes in HF patients, accurately stratified patients at risk of adverse outcomes, and found the top factors of adverse outcomes. These models and factors emphasize the importance of health status data in determining adverse outcomes in patients with HF.

Artificial intelligence predicts disk re-herniation following lumbar microdiscectomy: development of the "RAD" risk profile

PURPOSE

Surgical treatment of herniated lumbar intervertebral disks is a common procedure worldwide. However, recurrent herniated nucleus pulposus (re-HNP) may develop, complicating outcomes and patient management. The purpose of this study was to utilize machine-learning (ML) analytics to predict lumbar re-HNP, whereby a personalized risk prediction can be developed as a clinical tool.

METHODS

A retrospective, single center study was conducted of 2630 consecutive patients that underwent lumbar microdiscectomy (mean follow-up: 22-months). Various preoperative patient pain/disability/functional profiles, imaging parameters, and anthropomorphic/demographic metrics were noted. An Extreme Gradient Boost (XGBoost) classifier was implemented to develop a predictive model identifying patients at risk for re-HNP. The model was exported to a web application software for clinical utility.

RESULTS

There were 1608 males and 1022 females, 114 of whom experienced re-HNP. Primary herniations were central (65.8%), paracentral (17.6%), and far lateral (17.1%). The XGBoost algorithm identified multiple re-HNP predictors and was incorporated into an open-access web application software, identifying patients at low or high risk for re-HNP. Preoperative VAS leg, disability, alignment parameters, elevated body mass index, symptom duration, and age were the strongest predictors.

CONCLUSIONS

Our predictive modeling via an ML approach of our large-scale cohort is the first study, to our knowledge, that has identified significant risk factors for the development of re-HNP after initial lumbar decompression. We developed the re-herniation after decompression (RAD) profile index that has been translated into an online screening tool to identify low-high risk patients for re-HNP. Additional validation is needed for potential global implementation.

Prospective study for establishing minimal clinically important differences in patients with surgery for lower extremity metastases

BACKGROUND

The clinical relevance of patient-reported outcomes score changes is often unclear. Especially in patients undergoing surgery due to lower extremity metastases - where surgery is performed in the palliative setting and the goal is to optimize functional mobility, relieve pain and improve overall quality of life. This study assessed the minimal clinically important difference (MCID) of Patient-Reported Outcomes Measurement Information System (PROMIS) Pain Interference, Cancer-specific Physical Function, and Global (Physical and Mental Health) in patients treated surgically for impending or completed pathologic fractures.

METHODS

Patients undergoing surgery for osseous metastasis of the lower extremity because of an impending or completed pathologic fracture were consecutively enrolled in this tertiary center study. Patients completed the three PROMIS questionnaires preoperatively (n = 56) and at postoperative follow-up (n = 33) assessment one to three months later. Of the 23 patients that did not complete the postoperative survey, 5 patients died within 1-3 months and 18 patients were alive at 3-months but did not respond or show up at their postoperative consult. Thirty-one patients (94%) of the 33 included patients reported at least minimal improvement and two patients (6.1%) no change 1-3 months after the surgery based on an anchor-based approach.

RESULTS

The PROMIS MCIDs (95% confidence interval) for Pain Interference was 7.5 (3.4-12), Physical Function 4.1 (0.6-7.6), Global Physical Health 4.2 (2.0-6.6), and Global Mental Health 0.8 (-4.5-2.9).

CONCLUSION

This prospective study successfully defined a MCID for PROMIS Pain Interference of 7.5 (3.4-12), PROMIS Physical Function of 4.1 (0.6-7.6), and Global Physical Health of 4.2 (2.0-6.6) in patients with (impending) pathological fractures due to osseous metastases in the lower extremity; no MCID could be established for PROMIS Global Mental Health. Defining a narrower MCID value for each subpopulation requires a large, prospective, multicenter study. Nevertheless, the provided MCID values allow guidance to clinicians to evaluate the impact of surgical treatment on a patient's QoL.

LEVEL OF EVIDENCE

Level II Diagnostic study.

Development of machine learning model algorithm for prediction of 5-year soft tissue myxoid liposarcoma survival

BACKGROUND

Predicting survival in myxoid liposarcoma (MLS) patients is very challenging given its propensity to metastasize and the controversial role of adjuvant therapy. The purpose of this study was to develop a machine-learning algorithm for the prediction of survival at five years for patients with MLS and externally validate it using our institutional cohort.

METHODS

Two databases, the surveillance, epidemiology, and end results program (SEER) database and an institutional database, were used in this study. Five machine learning models were created based on the SEER database and performance was rated using the TRIPOD criteria. The model that performed best on the SEER data was again tested on our institutional database.

RESULTS

The net-elastic penalized logistic regression model was the best according to our performance indicators. This model had an area under the curve (AUC) of 0.85 when compared to the SEER testing data and an AUC of 0.76 when tested against institutional database. An application to use this calculator is available at https://sorg-apps.shinyapps.io/myxoid_liposarcoma/.

CONCLUSION

MLS is a soft-tissue sarcoma with adjunct treatment options that are, in part, decided by prognostic survival. We developed the first machine-learning predictive algorithm specifically for MLS using the SEER registry that retained performance during external validation with institutional data.

Diagnostic Performance of Artificial Intelligence for Detection of Anterior Cruciate Ligament and Meniscus Tears: A Systematic Review

PURPOSE

To (1) determine the diagnostic efficacy of artificial intelligence (AI) methods for detecting anterior cruciate ligament (ACL) and meniscus tears and to (2) compare the efficacy to human clinical experts.

METHODS

PubMed, OVID/Medline, and Cochrane libraries were queried in November 2019 for research articles pertaining to AI use for detection of ACL and meniscus tears. Information regarding AI model, prediction accuracy/area under the curve (AUC), sample sizes of testing/training sets, and imaging modalities were recorded.

RESULTS

A total of 11 AI studies were identified: 5 investigated ACL tears, 5 investigated meniscal tears, and 1 investigated both. The AUC of AI models for detecting ACL tears ranged from 0.895 to 0.980, and the prediction accuracy ranged from 86.7% to 100%. Of these studies, 3 compared AI models to clinical experts. Two found no significant differences in diagnostic capability, whereas one found that radiologists had a significantly greater sensitivity for detecting ACL tears (P = .002) and statistically similar specificity and accuracy. Of the 5 studies investigating the meniscus, the AUC for AI models ranged from 0.847 to 0.910 and prediction accuracy ranged from 75.0% to 90.0%. Of these studies, 2 compared AI models with clinical experts. One found no significant differences in diagnostic accuracy, whereas one found that the AI model had a significantly lower specificity (P = .003) and accuracy (P = .015) than radiologists. Two studies reported that the addition of AI models significantly increased the diagnostic performance of clinicians compared to their efforts without these models.

CONCLUSIONS

AI prediction capabilities were excellent and may enhance the diagnosis of ACL and meniscal pathology; however, AI did not outperform clinical experts.

CLINICAL RELEVANCE

AI models promise to improve diagnosing certain pathologies as well as or better than human experts, are excellent for detecting ACL and meniscus tears, and may enhance the diagnostic capabilities of human experts; however, when compared with these experts, they may not offer any significant advantage.

Does Artificial Intelligence Outperform Natural Intelligence in Interpreting Musculoskeletal Radiological Studies? A Systematic Review

BACKGROUND

Machine learning (ML) is a subdomain of artificial intelligence that enables computers to abstract patterns from data without explicit programming. A myriad of impactful ML applications already exists in orthopaedics ranging from predicting infections after surgery to diagnostic imaging. However, no systematic reviews that we know of have compared, in particular, the performance of ML models with that of clinicians in musculoskeletal imaging to provide an up-to-date summary regarding the extent of applying ML to imaging diagnoses. By doing so, this review delves into where current ML developments stand in aiding orthopaedists in assessing musculoskeletal images.

QUESTIONS/PURPOSES

This systematic review aimed (1) to compare performance of ML models versus clinicians in detecting, differentiating, or classifying orthopaedic abnormalities on imaging by (A) accuracy, sensitivity, and specificity, (B) input features (for example, plain radiographs, MRI scans, ultrasound), (C) clinician specialties, and (2) to compare the performance of clinician-aided versus unaided ML models.

METHODS

A systematic review was performed in PubMed, Embase, and the Cochrane Library for studies published up to October 1, 2019, using synonyms for machine learning and all potential orthopaedic specialties. We included all studies that compared ML models head-to-head against clinicians in the binary detection of abnormalities in musculoskeletal images. After screening 6531 studies, we ultimately included 12 studies. We conducted quality assessment using the Methodological Index for Non-randomized Studies (MINORS) checklist. All 12 studies were of comparable quality, and they all clearly included six of the eight critical appraisal items (study aim, input feature, ground truth, ML versus human comparison, performance metric, and ML model description). This justified summarizing the findings in a quantitative form by calculating the median absolute improvement of the ML models compared with clinicians for the following metrics of performance: accuracy, sensitivity, and specificity.

RESULTS

ML models provided, in aggregate, only very slight improvements in diagnostic accuracy and sensitivity compared with clinicians working alone and were on par in specificity (3% (interquartile range [IQR] -2.0% to 7.5%), 0.06% (IQR -0.03 to 0.14), and 0.00 (IQR -0.048 to 0.048), respectively). Inputs used by the ML models were plain radiographs (n = 8), MRI scans (n = 3), and ultrasound examinations (n = 1). Overall, ML models outperformed clinicians more when interpreting plain radiographs than when interpreting MRIs (17 of 34 and 3 of 16 performance comparisons, respectively). Orthopaedists and radiologists performed similarly to ML models, while ML models mostly outperformed other clinicians (outperformance in 7 of 19, 7 of 23, and 6 of 10 performance comparisons, respectively). Two studies evaluated the performance of clinicians aided and unaided by ML models; both demonstrated considerable improvements in ML-aided clinician performance by reporting a 47% decrease of misinterpretation rate (95% confidence interval [CI] 37 to 54; p < 0.001) and a mean increase in specificity of 0.048 (95% CI 0.029 to 0.068; p < 0.001) in detecting abnormalities on musculoskeletal images.

CONCLUSIONS

At present, ML models have comparable performance to clinicians in assessing musculoskeletal images. ML models may enhance the performance of clinicians as a technical supplement rather than as a replacement for clinical intelligence. Future ML-related studies should emphasize how ML models can complement clinicians, instead of determining the overall superiority of one versus the other. This can be accomplished by improving transparent reporting, diminishing bias, determining the feasibility of implantation in the clinical setting, and appropriately tempering conclusions.

LEVEL OF EVIDENCE

Level III, diagnostic study.

Compilation and Analysis of Web-Based Orthopedic Personalized Predictive Tools: A Scoping Review

Web-based personalized predictive tools in orthopedic surgery are becoming more widely available. Despite rising numbers of these tools, many orthopedic surgeons may not know what tools are available, how these tools were developed, and how they can be utilized. The aim of this scoping review is to compile and synthesize the profile of existing web-based orthopedic tools. We conducted two separate PubMed searches-one a broad search and the second a more targeted one involving high impact journals-with the aim of comprehensively identifying all existing tools. These articles were then screened for functional tool URLs, methods regarding the tool's creation, and general inputs and outputs required for the tool to function. We identified 57 articles, which yielded 31 unique web-based tools. These tools involved various orthopedic conditions (e.g., fractures, osteoarthritis, musculoskeletal neoplasias); interventions (e.g., fracture fixation, total joint arthroplasty); outcomes (e.g., mortality, clinical outcomes). This scoping review highlights the availability and utility of a vast array of web-based personalized predictive tools for orthopedic surgeons. Increased awareness and access to these tools may allow for better decision support, surgical planning, post-operative expectation management, and improved shared decision-making.