Machine Learning Approaches for Fracture Risk Assessment: A Comparative Analysis of Genomic and Phenotypic Data in 5130 Older Men

Enhanced osteoporotic fracture prediction in postmenopausal women using Bayesian optimization of machine learning models with genetic risk score

This study aimed to enhance the fracture risk prediction accuracy in major osteoporotic fractures (MOFs) and hip fractures (HFs) by integrating genetic profiles, machine learning (ML) techniques, and Bayesian optimization. The genetic risk score (GRS), derived from 1,103 risk single nucleotide polymorphisms (SNPs) from genome-wide association studies (GWAS), was formulated for 25,772 postmenopausal women from the Women's Health Initiative dataset. We developed four ML models: Support Vector Machine (SVM), Random Forest, XGBoost, and Artificial Neural Network (ANN) for binary fracture outcome and 10-year fracture risk prediction. GRS and FRAX clinical risk factors (CRFs) were used as predictors. Death as a competing risk was accounted for in ML models for time-to-fracture data. ML models were subsequently fine-tuned through Bayesian optimization, which displayed marked superiority over traditional grid search. Evaluation of the models' performance considered an array of metrics such as accuracy, weighted F1 Score, the area under the precision-recall curve (PRAUC), and the area under the receiver operating characteristic curve (AUC) for binary fracture predictions, and the C-index, Brier score, and dynamic mean AUC over a 10-year follow-up period for fracture risk predictions. We found that GRS-integrated XGBoost with Bayesian optimization is the most effective model, with an accuracy of 91.2% (95% CI: 90.4-92.0%) and an AUC of 0.739 (95% CI: 0.731-0.746) in MOF binary predictions. For 10-year fracture risk modeling, the XGBoost model attained a C-index of 0.795 (95% CI: 0.783-0.806) and a mean dynamic AUC of 0.799 (95% CI: 0.788-0.809). Compared to FRAX, the XGBoost model exhibited a categorical net reclassification improvement (NRI) of 22.6% (P = .004). A sensitivity analysis, which included BMD but lacked GRS, reaffirmed these findings. Furthermore, portability tests in diverse non-European groups, including Asians and African Americans, underscored the model's robustness and adaptability. This study accentuates the potential of combining genetic insights and optimized ML in strengthening fracture predictions, heralding new preventive strategies for postmenopausal women.

AI algorithms for accurate prediction of osteoporotic fractures in patients with diabetes: an up-to-date review

Osteoporotic fractures impose a substantial burden on patients with diabetes due to their unique characteristics in bone metabolism, limiting the efficacy of conventional fracture prediction tools. Artificial intelligence (AI) algorithms have shown great promise in predicting osteoporotic fractures. This review aims to evaluate the application of traditional fracture prediction tools (FRAX, QFracture, and Garvan FRC) in patients with diabetes and osteoporosis, review AI-based fracture prediction achievements, and assess the potential efficiency of AI algorithms in this population. This comprehensive literature search was conducted in Pubmed and Web of Science. We found that conventional prediction tools exhibit limited accuracy in predicting fractures in patients with diabetes and osteoporosis due to their distinct bone metabolism characteristics. Conversely, AI algorithms show remarkable potential in enhancing predictive precision and improving patient outcomes. However, the utilization of AI algorithms for predicting osteoporotic fractures in diabetic patients is still in its nascent phase, further research is required to validate their efficacy and assess the potential advantages of their application in clinical practice.

Predictive value of machine learning on fracture risk in osteoporosis: a systematic review and meta-analysis

OBJECTIVES

Early identification of fracture risk in patients with osteoporosis is essential. Machine learning (ML) has emerged as a promising technique to predict the risk, whereas its predictive performance remains controversial. Therefore, we conducted this systematic review and meta-analysis to explore the predictive efficiency of ML for the risk of fracture in patients with osteoporosis.

METHODS

Relevant studies were retrieved from four databases (PubMed, Embase, Cochrane Library and Web of Science) until 31 May 2023. A meta-analysis of the C-index was performed using a random-effects model, while a bivariate mixed-effects model was used for the meta-analysis of sensitivity and specificity. In addition, subgroup analysis was performed according to the types of ML models and fracture sites.

RESULTS

Fifty-three studies were included in our meta-analysis, involving 15 209 268 patients, 86 prediction models specifically developed for the osteoporosis population and 41 validation sets. The most commonly used predictors in these models encompassed age, BMI, past fracture history, bone mineral density T-score, history of falls, BMD, radiomics data, weight, height, gender and other chronic diseases. Overall, the pooled C-index of ML was 0.75 (95% CI: 0.72, 0.78) and 0.75 (95% CI: 0.71, 0.78) in the training set and validation set, respectively; the pooled sensitivity was 0.79 (95% CI: 0.72, 0.84) and 0.76 (95% CI: 0.80, 0.81) in the training set and validation set, respectively; and the pooled specificity was 0.81 (95% CI: 0.75, 0.86) and 0.83 (95% CI: 0.72, 0.90) in the training set and validation set, respectively.

CONCLUSIONS

ML has a favourable predictive performance for fracture risk in patients with osteoporosis. However, most current studies lack external validation. Thus, external validation is required to verify the reliability of ML models.

PROSPERO REGISTRATION NUMBER

CRD42022346896.

Development and validation of sex-specific hip fracture prediction models using electronic health records: a retrospective, population-based cohort study

BACKGROUND

Hip fracture is associated with immobility, morbidity, mortality, and high medical cost. Due to limited availability of dual-energy X-ray absorptiometry (DXA), hip fracture prediction models without using bone mineral density (BMD) data are essential. We aimed to develop and validate 10-year sex-specific hip fracture prediction models using electronic health records (EHR) without BMD.

METHODS

In this retrospective, population-based cohort study, anonymized medical records were retrieved from the Clinical Data Analysis and Reporting System for public healthcare service users in Hong Kong aged ≥60 years as of 31 December 2005. A total of 161,051 individuals (91,926 female; 69,125 male) with complete follow-up from 1 January 2006 till the study end date on 31 December 2015 were included in the derivation cohort. The sex-stratified derivation cohort was randomly divided into 80% training and 20% internal testing datasets. An independent validation cohort comprised 3046 community-dwelling participants aged ≥60 years as of 31 December 2005 from the Hong Kong Osteoporosis Study, a prospective cohort which recruited participants between 1995 and 2010. With 395 potential predictors (age, diagnosis, and drug prescription records from EHR), 10-year sex-specific hip fracture prediction models were developed using stepwise selection by logistic regression (LR) and four machine learning (ML) algorithms (gradient boosting machine, random forest, eXtreme gradient boosting, and single-layer neural networks) in the training cohort. Model performance was evaluated in both internal and independent validation cohorts.

FINDINGS

In female, the LR model had the highest AUC (0.815; 95% Confidence Interval [CI]: 0.805-0.825) and adequate calibration in internal validation. Reclassification metrics showed the LR model had better discrimination and classification performance than the ML algorithms. Similar performance was attained by the LR model in independent validation, with high AUC (0.841; 95% CI: 0.807-0.87) comparable to other ML algorithms. In internal validation for male, LR model had high AUC (0.818; 95% CI: 0.801-0.834) and it outperformed all ML models as indicated by reclassification metrics, with adequate calibration. In independent validation, the LR model had high AUC (0.898; 95% CI: 0.857-0.939) comparable to ML algorithms. Reclassification metrics demonstrated that LR model had the best discrimination performance.

INTERPRETATION

Even without using BMD data, the 10-year hip fracture prediction models developed by conventional LR had better discrimination performance than the models developed by ML algorithms. Upon further validation in independent cohorts, the LR models could be integrated into the routine clinical workflow, aiding the identification of people at high risk for DXA scan.

FUNDING

Health and Medical Research Fund, Health Bureau, Hong Kong SAR Government (reference: 17181381).

Predictive models for small-for-gestational-age births in women exposed to pesticides before pregnancy based on multiple machine learning algorithms

Background

The association between prenatal pesticide exposures and a higher incidence of small-for-gestational-age (SGA) births has been reported. No prediction model has been developed for SGA neonates in pregnant women exposed to pesticides prior to pregnancy.

Methods

A retrospective cohort study was conducted using information from the National Free Preconception Health Examination Project between 2010 and 2012. A development set (n = 606) and a validation set (n = 151) of the dataset were split at random. Traditional logistic regression (LR) method and six machine learning classifiers were used to develop prediction models for SGA neonates. The Shapley Additive Explanation (SHAP) model was applied to determine the most influential variables that contributed to the outcome of the prediction.

Results

757 neonates in total were analyzed. SGA occurred in 12.9% (n = 98) of cases overall. With an area under the receiver-operating-characteristic curve (AUC) of 0.855 [95% confidence interval (CI): 0.752–0.959], the model based on category boosting (CatBoost) algorithm obtained the best performance in the validation set. With the exception of the LR model (AUC: 0.691, 95% CI: 0.554–0.828), all models had good AUCs. Using recursive feature elimination (RFE) approach to perform the feature selection, we included 15 variables in the final model based on CatBoost classifier, achieving the AUC of 0.811 (95% CI: 0.675–0.947).

Conclusions

Machine learning algorithms can develop satisfactory tools for SGA prediction in mothers exposed to pesticides prior to pregnancy, which might become a tool to predict SGA neonates in the high-risk population.

Using machine learning for mortality prediction and risk stratification in atezolizumab-treated cancer patients: Integrative analysis of eight clinical trials

BACKGROUND

Few models exist to predict mortality in cancer patients receiving immunotherapy. Our aim was to build a machine learning-based risk stratification model for predicting mortality in atezolizumab-treated cancer patients.

METHODS

Data from 2538 patients in eight atezolizumab-treated cancer clinical trials across three cancer types (non-small-cell lung cancer, bladder transitional cell carcinoma, and renal cell carcinoma) were included. The whole cohort was randomly split into development and validation cohorts in a 7:3 ratio. Machine-learning algorithms (extreme gradient boosting, random forest, logistic regression with lasso regularization, support vector machine, and K-nearest neighbor) were applied to develop prediction models. Model performance was mainly assessed by area under the receiver operating characteristic curve (AUC) value, calibration plot, and decision curve analysis. The probability of death risk was then stratified.

RESULTS

One thousand and three hundred and seventy-nine (54.33%) patients died. The random forest (RF) model was overall the best in terms of predictive performance, with the AUC of 0.844 (95% confidence interval [CI]: 0.826-0.862) in the development cohort and 0.786 (95% CI: 0.754-0.818) in the validation cohort for predicting mortality. Twelve baseline variables contributing to mortality prediction in the RF model were C-reactive protein, PD-L1 level, cancer type, prior liver metastasis, derived neutrophil-to-lymphocyte ratio, alkaline phosphatase, albumin, hemoglobin, white blood cell count, number of metastatic sites, pulse rate, and Eastern Cooperative Oncology Group (ECOG) performance status. A total of 1782 (70.2%) patients were separated into the high-risk and 756 (29.8%) low-risk groups. Patients in the high-risk group were significantly more likely to die, experience disease progression, discontinue study, and discontinue treatment than patients in the low-risk group (all p values < 0.001). Risk groups were not associated with immune-related adverse events and grades 3-5 treatment-related adverse events (all p values > 0.05).

CONCLUSION

RF model has good performance in mortality prediction and risk stratification for cancer patients receiving atezolizumab monotherapy.

Development and Evaluation of a Machine Learning Prediction Model for Small-for-Gestational-Age Births in Women Exposed to Radiation before Pregnancy

Exposure to radiation has been associated with increased risk of delivering small-for-gestational-age (SGA) newborns. There are no tools to predict SGA newborns in pregnant women exposed to radiation before pregnancy. Here, we aimed to develop an array of machine learning (ML) models to predict SGA newborns in women exposed to radiation before pregnancy. Patients’ data was obtained from the National Free Preconception Health Examination Project from 2010 to 2012. The data were randomly divided into a training dataset (n = 364) and a testing dataset (n = 91). Eight various ML models were compared for solving the binary classification of SGA prediction, followed by a post hoc explainability based on the SHAP model to identify and interpret the most important features that contribute to the prediction outcome. A total of 455 newborns were included, with the occurrence of 60 SGA births (13.2%). Overall, the model obtained by extreme gradient boosting (XGBoost) achieved the highest area under the receiver-operating-characteristic curve (AUC) in the testing set (0.844, 95% confidence interval (CI): 0.713–0.974). All models showed satisfied AUCs, except for the logistic regression model (AUC: 0.561, 95% CI: 0.355–0.768). After feature selection by recursive feature elimination (RFE), 15 features were included in the final prediction model using the XGBoost algorithm, with an AUC of 0.821 (95% CI: 0.650–0.993). ML algorithms can generate robust models to predict SGA newborns in pregnant women exposed to radiation before pregnancy, which may thus be used as a prediction tool for SGA newborns in high-risk pregnant women.

Automated Breast Volume Scanner (ABVS)-Based Radiomic Nomogram: A Potential Tool for Reducing Unnecessary Biopsies of BI-RADS 4 Lesions

Improving the assessment of breast imaging reporting and data system (BI-RADS) 4 lesions and reducing unnecessary biopsies are urgent clinical issues. In this prospective study, a radiomic nomogram based on the automated breast volume scanner (ABVS) was constructed to identify benign and malignant BI-RADS 4 lesions and evaluate its value in reducing unnecessary biopsies. A total of 223 histologically confirmed BI-RADS 4 lesions were enrolled and assigned to the training and validation cohorts. A radiomic score was generated from the axial, sagittal, and coronal ABVS images. Combining the radiomic score and clinical-ultrasound factors, a radiomic nomogram was developed by multivariate logistic regression analysis. The nomogram integrating the radiomic score, lesion size, and BI-RADS 4 subcategories showed good discrimination between malignant and benign BI-RADS 4 lesions in the training (AUC, 0.959) and validation (AUC, 0.925) cohorts. Moreover, 42.5% of unnecessary biopsies would be reduced by using the nomogram, but nine (4%) malignant BI-RADS 4 lesions were unfortunately missed, of which 4A (77.8%) and small-sized (<10 mm) lesions (66.7%) accounted for the majority. The ABVS radiomics nomogram may be a potential tool to reduce unnecessary biopsies of BI-RADS 4 lesions, but its ability to detect small BI-RADS 4A lesions needs to be improved.

Prediction Models for Osteoporotic Fractures Risk: A Systematic Review and Critical Appraisal

Osteoporotic fractures (OF) are a global public health problem currently. Many risk prediction models for OF have been developed, but their performance and methodological quality are unclear. We conducted this systematic review to summarize and critically appraise the OF risk prediction models. Three databases were searched until April 2021. Studies developing or validating multivariable models for OF risk prediction were considered eligible. Used the prediction model risk of bias assessment tool to appraise the risk of bias and applicability of included models. All results were narratively summarized and described. A total of 68 studies describing 70 newly developed prediction models and 138 external validations were included. Most models were explicitly developed (n=31, 44%) and validated (n=76, 55%) only for female. Only 22 developed models (31%) were externally validated. The most validated tool was Fracture Risk Assessment Tool. Overall, only a few models showed outstanding (n=3, 1%) or excellent (n=32, 15%) prediction discrimination. Calibration of developed models (n=25, 36%) or external validation models (n=33, 24%) were rarely assessed. No model was rated as low risk of bias, mostly because of an insufficient number of cases and inappropriate assessment of calibration. There are a certain number of OF risk prediction models. However, few models have been thoroughly internally validated or externally validated (with calibration being unassessed for most of the models), and all models showed methodological shortcomings. Instead of developing completely new models, future research is suggested to validate, improve, and analyze the impact of existing models.

Machine Learning Solutions for Osteoporosis-A Review

Osteoporosis and its clinical consequence, bone fracture, is a multifactorial disease that has been the object of extensive research. Recent advances in machine learning (ML) have enabled the field of artificial intelligence (AI) to make impressive breakthroughs in complex data environments where human capacity to identify high-dimensional relationships is limited. The field of osteoporosis is one such domain, notwithstanding technical and clinical concerns regarding the application of ML methods. This qualitative review is intended to outline some of these concerns and to inform stakeholders interested in applying AI for improved management of osteoporosis. A systemic search in PubMed and Web of Science resulted in 89 studies for inclusion in the review. These covered one or more of four main areas in osteoporosis management: bone properties assessment (n = 13), osteoporosis classification (n = 34), fracture detection (n = 32), and risk prediction (n = 14). Reporting and methodological quality was determined by means of a 12-point checklist. In general, the studies were of moderate quality with a wide range (mode score 6, range 2 to 11). Major limitations were identified in a significant number of studies. Incomplete reporting, especially over model selection, inadequate splitting of data, and the low proportion of studies with external validation were among the most frequent problems. However, the use of images for opportunistic osteoporosis diagnosis or fracture detection emerged as a promising approach and one of the main contributions that ML could bring to the osteoporosis field. Efforts to develop ML-based models for identifying novel fracture risk factors and improving fracture prediction are additional promising lines of research. Some studies also offered insights into the potential for model-based decision-making. Finally, to avoid some of the common pitfalls, the use of standardized checklists in developing and sharing the results of ML models should be encouraged. © 2021 American Society for Bone and Mineral Research (ASBMR).