External Validation of Clinical Prediction Models in Unilateral Primary Aldosteronism

BACKGROUND

Targeted treatment of primary aldosteronism (PA) is informed by adrenal vein sampling (AVS), which remains limited to specialized centers. Clinical prediction models have been developed to help select patients who would most likely benefit from AVS. Our aim was to assess the performance of these models for PA subtyping.

METHODS

This external validation study evaluated consecutive patients referred for PA who underwent AVS at a tertiary care referral center in Alberta, Canada during 2006–2018. In alignment with the original study designs and intended uses of the clinical prediction models, the primary outcome was the presence of lateralization on AVS. Model discrimination was evaluated using the C-statistic. Model calibration was assessed by comparing the observed vs. predicted probability of lateralization in the external validation cohort.

RESULTS

The validation cohort included 342 PA patients who underwent AVS (mean age, 52.1 years [SD, 11.5]; 201 [58.8%] male; 186 [54.4%] with lateralization). Six published models were assessed. All models demonstrated low-to-moderate discrimination in the validation set (C-statistics; range, 0.60–0.72), representing a marked decrease compared with the derivation sets (range, 0.80–0.87). Comparison of observed and predicted probabilities of unilateral PA revealed significant miscalibration. Calibration-in-the-large for every model was >0 (range, 0.35–1.67), signifying systematic underprediction of lateralizing disease. Calibration slopes were consistently <1 (range, 0.35–0.87), indicating poor performance at the extremes of risk.

CONCLUSIONS

Overall, clinical prediction models did not accurately predict AVS lateralization in this large cohort. These models cannot be reliably used to inform the decision to pursue AVS for most patients.

Predicting Glycated Hemoglobin Through Continuous Glucose Monitoring in Real-Life Conditions: Improved Estimation Methods

BACKGROUND

The adoption of continuous glucose monitoring (CGM) already helps to improve glycemic control in diabetes. When coupled with appropriate data analysis techniques, CGM also provides dependable estimates for significant metrics, like glycated hemoglobin (HbA1c). Findings from the REALISM-T1D study can boost HbA1c estimation methods in diabetes care and stimulate their use in clinical practice.

METHODS

Continuous glucose monitoring data of 27 adults affected by type-1 diabetes were acquired by means of G6 (Dexcom, San Diego, CA) sensors for a time span of 120 days. Glycated hemoglobin laboratory assays were performed during the concluding follow-up visits. Data were then analyzed to derive estimates of assay results, taken as the gold standard.

RESULTS

Bland-Altman (BA) plots show that smart interpolation to patch missing data and a wise choice of interstitial glucose (IG) weighting function, besides a proper mean interstitial glucose (MIG) to HbA1c regression equation, improve HbA1c estimation quality with respect to methods relying on MIG alone. A decrease in the BA plot-related variance of differences with respect to the gold standard confirms the improvement. Wilcoxon signed-rank tests on the bias-compensated mean squared error (MSE) with respect to conventional MIG-based methods show that the improvement is statistically significant with a confidence level better than 95% (P = .0179).

CONCLUSIONS

Improved HbA1c estimation methods result in better HbA1c prediction quality with respect to those based on MIG alone, thus providing quick, but still relatively accurate feedback to diabetologists. They alleviate the discordances reported in literature and, with further improvements, may become a viable complement/alternative to HbA1c assays.

Application of Soft Computing Techniques to Predict the Strength of Geopolymer Composites

Geopolymers may be the best alternative to ordinary Portland cement because they are manufactured using waste materials enriched in aluminosilicate. Research on geopolymer composites is accelerating. However, considerable work, expense, and time are needed to cast, cure, and test specimens. The application of computational methods to the stated objective is critical for speedy and cost-effective research. In this study, supervised machine learning approaches were employed to predict the compressive strength of geopolymer composites. One individual machine learning approach, decision tree, and two ensembled machine learning approaches, AdaBoost and random forest, were used. The coefficient correlation (R²), statistical tests, and k-fold analysis were used to determine the validity and comparison of all models. It was discovered that ensembled machine learning techniques outperformed individual machine learning techniques in forecasting the compressive strength of geopolymer composites. However, the outcomes of the individual machine learning model were also within the acceptable limit. R² values of 0.90, 0.90, and 0.83 were obtained for AdaBoost, random forest, and decision models, respectively. The models’ decreased error values, such as mean absolute error, mean absolute percentage error, and root-mean-square errors, further confirmed the ensembled machine learning techniques’ increased precision. Machine learning approaches will aid the building industry by providing quick and cost-effective methods for evaluating material properties.

A Roadmap for Boosting Model Generalizability for Predicting Hospital Encounters for Asthma

In the United States, ~9% of people have asthma. Each year, asthma incurs high health care cost and many hospital encounters covering 1.8 million emergency room visits and 439,000 hospitalizations. A small percentage of patients with asthma use most health care resources. To improve outcomes and cut resource use, many health care systems use predictive models to prospectively find high-risk patients and enroll them in care management for preventive care. For maximal benefit from costly care management with limited service capacity, only patients at the highest risk should be enrolled. However, prior models built by others miss >50% of true highest-risk patients and mislabel many low-risk patients as high risk, leading to suboptimal care and wasted resources. To address this issue, 3 site-specific models were recently built to predict hospital encounters for asthma, gaining up to >11% better performance. However, these models do not generalize well across sites and patient subgroups, creating 2 gaps before translating these models into clinical use. This paper points out these 2 gaps and outlines 2 corresponding solutions: (1) a new machine learning technique to create cross-site generalizable predictive models to accurately find high-risk patients and (2) a new machine learning technique to automatically raise model performance for poorly performing subgroups while maintaining model performance on other subgroups. This gives a roadmap for future research.

Machine learning based prediction models in male reproductive health: Development of a proof-of-concept model for Klinefelter Syndrome in azoospermic patients

BACKGROUND

Due to the highly variable clinical phenotype, Klinefelter Syndrome is underdiagnosed.

OBJECTIVE

Assessment of supervised machine learning based prediction models for identification of Klinefelter Syndrome among azoospermic patients, and comparison to expert clinical evaluation.

MATERIALS AND METHODS

Retrospective patient data (karyotype, age, height, weight, testis volume, follicle-stimulating hormone, luteinizing hormone, testosterone, estradiol, prolactin, semen pH and semen volume) collected between January 2005 and June 2019 were retrieved from a patient data bank of a University Centre. Models were trained, validated and benchmarked based on different supervised machine learning algorithms. Models were then tested on an independent, prospectively acquired set of patient data (between July 2019 and July 2020). Benchmarking against physicians was performed in addition.

RESULTS

Based on average performance, support vector machines and CatBoost were particularly well-suited models, with 100% sensitivity and >93% specificity on the test dataset. Compared to a group of 18 expert clinicians, the machine learning models had significantly better median sensitivity (100% vs. 87.5%, p = 0.0455) and fared comparably with regards to specificity (90% vs. 89.9%, p = 0.4795), thereby possibly improving diagnosis rate. A Klinefelter Syndrome Score Calculator based on the prediction models is available on http://klinefelter-score-calculator.uni-muenster.de.

DISCUSSION

Differentiating Klinefelter Syndrome patients from azoospermic patients with normal karyotype (46,XY) is a problem that can be solved with supervised machine learning techniques, improving patient care.

CONCLUSIONS

Machine learning could improve the diagnostic rate of Klinefelter Syndrome among azoospermic patients, even more for less-experienced physicians.

Machine learning prediction models in orthopedic surgery: A systematic review in transparent reporting

Machine learning (ML) studies are becoming increasingly popular in orthopedics but lack a critically appraisal of their adherence to peer-reviewed guidelines. The objective of this review was to (1) evaluate quality and transparent reporting of ML prediction models in orthopedic surgery based on the transparent reporting of multivariable prediction models for individual prognosis or diagnosis (TRIPOD), and (2) assess risk of bias with the Prediction model Risk Of Bias ASsessment Tool. A systematic review was performed to identify all ML prediction studies published in orthopedic surgery through June 18th, 2020. After screening 7138 studies, 59 studies met the study criteria and were included. Two reviewers independently extracted data and discrepancies were resolved by discussion with at least two additional reviewers present. Across all studies, the overall median completeness for the TRIPOD checklist was 53% (interquartile range 47%-60%). The overall risk of bias was low in 44% (n = 26), high in 41% (n = 24), and unclear in 15% (n = 9). High overall risk of bias was driven by incomplete reporting of performance measures, inadequate handling of missing data, and use of small datasets with inadequate outcome numbers. Although the number of ML studies in orthopedic surgery is increasing rapidly, over 40% of the existing models are at high risk of bias. Furthermore, over half incompletely reported their methods and/or performance measures. Until these issues are adequately addressed to give patients and providers trust in ML models, a considerable gap remains between the development of ML prediction models and their implementation in orthopedic practice.

Prediction of Physical Frailty in Orthogeriatric Patients Using Sensor Insole-Based Gait Analysis and Machine Learning Algorithms: Cross-sectional Study

Background

Assessment of the physical frailty of older patients is of great importance in many medical disciplines to be able to implement individualized therapies. For physical tests, time is usually used as the only objective measure. To record other objective factors, modern wearables offer great potential for generating valid data and integrating the data into medical decision-making.

Objective

The aim of this study was to compare the predictive value of insole data, which were collected during the Timed-Up-and-Go (TUG) test, to the benchmark standard questionnaire for sarcopenia (SARC-F: strength, assistance with walking, rising from a chair, climbing stairs, and falls) and physical assessment (TUG test) for evaluating physical frailty, defined by the Short Physical Performance Battery (SPPB), using machine learning algorithms.

Methods

This cross-sectional study included patients aged >60 years with independent ambulation and no mental or neurological impairment. A comprehensive set of parameters associated with physical frailty were assessed, including body composition, questionnaires (European Quality of Life 5-dimension [EQ 5D 5L], SARC-F), and physical performance tests (SPPB, TUG), along with digital sensor insole gait parameters collected during the TUG test. Physical frailty was defined as an SPPB score≤8. Advanced statistics, including random forest (RF) feature selection and machine learning algorithms (K-nearest neighbor [KNN] and RF) were used to compare the diagnostic value of these parameters to identify patients with physical frailty.

Results

Classified by the SPPB, 23 of the 57 eligible patients were defined as having physical frailty. Several gait parameters were significantly different between the two groups (with and without physical frailty). The area under the receiver operating characteristic curve (AUROC) of the TUG test was superior to that of the SARC-F (0.862 vs 0.639). The recursive feature elimination algorithm identified 9 parameters, 8 of which were digital insole gait parameters. Both the KNN and RF algorithms trained with these parameters resulted in excellent results (AUROC of 0.801 and 0.919, respectively).

Conclusions

A gait analysis based on machine learning algorithms using sensor soles is superior to the SARC-F and the TUG test to identify physical frailty in orthogeriatric patients.

Alternative Splicing: A New Therapeutic Target for Ovarian Cancer

Background: Increasing evidences have shown that abnormal alternative splicing (AS) events are closely related to the prognosis of various tumors. However, the role of AS in ovarian cancer (OV) is poorly understood. This study aims to explore the correlation between AS and the prognosis of OV and establish a prognostic model for OV. Methods: We downloaded the RNA-seq data of OV from The Cancer Genome Atlas databases and assessed cancer-specific AS through the SpliceSeq software. Then systemically investigated the overall survival (OS)-related AS and splicing factors (SFs) by bioinformatics analysis. The nomogram was established based on the clinical information, and the clinical practicability of the nomogram was verified through the calibration curve. Finally, a splicing correlation network was constructed to reveal the relationship between OS-related AS and SFs. Results: A total of 48,049 AS events were detected from 10,582 genes, of which 1523 were significantly associated with OS. The area under the curve of the final prediction model was 0.785, 0.681, and 0.781 in 1, 3, and 5 years, respectively. Moreover, the nomogram showed high calibration and discrimination in OV patients. Spearman correlation analysis was used to determine 8 SFs significantly related to survival, including major facilitator superfamily domain containing 11, synaptotagmin binding cytoplasmic RNA interacting protein, DEAH-box helicase 35, CWC15, integrator complex subunit 1, LUC7 like 2, cell cycle and apoptosis regulator 1, and heterogeneous nuclear ribonucleoprotein A2/B1. Conclusion: This study provides a prognostic model related to AS in OV, and constructs an AS-clinicopathological nomogram, which provides the possibility to predict the long-term prognosis of OV patients. We have explored the wealth of RNA splicing networks and regulation patterns related to the prognosis of OV, which provides a large number of biomarkers and potential targets for the treatment of OV. Put forward the potential possibility of interfering with the AS of OV in the comprehensive treatment of OV.

A chronological and geographical analysis of personal reports of COVID-19 on Twitter from the UK

Objective

Given the uncertainty about the trends and extent of the rapidly evolving COVID-19 outbreak, and the lack of extensive testing in the United Kingdom, our understanding of COVID-19 transmission is limited. We proposed to use Twitter to identify personal reports of COVID-19 to assess whether this data can help inform as a source of data to help us understand and model the transmission and trajectory of COVID-19.

Methods

We used natural language processing and machine learning framework. We collected tweets (excluding retweets) from the Twitter Streaming API that indicate that the user or a member of the user's household had been exposed to COVID-19. The tweets were required to be geo-tagged or have profile location metadata in the UK.

Results

We identified a high level of agreement between personal reports from Twitter and lab-confirmed cases by geographical region in the UK. Temporal analysis indicated that personal reports from Twitter appear up to 2 weeks before UK government lab-confirmed cases are recorded.

Conclusions

Analysis of tweets may indicate trends in COVID-19 in the UK and provide signals of geographical locations where resources may need to be targeted or where regional policies may need to be put in place to further limit the spread of COVID-19. It may also help inform policy makers of the restrictions in lockdown that are most effective or ineffective.

Integrating Google Trends Search Engine Query Data Into Adult Emergency Department Volume Forecasting: Infodemiology Study

Background

The search for health information from web-based resources raises opportunities to inform the service operations of health care systems. Google Trends search query data have been used to study public health topics, such as seasonal influenza, suicide, and prescription drug abuse; however, there is a paucity of literature using Google Trends data to improve emergency department patient-volume forecasting.

Objective

We assessed the ability of Google Trends search query data to improve the performance of adult emergency department daily volume prediction models.

Methods

Google Trends search query data related to chief complaints and health care facilities were collected from Chicago, Illinois (July 2015 to June 2017). We calculated correlations between Google Trends search query data and emergency department daily patient volumes from a tertiary care adult hospital in Chicago. A baseline multiple linear regression model of emergency department daily volume with traditional predictors was augmented with Google Trends search query data; model performance was measured using mean absolute error and mean absolute percentage error.

Results

There were substantial correlations between emergency department daily volume and Google Trends "hospital" (r=0.54), combined terms (r=0.50), and "Northwestern Memorial Hospital" (r=0.34) search query data. The final Google Trends data-augmented model included the predictors Combined 3-day moving average and Hospital 3-day moving average and performed better (mean absolute percentage error 6.42%) than the final baseline model (mean absolute percentage error 6.67%)-an improvement of 3.1%.

Conclusions

The incorporation of Google Trends search query data into an adult tertiary care hospital emergency department daily volume prediction model modestly improved model performance. Further development of advanced models with comprehensive search query terms and complementary data sources may improve prediction performance and could be an avenue for further research.

Systematic review of prediction models for postacute care destination decision-making

OBJECTIVE

This article reports a systematic review of studies containing development and validation of models predicting postacute care destination after adult inpatient hospitalization, summarizes clinical populations and variables, evaluates model performance, assesses risk of bias and applicability, and makes recommendations to reduce bias in future models.

MATERIALS AND METHODS

A systematic literature review was conducted following PRISMA guidelines and the Cochrane Prognosis Methods Group criteria. Online databases were searched in June 2020 to identify all published studies in this area. Data were extracted based on the CHARMS checklist, and studies were evaluated based on predictor variables, validation, performance in validation, risk of bias, and applicability using the Prediction Model Risk of Bias Assessment Tool (PROBAST) tool.

RESULTS

The final sample contained 28 articles with 35 models for evaluation. Models focused on surgical (22), medical (5), or both (8) populations. Eighteen models were internally validated, 10 were externally validated, and 7 models underwent both types. Model performance varied within and across populations. Most models used retrospective data, the median number of predictors was 8.5, and most models demonstrated risk of bias.

DISCUSSION AND CONCLUSION

Prediction modeling studies for postacute care destinations are becoming more prolific in the literature, but model development and validation strategies are inconsistent, and performance is variable. Most models are developed using regression, but machine learning methods are increasing in frequency. Future studies should ensure the rigorous variable selection and follow TRIPOD guidelines. Only 14% of the models have been tested or implemented beyond original studies, so translation into practice requires further investigation.

Computer-Aided Diagnosis of Gastrointestinal Ulcer and Hemorrhage Using Wireless Capsule Endoscopy: Systematic Review and Diagnostic Test Accuracy Meta-analysis

BACKGROUND

Interpretation of capsule endoscopy images or movies is operator-dependent and time-consuming. As a result, computer-aided diagnosis (CAD) has been applied to enhance the efficacy and accuracy of the review process. Two previous meta-analyses reported the diagnostic performance of CAD models for gastrointestinal ulcers or hemorrhage in capsule endoscopy. However, insufficient systematic reviews have been conducted, which cannot determine the real diagnostic validity of CAD models.

OBJECTIVE

To evaluate the diagnostic test accuracy of CAD models for gastrointestinal ulcers or hemorrhage using wireless capsule endoscopic images.

METHODS

We conducted core databases searching for studies based on CAD models for the diagnosis of ulcers or hemorrhage using capsule endoscopy and presenting data on diagnostic performance. Systematic review and diagnostic test accuracy meta-analysis were performed.

RESULTS

Overall, 39 studies were included. The pooled area under the curve, sensitivity, specificity, and diagnostic odds ratio of CAD models for the diagnosis of ulcers (or erosions) were .97 (95% confidence interval, .95-.98), .93 (.89-.95), .92 (.89-.94), and 138 (79-243), respectively. The pooled area under the curve, sensitivity, specificity, and diagnostic odds ratio of CAD models for the diagnosis of hemorrhage (or angioectasia) were .99 (.98-.99), .96 (.94-0.97), .97 (.95-.99), and 888 (343-2303), respectively. Subgroup analyses showed robust results. Meta-regression showed that published year, number of training images, and target disease (ulcers vs erosions, hemorrhage vs angioectasia) was found to be the source of heterogeneity. No publication bias was detected.

CONCLUSIONS

CAD models showed high performance for the optical diagnosis of gastrointestinal ulcer and hemorrhage in wireless capsule endoscopy.

Added Value of Cognition in the Prediction of Survival in Low and High Grade Glioma

Background: Diffuse gliomas, which are at WHO grade II-IV, are progressive primary brain tumors with great variability in prognosis. Our aim was to investigate whether pre-operative cognitive functioning is of added value in survival prediction in these patients.

Methods: In a retrospective cohort study of patients undergoing awake craniotomy between 2010 and 2019 we performed pre-operative neuropsychological assessments in five cognitive domains. Their added prognostic value on top of known prognostic factors was assessed in two patient groups [low- (LGG) and high-grade gliomas (HGG]). We compared Cox proportional hazards regression models with and without the cognitive domain by means of loglikelihood ratios tests (LRT), discriminative performance measures (by AUC), and risk classification [by Integrated Discrimination Index (IDI)].

Results: We included 109 LGG and 145 HGG patients with a median survival time of 1,490 and 511 days, respectively. The domain memory had a significant added prognostic value in HGG as indicated by an LRT (p-value = 0.018). The cumulative AUC for HGG with memory included was.78 (SD = 0.017) and without cognition 0.77 (SD = 0.018), IDI was 0.043 (0.000–0.102). In LGG none of the cognitive domains added prognostic value.

Conclusions: Our findings indicated that memory deficits, which were revealed with the neuropsychological examination, were of additional prognostic value in HGG to other well-known predictors of survival.

Use and performance of machine learning models for type 2 diabetes prediction in clinical and community care settings: Protocol for a systematic review and meta-analysis of predictive modeling studies

Objective

Machine learning involves the use of algorithms without explicit instructions. Of late, machine learning models have been widely applied for the prediction of type 2 diabetes. However, no evidence synthesis of the performance of these prediction models of type 2 diabetes is available. We aim to identify machine learning prediction models for type 2 diabetes in clinical and community care settings and determine their predictive performance.

Methods

The systematic review of English language machine learning predictive modeling studies in 12 databases will be conducted. Studies predicting type 2 diabetes in predefined clinical or community settings are eligible. Standard CHARMS and TRIPOD guidelines will guide data extraction. Methodological quality will be assessed using a predefined risk of bias assessment tool. The extent of validation will be categorized by Reilly–Evans levels. Primary outcomes include model performance metrics of discrimination ability, calibration, and classification accuracy. Secondary outcomes include candidate predictors, algorithms used, level of validation, and intended use of models. The random-effects meta-analysis of c-indices will be performed to evaluate discrimination abilities. The c-indices will be pooled per prediction model, per model type, and per algorithm. Publication bias will be assessed through funnel plots and regression tests. Sensitivity analysis will be conducted to estimate the effects of study quality and missing data on primary outcome. The sources of heterogeneity will be assessed through meta-regression. Subgroup analyses will be performed for primary outcomes.

Ethics and dissemination

No ethics approval is required, as no primary or personal data are collected. Findings will be disseminated through scientific sessions and peer-reviewed journals.

PROSPERO registration number

CRD42019130886

Comparison of Machine Learning Techniques for Mortality Prediction in a Prospective Cohort of Older Adults

As global demographics change, ageing is a global phenomenon which is increasingly of interest in our modern and rapidly changing society. Thus, the application of proper prognostic indices in clinical decisions regarding mortality prediction has assumed a significant importance for personalized risk management (i.e., identifying patients who are at high or low risk of death) and to help ensure effective healthcare services to patients. Consequently, prognostic modelling expressed as all-cause mortality prediction is an important step for effective patient management. Machine learning has the potential to transform prognostic modelling. In this paper, results on the development of machine learning models for all-cause mortality prediction in a cohort of healthy older adults are reported. The models are based on features covering anthropometric variables, physical and lab examinations, questionnaires, and lifestyles, as well as wearable data collected in free-living settings, obtained for the "Healthy Ageing Initiative" study conducted on 2291 recruited participants. Several machine learning techniques including feature engineering, feature selection, data augmentation and resampling were investigated for this purpose. A detailed empirical comparison of the impact of the different techniques is presented and discussed. The achieved performances were also compared with a standard epidemiological model. This investigation showed that, for the dataset under consideration, the best results were achieved with Random UnderSampling in conjunction with Random Forest (either with or without probability calibration). However, while including probability calibration slightly reduced the average performance, it increased the model robustness, as indicated by the lower 95% confidence intervals. The analysis showed that machine learning models could provide comparable results to standard epidemiological models while being completely data-driven and disease-agnostic, thus demonstrating the opportunity for building machine learning models on health records data for research and clinical practice. However, further testing is required to significantly improve the model performance and its robustness.

Radiomics Analysis Based on Ultrasound Images to Distinguish the Tumor Stage and Pathological Grade of Bladder Cancer

OBJECTIVES

To identify the clinical value of ultrasound radiomic features in the preoperative prediction of tumor stage and pathological grade of bladder cancer (BLCA) patients.

METHODS

We retrospectively collected patients who had been diagnosed with BLCA by pathology. Ultrasound-based radiomic features were extracted from manually segmented regions of interest. Participants were randomly assigned to a training cohort and a validation cohort at a ratio of 7:3. Radiomic features were Z-score normalized and submitted to dimensional reduction analysis (including Spearman's correlation coefficient analysis, the random forest algorithm, and statistical testing) for core feature selection. Classifiers for tumor stage and pathological grade prediction were then constructed. Prediction performance was estimated by the area under the curve (AUC) of the receiver operating characteristic curve and was verified by the validation cohort.

RESULTS

A total of 5936 radiomic features were extracted from each of the ultrasound images obtained from 157 patients. The BLCA tumor stage and pathological grade prediction models were developed based on 30 and 35 features, respectively. Both models showed good predictive ability. For the tumor stage prediction model, the AUC was 0.94 in the training cohort and 0.84 in the validation cohort. For the pathological grade model, the AUCs obtained were 0.84 in the training cohort and 0.75 in the validation cohort.

CONCLUSIONS

The ultrasound-based radiomics models performed well in the preoperative tumor staging and pathological grading of BLCA. These findings should be applied clinically to optimize treatment and to assess prognoses for BLCA.

Predictive model for atrial fibrillation in hypertensive diabetic patients

BACKGROUND

Several scores to identify patients at high risk of suffering atrial fibrillation have been developed. Their applicability in hypertensive diabetic patients, however, remains uncertain. Our aim is to develop and validate a diagnostic predictive model to calculate the risk of developing atrial fibrillation at five years in a hypertensive diabetic population.

METHODS

The derivation cohort consisted of patients with both hypertension and diabetes attended in any of the 52 primary healthcare centres of Barcelona; the validation cohort came from the 11 primary healthcare centres of Terres de l'Ebre (Catalonia South) from January 2013 to December 2017. Multivariable Cox regression identified clinical risk factors associated with the development of atrial fibrillation. The overall performance, discrimination and calibration of the model were carried out.

RESULTS

The derivation data set comprised 54 575 patients. The atrial fibrillation rate incidence was 15.3 per 1000 person/year. A 5-year predictive model included age, male gender, overweight, heart failure, valvular heart disease, peripheral vascular disease, chronic kidney disease, number of antihypertensive drugs, systolic and diastolic blood pressure, heart rate, thromboembolism, stroke and previous history of myocardial infarction. The discrimination of the model was good (c-index = 0.692; 95% confidence interval, 0.684-0.700), and calibration was adequate. In the validation cohort, the discrimination was lower (c-index = 0.670).

CONCLUSIONS

The model accurately predicts future atrial fibrillation in a population with both diabetes and hypertension. Early detection allows the prevention of possible complications arising from this disease.

An Explainable Machine Learning Model for Material Backorder Prediction in Inventory Management

Global competition among businesses imposes a more effective and low-cost supply chain allowing firms to provide products at a desired quality, quantity, and time, with lower production costs. The latter include holding cost, ordering cost, and backorder cost. Backorder occurs when a product is temporarily unavailable or out of stock and the customer places an order for future production and shipment. Therefore, stock unavailability and prolonged delays in product delivery will lead to additional production costs and unsatisfied customers, respectively. Thus, it is of high importance to develop models that will effectively predict the backorder rate in an inventory system with the aim of improving the effectiveness of the supply chain and, consequentially, the performance of the company. However, traditional approaches in the literature are based on stochastic approximation, without incorporating information from historical data. To this end, machine learning models should be employed for extracting knowledge of large historical data to develop predictive models. Therefore, to cover this need, in this study, the backorder prediction problem was addressed. Specifically, various machine learning models were compared for solving the binary classification problem of backorder prediction, followed by model calibration and a post-hoc explainability based on the SHAP model to identify and interpret the most important features that contribute to material backorder. The results showed that the RF, XGB, LGBM, and BB models reached an AUC score of 0.95, while the best-performing model was the LGBM model after calibration with the Isotonic Regression method. The explainability analysis showed that the inventory stock of a product, the volume of products that can be delivered, the imminent demand (sales), and the accurate prediction of the future demand can significantly contribute to the correct prediction of backorders.

Assessing the Value of Unsupervised Clustering in Predicting Persistent High Health Care Utilizers: Retrospective Analysis of Insurance Claims Data

BACKGROUND

A high proportion of health care services are persistently utilized by a small subpopulation of patients. To improve clinical outcomes while reducing costs and utilization, population health management programs often provide targeted interventions to patients who may become persistent high users/utilizers (PHUs). Enhanced prediction and management of PHUs can improve health care system efficiencies and improve the overall quality of patient care.

OBJECTIVE

The aim of this study was to detect key classes of diseases and medications among the study population and to assess the predictive value of these classes in identifying PHUs.

METHODS

This study was a retrospective analysis of insurance claims data of patients from the Johns Hopkins Health Care system. We defined a PHU as a patient incurring health care costs in the top 20% of all patients' costs for 4 consecutive 6-month periods. We used 2013 claims data to predict PHU status in 2014-2015. We applied latent class analysis (LCA), an unsupervised clustering approach, to identify patient subgroups with similar diagnostic and medication patterns to differentiate variations in health care utilization across PHUs. Logistic regression models were then built to predict PHUs in the full population and in select subpopulations. Predictors included LCA membership probabilities, demographic covariates, and health utilization covariates. Predictive powers of the regression models were assessed and compared using standard metrics.

RESULTS

We identified 164,221 patients with continuous enrollment between 2013 and 2015. The mean study population age was 19.7 years, 55.9% were women, 3.3% had ≥1 hospitalization, and 19.1% had 10+ outpatient visits in 2013. A total of 8359 (5.09%) patients were identified as PHUs in both 2014 and 2015. The LCA performed optimally when assigning patients to four probability disease/medication classes. Given the feedback provided by clinical experts, we further divided the population into four diagnostic groups for sensitivity analysis: acute upper respiratory infection (URI) (n=53,232; 4.6% PHUs), mental health (n=34,456; 12.8% PHUs), otitis media (n=24,992; 4.5% PHUs), and musculoskeletal (n=24,799; 15.5% PHUs). For the regression models predicting PHUs in the full population, the F1-score classification metric was lower using a parsimonious model that included LCA categories (F1=38.62%) compared to that of a complex risk stratification model with a full set of predictors (F1=48.20%). However, the LCA-enabled simple models were comparable to the complex model when predicting PHUs in the mental health and musculoskeletal subpopulations (F1-scores of 48.69% and 48.15%, respectively). F1-scores were lower than that of the complex model when the LCA-enabled models were limited to the otitis media and acute URI subpopulations (45.77% and 43.05%, respectively).

CONCLUSIONS

Our study illustrates the value of LCA in identifying subgroups of patients with similar patterns of diagnoses and medications. Our results show that LCA-derived classes can simplify predictive models of PHUs without compromising predictive accuracy. Future studies should investigate the value of LCA-derived classes for predicting PHUs in other health care settings.

Algorithm for Individual Prediction of COVID-19-Related Hospitalization Based on Symptoms: Development and Implementation Study

BACKGROUND

The COVID-19 pandemic has placed a huge strain on the health care system globally. The metropolitan area of Milan, Italy, was one of the regions most impacted by the COVID-19 pandemic worldwide. Risk prediction models developed by combining administrative databases and basic clinical data are needed to stratify individual patient risk for public health purposes.

OBJECTIVE

This study aims to develop a stratification tool aimed at improving COVID-19 patient management and health care organization.

METHODS

A predictive algorithm was developed and applied to 36,834 patients with COVID-19 in Italy between March 8 and the October 9, 2020, in order to foresee their risk of hospitalization. Exposures considered were age, sex, comorbidities, and symptoms associated with COVID-19 (eg, vomiting, cough, fever, diarrhea, myalgia, asthenia, headache, anosmia, ageusia, and dyspnea). The outcome was hospitalizations and emergency department admissions for COVID-19. Discrimination and calibration of the model were also assessed.

RESULTS

The predictive model showed a good fit for predicting COVID-19 hospitalization (C-index 0.79) and a good overall prediction accuracy (Brier score 0.14). The model was well calibrated (intercept -0.0028, slope 0.9970). Based on these results, 118,804 patients diagnosed with COVID-19 from October 25 to December 11, 2020, were stratified into low, medium, and high risk for COVID-19 severity. Among the overall study population, 67,030 (56.42%) were classified as low-risk patients; 43,886 (36.94%), as medium-risk patients; and 7888 (6.64%), as high-risk patients. In all, 89.37% (106,179/118,804) of the overall study population was being assisted at home, 9% (10,695/118,804) was hospitalized, and 1.62% (1930/118,804) died. Among those assisted at home, most people (63,983/106,179, 60.26%) were classified as low risk, whereas only 3.63% (3858/106,179) were classified at high risk. According to ordinal logistic regression, the odds ratio (OR) of being hospitalized or dead was 5.0 (95% CI 4.6-5.4) among high-risk patients and 2.7 (95% CI 2.6-2.9) among medium-risk patients, as compared to low-risk patients.

CONCLUSIONS

A simple monitoring system, based on primary care data sets linked to COVID-19 testing results, hospital admissions data, and death records may assist in the proper planning and allocation of patients and resources during the ongoing COVID-19 pandemic.

A Serum Resistin and Multicytokine Inflammatory Pathway Is Linked With and Helps Predict All-cause Death in Diabetes

CONTEXT

Type 2 diabetes (T2D) shows a high mortality rate, partly mediated by atherosclerotic plaque instability. Discovering novel biomarkers may help identify high-risk patients who would benefit from more aggressive and specific managements. We recently described a serum resistin and multicytokine inflammatory pathway (REMAP), including resistin, interleukin (IL)-1β, IL-6, IL-8, and TNF-α, that is associated with cardiovascular disease.

OBJECTIVE

We investigated whether REMAP is associated with and improves the prediction of mortality in T2D.

METHODS

A REMAP score was investigated in 3 cohorts comprising 1528 patients with T2D (409 incident deaths) and in 59 patients who underwent carotid endarterectomy (CEA; 24 deaths). Plaques were classified as unstable/stable according to the modified American Heart Association atherosclerosis classification.

RESULTS

REMAP was associated with all-cause mortality in each cohort and in all 1528 individuals (fully adjusted hazard ratio [HR] for 1 SD increase = 1.34, P < .001). In CEA patients, REMAP was associated with mortality (HR = 1.64, P = .04) and a modest change was observed when plaque stability was taken into account (HR = 1.58; P = .07). REMAP improved discrimination and reclassification measures of both Estimation of Mortality Risk in Type 2 Diabetic Patients and Risk Equations for Complications of Type 2 Diabetes, well-established prediction models of mortality in T2D (P < .05-< .001).

CONCLUSION

REMAP is independently associated with and improves predict all-cause mortality in T2D; it can therefore be used to identify high-risk individuals to be targeted with more aggressive management. Whether REMAP can also identify patients who are more responsive to IL-6 and IL-1β monoclonal antibodies that reduce cardiovascular burden and total mortality is an intriguing possibility to be tested.

Use of Machine Learning and Statistical Algorithms to Predict Hospital Length of Stay Following Colorectal Cancer Resection: A South African Pilot Study

The aim of this pilot study was to develop logistic regression (LR) and support vector machine (SVM) models that differentiate low from high risk for prolonged hospital length of stay (LOS) in a South African cohort of 383 colorectal cancer patients who underwent surgical resection with curative intent. Additionally, the impact of 10-fold cross-validation (CV), Monte Carlo CV, and bootstrap internal validation methods on the performance of the two models was evaluated. The median LOS was 9 days, and prolonged LOS was defined as greater than 9 days post-operation. Preoperative factors associated with prolonged LOS were a prior history of hypertension and an Eastern Cooperative Oncology Group score between 2 and 4. Postoperative factors related to prolonged LOS were the need for a stoma as part of the surgical procedure and the development of post-surgical complications. The risk of prolonged LOS was higher in male patients and in any patient with lower preoperative hemoglobin. The highest area under the receiving operating characteristics (AU-ROC) was achieved using LR of 0.823 (CI = 0.798–0.849) and SVM of 0.821 (CI = 0.776–0.825), with each model using the Monte Carlo CV method for internal validation. However, bootstrapping resulted in models with slightly lower variability. We found no significant difference between the models across the three internal validation methods. The LR and SVM algorithms used in this study required incorporating important features for optimal hospital LOS predictions. The factors identified in this study, especially postoperative complications, can be employed as a simple and quick test clinicians may flag a patient at risk of prolonged LOS.

Toxicometabolomics: Small Molecules to Answer Big Toxicological Questions

Given the high biological impact of classical and emerging toxicants, a sensitive and comprehensive assessment of the hazards and risks of these substances to organisms is urgently needed. In this sense, toxicometabolomics emerged as a new and growing field in life sciences, which use metabolomics to provide new sets of susceptibility, exposure, and/or effects biomarkers; and to characterize in detail the metabolic responses and altered biological pathways that various stressful stimuli cause in many organisms. The present review focuses on the analytical platforms and the typical workflow employed in toxicometabolomic studies, and gives an overview of recent exploratory research that applied metabolomics in various areas of toxicology.

Risk prediction models for esophageal cancer: A systematic review and critical appraisal

BACKGROUND AND AIMS

Esophageal cancer risk prediction models allow for risk-stratified endoscopic screening. We aimed to assess the quality of these models developed in the general population.

METHODS

A systematic search of the PubMed and Embase databases from January 2000 through May 2021 was performed. Studies that developed or validated a model of esophageal cancer in the general population were included. Screening, data extraction, and risk of bias (ROB) assessment by the Prediction model Risk Of Bias Assessment Tool (PROBAST) were performed independently by two reviewers.

RESULTS

Of the 13 models included in the qualitative analysis, 8 were developed for esophageal squamous cell carcinoma (ESCC) and the other 5 were developed for esophageal adenocarcinoma (EAC). Only two models conducted external validation. In the ESCC models, cigarette smoking was included in each model, followed by age, sex, and alcohol consumption. For EAC models, cigarette smoking and body mass index were included in each model, and gastroesophageal reflux disease, uses of acid-suppressant medicine, and nonsteroidal anti-inflammatory drug were exclusively included. The discriminative performance was reported in all studies, with C statistics ranging from 0.71 to 0.88, whereas only six models reported calibration. For ROB, all the models had a low risk in participant and outcome, but all models showed high risk in analysis, and 60% of models showed a high risk in predictors, which resulted in all models being classified as having overall high ROB. For model applicability, about 60% of these models had an overall low risk, with 30% of models of high risk and 10% of models of unclear risk, concerning the assessment of participants, predictors, and outcomes.

CONCLUSIONS

Most current risk prediction models of esophageal cancer have a high ROB. Prediction models need further improvement in their quality and applicability to benefit esophageal cancer screening.