A tutorial on calibration measurements and calibration models for clinical prediction models

Differential gene expression analysis pipelines and bioinformatic tools for the identification of specific biomarkers: A review

In recent years, the role of bioinformatics and computational biology together with omics techniques and transcriptomics has gained tremendous importance in biomedicine and healthcare, particularly for the identification of biomarkers for precision medicine and drug discovery. Differential gene expression (DGE) analysis is one of the most used techniques for RNA-sequencing (RNA-seq) data analysis. This tool, which is typically used in various RNA-seq data processing applications, allows the identification of differentially expressed genes across two or more sample sets. Functional enrichment analyses can then be performed to annotate and contextualize the resulting gene lists. These studies provide valuable information about disease-causing biological processes and can help in identifying molecular targets for novel therapies. This review focuses on differential gene expression (DGE) analysis pipelines and bioinformatic techniques commonly used to identify specific biomarkers and discuss the advantages and disadvantages of these techniques.

Use of mycophenolate mofetil for the treatment of fibrotic hypersensitivity pneumonitis

INTRODUCTION

The optimal treatment of fibrosing hypersensitivity pneumonitis (fHP) is not well understood. The aim of the study was to obtain information about the usefulness of mycophenolate mofetil (MMF) in its treatment.

MATERIAL AND METHODS

Quasi-experimental analysis of patients diagnosed with fHP and treated with MMF for one year, in a single centre. From the start of treatment, data collection was prospective.

RESULTS

73 were included and 58 completed the study. FVC% and DLCO% decreased until starting MMF (year -1 to year 0). After completion of treatment (year 1), FVC% stabilised (p=0.336) and DLCO% improved significantly (p=0.004) compared to year 0. Dyspnoea, number of patients without corticosteroids and mean corticosteroid dose also improved significantly (p<0.001 in all cases). Being male and having a history of tuberculosis were predictors of poor drug response [AUC = 0.89 (95% CI: 0.80-0.98)]. 45 adverse effects were observed in 34 patients (46.6%). In 4 cases (5.5%), the adverse effect was severe and required discontinuation of treatment.

CONCLUSIONS

In patients with fHP, MMF improves lung function and dyspnoea and reduces both the number of patients requiring oral corticosteroids and their mean dose in those who completed 1 year of treatment. The model constructed predicts which patients will respond poorly to treatment, with good discriminative ability and only a small percentage of patients will not tolerate treatment. Further prospective, randomised clinical trials are needed to define the role of this treatment in fHP.

Development and Validation of the Hospital Medicine Safety Sepsis Initiative Mortality Model

BACKGROUND

When comparing outcomes after sepsis, it is essential to account for patient case mix to make fair comparisons. We developed a model to assess risk-adjusted 30-day mortality in the Michigan Hospital Medicine Safety sepsis initiative (HMS-Sepsis).

RESEARCH QUESTION

Can HMS-Sepsis registry data adequately predict risk of 30-day mortality? Do performance assessments using adjusted vs unadjusted data differ?

STUDY DESIGN AND METHODS

Retrospective cohort of community-onset sepsis hospitalizations in the HMS-Sepsis registry (April 2022-September 2023), with split-derivation (70%) and validation (30%) cohorts. We fit a risk-adjustment model (HMS-Sepsis mortality model) incorporating acute physiologic, demographic, and baseline health data and assessed model performance using concordance (C) statistics, Brier's scores, and comparisons of predicted vs observed mortality by deciles of risk. We compared hospital performance (first quintile, middle quintiles, fifth quintile) using observed vs adjusted mortality to understand the extent to which risk adjustment impacted hospital performance assessment.

RESULTS

Among 17,514 hospitalizations from 66 hospitals during the study period, 12,260 hospitalizations (70%) were used for model derivation and 5,254 hospitalizations (30%) were used for model validation. Thirty-day mortality for the total cohort was 19.4%. The final model included 13 physiologic variables, two physiologic interactions, and 16 demographic and chronic health variables. The most significant variables were age, metastatic solid tumor, temperature, altered mental status, and platelet count. The model C statistic was 0.82 for the derivation cohort, 0.81 for the validation cohort, and ≥ 0.78 for all subgroups assessed. Overall calibration error was 0.0%, and mean calibration error across deciles of risk was 1.5%. Standardized mortality ratios yielded different assessments than observed mortality for 33.9% of hospitals.

INTERPRETATION

The HMS-Sepsis mortality model showed strong discrimination and adequate calibration and reclassified one-third of hospitals to a different performance category from unadjusted mortality. Based on its strong performance, the HMS-Sepsis mortality model can aid in fair hospital benchmarking, assessment of temporal changes, and observational causal inference analysis.

Improving Inverse Probability Weighting by Post-calibrating Its Propensity Scores

Theoretical guarantees for causal inference using propensity scores are partially based on the scores behaving like conditional probabilities. However, prediction scores between zero and one do not necessarily behave like probabilities, especially when output by flexible statistical estimators. We perform a simulation study to assess the error in estimating the average treatment effect before and after applying a simple and well-established postprocessing method to calibrate the propensity scores. We observe that postcalibration reduces the error in effect estimation and that larger improvements in calibration result in larger improvements in effect estimation. Specifically, we find that expressive tree-based estimators, which are often less calibrated than logistic regression-based models initially, tend to show larger improvements relative to logistic regression-based models. Given the improvement in effect estimation and that postcalibration is computationally cheap, we recommend its adoption when modeling propensity scores with expressive models.

An elastic net regression model for predicting the risk of ICU admission and death for hospitalized patients with COVID-19

This study aimed to develop and validate prediction models to estimate the risk of death and intensive care unit admission in COVID-19 inpatients. All RT-PCR-confirmed adult COVID-19 inpatients admitted to Fujian Provincial Hospital from October 2022 to April 2023 were considered. Elastic Net Regression was used to derive the risk prediction models. Potential risk factors were considered, which included demographic characteristics, clinical symptoms, comorbidities, laboratory results, treatment process, prognosis. A total of 1906 inpatients were included finally by inclusion/exclusion criteria and were divided into derivation and test cohorts in a ratio of 8:2, where 1526 (80%) samples were used to develop prediction models under a repeated cross-validation framework and the remaining 380 (20%) samples were used for performance evaluation. Overall performance, discrimination and calibration were evaluated in the validation set and test cohort and quantified by accuracy, scaled Brier score (SbrS), the area under the ROC curve (AUROC), and Spiegelhalter-Z statistics. The models performed well, with high levels of discrimination (AUROCICU [95%CI]: 0.858 [0.803,0.899]; AUROCdeath [95%CI]: 0.906 [0.850,0.948]); and good calibrations (Spiegelhalter-ZICU: - 0.821 (p-value: 0.412); Spiegelhalter-Zdeath: 0.173) in the test set. We developed and validated prediction models to help clinicians identify high risk patients for death and ICU admission after COVID-19 infection.

Prognosticating the outcome of intensive care in older patients-a narrative review

Prognosis determines major decisions regarding treatment for critically ill patients. Statistical models have been developed to predict the probability of survival and other outcomes of intensive care. Although they were trained on the characteristics of large patient cohorts, they often do not represent very old patients (age ≥ 80 years) appropriately. Moreover, the heterogeneity within this particular group impairs the utility of statistical predictions for informing decision-making in very old individuals. In addition to these methodological problems, the diversity of cultural attitudes, available resources as well as variations of legal and professional norms limit the generalisability of prediction models, especially in patients with complex multi-morbidity and pre-existing functional impairments. Thus, current approaches to prognosticating outcomes in very old patients are imperfect and can generate substantial uncertainty about optimal trajectories of critical care in the individual. This article presents the state of the art and new approaches to predicting outcomes of intensive care for these patients. Special emphasis has been given to the integration of predictions into the decision-making for individual patients. This requires quantification of prognostic uncertainty and a careful alignment of decisions with the preferences of patients, who might prioritise functional outcomes over survival. Since the performance of outcome predictions for the individual patient may improve over time, time-limited trials in intensive care may be an appropriate way to increase the confidence in decisions about life-sustaining treatment.

Practical guide to building machine learning-based clinical prediction models using imbalanced datasets

Clinical prediction models often aim to predict rare, high-risk events, but building such models requires robust understanding of imbalance datasets and their unique study design considerations. This practical guide highlights foundational prediction model principles for surgeon-data scientists and readers who encounter clinical prediction models, from feature engineering and algorithm selection strategies to model evaluation and design techniques specific to imbalanced datasets. We walk through a clinical example using readable code to highlight important considerations and common pitfalls in developing machine learning-based prediction models. We hope this practical guide facilitates developing and critically appraising robust clinical prediction models for the surgical community.

Towards proactive palliative care in oncology: developing an explainable EHR-based machine learning model for mortality risk prediction

BACKGROUND

Ex-ante identification of the last year in life facilitates a proactive palliative approach. Machine learning models trained on electronic health records (EHR) demonstrate promising performance in cancer prognostication. However, gaps in literature include incomplete reporting of model performance, inadequate alignment of model formulation with implementation use-case, and insufficient explainability hindering trust and adoption in clinical settings. Hence, we aim to develop an explainable machine learning EHR-based model that prompts palliative care processes by predicting for 365-day mortality risk among patients with advanced cancer within an outpatient setting.

METHODS

Our cohort consisted of 5,926 adults diagnosed with Stage 3 or 4 solid organ cancer between July 1, 2017, and June 30, 2020 and receiving ambulatory cancer care within a tertiary center. The classification problem was modelled using Extreme Gradient Boosting (XGBoost) and aligned to our envisioned use-case: "Given a prediction point that corresponds to an outpatient cancer encounter, predict for mortality within 365-days from prediction point, using EHR data up to 365-days prior." The model was trained with 75% of the dataset (n = 39,416 outpatient encounters) and validated on a 25% hold-out dataset (n = 13,122 outpatient encounters). To explain model outputs, we used Shapley Additive Explanations (SHAP) values. Clinical characteristics, laboratory tests and treatment data were used to train the model. Performance was evaluated using area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC), while model calibration was assessed using the Brier score.

RESULTS

In total, 17,149 of the 52,538 prediction points (32.6%) had a mortality event within the 365-day prediction window. The model demonstrated an AUROC of 0.861 (95% CI 0.856-0.867) and AUPRC of 0.771. The Brier score was 0.147, indicating slight overestimations of mortality risk. Explanatory diagrams utilizing SHAP values allowed visualization of feature impacts on predictions at both the global and individual levels.

CONCLUSION

Our machine learning model demonstrated good discrimination and precision-recall in predicting 365-day mortality risk among individuals with advanced cancer. It has the potential to provide personalized mortality predictions and facilitate earlier integration of palliative care.

Using Machine Learning to Evaluate the Value of Genetic Liabilities in the Classification of Hypertension within the UK Biobank

Background and Objective: Hypertension increases the risk of cardiovascular diseases (CVD) such as stroke, heart attack, heart failure, and kidney disease, contributing to global disease burden and premature mortality. Previous studies have utilized statistical and machine learning techniques to develop hypertension prediction models. Only a few have included genetic liabilities and evaluated their predictive values. This study aimed to develop an effective hypertension classification model and investigate the potential influence of genetic liability for multiple risk factors linked to CVD on hypertension risk using the random forest and the neural network. Materials and Methods: The study involved 244,718 European participants, who were divided into training and testing sets. Genetic liabilities were constructed using genetic variants associated with CVD risk factors obtained from genome-wide association studies (GWAS). Various combinations of machine learning models before and after feature selection were tested to develop the best classification model. The models were evaluated using area under the curve (AUC), calibration, and net reclassification improvement in the testing set. Results: The models without genetic liabilities achieved AUCs of 0.70 and 0.72 using the random forest and the neural network methods, respectively. Adding genetic liabilities improved the AUC for the random forest but not for the neural network. The best classification model was achieved when feature selection and classification were performed using random forest (AUC = 0.71, Spiegelhalter z score = 0.10, p-value = 0.92, calibration slope = 0.99). This model included genetic liabilities for total cholesterol and low-density lipoprotein (LDL). Conclusions: The study highlighted that incorporating genetic liabilities for lipids in a machine learning model may provide incremental value for hypertension classification beyond baseline characteristics.

Generalizability of polygenic prediction models: how is the R2 defined on test data?

BACKGROUND

Polygenic risk scores (PRS) quantify an individual's genetic predisposition for different traits and are expected to play an increasingly important role in personalized medicine. A crucial challenge in clinical practice is the generalizability and transferability of PRS models to populations with different ancestries. When assessing the generalizability of PRS models for continuous traits, the R 2 is a commonly used measure to evaluate prediction accuracy. While the R 2 is a well-defined goodness-of-fit measure for statistical linear models, there exist different definitions for its application on test data, which complicates interpretation and comparison of results.

METHODS

Based on large-scale genotype data from the UK Biobank, we compare three definitions of the R 2 on test data for evaluating the generalizability of PRS models to different populations. Polygenic models for several phenotypes, including height, BMI and lipoprotein A, are derived based on training data with European ancestry using state-of-the-art regression methods and are evaluated on various test populations with different ancestries.

RESULTS

Our analysis shows that the choice of the R 2  definition can lead to considerably different results on test data, making the comparison of R 2  values from the literature problematic. While the definition as the squared correlation between predicted and observed phenotypes solely addresses the discriminative performance and always yields values between 0 and 1, definitions of the R 2 based on the mean squared prediction error (MSPE) with reference to intercept-only models assess both discrimination and calibration. These MSPE-based definitions can yield negative values indicating miscalibrated predictions for out-of-target populations. We argue that the choice of the most appropriate definition depends on the aim of PRS analysis - whether it primarily serves for risk stratification or also for individual phenotype prediction. Moreover, both correlation-based and MSPE-based definitions of R 2 can provide valuable complementary information.

CONCLUSIONS

Awareness of the different definitions of the R 2 on test data is necessary to facilitate the reporting and interpretation of results on PRS generalizability. It is recommended to explicitly state which definition was used when reporting R 2 values on test data. Further research is warranted to develop and evaluate well-calibrated polygenic models for diverse populations.

Validation of the European Society of Cardiology pretest probability models for obstructive coronary artery disease in high-risk population

OBJECTIVE

The pre-test probability (PTP) model for obstructive coronary artery disease (CAD) was updated in 2019 by the European Society of Cardiology (ESC). To our knowledge, this model was never externally validated in a population with a high incidence of CAD. The aim of this study is to validate the new PTP ESC model in our population, which has a high CAD incidence, and to compare it with the previous PTP ESC model from 2013.

METHODS

We retrospectively analysed 1294 symptomatic patients with suspected CAD referred to our centre between 2015 and 2019. In all patients, the PTP score was calculated based on age, gender, and symptoms according to the ESC model from 2013 (2013-ESC-PTP) and 2019 (2019-ESC-PTP). All patients underwent invasive coronary angiography (ICA).

RESULTS

Of the 1294 patients, obstructive CAD was diagnosed in 533 patients (41.2%). The 2019-ESC-PTP model categorised significantly more patients into the low probability group (PTP < 15%) than the 2013-ESC-PTP model (39.8% vs. 5.6%, p < 0.001). Obstructive CAD prevalence was underestimated using 2019-ESC-PTP at all PTP levels (calibration intercept 1.15, calibration slope 0.96). The 2013-ESC-PTP overestimated obstructive CAD prevalence (calibration intercept -0.24, calibration slope 0.73). The discrimination measured with an area under the curve was similar for both models, indicating moderate accuracy of the models.

CONCLUSION

In high-risk Serbian population, both the 2013 and 2019 ESC-PTP models had moderate accuracy in diagnosing CAD, with the 2019-ESC-PTP underestimating the prevalence of CAD and the 2013-ESC-PTP overestimating it. Further studies are warranted to establish PTP models for high-risk countries.

Association of Blood Copper With the Subclinical Carotid Atherosclerosis: An Observational Study

BACKGROUND

Copper exposure is reported to be associated with increased risk of stroke. However, the association of copper exposure with subclinical carotid atherosclerosis remains unclear.

METHODS AND RESULTS

This observational study included consecutive participants from Xinqiao Hospital between May 2020 and August 2021. Blood metals were measured using inductively coupled plasma mass spectrometry and carotid atherosclerosis was assessed using ultrasound. Modified Poisson regression was performed to evaluate the associations of copper and other metals with subclinical carotid plaque presence. Blood metals were analyzed as categorical according to the quartiles. Multivariable models were adjusted for age, sex, body mass index, education, smoking, drinking, hypertension, diabetes, dyslipidemia, estimated glomerular filtration rate, and coronary artery disease history. Bayesian Kernel Machine Regression was conducted to evaluate the overall association of metal mixture with subclinical carotid plaque presence. One thousand five hundred eighty-five participants were finally enrolled in our study, and carotid plaque was found in 1091 subjects. After adjusting for potential confounders, metal-progressively-adjusted models showed that blood copper was positively associated with subclinical carotid plaque (relative risk according to comparing quartile 4 to quartile 1 was 1.124 [1.021-1.238], relative risk according to per interquartile increment was 1.039 [1.008-1.071]). Blood cadmium and lead were also significantly associated with subclinical carotid plaque. Bayesian Kernel Machine Regression analyses suggested a synergistic effect of copper-cadmium-lead mixture on subclinical carotid plaque presence.

CONCLUSIONS

Our findings identify copper as a novel risk factor of subclinical carotid atherosclerosis and show the potential synergistic proatherogenic effect of copper, cadmium, and lead mixture.

Prediction Models for Forecasting Risk of Development of Surgical Site Infection after Lower Limb Revascularization Surgery: A Systematic Review

BACKGROUND

Surgical site infections (SSIs) are a common and potentially preventable complication of lower limb revascularization surgery associated with increased healthcare resource utilization and patient morbidity. We conducted a systematic review to evaluate multivariable prediction models designed to forecast risk of SSI development after these procedures.

METHODS

After protocol registration (CRD42022331292), we searched MEDLINE, EMBASE, CENTRAL, and Evidence-Based Medicine Reviews (inception to April 4th, 2023) for studies describing multivariable prediction models designed to forecast risk of SSI in adults after lower limb revascularization surgery. Two investigators independently screened abstracts and full-text articles, extracted data, and assessed risk of bias. A narrative synthesis was performed to summarize predictors included in the models and their calibration and discrimination, validation status, and clinical applicability.

RESULTS

Among the 6,671 citations identified, we included 5 studies (n = 23,063 patients). The included studies described 5 unique multivariable prediction models generated through forward selection, backward selection, or Akaike Information Criterion-based methods. Two models were designed to predict any SSI and 3 Szyilagyi grade II (extending into subcutaneous tissue) SSI. Across the 5 models, 18 adjusted predictors (10 of which were preoperative, 3 intraoperative, and 5 postoperative) significantly predicted any SSI and 14 adjusted predictors significantly predict Szilagyi grade II SSI. Female sex, obesity, and chronic obstructive pulmonary disease significantly predicted SSI in more than one model. All models had a "good fit" according to the Hosmer-Lemeshow test (P > 0.05). Model discrimination was quantified using the area under the curve, which ranged from 0.66 to 0.75 across models. Two models were internally validated using non-exhaustive twofold cross-validation and bootstrap resampling. No model was externally validated. Three studies had a high overall risk of bias according to the Prediction model Risk Of Bias ASsessment Tool (PROBAST).

CONCLUSIONS

Five multivariable prediction models with moderate discrimination have been developed to forecast risk of SSI development after lower limb revascularization surgery. Given the frequency and consequences of SSI after these procedures, development and external validation of novel prediction models and comparison of these models to the existing models evaluated in this systematic review is warranted.

Early prediction of long hospital stay for Intensive Care units readmission patients using medication information

OBJECTIVE

Predicting Intensive Care Unit (ICU) Length of Stay (LOS) accurately can improve patient wellness, hospital operations, and the health system's financial status. This study focuses on predicting the prolonged ICU LOS (≥3 days) of the 2nd admission, utilizing short historical data (1st admission only) for early-stage prediction, as well as incorporating medication information.

MATERIALS AND METHODS

We selected 18,572 ICU patients' records from the MIMIC-IV database for this study. We applied five machine learning classifiers: Logistic regression (LR), Random Forest (RF), Support Vector Machine (SVM), AdaBoost (AB) and XGBoost (XGB). We computed both the sum dose and the average dose for the medication and included them in our model.

RESULTS

The performance of the RF model demonstrates the highest level of accuracy compared to other models, as indicated by an Area Under the Receiver Operating Characteristic Curve (AUC) of 0.716 and an Expected Calibration Error (ECE) of 0.023.

DISCUSSION

The calibration improved all five classifiers (LR, RF, SVC, AB, XGB) in terms of ECE. The most important two features for RF are the length of 1st admission and the patient's age when they visited the hospital. The most important medication features are Phytonadione and Metoprolol Succinate XL. Also, both the sum and the average dose for the medication features contributed to the prediction task.

CONCLUSION

Our model showed the capability to predict the prolonged ICU LOS of the 2nd admission by utilizing the demographic, diagnosis, and medication information from the 1st admission. This method can potentially support the prevention of patient complications and enhance resource allocation in hospitals.

Benchmarking clinical risk prediction algorithms with ensemble machine learning for the noninvasive diagnosis of liver fibrosis in NAFLD

BACKGROUND AND AIMS

Ensemble machine-learning methods, like the superlearner, combine multiple models into a single one to enhance predictive accuracy. Here we explore the potential of the superlearner as a benchmarking tool for clinical risk prediction, illustrating the approach to identifying significant liver fibrosis among patients with NAFLD.

APPROACH AND RESULTS

We used 23 demographic/clinical variables to train superlearner(s) on data from the NASH-clinical research network observational study (n = 648) and validated models with data from the FLINT trial (n = 270) and National Health and Nutrition Examination Survey (NHANES) participants with NAFLD (n = 1244). Comparing the superlearner's performance to existing models (Fibrosis-4 [FIB-4], NAFLD fibrosis score, Forns, AST to Platelet Ratio Index [APRI], BARD, and Steatosis-Associated Fibrosis Estimator [SAFE]), it exhibited strong discriminative ability in the FLINT and NHANES validation sets, with AUCs of 0.79 (95% CI: 0.73-0.84) and 0.74 (95% CI: 0.68-0.79) respectively.

CONCLUSIONS

Notably, the SAFE score performed similarly to the superlearner, both of which outperformed FIB-4, APRI, Forns, and BARD scores in the validation data sets. Surprisingly, the superlearner derived from 12 base models matched the performance of one with 90 base models. Overall, the superlearner, being the "best-in-class" machine-learning predictor, excelled in detecting fibrotic NASH, and this approach can be used to benchmark the performance of conventional clinical risk prediction models.

Predictive model for non-malignant portal vein thrombosis associated with cirrhosis based on inflammatory biomarkers

BACKGROUND

Portal vein thrombosis (PVT), a complication of liver cirrhosis, is a major public health concern. PVT prediction is the most effective method for PVT diagnosis and treatment.

AIM

To develop and validate a nomogram and network calculator based on clinical indicators to predict PVT in patients with cirrhosis.

METHODS

Patients with cirrhosis hospitalized between January 2016 and December 2021 at the First Hospital of Lanzhou University were screened and 643 patients with cirrhosis who met the eligibility criteria were retrieved. Following a 1:1 propensity score matching 572 patients with cirrhosis were screened, and relevant clinical data were collected. PVT risk factors were identified using the least absolute shrinkage and selection operator (LASSO) and multivariate logistic regression analysis. Variance inflation factors and correlation matrix plots were used to analyze multicollinearity among the variables. A nomogram was constructed to predict the probability of PVT based on independent risk factors for PVT, and its predictive performance was verified using a receiver operating characteristic curve (ROC), calibration curves, and decision curve analysis (DCA). Finally, a network calculator was constructed based on the nomograms.

RESULTS

This study enrolled 286 cirrhosis patients with PVT and 286 without PVT. LASSO analysis revealed 13 variables as strongly associated with PVT occurrence. Multivariate logistic regression analysis revealed nine indicators as independent PVT risk factors, including etiology, ascites, gastroesophageal varices, platelet count, D-dimer, portal vein diameter, portal vein velocity, aspartate transaminase to neutrophil ratio index, and platelet-to-lymphocyte ratio. LASSO and correlation matrix plot results revealed no significant multicollinearity or correlation among the variables. A nomogram was constructed based on the screened independent risk factors. The nomogram had excellent predictive performance, with an area under the ROC curve of 0.821 and 0.829 in the training and testing groups, respectively. Calibration curves and DCA revealed its good clinical performance. Finally, the optimal cutoff value for the total nomogram score was 0.513. The sensitivity and specificity of the optimal cutoff values were 0.822 and 0.706, respectively.

CONCLUSION

A nomogram for predicting PVT occurrence was successfully developed and validated, and a network calculator was constructed. This can enable clinicians to rapidly and easily identify high PVT risk groups.

T cell-mediated tumor killing sensitivity gene signature-based prognostic score for acute myeloid leukemia

BACKGROUND AND OBJECTIVE

Acute myeloid leukemia (AML) is an aggressive, heterogenous hematopoetic malignancies with poor long-term prognosis. T-cell mediated tumor killing plays a key role in tumor immunity. Here, we explored the prognostic performance and functional significance of a T-cell mediated tumor killing sensitivity gene (GSTTK)-based prognostic score (TTKPI).

METHODS

Publicly available transcriptomic data for AML were obtained from TCGA and NCBI-GEO. GSTTK were identified from the TISIDB database. Signature GSTTK for AML were identified by differential expression analysis, COX proportional hazards and LASSO regression analysis and a comprehensive TTKPI score was constructed. Prognostic performance of the TTKPI was examined using Kaplan-Meier survival analysis, Receiver operating curves, and nomogram analysis. Association of TTKPI with clinical phenotypes, tumor immune cell infiltration patterns, checkpoint expression patterns were analysed. Drug docking was used to identify important candidate drugs based on the TTKPI-component genes.

RESULTS

From 401 differentially expressed GSTTK in AML, 24 genes were identified as signature genes and used to construct the TTKPI score. High-TTKPI risk score predicted worse survival and good prognostic accuracy with AUC values ranging from 75 to 96%. Higher TTKPI scores were associated with older age and cancer stage, which showed improved prognostic performance when combined with TTKPI. High TTKPI was associated with lower naïve CD4 T cell and follicular helper T cell infiltrates and higher M2 macrophages/monocyte infiltration. Distinct patterns of immune checkpoint expression corresponded with TTKPI score groups. Three agents; DB11791 (Capmatinib), DB12886 (GSK-1521498) and DB14773 (Lifirafenib) were identified as candidates for AML.

CONCLUSION

A T-cell mediated killing sensitivity gene-based prognostic score TTKPI showed good accuracy in predicting survival in AML. TTKPI corresponded to functional and immunological features of the tumor microenvironment including checkpoint expression patterns and should be investigated for precision medicine approaches.

Development and Validation of Machine Learning Algorithms to Predict 1-Year Ischemic Stroke and Bleeding Events in Patients with Atrial Fibrillation and Cancer

In this study, we leveraged machine learning (ML) approach to develop and validate new assessment tools for predicting stroke and bleeding among patients with atrial fibrillation (AFib) and cancer. We conducted a retrospective cohort study including patients who were newly diagnosed with AFib with a record of cancer from the 2012-2018 Surveillance, Epidemiology, and End Results (SEER)-Medicare database. The ML algorithms were developed and validated separately for each outcome by fitting elastic net, random forest (RF), extreme gradient boosting (XGBoost), support vector machine (SVM), and neural network models with tenfold cross-validation (train:test = 7:3). We obtained area under the curve (AUC), sensitivity, specificity, and F2 score as performance metrics. Model calibration was assessed using Brier score. In sensitivity analysis, we resampled data using Synthetic Minority Oversampling Technique (SMOTE). Among 18,388 patients with AFib and cancer, 523 (2.84%) had ischemic stroke and 221 (1.20%) had major bleeding within one year after AFib diagnosis. In prediction of ischemic stroke, RF significantly outperformed other ML models [AUC (0.916, 95% CI 0.887-0.945), sensitivity 0.868, specificity 0.801, F2 score 0.375, Brier score = 0.035]. However, the performance of ML algorithms in prediction of major bleeding was low with highest AUC achieved by RF (0.623, 95% CI 0.554-0.692). RF models performed better than CHA2DS2-VASc and HAS-BLED scores. SMOTE did not improve the performance of the ML algorithms. Our study demonstrated a promising application of ML in stroke prediction among patients with AFib and cancer. This tool may be leveraged in assisting clinicians to identify patients at high risk of stroke and optimize treatment decisions.

Incidence, Management, and Outcomes of Pulmonary Embolism at Tertiary Pediatric Hospitals in the United States

Background

Pediatric pulmonary embolism (PE) is rare and potentially life-threatening. Though thrombolysis and thrombectomy are increasingly used in adult PE, trends in pediatric treatment and outcomes remain incompletely described.

Objectives

The purpose of this study was to describe the incidence of PE, proportion of cases treated with anticoagulation alone, systemic thrombolysis, and directed therapy (local thrombolysis and thrombectomy), clinical outcomes, and total costs.

Methods

A multicenter observational study was performed using administrative data from the Pediatric Health Information System database to study PE treated at U.S. pediatric hospitals from 2015 to 2021. Outcomes by treatment were evaluated using multivariable generalized linear mixed effects models.

Results

Of 3,136 subjects, 70% were at least 12 years of age, and 46% were male. Sixty-two percent had at least 1 comorbidity, and congenital heart disease of any kind was the most prevalent (20%). Eighty-eight percent of subjects received anticoagulation alone, 7% received systemic thrombolysis, and 5% received directed therapy. Overall in-hospital mortality was 7.5%. Treatment approach did not change over time (P = 0.98). After adjusting for patient characteristics, directed therapy was associated with a lower risk of mortality (adjusted percentage -3%, [95% CI: -5% to 0%]) than anticoagulation alone. Systemic thrombolysis was associated with a greater total cost of hospitalization ($113,043 greater [95% CI: $62,866, $163,219]). Length of hospital stay did not differ by treatment.

Conclusions

Pediatric patients with PE have a high incidence of underlying chronic disease. Anticoagulation alone remains the mainstay of treatment, with thrombolysis and thrombectomy rarely being used. Given the relative rarity of pediatric PE, additional research requiring innovative study designs is paramount.

Deep learning-based facial analysis for predicting difficult videolaryngoscopy: a feasibility study

While videolaryngoscopy has resulted in better overall success rates of tracheal intubation, airway assessment is still an important prerequisite for safe airway management. This study aimed to create an artificial intelligence model to identify difficult videolaryngoscopy using a neural network. Baseline characteristics, medical history, bedside examination and seven facial images were included as predictor variables. ResNet-18 was introduced to recognise images and extract features. Different machine learning algorithms were utilised to develop predictive models. A videolaryngoscopy view of Cormack-Lehane grade of 1 or 2 was classified as 'non-difficult', while grade 3 or 4 was classified as 'difficult'. A total of 5849 patients were included, of whom 5335 had non-difficult and 514 had difficult videolaryngoscopy. The facial model (only including facial images) using the Light Gradient Boosting Machine algorithm showed the highest area under the curve (95%CI) of 0.779 (0.733-0.825) with a sensitivity (95%CI) of 0.757 (0.650-0.845) and specificity (95%CI) of 0.721 (0.626-0.794) in the test set. Compared with bedside examination and multivariate scores (El-Ganzouri and Wilson), the facial model had significantly higher predictive performance (p < 0.001). Artificial intelligence-based facial analysis is a feasible technique for predicting difficulty during videolaryngoscopy, and the model developed using neural networks has higher predictive performance than traditional methods.

Early prediction of mortality at sepsis diagnosis time in critically ill patients by using interpretable machine learning

This study applied machine learning for the early prediction of 30-day mortality at sepsis diagnosis time in critically ill patients. Retrospective study using data collected from the Medical Information Mart for Intensive Care IV database. The data of the patient cohort was divided on the basis of the year of hospitalization, into training (2008-2013), validation (2014-2016), and testing (2017-2019) datasets. 24,377 patients with the sepsis diagnosis time < 24 h after intensive care unit (ICU) admission were included. A gradient boosting tree-based algorithm (XGBoost) was used for training the machine learning model to predict 30-day mortality at sepsis diagnosis time in critically ill patients. Model performance was measured in both discrimination and calibration aspects. The model was interpreted using the SHapley Additive exPlanations (SHAP) module. The 30-day mortality rate of the testing dataset was 17.9%, and 39 features were selected for the machine learning model. Model performance on the testing dataset achieved an area under the receiver operating characteristic curve (AUROC) of 0.853 (95% CI 0.837-0.868) and an area under the precision-recall curves of 0.581 (95% CI 0.541-0.619). The calibration plot for the model revealed a slope of 1.03 (95% CI 0.94-1.12) and intercept of 0.14 (95% CI 0.04-0.25). The SHAP revealed the top three most significant features, namely age, increased red blood cell distribution width, and respiratory rate. Our study demonstrated the feasibility of using the interpretable machine learning model to predict mortality at sepsis diagnosis time.

A Vulnerability Index to Assess the Risk of SARS-CoV-2-Related Hospitalization/Death: Urgent Need for an Update after Diffusion of Anti-COVID Vaccines

Background: There are algorithms to predict the risk of SARS-CoV-2-related complications. Given the spread of anti-COVID vaccination, which sensibly modified the burden of risk of the infection, these tools need to be re-calibrated. Therefore, we updated our vulnerability index, namely, the Health Search (HS)-CoVulnerabiltyIndex (VI)d (HS-CoVId), to predict the risk of SARS-CoV-2-related hospitalization/death in the primary care setting. Methods: We formed a cohort of individuals aged ≥15 years and diagnosed with COVID-19 between 1 January and 31 December 2021 in the HSD. The date of COVID-19 diagnosis was the study index date. These patients were eligible if they had received an anti-COVID vaccine at least 15 days before the index date. Patients were followed up from the index date until one of the following events, whichever came first: COVID-19-related hospitalization/death (event date), end of registration with their GPs, and end of the study period (31 December 2022). To calculate the incidence rate of COVID-19-related hospitalization/death, a patient-specific score was derived through linear combination of the coefficients stemming from a multivariate Cox regression model. Its prediction performance was evaluated by obtaining explained variation, discrimination, and calibration measures. Results: We identified 2192 patients who had received an anti-COVID vaccine from 1 January to 31 December 2021. With this cohort, we re-calibrated the HS-CoVId by calculating optimism-corrected pseudo-R2, AUC, and calibration slope. The final model reported a good predictive performance by explaining 58% (95% CI: 48-71%) of variation in the occurrence of hospitalizations/deaths, the AUC was 83 (95% CI: 77-93%), and the calibration slope did not reject the equivalence hypothesis (p-value = 0.904). Conclusions: Two versions of HS-CoVId need to be differentially adopted to assess the risk of COVID-19-related complications among vaccinated and unvaccinated subjects. Therefore, this functionality should be operationalized in related patient- and population-based informatic tools intended for general practitioners.

Emergency Department Utilization for Pediatric Gastrostomy Tubes Across the United States

INTRODUCTION

Emergency Department (ED) visits for gastrostomy tube complications in children represent a substantial health-care burden, and many ED visits are potentially preventable. The number and nature of ED visits to community hospitals for pediatric gastrostomy tube complications is unknown.

METHODS

Using the 2019 Nationwide Emergency Department Sample, we performed a retrospective cross-sectional analysis of pediatric patients (<18 y) with a primary diagnosis of gastrostomy tube complication. Our primary outcome was a potentially preventable ED visit, defined as an encounter that did not result in any imaging, procedures, or an inpatient admission. Univariate and multivariable logistic regression analyses were used to determine the associations between patient factors and our primary outcome.

RESULTS

We observed 32,036 ED visits at 535 hospitals and 15,165 (47.3%) were potentially preventable. The median (interquartile range) age was 2 (1, 6) years and 17,707 (55%) were male. Compared to White patients, patients with higher odds of potentially preventable visits were Black (adjusted odds ratio (aOR) [95% confidence interval {CI}]: 1.07 [1.05-1.11], P < 0.001) and Hispanic (aOR [95% CI]: 1.05 [1.02-1.08], P = 0.004). Patients with residential zip codes in the first (aOR [95% CI]: 1.08 [1.04, 1.12], P < 0.001), second (aOR [95% CI]: 1.07 [1.03, 1.11], P < 0.001), and third (aOR [95% CI]: 1.09 [1.05, 1.13], P < 0.001) median household income quartiles had higher odds of potentially preventable visits compared to the highest.

CONCLUSIONS

In a nationally representative sample of EDs, 47.3% of visits for pediatric gastrostomy tubes were potentially preventable. Efforts to improve outpatient management are warranted to reduce health-care utilization for these patients.

A review on statistical and machine learning competing risks methods

When modeling competing risks (CR) survival data, several techniques have been proposed in both the statistical and machine learning literature. State-of-the-art methods have extended classical approaches with more flexible assumptions that can improve predictive performance, allow high-dimensional data and missing values, among others. Despite this, modern approaches have not been widely employed in applied settings. This article aims to aid the uptake of such methods by providing a condensed compendium of CR survival methods with a unified notation and interpretation across approaches. We highlight available software and, when possible, demonstrate their usage via reproducible R vignettes. Moreover, we discuss two major concerns that can affect benchmark studies in this context: the choice of performance metrics and reproducibility.

Nomogram for predicting neutropenia in patients with esophageal, gastric, or colorectal cancer treated by chemotherapy in the first cycle

OBJECTIVES

Development and validation of a predictive model including serum vitamin concentration to estimate the risk of chemotherapy-induced grade 3/4 neutropenia in esophageal cancer, gastric cancer, or colorectal cancer patients who receive the first cycle of chemotherapy.

METHODS

Data from 535 patients treated at the Affiliated Fuyang People's Hospital of Anhui Medical University from January 1, 2020, to March 2, 2022, were used to derive the predictive model. Least absolute shrinkage and selection operator regression analysis was performed to screen potential risk characteristics, and multivariate logistic regression was utilized to investigate efficient factors associated with chemotherapy-induced neutropenia. A nomogram was constructed using this logistic model. This nomogram was then tested on a temporal validation cohort containing 212 consecutive patients.

RESULTS

In the cohort of all 747 eligible patients, grade 3/4 neutropenia incidence was 45.2%. Age, Eastern Cooperative Oncology Group-performance status, neutrophil count, serum albumin, and hemoglobin data were entered into the final model. The performance of the final predictive nomogram was assessed by the area under the receiver operating characteristic curve in both the development and validation datasets. The calibration curves indicated that the estimated risks were accurate. Decision curve analysis for the predictive model exhibited improved clinical practicality.

CONCLUSION

In the present study, we established an accessible risk predictive model and identified valuable serum vitamin concentration parameters associated with chemotherapy-induced neutropenia. The predictive model may improve the grade 3/4 neutropenia risk prediction in patients with gastrointestinal malignancies who receive oxaliplatin- and fluoropyrimidine-based chemotherapy and help physicians make appropriate decisions for disease management.

Leveraging artificial intelligence for perioperative cancer risk assessment of oral potentially malignant disorders

Oral potentially malignant disorders (OPMDs) are mucosal conditions with an inherent disposition to develop oral squamous cell carcinoma. Surgical management is the most preferred strategy to prevent malignant transformation in OPMDs, and surgical approaches to treatment include conventional scalpel excision, laser surgery, cryotherapy, and photodynamic therapy. However, in reality, since all patients with OPMDs will not develop oral squamous cell carcinoma in their lifetime, there is a need to stratify patients according to their risk of malignant transformation to streamline surgical intervention for patients with the highest risks. Artificial intelligence (AI) has the potential to integrate disparate factors influencing malignant transformation for robust, precise, and personalized cancer risk stratification of OPMD patients than current methods to determine the need for surgical resection, excision, or re-excision. Therefore, this article overviews existing AI models and tools, presents a clinical implementation pathway, and discusses necessary refinements to aid the clinical application of AI-based platforms for cancer risk stratification of OPMDs in surgical practice.

Investigation of statistical methods used in prognostic prediction models for obstetric care: A 10 year-span cross-sectional study

INTRODUCTION

Obstetric care is a highly active area in the development and application of prognostic prediction models. The development and validation of these models often require the utilization of advanced statistical techniques. However, failure to adhere to rigorous methodological standards could greatly undermine the reliability and trustworthiness of the resultant models. Consequently, the aim of our study was to examine the current statistical practices employed in obstetric care and offer recommendations to enhance the utilization of statistical methods in the development of prognostic prediction models.

MATERIAL AND METHODS

We conducted a cross-sectional survey using a sample of studies developing or validating prognostic prediction models for obstetric care published in a 10-year span (2011-2020). A structured questionnaire was developed to investigate the statistical issues in five domains, including model derivation (predictor selection and algorithm development), model validation (internal and external), model performance, model presentation, and risk threshold setting. On the ground of survey results and existing guidelines, a list of recommendations for statistical methods in prognostic models was developed.

RESULTS

A total of 112 eligible studies were included, with 107 reporting model development and five exclusively reporting external validation. During model development, 58.9% of the studies did not include any form of validation. Of these, 46.4% used stepwise regression in a crude manner for predictor selection, while two-thirds made decisions on retaining or dropping candidate predictors solely based on p-values. Additionally, 26.2% transformed continuous predictors into categorical variables, and 80.4% did not consider nonlinear relationships between predictors and outcomes. Surprisingly, 94.4% of the studies did not examine the correlation between predictors. Moreover, 47.1% of the studies did not compare population characteristics between the development and external validation datasets, and only one-fifth evaluated both discrimination and calibration. Furthermore, 53.6% of the studies did not clearly present the model, and less than half established a risk threshold to define risk categories. In light of these findings, 10 recommendations were formulated to promote the appropriate use of statistical methods.

CONCLUSIONS

The use of statistical methods is not yet optimal. Ten recommendations were offered to assist the statistical methods of prognostic prediction models in obstetric care.

Semi-supervised ROC analysis for reliable and streamlined evaluation of phenotyping algorithms

OBJECTIVE

High-throughput phenotyping will accelerate the use of electronic health records (EHRs) for translational research. A critical roadblock is the extensive medical supervision required for phenotyping algorithm (PA) estimation and evaluation. To address this challenge, numerous weakly-supervised learning methods have been proposed. However, there is a paucity of methods for reliably evaluating the predictive performance of PAs when a very small proportion of the data is labeled. To fill this gap, we introduce a semi-supervised approach (ssROC) for estimation of the receiver operating characteristic (ROC) parameters of PAs (eg, sensitivity, specificity).

MATERIALS AND METHODS

ssROC uses a small labeled dataset to nonparametrically impute missing labels. The imputations are then used for ROC parameter estimation to yield more precise estimates of PA performance relative to classical supervised ROC analysis (supROC) using only labeled data. We evaluated ssROC with synthetic, semi-synthetic, and EHR data from Mass General Brigham (MGB).

RESULTS

ssROC produced ROC parameter estimates with minimal bias and significantly lower variance than supROC in the simulated and semi-synthetic data. For the 5 PAs from MGB, the estimates from ssROC are 30% to 60% less variable than supROC on average.

DISCUSSION

ssROC enables precise evaluation of PA performance without demanding large volumes of labeled data. ssROC is also easily implementable in open-source R software.

CONCLUSION

When used in conjunction with weakly-supervised PAs, ssROC facilitates the reliable and streamlined phenotyping necessary for EHR-based research.

Development and validation of a risk calculator for postoperative diplopia following orbital fracture repair in adults

Postoperative diplopia is the most common complication following orbital fracture repair (OFR). Existing evidence on its risk factors is based on single-institution studies and small sample sizes. Our study is the first multi-center study to develop and validate a risk calculator for the prediction of postoperative diplopia following OFR. We reviewed trauma patients who underwent OFR at two high-volume trauma centers (2015-2019). Excluded were patients < 18 years old and those with postoperative follow-up < 2 weeks. Our primary outcome was incidence/persistence of postoperative diplopia at ≥ 2 weeks. A risk model for the prediction of postoperative diplopia was derived using a development dataset (70% of population) and validated using a validation dataset (remaining 30%). The C-statistic and Hosmer-Lemeshow tests were used to assess the risk model accuracy. A total of n = 254 adults were analyzed. The factors that predicted postoperative diplopia were: age at injury, preoperative enophthalmos, fracture size/displacement, surgical timing, globe/soft tissue repair, and medial wall involvement. Our predictive model had excellent discrimination (C-statistic = 80.4%), calibration (P = 0.2), and validation (C-statistic = 80%). Our model rules out postoperative diplopia with a 100% sensitivity and negative predictive value (NPV) for a probability < 8.9%. Our predictive model rules out postoperative diplopia with an 87.9% sensitivity and a 95.8% NPV for a probability < 13.4%. We designed the first validated risk calculator that can be used as a powerful screening tool to rule out postoperative diplopia following OFR in adults.

Improved prediction of clinical pregnancy using artificial intelligence with enhanced inner cell mass and trophectoderm images

This study aimed to assess the performance of an artificial intelligence (AI) model for predicting clinical pregnancy using enhanced inner cell mass (ICM) and trophectoderm (TE) images. In this retrospective study, we included static images of 2555 day-5-blastocysts from seven in vitro fertilization centers in South Korea. The main outcome of the study was the predictive capability of the model to detect clinical pregnancies (gestational sac). Compared with the original embryo images, the use of enhanced ICM and TE images improved the average area under the receiver operating characteristic curve for the AI model from 0.716 to 0.741. Additionally, a gradient-weighted class activation mapping analysis demonstrated that the enhanced image-trained AI model was able to extract features from crucial areas of the embryo in 99% (506/512) of the cases. Particularly, it could extract the ICM and TE. In contrast, the AI model trained on the original images focused on the main areas in only 86% (438/512) of the cases. Our results highlight the potential efficacy of using ICM- and TE-enhanced embryo images when training AI models to predict clinical pregnancy.

Heart rate complexity helps mortality prediction in the intensive care unit: A pilot study using artificial intelligence

BACKGROUND

In intensive care units (ICUs), accurate mortality prediction is crucial for effective patient management and resource allocation. The Simplified Acute Physiology Score II (SAPS-2), though commonly used, relies heavily on comprehensive clinical data and blood samples. This study sought to develop an artificial intelligence (AI) model utilizing key hemodynamic parameters to predict ICU mortality within the first 24 h and assess its performance relative to SAPS-2.

METHODS

We conducted an analysis of select hemodynamic parameters and the structure of heart rate curves to identify potential predictors of ICU mortality. A machine-learning model was subsequently trained and validated on distinct patient cohorts. The AI algorithm's performance was then compared to the SAPS-2, focusing on classification accuracy, calibration, and generalizability.

MEASUREMENTS AND MAIN RESULTS

The study included 1298 ICU admissions from March 27th, 2015, to March 27th, 2017. An additional cohort from 2022 to 2023 comprised 590 patients, resulting in a total dataset of 1888 patients. The observed mortality rate stood at 24.0%. Key determinants of mortality were the Glasgow Coma Scale score, heart rate complexity, patient age, duration of diastolic blood pressure below 50 mmHg, heart rate variability, and specific mean and systolic blood pressure thresholds. The AI model, informed by these determinants, exhibited a performance profile in predicting mortality that was comparable, if not superior, to the SAPS-2.

CONCLUSIONS

The AI model, which integrates heart rate and blood pressure curve analyses with basic clinical parameters, provides a methodological approach to predict in-hospital mortality in ICU patients. This model offers an alternative to existing tools that depend on extensive clinical data and laboratory inputs. Its potential integration into ICU monitoring systems may facilitate more streamlined mortality prediction processes.

Artificial Intelligence Applications in the Treatment of Colorectal Cancer: A Narrative Review

Colorectal cancer is the third most prevalent cancer worldwide, and its treatment has been a demanding clinical problem. Beyond traditional surgical therapy and chemotherapy, newly revealed molecular mechanisms diversify therapeutic approaches for colorectal cancer. However, the selection of personalized treatment among multiple treatment options has become another challenge in the era of precision medicine. Artificial intelligence has recently been increasingly investigated in the treatment of colorectal cancer. This narrative review mainly discusses the applications of artificial intelligence in the treatment of colorectal cancer patients. A comprehensive literature search was conducted in MEDLINE, EMBASE, and Web of Science to identify relevant papers, resulting in 49 articles being included. The results showed that, based on different categories of data, artificial intelligence can predict treatment outcomes and essential guidance information of traditional and novel therapies, thus enabling individualized treatment strategy selection for colorectal cancer patients. Some frequently implemented machine learning algorithms and deep learning frameworks have also been employed for long-term prognosis prediction in patients with colorectal cancer. Overall, artificial intelligence shows encouraging results in treatment strategy selection and prognosis evaluation for colorectal cancer patients.

Construction and evaluation of neonatal respiratory failure risk prediction model for neonatal respiratory distress syndrome

BACKGROUND

Neonatal respiratory distress syndrome (NRDS) is a common respiratory disease in preterm infants, often accompanied by respiratory failure. The aim of this study was to establish and validate a nomogram model for predicting the probability of respiratory failure in NRDS patients.

METHODS

Patients diagnosed with NRDS were extracted from the MIMIC-iv database. The patients were randomly assigned to a training and a validation cohort. Univariate and stepwise Cox regression analyses were used to determine the prognostic factors of NRDS. A nomogram containing these factors was established to predict the incidence of respiratory failure in NRDS patients. The area under the receiver operating characteristic curve (AUC), receiver operating characteristic curve (ROC), calibration curves and decision curve analysis were used to determine the effectiveness of this model.

RESULTS

The study included 2,705 patients with NRDS. Univariate and multivariate stepwise Cox regression analysis showed that the independent risk factors for respiratory failure in NRDS patients were gestational age, pH, partial pressure of oxygen (PO2), partial pressure of carbon dioxide (PCO2), hemoglobin, blood culture, infection, neonatal intracranial hemorrhage, Pulmonary surfactant (PS), parenteral nutrition and respiratory support. Then, the nomogram was constructed and verified.

CONCLUSIONS

This study identified the independent risk factors of respiratory failure in NRDS patients and used them to construct and evaluate respiratory failure risk prediction model for NRDS. The present findings provide clinicians with the judgment of patients with respiratory failure in NRDS and help clinicians to identify and intervene in the early stage.

Comparing linear discriminant analysis and supervised learning algorithms for binary classification-A method comparison study

In psychology, linear discriminant analysis (LDA) is the method of choice for two-group classification tasks based on questionnaire data. In this study, we present a comparison of LDA with several supervised learning algorithms. In particular, we examine to what extent the predictive performance of LDA relies on the multivariate normality assumption. As nonparametric alternatives, the linear support vector machine (SVM), classification and regression tree (CART), random forest (RF), probabilistic neural network (PNN), and the ensemble k conditional nearest neighbor (EkCNN) algorithms are applied. Predictive performance is determined using measures of overall performance, discrimination, and calibration, and is compared in two reference data sets as well as in a simulation study. The reference data are Likert-type data, and comprise 5 and 10 predictor variables, respectively. Simulations are based on the reference data and are done for a balanced and an unbalanced scenario in each case. In order to compare the algorithms' performance, data are simulated from multivariate distributions with differing degrees of nonnormality. Results differ depending on the specific performance measure. The main finding is that LDA is always outperformed by RF in the bimodal data with respect to overall performance. Discriminative ability of the RF algorithm is often higher compared to LDA, but its model calibration is usually worse. Still LDA mostly ranges second in cases it is outperformed by another algorithm, or the differences are only marginal. In consequence, we still recommend LDA for this type of application.

Transportability of bacterial infection prediction models for critically ill patients

OBJECTIVE

Bacterial infections (BIs) are common, costly, and potentially life-threatening in critically ill patients. Patients with suspected BIs may require empiric multidrug antibiotic regimens and therefore potentially be exposed to prolonged and unnecessary antibiotics. We previously developed a BI risk model to augment practices and help shorten the duration of unnecessary antibiotics to improve patient outcomes. Here, we have performed a transportability assessment of this BI risk model in 2 tertiary intensive care unit (ICU) settings and a community ICU setting. We additionally explored how simple multisite learning techniques impacted model transportability.

METHODS

Patients suspected of having a community-acquired BI were identified in 3 datasets: Medical Information Mart for Intensive Care III (MIMIC), Northwestern Medicine Tertiary (NM-T) ICUs, and NM "community-based" ICUs. ICU encounters from MIMIC and NM-T datasets were split into 70/30 train and test sets. Models developed on training data were evaluated against the NM-T and MIMIC test sets, as well as NM community validation data.

RESULTS

During internal validations, models achieved AUROCs of 0.78 (MIMIC) and 0.81 (NM-T) and were well calibrated. In the external community ICU validation, the NM-T model had robust transportability (AUROC 0.81) while the MIMIC model transported less favorably (AUROC 0.74), likely due to case-mix differences. Multisite learning provided no significant discrimination benefit in internal validation studies but offered more stability during transport across all evaluation datasets.

DISCUSSION

These results suggest that our BI risk models maintain predictive utility when transported to external cohorts.

CONCLUSION

Our findings highlight the importance of performing external model validation on myriad clinically relevant populations prior to implementation.

Calibrating machine learning approaches for probability estimation: A comprehensive comparison

Statistical prediction models have gained popularity in applied research. One challenge is the transfer of the prediction model to a different population which may be structurally different from the model for which it has been developed. An adaptation to the new population can be achieved by calibrating the model to the characteristics of the target population, for which numerous calibration techniques exist. In view of this diversity, we performed a systematic evaluation of various popular calibration approaches used by the statistical and the machine learning communities for estimating two-class probabilities. In this work, we first provide a review of the literature and, second, present the results of a comprehensive simulation study. The calibration approaches are compared with respect to their empirical properties and relationships, their ability to generalize precise probability estimates to external populations and their availability in terms of easy-to-use software implementations. Third, we provide code from real data analysis allowing its application by researchers. Logistic calibration and beta calibration, which estimate an intercept plus one and two slope parameters, respectively, consistently showed the best results in the simulation studies. Calibration on logit transformed probability estimates generally outperformed calibration methods on nontransformed estimates. In case of structural differences between training and validation data, re-estimation of the entire prediction model should be outweighted against sample size of the validation data. We recommend regression-based calibration approaches using transformed probability estimates, where at least one slope is estimated in addition to an intercept for updating probability estimates in validation studies.

Adjusting for Congenital Heart Surgery Risk Using Administrative Data

BACKGROUND

Congenital heart surgery (CHS) encompasses a heterogeneous population of patients and surgeries. Risk standardization models that adjust for patient and procedural characteristics can allow for collective study of these disparate patients and procedures.

OBJECTIVES

We sought to develop a risk-adjustment model for CHS using the newly developed Risk Stratification for Congenital Heart Surgery for ICD-10 Administrative Data (RACHS-2) methodology.

METHODS

Within the Kids' Inpatient Database 2019, we identified all CHSs that could be assigned a RACHS-2 score. Hierarchical logistic regression (clustered on hospital) was used to identify patient and procedural characteristics associated with in-hospital mortality. Model validation was performed using data from 24 State Inpatient Databases during 2017.

RESULTS

Of 5,902,538 total weighted hospital discharges in the Kids' Inpatient Database 2019, 22,310 pediatric cardiac surgeries were identified and assigned a RACHS-2 score. In-hospital mortality occurred in 543 (2.4%) of cases. Using only RACHS-2, the mortality mode had a C-statistic of 0.81 that improved to 0.83 with the addition of age. A final multivariable model inclusive of RACHS-2, age, payer, and presence of a complex chronic condition outside of congenital heart disease further improved model discrimination to 0.87 (P < 0.001). Discrimination in the validation cohort was also very good with a C-statistic of 0.83.

CONCLUSIONS

We created and validated a risk-adjustment model for CHS that accounts for patient and procedural characteristics associated with in-hospital mortality available in administrative data, including the newly developed RACHS-2. Our risk model will be critical for use in health services research and quality improvement initiatives.

Deep Learning of Electrocardiograms in Sinus Rhythm From US Veterans to Predict Atrial Fibrillation

Importance

Early detection of atrial fibrillation (AF) may help prevent adverse cardiovascular events such as stroke. Deep learning applied to electrocardiograms (ECGs) has been successfully used for early identification of several cardiovascular diseases.

Objective

To determine whether deep learning models applied to outpatient ECGs in sinus rhythm can predict AF in a large and diverse patient population.

Design, Setting, and Participants

This prognostic study was performed on ECGs acquired from January 1, 1987, to December 31, 2022, at 6 US Veterans Affairs (VA) hospital networks and 1 large non-VA academic medical center. Participants included all outpatients with 12-lead ECGs in sinus rhythm.

Main Outcomes and Measures

A convolutional neural network using 12-lead ECGs from 2 US VA hospital networks was trained to predict the presence of AF within 31 days of sinus rhythm ECGs. The model was tested on ECGs held out from training at the 2 VA networks as well as 4 additional VA networks and 1 large non-VA academic medical center.

Results

A total of 907 858 ECGs from patients across 6 VA sites were included in the analysis. These patients had a mean (SD) age of 62.4 (13.5) years, 6.4% were female, and 93.6% were male, with a mean (SD) CHA2DS2-VASc (congestive heart failure, hypertension, age, diabetes mellitus, prior stroke or transient ischemic attack or thromboembolism, vascular disease, age, sex category) score of 1.9 (1.6). A total of 0.2% were American Indian or Alaska Native, 2.7% were Asian, 10.7% were Black, 4.6% were Latinx, 0.7% were Native Hawaiian or Other Pacific Islander, 62.4% were White, 0.4% were of other race or ethnicity (which is not broken down into subcategories in the VA data set), and 18.4% were of unknown race or ethnicity. At the non-VA academic medical center (72 483 ECGs), the mean (SD) age was 59.5 (15.4) years and 52.5% were female, with a mean (SD) CHA2DS2-VASc score of 1.6 (1.4). A total of 0.1% were American Indian or Alaska Native, 7.9% were Asian, 9.4% were Black, 2.9% were Latinx, 0.03% were Native Hawaiian or Other Pacific Islander, 74.8% were White, 0.1% were of other race or ethnicity, and 4.7% were of unknown race or ethnicity. A deep learning model predicted the presence of AF within 31 days of a sinus rhythm ECG on held-out test ECGs at VA sites with an area under the receiver operating characteristic curve (AUROC) of 0.86 (95% CI, 0.85-0.86), accuracy of 0.78 (95% CI, 0.77-0.78), and F1 score of 0.30 (95% CI, 0.30-0.31). At the non-VA site, AUROC was 0.93 (95% CI, 0.93-0.94); accuracy, 0.87 (95% CI, 0.86-0.88); and F1 score, 0.46 (95% CI, 0.44-0.48). The model was well calibrated, with a Brier score of 0.02 across all sites. Among individuals deemed high risk by deep learning, the number needed to screen to detect a positive case of AF was 2.47 individuals for a testing sensitivity of 25% and 11.48 for 75%. Model performance was similar in patients who were Black, female, or younger than 65 years or who had CHA2DS2-VASc scores of 2 or greater.

Conclusions and Relevance

Deep learning of outpatient sinus rhythm ECGs predicted AF within 31 days in populations with diverse demographics and comorbidities. Similar models could be used in future AF screening efforts to reduce adverse complications associated with this disease.

The predictive accuracy of coronary heart disease risk prediction models in rural Northwestern China

Cardiovascular risk models developed may have limitations when applied to rural Chinese. This study validated and compared the Framingham Risk Score (FRS) and Prediction for Atherosclerotic Cardiovascular Disease Risk in China (PAR) models in predicting 10-year risk of coronary heart disease (CHD) in a rural cohort in Ningxia, China from 2008 to 2019. The FRS and PAR models were validated by estimating predicted events, C index, calibration χ2 and plots. 1381 adults without CHD at baseline were followed up for 9.75 years on average. 168 CHD cases were observed. The FRS and PAR underestimated CHD events by 22 % and 46 % for the total population, while overestimated for males by 152 % and 78 %, respectively. The C index was slightly higher for PAR than FRS. Both models showed weak calibration with chi-square values above 20 (p < 0.001). Bland-Altman plots indicated FRS predicted higher CHD risk than PAR, lacking consistency. Overall, FRS and PAR demonstrated limited performance in predicting 10-year CHD risk in this rural population. PAR had slightly better discrimination than FRS, but require further improvement in calibration and individual risk estimation to suit the rural population in Northwest China.

Impact of COVID-19 infection experience on mental health status of intensive care unit patients' family members: a real-world study

PURPOSE

Family members of patients hospitalized in intensive care unit (ICU) are susceptible to adverse psychological outcomes. However, there is a paucity of studies specifically examining the mental health symptoms in ICU patients' family members with a prior history of coronavirus disease 2019 (COVID-19) infection.

AIM

This study aimed to investigate mental health status and its influencing factors of ICU patients' family members with previous COVID-19 infection experience in China.

DESIGN

Nationwide, cross-sectional cohort of consecutive participants of family members of ICU patients from 10 provinces randomly selected in mainland China conducted between October 2022 and May 2023.

METHODS

The basic information scale, Self-rating depression scale, Self-rating Anxiety Scale, Impact of Event Scale-Revised, Pittsburgh sleep quality index, Perceived Stress Scale, Connor-Davidson resilience scale, Simplified Coping Style Questionnaire were employed to explore mental health status among participants.

RESULTS

A total of 463 participants, comprising 156 individuals in Covid-19 family member cohort (infection group) and 307 individuals in control family member cohort (control group), met inclusion criteria. The infection group exhibited significantly higher incidence of composite mental health symptoms compared to control group (P = 0.017). Multivariable logistic regression analysis revealed that being female, engaging in physical/mental labor, residing in rural areas, and having children were identified as risk factors for the development of depression, anxiety, and post-traumatic stress disorder symptoms, while medical history of surgery was protective factor. A predictive model demonstrated a favorable discriminative ability and excellent calibration.

CONCLUSION

COVID-19 infection experience regarded as new traumatic stressors worsen mental health status of ICU patients' family members.

An Explainable Artificial Intelligence Model Proposed for the Prediction of Myalgic Encephalomyelitis/Chronic Fatigue Syndrome and the Identification of Distinctive Metabolites

BACKGROUND

Myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS) is a complex and debilitating illness with a significant global prevalence, affecting over 65 million individuals. It affects various systems, including the immune, neurological, gastrointestinal, and circulatory systems. Studies have shown abnormalities in immune cell types, increased inflammatory cytokines, and brain abnormalities. Further research is needed to identify consistent biomarkers and develop targeted therapies. This study uses explainable artificial intelligence and machine learning techniques to identify discriminative metabolites for ME/CFS.

MATERIAL AND METHODS

The model investigates a metabolomics dataset of CFS patients and healthy controls, including 26 healthy controls and 26 ME/CFS patients aged 22-72. The dataset encapsulated 768 metabolites into nine metabolic super-pathways: amino acids, carbohydrates, cofactors, vitamins, energy, lipids, nucleotides, peptides, and xenobiotics. Random forest methods together with other classifiers were applied to the data to classify individuals as ME/CFS patients and healthy individuals. The classification learning algorithms' performance in the validation step was evaluated using a variety of methods, including the traditional hold-out validation method, as well as the more modern cross-validation and bootstrap methods. Explainable artificial intelligence approaches were applied to clinically explain the optimum model's prediction decisions.

RESULTS

The metabolomics of C-glycosyltryptophan, oleoylcholine, cortisone, and 3-hydroxydecanoate were determined to be crucial for ME/CFS diagnosis. The random forest model outperformed the other classifiers in ME/CFS prediction using the 1000-iteration bootstrapping method, achieving 98% accuracy, precision, recall, F1 score, 0.01 Brier score, and 99% AUC. According to the obtained results, the bootstrap validation approach demonstrated the highest classification outcomes.

CONCLUSION

The proposed model accurately classifies ME/CFS patients based on the selected biomarker candidate metabolites. It offers a clear interpretation of risk estimation for ME/CFS, aiding physicians in comprehending the significance of key metabolomic features within the model.

Fairness of Machine Learning Algorithms for Predicting Foregone Preventive Dental Care for Adults

Importance

Access to routine dental care prevents advanced dental disease and improves oral and overall health. Identifying individuals at risk of foregoing preventive dental care can direct prevention efforts toward high-risk populations.

Objective

To predict foregone preventive dental care among adults overall and in sociodemographic subgroups and to assess the algorithmic fairness.

Design, Setting, and Participants

This prognostic study was a secondary analyses of longitudinal data from the US Medical Expenditure Panel Survey (MEPS) from 2016 to 2019, each with 2 years of follow-up. Participants included adults aged 18 years and older. Data analysis was performed from December 2022 to June 2023.

Exposure

A total of 50 predictors, including demographic and socioeconomic characteristics, health conditions, behaviors, and health services use, were assessed.

Main Outcomes and Measures

The outcome of interest was foregoing preventive dental care, defined as either cleaning, general examination, or an appointment with the dental hygienist, in the past year.

Results

Among 32 234 participants, the mean (SD) age was 48.5 (18.2) years and 17 386 participants (53.9%) were female; 1935 participants (6.0%) were Asian, 5138 participants (15.9%) were Black, 7681 participants (23.8%) were Hispanic, 16 503 participants (51.2%) were White, and 977 participants (3.0%) identified as other (eg, American Indian and Alaska Native) or multiple racial or ethnic groups. There were 21 083 (65.4%) individuals who missed preventive dental care in the past year. The algorithms demonstrated high performance, achieving an area under the receiver operating characteristic curve (AUC) of 0.84 (95% CI, 0.84-0.85) in the overall population. While the full sample model performed similarly when applied to White individuals and older adults (AUC, 0.88; 95% CI, 0.87-0.90), there was a loss of performance for other subgroups. Removing the subgroup-sensitive predictors (ie, race and ethnicity, age, and income) did not impact model performance. Models stratified by race and ethnicity performed similarly or worse than the full model for all groups, with the lowest performance for individuals who identified as other or multiple racial groups (AUC, 0.76; 95% CI, 0.70-0.81). Previous pattern of dental visits, health care utilization, dental benefits, and sociodemographic characteristics were the highest contributing predictors to the models' performance.

Conclusions and Relevance

Findings of this prognostic study using cohort data suggest that tree-based ensemble machine learning models could accurately predict adults at risk of foregoing preventive dental care and demonstrated bias against underrepresented sociodemographic groups. These results highlight the importance of evaluating model fairness during development and testing to avoid exacerbating existing biases.

Development and Validation of a Nomogram to Predict the Risk of Recurrent Lower Extremity Radiating Pain Within 1 Week Following Full-Endoscopic Lumbar Discectomy

BACKGROUND

Accurately predicting the risk of lower extremity (LE) radiating pain after surgery is an important endeavor for spinal surgeons. Our study aimed to identify risk factors for LE radiating pain after decompression with full-endoscopic lumbar discectomy (FELD) and develop a nomogram.

METHODS

We retrospectively reviewed the medical data of patients with lumbar disc herniation who underwent FELD. Two hundred thirty-five patients diagnosed at our hospital from January 2015 to December 2020 were used for model development. The independent risk factors for LE radiating pain after surgery were determined by least absolute shrinkage and selection operator logistic regression and multivariate logistic regression analysis. A nomogram was developed to predict the risk of LE radiating pain based on independent risk factors. Receiver operating characteristic curve, calibration curve, and decision curve analyses were used to evaluate the predictive performance. The nomogram was further verified by an independent cohort.

RESULTS

Three hundred seventy-five patients were enrolled in this study, with 102 patients in the training cohort reporting LE radiating pain after FELD, while 133 patients did not. In the validation cohort, 57 patients reported LE radiating pain after FELD, while 83 patients did not. The model was established by multivariate logistic regression analysis. The risk factors included a higher Michigan State University classification of herniated discs, increased disease course, increased time of surgery, reduced lateral recess width, and an interlaminar surgical approach, compared to transforaminal approach. The C-indices and the area under the receiver operating characteristic curve of the predictive model demonstrated good discrimination. Good predictive performance and accuracy were also observed in the validation cohort.

CONCLUSIONS

A novel nomogram for predicting recurrent LE radiating pain within 1 week after FELD was established and validated. More aggressive pain management strategies should be considered for patients at high risk of LE radiating pain after surgery, as predicted by this model.

Low dose tacrolimus exposure and early steroid withdrawal with strict body weight control can improve post kidney transplant glucose tolerance in Japanese patients

The development of diabetes mellitus (DM) after living donor kidney transplantation (KT) is a risk factor for worsening transplant kidney function, cardiac disease, and cerebrovascular disease, which may affect prognosis after KT. At our institution, all patients' glucose tolerance is evaluated perioperatively by oral glucose tolerance tests (OGTTs) at pre-KT, and 3, 6, and 12 month (mo.) after KT. We analyzed the insulinogenic index (ISI) and homeostasis model assessment beta cell (HOMA-β) based on the immunoreactive insulin (IRI) levels to determine how glucose tolerance changed after KT in 214 patients who had not been diagnosed with DM before KT. In addition, we analyzed the body mass index (BMI) which may also influence glucose tolerance after KT. The concentration of tacrolimus (TAC) in blood was also measured as the area under the curve (AUC) to examine its effects at each sampling point. The preoperative-OGTTs showed that DM was newly diagnosed in 22 of 214 patients (10.3%) who had not been given a diagnosis of DM by the pre-KT fasting blood sugar (FBS) tests. The glucose tolerance was improved in 15 of 22 DM patients at 12 mo. after KT. ISI and IRI deteriorated only at 3 mo. after KT but improved over time. There was a trend of an inverse correlation between HOMA-β and TAC-AUC. We also found inverse correlations between IRI and an increase in BMI from 3 to 12 mo. after KT. Early corticosteroid withdrawal or the steroid minimization protocol with tacrolimus to maintain a low level of diabetogenic tacrolimus and BMI decrease after KT used by our hospital individualizes lifestyle interventions for each patient might contribute to an improvement in post-KT glucose tolerance.

Temporal validation of the MMCD score to predict kidney replacement therapy and in-hospital mortality in COVID-19 patients

BACKGROUND

Acute kidney injury has been described as a common complication in patients hospitalized with COVID-19, which may lead to the need for kidney replacement therapy (KRT) in its most severe forms. Our group developed and validated the MMCD score in Brazilian COVID-19 patients to predict KRT, which showed excellent performance using data from 2020. This study aimed to validate the MMCD score in a large cohort of patients hospitalized with COVID-19 in a different pandemic phase and assess its performance to predict in-hospital mortality.

METHODS

This study is part of the "Brazilian COVID-19 Registry", a retrospective observational cohort of consecutive patients hospitalized for laboratory-confirmed COVID-19 in 25 Brazilian hospitals between March 2021 and August 2022. The primary outcome was KRT during hospitalization and the secondary was in-hospital mortality. We also searched literature for other prediction models for KRT, to assess the results in our database. Performance was assessed using area under the receiving operator characteristic curve (AUROC) and the Brier score.

RESULTS

A total of 9422 patients were included, 53.8% were men, with a median age of 59 (IQR 48-70) years old. The incidence of KRT was 8.8% and in-hospital mortality was 18.1%. The MMCD score had excellent discrimination and overall performance to predict KRT (AUROC: 0.916 [95% CI 0.909-0.924]; Brier score = 0.057). Despite the excellent discrimination and overall performance (AUROC: 0.922 [95% CI 0.914-0.929]; Brier score = 0.100), the calibration was not satisfactory concerning in-hospital mortality. A random forest model was applied in the database, with inferior performance to predict KRT requirement (AUROC: 0.71 [95% CI 0.69-0.73]).

CONCLUSION

The MMCD score is not appropriate for in-hospital mortality but demonstrates an excellent predictive ability to predict KRT in COVID-19 patients. The instrument is low cost, objective, fast and accurate, and can contribute to supporting clinical decisions in the efficient allocation of assistance resources in patients with COVID-19.

Clinical Grading Scales and Neuroprognostication in Acute Brain Injury

Prediction of neurological clinical outcome after acute brain injury is critical because it helps guide discussions with patients and families and informs treatment plans and allocation of resources. Numerous clinical grading scales have been published that aim to support prognostication after acute brain injury. However, the development and validation of clinical scales lack a standardized approach. This in turn makes it difficult for clinicians to rely on prognostic grading scales and to integrate them into clinical practice. In this review, we discuss quality measures of score development and validation and summarize available scales to prognosticate outcomes after acute brain injury. These include scales developed for patients with coma, cardiac arrest, ischemic stroke, nontraumatic intracerebral hemorrhage, subarachnoid hemorrhage, and traumatic brain injury; for each scale, we discuss available validation studies.

A convolutional neural network-based, quantitative complete blood count scattergram-mapping framework promptly screens acute promyelocytic leukemia with high sensitivity

BACKGROUND

Acute promyelocytic leukemia (APL) is a subtype of acute myeloid leukemia (AML) characterized by its rapidly progressive and fatal clinical course if untreated, although it is curable if treated in a timely manner. Promptly screening patients who have results that are suspicious for APL is vital to overcome early death.

METHODS

The authors developed an innovative framework consisting of ResNet-18, a convolutional neural network architecture, with the objective of quantitatively mapping a complete blood count (CBC) scattergram to quickly and robustly indicate a probable susceptibility to APL. Three hundred and twenty scattergrams of the white blood cell differential channel from 51 patients with APL, 510 scattergrams from 105 patients who had non-APL AML, and 320 scattergrams from 320 healthy controls were randomly stratified at a ratio of 4:1 and split into training and testing data sets to accomplish five-fold cross-validation.

RESULTS

Both the area under the curve and the average precision of >0.99 were achieved in each fold. Three hundred four of the 320 APL scattergrams (95%) were correctly flagged by the model, which outcompeted the CBC review rules recommended by the International Society of Laboratory Hematology (all p < .001). External validation based on an independent testing data set that included 56 scattergrams from 31 patients with APL, 56 scattergrams from 55 patients with non-APL AML, and 64 scattergrams from 64 healthy controls also confirmed the sensitivity and specificity of the framework.

CONCLUSIONS

To the authors' knowledge, their convolutional neural network-based framework is the first to use scattergram output from routine CBC analysis to map suspicious APL early with outstanding sensitivity, specificity, and precision. The authors also describe a new CBC workflow incorporating this framework upstream of the morphologic review, which would provide the earliest flag for APL.

PLAIN LANGUAGE SUMMARY

The authors propose an innovative way to visualize complete blood counts (CBCs) by mapping the difference in white blood cell counts using automated CBC analysis to identify potential acute promyelocytic leukemia (APL) using a convolutional neural network (CNN), which can eliminate the potential pitfalls of manual observation. Analyses of an unprecedented, realistic data set validated that the quantitative relationship between the CBC scattergram and an APL abnormality is highly consistent. This is the first study to date focusing on screening for APL using scattergrams of the difference in white blood cell counts from routine CBC tests and has significant clinical relevance. The authors recommend using this method even before analyzing cell images, which could provide the earliest way to screen for APL in a sensitive and accurate way.

A methodology to assess the population size and estimate the needed resources for new licensed medications by combining clinical and administrative databases: The example of glycated haemoglobin in type 2 diabetes

PURPOSE

To develop and validate a model to estimate glycated haemoglobin (HbA1c) values in patients with type 2 diabetes mellitus (T2DM) using a clinical data source, with the aim to apply this equation to administrative databases.

METHODS

Using a primary care and administrative Italian databases, namely the Health Search database (HSD) and the ReS (Ricerca e Salute) database, we selected all patients aged 18 years or older on 31 December 2018 being diagnosed with T2DM and without prior prescription of sodium-glucose cotransporter-2 (SGLT-2) inhibitors. We included patients prescribed with and adherent to metformin. HSD was used to develop and test (using 2019 data as well) the algorithm imputing HbA1c values ≥7% according to a series of covariates. The algorithm was gathered by combining beta-coefficients being estimated by logistic regression models using complete case (excluding missing values) and imputed (after multiple imputation) dataset. The final algorithm was applied to ReS database using the same covariates.

RESULTS

The tested algorithms were able to explain 17%-18% variation in assessing HbA1c values. Good discrimination (70%) and calibration were obtained as well. The best algorithm (three) cut-offs, namely those providing correct classifications ranging 66%-70% was therefore calculated and applied to ReS database. By doing so, from 52 999 (27.9, 95% CI: 27.7%-28.1%) to 74 250 (40.1%, 95% CI: 38.9%-39.3%) patients were estimated with HbA1c ≥7%.

CONCLUSION

Through this methodology, healthcare authorities should be able to quantify the population eligible to a new licensed medication, such as SGLT-2 inhibitors, and to simulate scenarios to assess reimbursement criteria according to precise estimates.

Machine learning and EEG can classify passive viewing of discrete categories of visual stimuli but not the observation of pain

Previous studies have demonstrated the potential of machine learning (ML) in classifying physical pain from non-pain states using electroencephalographic (EEG) data. However, the application of ML to EEG data to categorise the observation of pain versus non-pain images of human facial expressions or scenes depicting pain being inflicted has not been explored. The present study aimed to address this by training Random Forest (RF) models on cortical event-related potentials (ERPs) recorded while participants passively viewed faces displaying either pain or neutral expressions, as well as action scenes depicting pain or matched non-pain (neutral) scenarios. Ninety-one participants were recruited across three samples, which included a model development group (n = 40) and a cross-subject validation group (n = 51). Additionally, 25 participants from the model development group completed a second experimental session, providing a within-subject temporal validation sample. The analysis of ERPs revealed an enhanced N170 component in response to faces compared to action scenes. Moreover, an increased late positive potential (LPP) was observed during the viewing of pain scenes compared to neutral scenes. Additionally, an enhanced P3 response was found when participants viewed faces displaying pain expressions compared to neutral expressions. Subsequently, three RF models were developed to classify images into faces and scenes, neutral and pain scenes, and neutral and pain expressions. The RF model achieved classification accuracies of 75%, 64%, and 69% for cross-validation, cross-subject, and within-subject classifications, respectively, along with reasonably calibrated predictions for the classification of face versus scene images. However, the RF model was unable to classify pain versus neutral stimuli above chance levels when presented with subsequent tasks involving images from either category. These results expand upon previous findings by externally validating the use of ML in classifying ERPs related to different categories of visual images, namely faces and scenes. The results also indicate the limitations of ML in distinguishing pain and non-pain connotations using ERP responses to the passive viewing of visually similar images.