External validation of the prediction model of intradialytic hypotension: a multicenter prospective cohort study

OBJECTIVE

Intradialytic hypotension (IDH) is a common and serious complication in patients with Maintenance Hemodialysis (MHD). The purpose of this study is to externally verify three IDH risk prediction models recently developed by Ma et al. and recalibrate, update and present the optimal model to improve the accuracy and applicability of the model in clinical environment.

METHODS

A multicenter prospective cohort study of patients from 11 hemodialysis centers in Sichuan Province, China, was conducted using convenience sampling from March 2022 to July 2022, with a follow-up period of 1 month. Model performance was assessed by: (1) Discrimination: Evaluated through the computation of the Area Under Curve (AUC) and its corresponding 95% confidence intervals. (2) Calibration: scrutinized through visual inspection of the calibration plot and utilization of the Brier score. (3) The incremental value of risk prediction and the utility of updating the model were gauged using NRI (Net Reclassification Improvement) and IDI (Integrated Discrimination Improvement). Decision Curve Analysis (DCA) was employed to evaluate the clinical benefit of updating the model.

RESULTS

The final cohort comprised 2235 individuals undergoing maintenance hemodialysis, exhibiting a 14.6% occurrence rate of IDH. The externally validated Area Under the Curve (AUC) values for the three original prediction models were 0.746 (95% CI: 0.718 to 0.775), 0.709 (95% CI: 0.679 to 0.739), and 0.735 (95% CI: 0.706 to 0.764) respectively. Conversely, the AUC value for the recalibrated and updated columnar plot model reached 0.817 (95% CI: 0.791 to 0.842), accompanied by a Brier score of 0.081. Furthermore, Decision Curve Analysis (DCA) exhibited a net benefit within the threshold probability range of 15.2% to 87.1%.

CONCLUSION

Externally validated, recalibrated, updated, and presented IDH prediction models may serve as a valuable instrument for evaluating IDH risk in clinical practice. Furthermore, they hold the potential to guide clinical providers in discerning individuals at risk and facilitating judicious clinical intervention decisions.

Prognosis Impact and Prediction of Trans-Radial Access Failure in Patients With STEMI, A Nationwide Observational Study

Trans-radial access (TRA) is the primary arterial approach for percutaneous coronary intervention (PCI) in ST-elevation myocardial infarction (STEMI). However, occasionally, a crossover to trans-femoral access is necessary because of unsuccessful TRA. The impact of failed TRA on the prognosis in STEMI patients and the utility of predictive models for TRA failure remains uncertain. Data from the Hungarian Myocardial Infarction Registry (January 2014 to December 2020) were analyzed. Primary endpoints were 1-year mortality and major adverse cardiovascular events. Propensity score matching was employed to create a balanced cohort for comparing successful and failed TRA. The impact of unsuccessful TRA on prognosis was evaluated using Cox regression analysis. Machine learning techniques were applied to predict TRA failure. The performance and the clinical applicability of the novel and previous prediction models were comprehensively evaluated. Of 76,625 registered patients, 34,293 (69.8 ± 13.4 years, male/female: 21,893/12,400) underwent TRA (33,573) or failed TRA (720) PCI for STEMI. After propensity score matching, in the unsuccessful TRA group, the risk of mortality (34.3% vs 22.5%, hazard ratio 1.6, 95% confidence interval 1.3 to 2.0, p <0.001) and major adverse cardiovascular events (37.4% vs 26.8%, hazard ratio 1.5, 95% confidence interval 1.3 to 1.8, p <0.001) were significantly higher. Door-to-balloon time did not differ significantly (p = 0.835). In predictive analysis, Regularized Discriminant Analysis emerged as the most promising model, surpassing previous prediction models (area under the curve: 0.66, sensitivity: 0.32, specificity: 0.86). Nevertheless, Global Registry of Acute Coronary Events (GRACE) 2.0 score demonstrated a remarkable performance (area under the curve: 0.65, sensitivity: 0.51, specificity: 0.73). This study underscores the pivotal role of successful TRA in enhancing outcomes in STEMI cases, advocating for its prioritization. The inability to conclude interventions through this approach is linked to a poorer prognosis, even in risk-adjusted analyses. Our findings indicate that prediction models utilizing clinical parameters do not outperform the established GRACE 2.0 algorithm, questioning their utility. In conclusion, the results emphasize the significance of TRA success and the continued relevance of the GRACE score in clinical decision-making to optimize patient outcomes.

The Prediction of Brain Activity from Connectivity: Advances and Applications

The human brain is composed of multiple, discrete, functionally specialized regions that are interconnected to form large-scale distributed networks. Using advanced brain-imaging methods and machine-learning analytical approaches, recent studies have demonstrated that regional brain activity during the performance of various cognitive tasks can be accurately predicted from patterns of task-independent brain connectivity. In this review article, we first present evidence for the predictability of brain activity from structural connectivity (i.e., white matter connections) and functional connectivity (i.e., temporally synchronized task-free activations). We then discuss the implications of such predictions to clinical populations, such as patients diagnosed with psychiatric disorders or neurologic diseases, and to the study of brain-behavior associations. We conclude that connectivity may serve as an infrastructure that dictates brain activity, and we pinpoint several open questions and directions for future research.

External Validation of the International IgA Nephropathy Prediction Tool in Older Adult Patients

Purpose

The International IgA Nephropathy Prediction Tool (IIgAN-PT) can predict the risk of End-stage renal disease (ESRD) or estimated glomerular filtration rate (eGFR) decline ≥ 50% for adult IgAN patients. Considering the differential progression between older adult and adult patients, this study aims to externally validate its performance in the older adult cohort.

Patients and Methods

We analyzed 165 IgAN patients aged 60 and above from six medical centers, categorizing them by their predicted risk. The primary outcome was a ≥50% reduction in estimated glomerular filtration rate (eGFR) or kidney failure. Evaluation of both models involved concordance statistics (C-statistics), time-dependent receiver operating characteristic (ROC) curves, Kaplan-Meier survival curves, and calibration plots. Comparative reclassification was conducted using net reclassification improvement (NRI) and integrated discrimination improvement (IDI).

Results

The study included 165 Chinese patients (median age 64, 60% male), with a median follow-up of 5.1 years. Of these, 21% reached the primary outcome. Both models with or without race demonstrated good discrimination (C-statistics 0.788 and 0.790, respectively). Survival curves for risk groups were well-separated. The full model without race more accurately predicted 5-year risks, whereas the full model with race tended to overestimate risks after 3 years. No significant reclassification improvement was noted in the full model without race (NRI 0.09, 95% CI: -0.27 to 0.34; IDI 0.003, 95% CI: -0.009 to 0.019).

Conclusion

: Both models exhibited excellent discrimination among older adult IgAN patients. The full model without race demonstrated superior calibration in predicting the 5-year risk.

Contribution to Characterizing the Meat Quality of Protected Designation of Origin Serrana and Preta de Montesinho Kids Using the Near-Infrared Reflectance Methodology

The aims of this study were to describe and compare the meat quality characteristics of male and female kids from the "Serrana" and "Preta de Montesinho" breeds certified as "Cabrito Transmontano" and reinforce the performance of near-infrared reflectance (NIR) spectra in predicting these quality characteristics and discriminating among breeds. Samples of Longissimus thoracis (n = 32; sixteen per breed; eight males and eight females) were used. Breed significantly affected meat quality characteristics, with only color and fatty acid (FA) (C12:0) being influenced by sex. The meat of the "Serrana" breed proved to be more tender than that of the "Preta de Montesinho". However, the meat from the "Preta de Montesinho" breed showed higher intramuscular fat content and was lighter than that from the "Serrana" breed, which favors its quality of color and juiciness. The use of NIR with the linear support vector machine regression (SVMR) classification model demonstrated its capability to quantify meat quality characteristics such as pH, CIELab color, protein, moisture, ash, fat, texture, water-holding capacity, and lipid profile. Discriminant analysis was performed by dividing the sample spectra into calibration sets (75 percent) and prediction sets (25 percent) and applying the Kennard-Stone algorithm to the spectra. This resulted in 100% correct classifications with the training data and 96.7% accuracy with the test data. The test data showed acceptable estimation models with R2 > 0.99.

Development and Internal Validation of Nomograms for Survival of Advanced Epithelial Ovarian Cancer Based on Established Prognostic Factors and Hematologic Parameters

Objective: To assess the association between pretreatment thrombocytosis, anemia, and leukocytosis and overall survival (OS) of advanced-stage EOC. Furthermore, to develop nomograms using established prognostic factors and pretreatment hematologic parameters to predict the OS of advanced EOC patients. Methods: Advanced-stage EOC patients treated between January 1996 and January 2010 in eastern Netherlands were included. Survival outcomes were compared between patients with and without pretreatment thrombocytosis (≥450,000 platelets/µL), anemia (hemoglobin level of <7.5 mmol/L), or leukocytosis (≥11.0 × 109 leukocytes/L). Three nomograms (for ≤3-, ≥5-, and ≥10-year OS) were developed. Candidate predictors were fitted into multivariable logistic regression models. Multiple imputation was conducted. Model performance was assessed on calibration, discrimination, and Brier scores. Bootstrap validation was used to correct for model optimism. Results: A total of 773 advanced-stage (i.e., FIGO stages IIB-IV) EOC patients were included. The median [interquartile range, IQR] OS was 2.3 [1.3-4.2] and 3.0 [1.4-7.0] years for patients with and without pretreatment thrombocytosis (p < 0.01). The median OS was not notably different for patients with and without pretreatment leukocytosis (p = 0.58) or patients with and without pretreatment anemia (p = 0.07). The final nomograms comprised established predictors with either pretreatment leukocyte or platelet count. The ≥5- and ≥10-year OS models demonstrated good calibration and adequate discrimination with optimism-corrected c-indices [95%-CI] of 0.76 [0.72-0.80] and 0.78 [0.73-0.83], respectively. The ≤3-year OS model demonstrated suboptimal performance with an optimism-corrected c-index of 0.71 [0.66-0.75]. Conclusions: Pretreatment thrombocytosis is associated with poorer EOC survival. Two well-performing models predictive of ≥5-year and ≥10-year OS in advanced-stage EOC were developed and internally validated.

A multiview deep learning-based prediction pipeline augmented with confident learning can improve performance in determining knee arthroplasty candidates

PURPOSE

Preoperative prudent patient selection plays a crucial role in knee osteoarthritis management but faces challenges in appropriate referrals such as total knee arthroplasty (TKA), unicompartmental knee arthroplasty (UKA) and nonoperative intervention. Deep learning (DL) techniques can build prediction models for treatment decision-making. The aim is to develop and evaluate a knee arthroplasty prediction pipeline using three-view X-rays to determine the suitable candidates for TKA, UKA or are not arthroplasty candidates.

METHODS

A study was conducted using three-view (anterior-posterior, lateral and patellar) X-rays and surgical data of patients undergoing TKA, UKA or nonarthroplasty interventions from sites A and B. Data from site A were used to derive and validate models. Data from site B were used as external test set. A DL pipeline combining YOLOv3 and ResNet-18 with confident learning (CL) was developed. Multiview Convolutional Neural Network, EfficientNet-b4, ResNet-101 and the proposed model without CL were also trained and tested. The models were evaluated using metrics such as area under the receiver operating characteristic curve (AUC), accuracy, precision, specificity, sensitivity and F1 score.

RESULTS

The data set comprised a total of 1779 knees. Of which 1645 knees were from site A as a derivation set and an internal validation cohort. The external validation cohort consisted of 134 knees. The internal validation cohort demonstrated superior performance for the proposed model augmented with CL, achieving an AUC of 0.94 and an accuracy of 85.9%. External validation further confirmed the model's generalisation, with an AUC of 0.93 and an accuracy of 82.1%. Comparative analysis with other neural network models showed the proposed model's superiority.

CONCLUSIONS

The proposed DL pipeline, integrating YOLOv3, ResNet-18 and CL, provides accurate predictions for knee arthroplasty candidates based on three-view X-rays. This prediction model could be useful in performing decision making for the type of arthroplasty procedure in an automated fashion.

LEVEL OF EVIDENCE

Level III, diagnostic study.

Prediction of incident chronic kidney disease in community-based electronic health records: a systematic review and meta-analysis

Background

Chronic kidney disease (CKD) is a major global health problem and its early identification would allow timely intervention to reduce complications. We performed a systematic review and meta-analysis of multivariable prediction models derived and/or validated in community-based electronic health records (EHRs) for the prediction of incident CKD in the community.

Methods

Ovid Medline and Ovid Embase were searched for records from 1947 to 31 January 2024. Measures of discrimination were extracted and pooled by Bayesian meta-analysis, with heterogeneity assessed through a 95% prediction interval (PI). Risk of bias was assessed using Prediction model Risk Of Bias ASsessment Tool (PROBAST) and certainty in effect estimates by Grading of Recommendations, Assessment, Development and Evaluation (GRADE).

Results

Seven studies met inclusion criteria, describing 12 prediction models, with two eligible for meta-analysis including 2 173 202 patients. The Chronic Kidney Disease Prognosis Consortium (CKD-PC) (summary c-statistic 0.847; 95% CI 0.827-0.867; 95% PI 0.780-0.905) and SCreening for Occult REnal Disease (SCORED) (summary c-statistic 0.811; 95% CI 0.691-0.926; 95% PI 0.514-0.992) models had good model discrimination performance. Risk of bias was high in 64% of models, and driven by the analysis domain. No model met eligibility for meta-analysis if studies at high risk of bias were excluded, and certainty of effect estimates was 'low'. No clinical utility analyses or clinical impact studies were found for any of the models.

Conclusions

Models derived and/or externally validated for prediction of incident CKD in community-based EHRs demonstrate good prediction performance, but assessment of clinical usefulness is limited by high risk of bias, low certainty of evidence and a lack of impact studies.

Prognostic Models for Chronic Low Back Pain Outcomes in Primary Care Are at High Risk of Bias and Lack Validation-High-Quality Studies Are Needed: A Systematic Review

OBJECTIVE: To provide an updated overview of available prognostic models for people with chronic low back pain (LBP) in primary care. DESIGN: Prognosis systematic review LITERATURE SEARCH: We searched for relevant studies on MEDLINE, Embase, Web of Science, and CINAHL databases (up to July 13, 2022), and performed citation tracking in Web of Science. STUDY SELECTION CRITERIA: We included observational (cohort or nested case-control) studies and randomized controlled trials that developed or validated prognostic models for adults with chronic LBP in primary care. The outcomes of interest were physical functioning, pain intensity, and health-related quality of life at any follow-up time-point. DATA SYNTHESIS: Data were extracted using the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS), and the Prediction model Risk of Bias Assessment Tool (PROBAST) tool was used to evaluate the risk of bias of the models. Due to the number of studies retrieved and the heterogeneity, we reported the results descriptively. RESULTS: Ten studies (out of 5593 hits screened) with 34 models met our inclusion criteria, of which six are development studies and four are external validation studies. Five studies reported the area under the curve of the models (ranging from 0.48 to 0.84), whereas no study reported calibration indices. The most promising model is the Örebro Musculoskeletal Pain Screening Questionnaire Short-Form. CONCLUSIONS: Given the high risk of bias and lack of external validation, we cannot recommend that clinicians use prognostic models for patients with chronic LBP in primary care settings. J Orthop Sports Phys Ther 2024;54(5):1-13. Epub 15 February 2024. doi:10.2519/jospt.2024.12081.

Ventilatory efficiency as a prognostic factor for postoperative complications in patients undergoing elective major surgery: a systematic review

BACKGROUND

Major surgery is associated with high complication rates. Several risk scores exist to assess individual patient risk before surgery but have limited precision. Novel prognostic factors can be included as additional building blocks in existing prediction models. A candidate prognostic factor, measured by cardiopulmonary exercise testing, is ventilatory efficiency (VE/VCO2). The aim of this systematic review was to summarise evidence regarding VE/VCO2 as a prognostic factor for postoperative complications in patients undergoing major surgery.

METHODS

A medical library specialist developed the search strategy. No database-provided limits, considering study types, languages, publication years, or any other formal criteria were applied to any of the sources. Two reviewers assessed eligibility of each record and rated risk of bias in included studies.

RESULTS

From 10,082 screened records, 65 studies were identified as eligible. We extracted adjusted associations from 32 studies and unadjusted from 33 studies. Risk of bias was a concern in the domains 'study confounding' and 'statistical analysis'. VE/VCO2 was reported as a prognostic factor for short-term complications after thoracic and abdominal surgery. VE/VCO2 was also reported as a prognostic factor for mid- to long-term mortality. Data-driven covariable selection was applied in 31 studies. Eighteen studies excluded VE/VCO2 from the final multivariable regression owing to data-driven model-building approaches.

CONCLUSIONS

This systematic review identifies VE/VCO2 as a predictor for short-term complications after thoracic and abdominal surgery. However, the available data do not allow conclusions about clinical decision-making. Future studies should select covariables for adjustment a priori based on external knowledge.

SYSTEMATIC REVIEW PROTOCOL

PROSPERO (CRD42022369944).

Ensemble Machine Learning for Predicting 90-Day Outcomes and Analyzing Risk Factors in Acute Kidney Injury Requiring Dialysis

Purpose

Our objectives were to (1) employ ensemble machine learning algorithms utilizing real-world clinical data to predict 90-day prognosis, including dialysis dependence and mortality, following the first hospitalized dialysis and (2) identify the significant factors associated with overall outcomes.

Patients and Methods

We identified hospitalized patients with Acute kidney injury requiring dialysis (AKI-D) from a dataset of the Taipei Medical University Clinical Research Database (TMUCRD) from January 2008 to December 2020. The extracted data comprise demographics, comorbidities, medications, and laboratory parameters. Ensemble machine learning models were developed utilizing real-world clinical data through the Google Cloud Platform.

Results

The Study Analyzed 1080 Patients in the Dialysis-Dependent Module, Out of Which 616 Received Regular Dialysis After 90 Days. Our Ensemble Model, Consisting of 25 Feedforward Neural Network Models, Demonstrated the Best Performance with an Auroc of 0.846. We Identified the Baseline Creatinine Value, Assessed at Least 90 Days Before the Initial Dialysis, as the Most Crucial Factor. We selected 2358 patients, 984 of whom were deceased after 90 days, for the survival module. The ensemble model, comprising 15 feedforward neural network models and 10 gradient-boosted decision tree models, achieved superior performance with an AUROC of 0.865. The pre-dialysis creatinine value, tested within 90 days prior to the initial dialysis, was identified as the most significant factor.

Conclusion

Ensemble machine learning models outperform logistic regression models in predicting outcomes of AKI-D, compared to existing literature. Our study, which includes a large sample size from three different hospitals, supports the significance of the creatinine value tested before the first hospitalized dialysis in determining overall prognosis. Healthcare providers could benefit from utilizing our validated prediction model to improve clinical decision-making and enhance patient care for the high-risk population.

A Systematic Review of Predictor Composition, Outcomes, Risk of Bias, and Validation of COVID-19 Prognostic Scores

BACKGROUND

Numerous prognostic scores have been published to support risk stratification for patients with coronavirus disease 2019 (COVID-19).

METHODS

We performed a systematic review to identify the scores for confirmed or clinically assumed COVID-19 cases. An in-depth assessment and risk of bias (ROB) analysis (Prediction model Risk Of Bias ASsessment Tool [PROBAST]) was conducted for scores fulfilling predefined criteria ([I] area under the curve [AUC)] ≥ 0.75; [II] a separate validation cohort present; [III] training data from a multicenter setting [≥2 centers]; [IV] point-scale scoring system).

RESULTS

Out of 1522 studies extracted from MEDLINE/Web of Science (20/02/2023), we identified 242 scores for COVID-19 outcome prognosis (mortality 109, severity 116, hospitalization 14, long-term sequelae 3). Most scores were developed using retrospective (75.2%) or single-center (57.1%) cohorts. Predictor analysis revealed the primary use of laboratory data and sociodemographic information in mortality and severity scores. Forty-nine scores were included in the in-depth analysis. The results indicated heterogeneous quality and predictor selection, with only five scores featuring low ROB. Among those, based on the number and heterogeneity of validation studies, only the 4C Mortality Score can be recommended for clinical application so far.

CONCLUSIONS

The application and translation of most existing COVID scores appear unreliable. Guided development and predictor selection would have improved the generalizability of the scores and may enhance pandemic preparedness in the future.

Effects of IV fluid restriction according to site-specific intensity of standard fluid treatment-protocol

BACKGROUND

Variation in usual practice in fluid trials assessing lower versus higher volumes may affect overall comparisons. To address this, we will evaluate the effects of heterogeneity in treatment intensity in the Conservative versus Liberal Approach to Fluid Therapy of Septic Shock in Intensive Care trial. This will reflect the effects of differences in site-specific intensities of standard fluid treatment due to local practice preferences while considering participant characteristics.

METHODS

We will assess the effects of heterogeneity in treatment intensity across one primary (all-cause mortality) and three secondary outcomes (serious adverse events or reactions, days alive without life support and days alive out of hospital) after 90 days. We will classify sites based on the site-specific intensity of standard fluid treatment, defined as the mean differences in observed versus predicted intravenous fluid volumes in the first 24 h in the standard-fluid group while accounting for differences in participant characteristics. Predictions will be made using a machine learning model including 22 baseline predictors using the extreme gradient boosting algorithm. Subsequently, sites will be grouped into fluid treatment intensity subgroups containing at least 100 participants each. Subgroups differences will be assessed using hierarchical Bayesian regression models with weakly informative priors. We will present the full posterior distributions of relative (risk ratios and ratios of means) and absolute differences (risk differences and mean differences) in each subgroup.

DISCUSSION

This study will provide data on the effects of heterogeneity in treatment intensity while accounting for patient characteristics in critically ill adult patients with septic shock.

REGISTRATIONS

The European Clinical Trials Database (EudraCT): 2018-000404-42, ClinicalTrials. gov: NCT03668236.

A review of disease risk prediction methods and applications in the omics era

Risk prediction and disease prevention are the innovative care challenges of the 21st century. Apart from freeing the individual from the pain of disease, it will lead to low medical costs for society. Until very recently, risk assessments have ushered in a new era with the emergence of omics technologies, including genomics, transcriptomics, epigenomics, proteomics, and so on, which potentially advance the ability of biomarkers to aid prediction models. While risk prediction has achieved great success, there are still some challenges and limitations. We reviewed the general process of omics-based disease risk model construction and the applications in four typical diseases. Meanwhile, we highlighted the problems in current studies and explored the potential opportunities and challenges for future clinical practice.

Climate change impacts on temperate fruit and nut production: a systematic review

Temperate fruit and nut crops require distinctive cold and warm seasons to meet their physiological requirements and progress through their phenological stages. Consequently, they have been traditionally cultivated in warm temperate climate regions characterized by dry-summer and wet-winter seasons. However, fruit and nut production in these areas faces new challenging conditions due to increasingly severe and erratic weather patterns caused by climate change. This review represents an effort towards identifying the current state of knowledge, key challenges, and gaps that emerge from studies of climate change effects on fruit and nut crops produced in warm temperate climates. Following the PRISMA methodology for systematic reviews, we analyzed 403 articles published between 2000 and 2023 that met the defined eligibility criteria. A 44-fold increase in the number of publications during the last two decades reflects a growing interest in research related to both a better understanding of the effects of climate anomalies on temperate fruit and nut production and the need to find strategies that allow this industry to adapt to current and future weather conditions while reducing its environmental impacts. In an extended analysis beyond the scope of the systematic review methodology, we classified the literature into six main areas of research, including responses to environmental conditions, water management, sustainable agriculture, breeding and genetics, prediction models, and production systems. Given the rapid expansion of climate change-related literature, our analysis provides valuable information for researchers, as it can help them identify aspects that are well understood, topics that remain unexplored, and urgent questions that need to be addressed in the future.

A comparison of hyperparameter tuning procedures for clinical prediction models: A simulation study

Tuning hyperparameters, such as the regularization parameter in Ridge or Lasso regression, is often aimed at improving the predictive performance of risk prediction models. In this study, various hyperparameter tuning procedures for clinical prediction models were systematically compared and evaluated in low-dimensional data. The focus was on out-of-sample predictive performance (discrimination, calibration, and overall prediction error) of risk prediction models developed using Ridge, Lasso, Elastic Net, or Random Forest. The influence of sample size, number of predictors and events fraction on performance of the hyperparameter tuning procedures was studied using extensive simulations. The results indicate important differences between tuning procedures in calibration performance, while generally showing similar discriminative performance. The one-standard-error rule for tuning applied to cross-validation (1SE CV) often resulted in severe miscalibration. Standard non-repeated and repeated cross-validation (both 5-fold and 10-fold) performed similarly well and outperformed the other tuning procedures. Bootstrap showed a slight tendency to more severe miscalibration than standard cross-validation-based tuning procedures. Differences between tuning procedures were larger for smaller sample sizes, lower events fractions and fewer predictors. These results imply that the choice of tuning procedure can have a profound influence on the predictive performance of prediction models. The results support the application of standard 5-fold or 10-fold cross-validation that minimizes out-of-sample prediction error. Despite an increased computational burden, we found no clear benefit of repeated over non-repeated cross-validation for hyperparameter tuning. We warn against the potentially detrimental effects on model calibration of the popular 1SE CV rule for tuning prediction models in low-dimensional settings.

Estimation of aortic stiffness by finger photoplethysmography using enhanced pulse wave analysis and machine learning

Introduction

Aortic stiffness plays a critical role in the evolution of cardiovascular diseases, but the assessment requires specialized equipment. Photoplethysmography (PPG) and single-lead electrocardiogram (ECG) are readily available in healthcare and wearable devices. We studied whether a brief PPG registration, alone or in combination with single-lead ECG, could be used to reliably estimate aortic stiffness.

Methods

A proof-of-concept study with simultaneous high-resolution index finger recordings of infrared PPG, single-lead ECG, and finger blood pressure (Finapres) was performed in 33 participants [median age 44 (range 21-66) years, 19 men] and repeated within 2 weeks. Carotid-femoral pulse wave velocity (cfPWV; two-site tonometry with SphygmoCor) was used as a reference. A brachial single-cuff oscillometric device assessed aortic pulse wave velocity (aoPWV; Arteriograph) for further comparisons. We extracted 136 established PPG waveform features and engineered 13 new with improved coupling to the finger blood pressure curve. Height-normalized pulse arrival time (NPAT) was derived using ECG. Machine learning methods were used to develop prediction models.

Results

The best PPG-based models predicted cfPWV and aoPWV well (root-mean-square errors of 0.70 and 0.52 m/s, respectively), with minor improvements by adding NPAT. Repeatability and agreement were on par with the reference equipment. A new PPG feature, an amplitude ratio from the early phase of the waveform, was most important in modelling, showing strong correlations with cfPWV and aoPWV (r = -0.81 and -0.75, respectively, both P < 0.001).

Conclusion

Using new features and machine learning methods, a brief finger PPG registration can estimate aortic stiffness without requiring additional information on age, anthropometry, or blood pressure. Repeatability and agreement were comparable to those obtained using non-invasive reference equipment. Provided further validation, this readily available simple method could improve cardiovascular risk evaluation, treatment, and prognosis.

Using Artificial Intelligence to Predict Mechanical Ventilation Weaning Success in Patients with Respiratory Failure, Including Those with Acute Respiratory Distress Syndrome

The management of mechanical ventilation (MV) remains a challenge in intensive care units (ICUs). The digitalization of healthcare and the implementation of artificial intelligence (AI) and machine learning (ML) has significantly influenced medical decision-making capabilities, potentially enhancing patient outcomes. Acute respiratory distress syndrome, an overwhelming inflammatory lung disease, is common in ICUs. Most patients require MV. Prolonged MV is associated with an increased length of stay, morbidity, and mortality. Shortening the MV duration has both clinical and economic benefits and emphasizes the need for better MV weaning management. AI and ML models can assist the physician in weaning patients from MV by providing predictive tools based on big data. Many ML models have been developed in recent years, dealing with this unmet need. Such models provide an important prediction regarding the success of the individual patient's MV weaning. Some AI models have shown a notable impact on clinical outcomes. However, there are challenges in integrating AI models into clinical practice due to the unfamiliar nature of AI for many physicians and the complexity of some AI models. Our review explores the evolution of weaning methods up to and including AI and ML as weaning aids.

The Lung Cancer Prediction Model "Stress Test": Assessment of Models' Performance in a High-Risk Prospective Pulmonary Nodule Cohort

BACKGROUND

Pulmonary nodules represent a growing health care burden because of delayed diagnosis of malignant lesions and overtesting for benign processes. Clinical prediction models were developed to inform physician assessment of pretest probability of nodule malignancy but have not been validated in a high-risk cohort of nodules for which biopsy was ultimately performed.

RESEARCH QUESTION

Do guideline-recommended prediction models sufficiently discriminate between benign and malignant nodules when applied to cases referred for biopsy by navigational bronchoscopy?

STUDY DESIGN AND METHODS

We assembled a prospective cohort of 322 indeterminate pulmonary nodules in 282 patients referred to a tertiary medical center for diagnostic navigational bronchoscopy between 2017 and 2019. We calculated the probability of malignancy for each nodule using the Brock model, Mayo Clinic model, and Veterans Affairs (VA) model. On a subset of 168 patients who also had PET-CT scans before biopsy, we also calculated the probability of malignancy using the Herder model. The performance of the models was evaluated by calculating the area under the receiver operating characteristic curves (AUCs) for each model.

RESULTS

The study cohort contained 185 malignant and 137 benign nodules (57% prevalence of malignancy). The malignant and benign cohorts were similar in terms of size, with a median longest diameter for benign and malignant nodules of 15 and 16 mm, respectively. The Brock model, Mayo Clinic model, and VA model showed similar performance in the entire cohort (Brock AUC, 0.70; 95% CI, 0.64-0.76; Mayo Clinic AUC, 0.70; 95% CI, 0.64-0.76; VA AUC, 0.67; 95% CI, 0.62-0.74). For 168 nodules with available PET-CT scans, the Herder model had an AUC of 0.77 (95% CI, 0.68-0.85).

INTERPRETATION

Currently available clinical models provide insufficient discrimination between benign and malignant nodules in the common clinical scenario in which a patient is being referred for biopsy, especially when PET-CT scan information is not available.

Validation of PRE-DELIRIC and E-PRE-DELIRIC in a Danish population of intensive care unit patients-A prospective observational multicenter study

BACKGROUND

Delirium is a clinical condition characterized by an acute change in brain function and is frequently observed in critically ill patients. The condition has been associated with negative outcomes, making it crucial to identify patients who are at risk. Two recent prediction models have been developed to estimate the risk of delirium in intensive care unit (ICU) patients; the prediction model for delirium (PRE-DELIRIC) and the early prediction model for delirium (E-PRE-DELIRIC). We aimed to perform an external validation of these models in a Danish cohort of critically ill patients.

METHODS

We conducted a prospective, observational multicenter study to validate the PRE-DELIRIC and E-PRE-DELIRIC models in a population of patients admitted to four general ICUs in the Zealand Region of Denmark. From January 2022 to January 2023 all adult patients acutely admitted to the participating ICUs were assessed for eligibility. Patients had to be admitted to the ICU for >24 h to be included in the study. Included patients were screened with E-PRE-DELIRIC upon ICU admission and PRE-DELIRIC after 24 h of admission and followed throughout their ICU stay with CAM-ICU delirium assessments. Our primary outcomes were the prognostic accuracy measured by Area Under the Receiver Operating Characteristics (AUROC) and the calibration plot for the E-PRE-DELIRIC and PRE-DELIRIC prediction models.

RESULTS

We included 660 patients, of whom 660 were assessed with E-PRE-DELIRIC, and 622 were assessed with PRE-DELIRIC. PRE-DELIRIC showed acceptable discrimination with AUROC of 0.70 (95% CI 0.66 to 0.74) and good calibration. E-PRE-DELIRIC had inadequate discrimination AUROC of 0.63 (95% CI 0.58 to 0.67) and poor calibration.

CONCLUSION

In a Danish cohort, we found that the PRE-DELIRIC model demonstrated acceptable performance and E-PRE-DELIRIC demonstrated poor performance. In critically ill adult patients PRE-DELIRIC may be useful in identifying patients at high risk of delirium.

Predicting risk of preterm birth in singleton pregnancies using machine learning algorithms

We aimed to develop, train, and validate machine learning models for predicting preterm birth (<37 weeks' gestation) in singleton pregnancies at different gestational intervals. Models were developed based on complete data from 22,603 singleton pregnancies from a prospective population-based cohort study that was conducted in 51 midwifery clinics and hospitals in Wenzhou City of China between 2014 and 2016. We applied Catboost, Random Forest, Stacked Model, Deep Neural Networks (DNN), and Support Vector Machine (SVM) algorithms, as well as logistic regression, to conduct feature selection and predictive modeling. Feature selection was implemented based on permutation-based feature importance lists derived from the machine learning models including all features, using a balanced training data set. To develop prediction models, the top 10%, 25%, and 50% most important predictive features were selected. Prediction models were developed with the training data set with 5-fold cross-validation for internal validation. Model performance was assessed using area under the receiver operating curve (AUC) values. The CatBoost-based prediction model after 26 weeks' gestation performed best with an AUC value of 0.70 (0.67, 0.73), accuracy of 0.81, sensitivity of 0.47, and specificity of 0.83. Number of antenatal care visits before 24 weeks' gestation, aspartate aminotransferase level at registration, symphysis fundal height, maternal weight, abdominal circumference, and blood pressure emerged as strong predictors after 26 completed weeks. The application of machine learning on pregnancy surveillance data is a promising approach to predict preterm birth and we identified several modifiable antenatal predictors.

Prediction Models for Dysphagia in Intensive Care Unit after Mechanical Ventilation: A Systematic Review and Meta-analysis

OBJECTIVE

Dysphagia is a common condition that can independently lead to death in patients in the intensive care unit (ICU), particularly those who require mechanical ventilation. Despite extensive research on the predictors of dysphagia development, consistency across these studies is lacking. Therefore, this study aimed to identify predictors and summarize existing prediction models for dysphagia in ICU patients undergoing invasive mechanical ventilation.

METHODS

We searched five databases: PubMed, EMBASE, Web of Science, Cochrane Library, and the China National Knowledge Infrastructure. Studies that developed a post-extubation dysphagia risk prediction model in ICU were included. A meta-analysis of individual predictor variables was performed with mixed-effects models. The risk of bias was assessed using the prediction model risk of bias assessment tool (PROBAST).

RESULTS

After screening 1,923 references, we ultimately included nine studies in our analysis. The most commonly identified risk predictors included in the final risk prediction model were the length of indwelling endotracheal tube ≥72 h, Acute Physiology and Chronic Health Evaluation (APACHE) II score ≥15, age ≥65 years, and duration of gastric tube ≥72 h. However, PROBAST analysis revealed a high risk of bias in the performance of these prediction models, mainly because of the lack of external validation, inadequate pre-screening of variables, and improper treatment of continuous and categorical predictors.

CONCLUSIONS

These models are particularly susceptible to bias because of numerous limitations in their development and inadequate external validation. Future research should focus on externally validating the existing model in ICU patients with varying characteristics. Moreover, assessing the acceptance and effectiveness of the model in clinical practice is needed.

LEVEL OF EVIDENCE

NA Laryngoscope, 134:517-525, 2024.

A machine learning model to predict the risk of depression in US adults with obstructive sleep apnea hypopnea syndrome: a cross-sectional study

Objective

Depression is very common and harmful in patients with obstructive sleep apnea hypopnea syndrome (OSAHS). It is necessary to screen OSAHS patients for depression early. However, there are no validated tools to assess the likelihood of depression in patients with OSAHS. This study used data from the National Health and Nutrition Examination Survey (NHANES) database and machine learning (ML) methods to construct a risk prediction model for depression, aiming to predict the probability of depression in the OSAHS population. Relevant features were analyzed and a nomogram was drawn to visually predict and easily estimate the risk of depression according to the best performing model.

Study design

This is a cross-sectional study.

Methods

Data from three cycles (2005-2006, 2007-2008, and 2015-2016) were selected from the NHANES database, and 16 influencing factors were screened and included. Three prediction models were established by the logistic regression algorithm, least absolute shrinkage and selection operator (LASSO) algorithm, and random forest algorithm, respectively. The receiver operating characteristic (ROC) area under the curve (AUC), specificity, sensitivity, and decision curve analysis (DCA) were used to assess evaluate and compare the different ML models.

Results

The logistic regression model had lower sensitivity than the lasso model, while the specificity and AUC area were higher than the random forest and lasso models. Moreover, when the threshold probability range was 0.19-0.25 and 0.45-0.82, the net benefit of the logistic regression model was the largest. The logistic regression model clarified the factors contributing to depression, including gender, general health condition, body mass index (BMI), smoking, OSAHS severity, age, education level, ratio of family income to poverty (PIR), and asthma.

Conclusion

This study developed three machine learning (ML) models (logistic regression model, lasso model, and random forest model) using the NHANES database to predict depression and identify influencing factors among OSAHS patients. Among them, the logistic regression model was superior to the lasso and random forest models in overall prediction performance. By drawing the nomogram and applying it to the sleep testing center or sleep clinic, sleep technicians and medical staff can quickly and easily identify whether OSAHS patients have depression to carry out the necessary referral and psychological treatment.

Evaluation of gut microbiota predictive potential associated with phenotypic characteristics to identify multifactorial diseases

Gut microbiota has been implicated in various clinical conditions, yet the substantial heterogeneity in gut microbiota research results necessitates a more sophisticated approach than merely identifying statistically different microbial taxa between healthy and unhealthy individuals. Our study seeks to not only select microbial taxa but also explore their synergy with phenotypic host variables to develop novel predictive models for specific clinical conditions.

DESIGN

We assessed 50 healthy and 152 unhealthy individuals for phenotypic variables (PV) and gut microbiota (GM) composition by 16S rRNA gene sequencing. The entire modeling process was conducted in the R environment using the Random Forest algorithm. Model performance was assessed through ROC curve construction.

RESULTS

We evaluated 52 bacterial taxa and pre-selected PV (p < 0.05) for their contribution to the final models. Across all diseases, the models achieved their best performance when GM and PV data were integrated. Notably, the integrated predictive models demonstrated exceptional performance for rheumatoid arthritis (AUC = 88.03%), type 2 diabetes (AUC = 96.96%), systemic lupus erythematosus (AUC = 98.4%), and type 1 diabetes (AUC = 86.19%).

CONCLUSION

Our findings underscore that the selection of bacterial taxa based solely on differences in relative abundance between groups is insufficient to serve as clinical markers. Machine learning techniques are essential for mitigating the considerable variability observed within gut microbiota. In our study, the use of microbial taxa alone exhibited limited predictive power for health outcomes, while the integration of phenotypic variables into predictive models substantially enhanced their predictive capabilities.