Prognostic models for outcome prediction in patients with chronic obstructive pulmonary disease: systematic review and critical appraisal

Predictive models of Alzheimer's disease dementia risk in older adults with mild cognitive impairment: a systematic review and critical appraisal

BACKGROUND

Mild cognitive impairment has received widespread attention as a high-risk population for Alzheimer's disease, and many studies have developed or validated predictive models to assess it. However, the performance of the model development remains unknown.

OBJECTIVE

The objective of this review was to provide an overview of prediction models for the risk of Alzheimer's disease dementia in older adults with mild cognitive impairment.

METHOD

PubMed, EMBASE, Web of Science, and MEDLINE were systematically searched up to October 19, 2023. We included cohort studies in which risk prediction models for Alzheimer's disease dementia in older adults with mild cognitive impairment were developed or validated. The Predictive Model Risk of Bias Assessment Tool (PROBAST) was employed to assess model bias and applicability. Random-effects models combined model AUCs and calculated (approximate) 95% prediction intervals for estimations. Heterogeneity across studies was evaluated using the I2 statistic, and subgroup analyses were conducted to investigate sources of heterogeneity. Additionally, funnel plot analysis was utilized to identify publication bias.

RESULTS

The analysis included 16 studies involving 9290 participants. Frequency analysis of predictors showed that 14 appeared at least twice and more, with age, functional activities questionnaire, and Mini-mental State Examination scores of cognitive functioning being the most common predictors. From the studies, only two models were externally validated. Eleven studies ultimately used machine learning, and four used traditional modelling methods. However, we found that in many of the studies, there were problems with insufficient sample sizes, missing important methodological information, lack of model presentation, and all of the models were rated as having a high or unclear risk of bias. The average AUC of the 15 best-developed predictive models was 0.87 (95% CI: 0.83, 0.90).

DISCUSSION

Most published predictive modelling studies are deficient in rigour, resulting in a high risk of bias. Upcoming research should concentrate on enhancing methodological rigour and conducting external validation of models predicting Alzheimer's disease dementia. We also emphasize the importance of following the scientific method and transparent reporting to improve the accuracy, generalizability and reproducibility of study results.

REGISTRATION

This systematic review was registered in PROSPERO (Registration ID: CRD42023468780).

Smoking, respiratory symptoms, lung function and life expectancy: A longitudinal study of ageing

BACKGROUND AND OBJECTIVE

Prognostic indices have been developed to predict various outcomes, including mortality. These indices and hazard ratios may be difficult for patients to understand. We investigated the association between smoking, respiratory symptoms and lung function with remaining life expectancy (LE) in older adults.

METHODS

Data were from the 2004/05 English Longitudinal Study of Ageing (ELSA) (n = 8930), participants aged ≥50-years, with mortality data until 2012. Respiratory symptoms included were chronic phlegm and shortness of breath (SOB). The association between smoking, respiratory symptoms and FEV1/FVC, and remaining LE was estimated using a parametric survival function and adjusted for covariates including age at baseline and sex.

RESULTS

The extent to which symptoms and FEV1/FVC predicted differences in remaining LE varied by smoking. Compared to asymptomatic never smokers with normal lung function (the reference group), in never smokers, only those with SOB had a significant reduction in remaining LE. In former and current smokers, those with respiratory symptoms had significantly lower remaining LE compared to the reference group if they had FEV1/FVC <0.70 compared to those with FEV1/FVC ≥0.70. Males aged 50-years, current smokers with SOB and FEV1/FVC <0.70, had a remaining LE of 19.2 (95%CI: 16.5-22.2) years, a decrease of 8.1 (5.3-10.8) years, compared to the reference group.

CONCLUSION

Smoking, respiratory symptoms and FEV1/FVC are strongly associated with remaining LE in older people. The use of remaining LE to communicate mortality risk to patients needs further investigation.

PD-1+ T lymphocyte proportions and hospitalized exacerbation of COPD: a prospective cohort study

OBJECTIVE

To evaluate the predictive value of PD-1 expression in T lymphocytes for rehospitalization due to acute exacerbations of COPD (AECOPD) in discharged patients.

METHODS

115 participants hospitalized with COPD (average age 71.8 ± 6.0 years) were recruited at Fujian Provincial Hospital. PD1+T lymphocytes proportions (PD1+T%), baseline demographics and clinical data were recorded at hospital discharge. AECOPD re-admission were collected at 1-year follow-up. Kaplan-Meier analysis compared the time to AECOPD readmissions among groups stratified by PD1+T%. Multivariable Cox proportional hazards regression and stratified analysis determined the correlation between PD1+T%, potential confounders, and AECOPD re-admission. ROC and DCA evaluated PD1+T% in enhancing the clinical predictive values of Cox models, BODE and CODEX.

RESULTS

68 participants (59.1%) were AECOPD readmitted, those with AECOPD readmission exhibited significantly elevated baseline PD-1+CD4+T/CD4+T% and PD-1+CD8 + T/CD8 + T% compared to non-readmitted counterparts. PD1+ T lymphocyte levels statistically correlated with BODE and CODEX indices. Kaplan-Meier analysis demonstrated that those in Higher PD1+ T lymphocyte proportions had reduced time to AECOPD readmission (logRank p < 0.05). Cox analysis identified high PD1+CD4+T and PD1+CD8+T ratios as risk factors of AECOPD readmission, with hazard ratios of 1.384(95%CI [1.043-1.725]) and 1.401(95%CI [1.013-1.789]), respectively. Notably, in patients aged < 70 years and with fewer than twice AECOPD episodes in the previous year, high PD1+T lymphocyte counts significantly increased risk for AECOPD readmission(p < 0.05). The AECOPD readmission predictive model, incorporating PD1+T% exhibited superior discrimination to the Cox model, BODE index and CODEX index, AUC of ROC were 0.763(95%CI [0.633-0.893]) and 0.734(95%CI [0.570-0.899]) (DeLong's test p < 0.05).The DCA illustrates that integrating PD1+T% into models significantly enhances the utility in aiding clinical decision-making.

CONCLUSION

Evaluation of PD1+ lymphocyte proportions offer a novel perspective for identifying high-risk COPD patients, potentially providing insights for COPD management.

TRIAL REGISTRATION

Chinese Clinical Trial Registry (ChiCTR, URL: www.chictr.org.cn/ ), Registration number: ChiCTR2200055611 Date of Registration: 2022-01-14.

Individualized prediction models in ADHD: a systematic review and meta-regression

There have been increasing efforts to develop prediction models supporting personalised detection, prediction, or treatment of ADHD. We overviewed the current status of prediction science in ADHD by: (1) systematically reviewing and appraising available prediction models; (2) quantitatively assessing factors impacting the performance of published models. We did a PRISMA/CHARMS/TRIPOD-compliant systematic review (PROSPERO: CRD42023387502), searching, until 20/12/2023, studies reporting internally and/or externally validated diagnostic/prognostic/treatment-response prediction models in ADHD. Using meta-regressions, we explored the impact of factors affecting the area under the curve (AUC) of the models. We assessed the study risk of bias with the Prediction Model Risk of Bias Assessment Tool (PROBAST). From 7764 identified records, 100 prediction models were included (88% diagnostic, 5% prognostic, and 7% treatment-response). Of these, 96% and 7% were internally and externally validated, respectively. None was implemented in clinical practice. Only 8% of the models were deemed at low risk of bias; 67% were considered at high risk of bias. Clinical, neuroimaging, and cognitive predictors were used in 35%, 31%, and 27% of the studies, respectively. The performance of ADHD prediction models was increased in those models including, compared to those models not including, clinical predictors (β = 6.54, p = 0.007). Type of validation, age range, type of model, number of predictors, study quality, and other type of predictors did not alter the AUC. Several prediction models have been developed to support the diagnosis of ADHD. However, efforts to predict outcomes or treatment response have been limited, and none of the available models is ready for implementation into clinical practice. The use of clinical predictors, which may be combined with other type of predictors, seems to improve the performance of the models. A new generation of research should address these gaps by conducting high quality, replicable, and externally validated models, followed by implementation research.

Developing and validating prediction models for severe exacerbations and readmissions in patients hospitalised for COPD exacerbation (SERCO) in China: a prospective observational study

BACKGROUND

There is a lack of individualised prediction models for patients hospitalised with chronic obstructive pulmonary disease (COPD) for clinical practice. We developed and validated prediction models of severe exacerbations and readmissions in patients hospitalised for COPD exacerbation (SERCO).

METHODS

Data were obtained from the Acute Exacerbations of Chronic Obstructive Pulmonary Disease Inpatient Registry study (NCT02657525) in China. Cause-specific hazard models were used to estimate coefficients. C-statistic was used to evaluate the discrimination. Slope and intercept were used to evaluate the calibration and used for model adjustment. Models were validated internally by 10-fold cross-validation and externally using data from different regions. Risk-stratified scoring scales and nomograms were provided. The discrimination ability of the SERCO model was compared with the exacerbation history in the previous year.

RESULTS

Two sets with 2196 and 1869 patients from different geographical regions were used for model development and external validation. The 12-month severe exacerbations cumulative incidence rates were 11.55% (95% CI 10.06% to 13.16%) in development cohorts and 12.30% (95% CI 10.67% to 14.05%) in validation cohorts. The COPD-specific readmission incidence rates were 11.31% (95% CI 9.83% to 12.91%) and 12.26% (95% CI 10.63% to 14.02%), respectively. Demographic characteristics, medical history, comorbidities, drug usage, Global Initiative for Chronic Obstructive Lung Disease stage and interactions were included as predictors. C-indexes for severe exacerbations were 77.3 (95% CI 70.7 to 83.9), 76.5 (95% CI 72.6 to 80.4) and 74.7 (95% CI 71.2 to 78.2) at 1, 6 and 12 months. The corresponding values for readmissions were 77.1 (95% CI 70.1 to 84.0), 76.3 (95% CI 72.3 to 80.4) and 74.5 (95% CI 71.0 to 78.0). The SERCO model was consistently discriminative and accurate with C-indexes in the derivation and internal validation groups. In external validation, the C-indexes were relatively lower at 60-70 levels. The SERCO model discriminated outcomes better than prior severe exacerbation history. The slope and intercept after adjustment showed close agreement between predicted and observed risks. However, in external validation, the models may overestimate the risk in higher-risk groups. The model-driven risk groups showed significant disparities in prognosis.

CONCLUSION

The SERCO model provides individual predictions for severe exacerbation and COPD-specific readmission risk, which enables identifying high-risk patients and implementing personalised preventive intervention for patients with COPD.

The predictive accuracy of machine learning for the risk of death in HIV patients: a systematic review and meta-analysis

BACKGROUND

Early prediction of mortality in individuals with HIV (PWH) has perpetually posed a formidable challenge. With the widespread integration of machine learning into clinical practice, some researchers endeavor to formulate models predicting the mortality risk for PWH. Nevertheless, the diverse timeframes of mortality among PWH and the potential multitude of modeling variables have cast doubt on the efficacy of the current predictive model for HIV-related deaths. To address this, we undertook a systematic review and meta-analysis, aiming to comprehensively assess the utilization of machine learning in the early prediction of HIV-related deaths and furnish evidence-based support for the advancement of artificial intelligence in this domain.

METHODS

We systematically combed through the PubMed, Cochrane, Embase, and Web of Science databases on November 25, 2023. To evaluate the bias risk in the original studies included, we employed the Predictive Model Bias Risk Assessment Tool (PROBAST). During the meta-analysis, we conducted subgroup analysis based on survival and non-survival models. Additionally, we utilized meta-regression to explore the influence of death time on the predictive value of the model for HIV-related deaths.

RESULTS

After our comprehensive review, we analyzed a total of 24 pieces of literature, encompassing data from 401,389 individuals diagnosed with HIV. Within this dataset, 23 articles specifically delved into deaths during long-term follow-ups outside hospital settings. The machine learning models applied for predicting these deaths comprised survival models (COX regression) and other non-survival models. The outcomes of the meta-analysis unveiled that within the training set, the c-index for predicting deaths among people with HIV (PWH) using predictive models stands at 0.83 (95% CI: 0.75-0.91). In the validation set, the c-index is slightly lower at 0.81 (95% CI: 0.78-0.85). Notably, the meta-regression analysis demonstrated that neither follow-up time nor the occurrence of death events significantly impacted the performance of the machine learning models.

CONCLUSIONS

The study suggests that machine learning is a viable approach for developing non-time-based predictions regarding HIV deaths. Nevertheless, the limited inclusion of original studies necessitates additional multicenter studies for thorough validation.

Comparison of BAP65, DECAF, PEARL, and MEWS Scores in Predicting Respiratory Support Need in Hospitalized Exacerbation of Chronic Obstructive Lung Disease Patients

OBJECTIVE

Prognostic models aid clinical practice with decision-making on treatment and hospitalization in exacerbation of chronic obstructive lung disease (ECOPD). Although there are many studies with prognostic models, diagnostic accuracy is variable within and between models.

SUBJECTS AND METHODS

We compared the prognostic performance of the BAP65 score, DECAF score, PEARL score, and modified early warning score (MEWS) in hospitalized patients with ECOPD, to estimate ventilatory support need.

RESULTS

This cross-sectional study consisted of 139 patients. Patients in need of noninvasive or invasive mechanical ventilation support are grouped as ventilatory support groups (n = 54). Comparison between receiver operating characteristic curves revealed that the DECAF score is significantly superior to the PEARL score (p = 0.04) in discriminating patients in need of ventilatory support. DECAF score with a cutoff value of 1 presented the highest sensitivity and BAP65 score with a cutoff value of 2 presented the highest specificity in predicting ventilatory support need. Multivariable analysis revealed that gender played a significant role in COPD exacerbation outcome, and arterial pCO2 and RDW measurements were also predictors of ventilatory support need. Within severity indexes, only the DECAF score was independently associated with the outcome. One-point increase in DECAF score created a 1.43 times higher risk of ventilatory support need. All severity indexes showed a correlation with age, comorbidity index, and dyspnea. BAP65 and DECAF scores also showed a correlation with length of stay.

CONCLUSION

Objective and practical classifications are needed by clinicians to assess prognosis and initiate treatment accordingly. DECAF score is a strong candidate among severity indexes.

Prognostic Factors of Mortality in Nonchronic Obstructive Pulmonary Disease Chronic Lung Disease: A Scoping Review

Introduction: Patients with chronic lung disease (CLD) experience a heavy symptom burden at the end of life, but their uptake of palliative care is notably low. Having an understanding of a patient's prognosis would facilitate shared decision making on treatment options and care planning between patients, families, and their clinicians, and complement clinicians' assessments of patients' unmet palliative needs. While literature on prognostication in patients with chronic obstructive pulmonary disease (COPD) has been established and summarized, information for other CLDs remains less consolidated. Summarizing the mortality risk factors for non-COPD CLDs would be a novel contribution to literature. Hence, we aimed to identify and summarize the prognostic factors associated with non-COPD CLDs from the literature. Methods: We conducted a scoping review following published guidelines. We searched MEDLINE, Embase, PubMed, CINAHL, Cochrane Library, and Web of Science for studies published between 2000 and 2020 that described non-COPD CLD populations with an all-cause mortality risk period of up to three years. Only primary studies which reported associations with mortality adjusted through multivariable analysis were included. Results: Fifty-five studies were reviewed, with 53 based on interstitial lung disease (ILD) or connective tissue disease-associated ILD populations and two in bronchiectasis populations. Prognostic factors were classified into 10 domains, with pulmonary function and disease being the largest. Older age, lower forced vital capacity, and lower carbon monoxide diffusing capacity were most commonly investigated and associated with statistically significant increases in mortality risks. Conclusions: This comprehensive overview of prognostic factors for patients with non-COPD CLDs would facilitate the identification and prioritization of candidate factors to predict short-term mortality, supporting tool development for decision making and to identify high-risk patients for palliative needs assessments. Literature focused on patients with ILDs, and more studies should be conducted on other CLDs to bridge the knowledge gap.

Investigation of statistical methods used in prognostic prediction models for obstetric care: A 10 year-span cross-sectional study

INTRODUCTION

Obstetric care is a highly active area in the development and application of prognostic prediction models. The development and validation of these models often require the utilization of advanced statistical techniques. However, failure to adhere to rigorous methodological standards could greatly undermine the reliability and trustworthiness of the resultant models. Consequently, the aim of our study was to examine the current statistical practices employed in obstetric care and offer recommendations to enhance the utilization of statistical methods in the development of prognostic prediction models.

MATERIAL AND METHODS

We conducted a cross-sectional survey using a sample of studies developing or validating prognostic prediction models for obstetric care published in a 10-year span (2011-2020). A structured questionnaire was developed to investigate the statistical issues in five domains, including model derivation (predictor selection and algorithm development), model validation (internal and external), model performance, model presentation, and risk threshold setting. On the ground of survey results and existing guidelines, a list of recommendations for statistical methods in prognostic models was developed.

RESULTS

A total of 112 eligible studies were included, with 107 reporting model development and five exclusively reporting external validation. During model development, 58.9% of the studies did not include any form of validation. Of these, 46.4% used stepwise regression in a crude manner for predictor selection, while two-thirds made decisions on retaining or dropping candidate predictors solely based on p-values. Additionally, 26.2% transformed continuous predictors into categorical variables, and 80.4% did not consider nonlinear relationships between predictors and outcomes. Surprisingly, 94.4% of the studies did not examine the correlation between predictors. Moreover, 47.1% of the studies did not compare population characteristics between the development and external validation datasets, and only one-fifth evaluated both discrimination and calibration. Furthermore, 53.6% of the studies did not clearly present the model, and less than half established a risk threshold to define risk categories. In light of these findings, 10 recommendations were formulated to promote the appropriate use of statistical methods.

CONCLUSIONS

The use of statistical methods is not yet optimal. Ten recommendations were offered to assist the statistical methods of prognostic prediction models in obstetric care.

Development and validation of a nomogram model for mortality prediction in stable chronic obstructive pulmonary disease patients: A prospective observational study in the RealDTC cohort

Background

Chronic obstructive pulmonary disease (COPD) is the third leading cause of death worldwide. There is no nomogram model available for mortality prediction of stable COPD. We intended to develop and validate a nomogram model to predict mortality risk in stable COPD patients for personalised prognostic assessment.

Methods

A prospective observational study was made of COPD outpatients registered in the RealDTC study between December 2016 and December 2019. Patients were randomly assigned to the training cohort and validation cohort in a ratio of 7:3. We used Lasso regression to screen predicted variables. Further, we evaluated the prognostic performance using the area under the time-dependent receiver operating characteristic curve (AUC) and calibration curve. We used the AUC, concordance index, and decision curve analysis to evaluate the net benefits and utility of the nomogram compared with three earlier prediction models.

Results

Of 2499 patients, the median follow-up was 38 months. The characteristics of the patients between the training cohort (n = 1743) and the validation cohort (n = 756) were similar. ABEODS nomogram model, combining age, body mass index, educational level, airflow obstruction, dyspnoea, and severe exacerbation in the first year, was constructed to predict mortality in stable COPD patients. In the integrative analysis of training and validation cohorts of the nomogram model, the three-year mortality prediction achieved AUC = 0.84; 95% confidence interval (CI) = 0.81, 0.88 and AUC = 0.80; 95% CI = 0.74, 0.86, respectively. The ABEODS nomogram model preserved excellent calibration in both the training cohort and validation cohort. The time-dependent AUC, concordance index, and net benefit of the nomogram model were higher than those of BODEx, updated ADO, and DOSE, respectively.

Conclusions

We developed and validated a prognostic nomogram model that accurately predicts mortality across the COPD severity spectrum. The proposed ABEODS nomogram model performed better than earlier models, including BODEx, updated ADO, and DOSE in Chinese patients with COPD.

Registration

ChiCTR-POC-17010431.

Association of blood cadmium concentration with chronic obstructive pulmonary disease progression: a prospective cohort study

BACKGROUND

Prior studies in patients with chronic obstructive pulmonary disease (COPD) had indicated a potential correlation between cadmium (Cd) exposure and reduction in lung function. Nevertheless, the influence of Cd exposure on the progression of COPD remained unknown. Exploring the relationship between Cd exposure and the progression of COPD was the aim of this investigation.

METHODS

Stable COPD patients were enrolled. Blood samples were collected and lung function was evaluated. Regular professional follow-ups were conducted through telephone communications, outpatient services, and patients' hospitalization records.

RESULTS

Each additional unit of blood Cd was associated with upward trend in acute exacerbation, hospitalization, longer hospital stay, and death within 2 years. Even after adjusting for potential confounding factors, each 1 unit rise in blood Cd still correlated with a rise in the frequencies of acute exacerbation, longer hospital stay, and death. Moreover, COPD patients with less smoking amount, lower lung function and without comorbidities were more vulnerable to Cd-induced disease deterioration.

CONCLUSION

Patients with COPD who have higher blood Cd concentration are susceptible to worse disease progression.

Integrating Clinical and Air Quality Data to Improve Prediction of COPD Exacerbations

Several studies have found associations between air pollution and respiratory disease outcomes. However, there is minimal prognostic research exploring whether integrating air quality into clinical prediction models can improve accuracy and utility. In this study, we built models using both logistic regression and random forests to determine the benefits of including air quality data with meteorological and clinical data in prediction of COPD exacerbations requiring medical care. Logistic models were not improved by inclusion of air quality. However, the net benefit curves of random forest models showed greater clinical utility with the addition of air quality data. These models demonstrate a practical and relatively low-cost way to include environmental information into clinical prediction tools to improve the clinical utility of COPD prediction. Findings could be used to provide population level health warnings as well as individual-patient risk assessments.

Influence of physical activity on the prognosis of COPD patients: the HADO.2 score - health, activity, dyspnoea and obstruction

Objective

The aim of this study was to create a prognostic instrument for COPD with a multidimensional perspective that includes physical activity (PA). The score also included health status, dyspnoea and forced expiratory volume in 1 s (HADO.2 score).

Methods

A prospective, observational, non-intervention study was carried out. Patients were recruited from the six outpatient clinics of the respiratory service of a single university hospital. The component variables of the HADO.2 score and BODE index were studied, and PA was measured using an accelerometer. The outcomes for the HADO.2 score were mortality and hospitalisations during follow-up and an exploration of the correlation with health-related quality of life at the moment of inclusion in the study.

Results

401 patients were included in the study and followed up for three years. The HADO.2 score showed good predictive capacity for mortality: C-index 0.79 (0.72-0.85). The C-index for hospitalisations was 0.72 (0.66-0.77) and the predictive ability for quality of life, as measured by R2, was 0.63 and 0.53 respectively for the Saint George's Respiratory Questionnaire and COPD Assessment Test.

Conclusions

There was no statistically significant difference between the mortality predictive capacity of the HADO.2 score and the BODE index. Adding PA to the original BODE index significantly improved the predictive capacity of the index. The HADO.2 score, which includes PA as a key variable, showed good predictive capacity for mortality and hospitalisations. There were no differences in the predictive capacity of the HADO.2 score and the BODE index.

The value of machine learning in preoperative identification of lymph node metastasis status in endometrial cancer: a systematic review and meta-analysis

Background

The early identification of lymph node metastasis status in endometrial cancer (EC) is a serious challenge in clinical practice. Some investigators have introduced machine learning into the early identification of lymph node metastasis in EC patients. However, the predictive value of machine learning is controversial due to the diversity of models and modeling variables. To this end, we carried out this systematic review and meta-analysis to systematically discuss the value of machine learning for the early identification of lymph node metastasis in EC patients.

Methods

A systematic search was conducted in Pubmed, Cochrane, Embase, and Web of Science until March 12, 2023. PROBAST was used to assess the risk of bias in the included studies. In the process of meta-analysis, subgroup analysis was performed according to modeling variables (clinical features, radiomic features, and radiomic features combined with clinical features) and different types of models in various variables.

Results

This systematic review included 50 primary studies with a total of 103,752 EC patients, 12,579 of whom had positive lymph node metastasis. Meta-analysis showed that among the machine learning models constructed by the three categories of modeling variables, the best model was constructed by combining radiomic features with clinical features, with a pooled c-index of 0.907 (95%CI: 0.886-0.928) in the training set and 0.823 (95%CI: 0.757-0.890) in the validation set, and good sensitivity and specificity. The c-index of the machine learning model constructed based on clinical features alone was not inferior to that based on radiomic features only. In addition, logistic regression was found to be the main modeling method and has ideal predictive performance with different categories of modeling variables.

Conclusion

Although the model based on radiomic features combined with clinical features has the best predictive efficiency, there is no recognized specification for the application of radiomics at present. In addition, the logistic regression constructed by clinical features shows good sensitivity and specificity. In this context, large-sample studies covering different races are warranted to develop predictive nomograms based on clinical features, which can be widely applied in clinical practice.

Systematic review registration

https://www.crd.york.ac.uk/PROSPERO, identifier CRD42023420774.

Predictive value of machine learning on fracture risk in osteoporosis: a systematic review and meta-analysis

OBJECTIVES

Early identification of fracture risk in patients with osteoporosis is essential. Machine learning (ML) has emerged as a promising technique to predict the risk, whereas its predictive performance remains controversial. Therefore, we conducted this systematic review and meta-analysis to explore the predictive efficiency of ML for the risk of fracture in patients with osteoporosis.

METHODS

Relevant studies were retrieved from four databases (PubMed, Embase, Cochrane Library and Web of Science) until 31 May 2023. A meta-analysis of the C-index was performed using a random-effects model, while a bivariate mixed-effects model was used for the meta-analysis of sensitivity and specificity. In addition, subgroup analysis was performed according to the types of ML models and fracture sites.

RESULTS

Fifty-three studies were included in our meta-analysis, involving 15 209 268 patients, 86 prediction models specifically developed for the osteoporosis population and 41 validation sets. The most commonly used predictors in these models encompassed age, BMI, past fracture history, bone mineral density T-score, history of falls, BMD, radiomics data, weight, height, gender and other chronic diseases. Overall, the pooled C-index of ML was 0.75 (95% CI: 0.72, 0.78) and 0.75 (95% CI: 0.71, 0.78) in the training set and validation set, respectively; the pooled sensitivity was 0.79 (95% CI: 0.72, 0.84) and 0.76 (95% CI: 0.80, 0.81) in the training set and validation set, respectively; and the pooled specificity was 0.81 (95% CI: 0.75, 0.86) and 0.83 (95% CI: 0.72, 0.90) in the training set and validation set, respectively.

CONCLUSIONS

ML has a favourable predictive performance for fracture risk in patients with osteoporosis. However, most current studies lack external validation. Thus, external validation is required to verify the reliability of ML models.

PROSPERO REGISTRATION NUMBER

CRD42022346896.

Respiratory Interventions, Hospital Utilization, and Clinical Outcomes of Persons with COPD and COVID-19

Purpose

Coronavirus disease 2019 (COVID-19) impacted outcomes of persons with chronic respiratory diseases such as chronic obstructive pulmonary disease (COPD). This study investigated the differences in respiratory interventions, hospital utilization, smoking status, and 30-day readmission in those with COPD and COVID-19 based on hospital survival status.

Methods

A retrospective cross-sectional study was conducted from February 2020 to October 2020 and included persons with COPD and COVID-19 infection. We examined respiratory interventions, hospital utilization and outcomes, and 30-day hospital readmission. Chi-square test analysis was used to assess categorical variables, and t-test or Mann-Whitney was used to analyze continuous data based on normality.

Results

Ninety persons were included in the study, 78 (87%) were survivors. The most common comorbidity was hypertension 71 (78.9%) (p = 0.003). Twenty-two (24%) persons were intubated, from whom 12 (15%) survived (p < 0.001). There were 25 (32.1%) and 12 (100%), (p < 0.001) persons who required an ICU admission from the survivor and non-survivor groups, respectively. Among the survivor group, fifteen (19%) persons required 30-day hospital readmission.

Conclusion

Persons with COPD and COVID-19 had a lower mortality rate (13%) compared to other studies in the early pandemic phase. Non-survivors had increased ICU utilization, endotracheal intubation, and more frequent application of volume control mode. Discharging survivors to long-term acute care facilities may reduce 30-day hospital readmissions.

Development of Multivariable Prediction Models for the Identification of Patients Admitted to Hospital with an Exacerbation of COPD and the Prediction of Risk of Readmission: A Retrospective Cohort Study using Electronic Medical Record Data

BACKGROUND

Approximately 20% of patients who are discharged from hospital for an acute exacerbation of COPD (AECOPD) are readmitted within 30 days. To reduce this, it is important both to identify all individuals admitted with AECOPD and to predict those who are at higher risk for readmission.

OBJECTIVES

To develop two clinical prediction models using data available in electronic medical records: 1) identifying patients admitted with AECOPD and 2) predicting 30-day readmission in patients discharged after AECOPD.

METHODS

Two datasets were created using all admissions to General Internal Medicine from 2012 to 2018 at two hospitals: one cohort to identify AECOPD and a second cohort to predict 30-day readmissions. We fit and internally validated models with four algorithms.

RESULTS

Of the 64,609 admissions, 3,620 (5.6%) were diagnosed with an AECOPD. Of those discharged, 518 (15.4%) had a readmission to hospital within 30 days. For identification of patients with a diagnosis of an AECOPD, the top-performing models were LASSO and a four-variable regression model that consisted of specific medications ordered within the first 72 hours of admission. For 30-day readmission prediction, a two-variable regression model was the top performing model consisting of number of COPD admissions in the previous year and the number of non-COPD admissions in the previous year.

CONCLUSION

We generated clinical prediction models to identify AECOPDs during hospitalization and to predict 30-day readmissions after an acute exacerbation from a dataset derived from available EMR data. Further work is needed to improve and externally validate these models.

Development and Validation of Prediction Models for Exacerbation, Frequent Exacerbations and Severe Exacerbations of Chronic Obstructive Pulmonary Disease: A Registry Study in North China

In COPD patients, exacerbation has a detrimental influence on the quality of life, disease progression and socioeconomic burden. This study aimed to develop and validate models to predict exacerbation, frequent exacerbations and severe exacerbations in COPD patients. We conducted an observational prospective multicenter study. Clinical data of all outpatients with stable COPD were collected from Beijing Chaoyang Hospital and Beijing Renhe Hospital between January 2018 and December 2019. Patients were followed up for 1 year. The data from Chaoyang Hospital was used for modeling dataset, and that of Renhe Hospital was used for external validation dataset. The final dataset included 456 patients, with 326 patients as the model group and 130 patients as the validation group. Using LABA + ICS, frequent exacerbations in the past year and CAT score were independent risk factors for exacerbation in the next year (OR = 2.307, 2.722 and 1.147), and FVC %pred as a protective factor (OR = 0.975). Combined with chronic heart failure, frequent exacerbations in the past year, blood EOS counts and CAT score were independent risk factors for frequent exacerbations in the next year (OR = 4.818, 2.602, 1.015 and 1.342). Using LABA + ICS, combined with chronic heart failure, frequent exacerbations in the past year and CAT score were independent risk factors for severe exacerbations in the next year (OR = 1.950, 3.135, 2.980 and 1.133). Based on these prognostic models, nomograms were generated. The prediction models were simple and useful tools for predicting the risk of exacerbation, frequent exacerbations and severe exacerbations of COPD patients in North China.

Predictors of Readmission, for Patients with Chronic Obstructive Pulmonary Disease (COPD) - A Systematic Review

Introduction

Chronic obstructive pulmonary disease (COPD) is the third-leading cause of death globally and is responsible for over 3 million deaths annually. One of the factors contributing to the significant healthcare burden for these patients is readmission. The aim of this review is to describe significant predictors and prediction scores for all-cause and COPD-related readmission among patients with COPD.

Methods

A search was conducted in Ovid MEDLINE, Ovid Embase, Cochrane Database of Systematic Reviews, and Cochrane Central Register of Controlled Trials, from database inception to June 7, 2022. Studies were included if they reported on patients at least 40 years old with COPD, readmission data within 1 year, and predictors of readmission. Study quality was assessed. Significant predictors of readmission and the degree of significance, as noted by the p-value, were extracted for each study. This review was registered on PROSPERO (CRD42022337035).

Results

In total, 242 articles reporting on 16,471,096 patients were included. There was a low risk of bias across the literature. Of these, 153 studies were observational, reporting on predictors; 57 studies were observational studies reporting on interventions; and 32 were randomized controlled trials of interventions. Sixty-four significant predictors for all-cause readmission and 23 for COPD-related readmission were reported across the literature. Significant predictors included 1) pre-admission patient characteristics, such as male sex, prior hospitalization, poor performance status, number and type of comorbidities, and use of long-term oxygen; 2) hospitalization details, such as length of stay, use of corticosteroids, and use of ventilatory support; 3) results of investigations, including anemia, lower FEV1, and higher eosinophil count; and 4) discharge characteristics, including use of home oxygen and discharge to long-term care or a skilled nursing facility.

Conclusion

The findings from this review may enable better predictive modeling and can be used by clinicians to better inform their clinical gestalt of readmission risk.

Prediction of fetal and neonatal outcomes after preterm manifestations of placental insufficiency: systematic review of prediction models

OBJECTIVES

To identify all prediction models for fetal and neonatal outcomes in pregnancies with preterm manifestations of placental insufficiency (gestational hypertension, pre-eclampsia, HELLP syndrome or fetal growth restriction with its onset before 37 weeks' gestation) and to assess the quality of the models and their performance on external validation.

METHODS

A systematic literature search was performed in PubMed, Web of Science and EMBASE. Studies describing prediction models for fetal/neonatal mortality or significant neonatal morbidity in patients with preterm placental insufficiency disorders were included. Data extraction was performed using the CHARMS checklist. Risk of bias was assessed using PROBAST. Literature selection and data extraction were performed by two researchers independently.

RESULTS

Our literature search yielded 22 491 unique publications. Fourteen were included after full-text screening of 218 articles that remained after initial exclusions. The studies derived a total of 41 prediction models, including four models in the setting of pre-eclampsia or HELLP, two models in the setting of fetal growth restriction and/or pre-eclampsia and 35 models in the setting of fetal growth restriction. None of the models was validated externally, and internal validation was performed in only two studies. The final models contained mainly ultrasound (Doppler) markers as predictors of fetal/neonatal mortality and neonatal morbidity. Discriminative properties were reported for 27/41 models (c-statistic between 0.6 and 0.9). Only two studies presented a calibration plot. The risk of bias was assessed as unclear in one model and high for all other models, mainly owing to the use of inappropriate statistical methods.

CONCLUSIONS

We identified 41 prediction models for fetal and neonatal outcomes in pregnancies with preterm manifestations of placental insufficiency. All models were considered to be of low methodological quality, apart from one that had unclear methodological quality. Higher-quality models and external validation studies are needed to inform clinical decision-making based on prediction models. © 2023 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of International Society of Ultrasound in Obstetrics and Gynecology.

Adipocyte dysfunction promotes lung inflammation and aberrant repair: a potential target of COPD

Background

Obesity and chronic obstructive pulmonary disease (COPD) are prevailing worldwide, bringing a heavy medical burden. Clinical and pathophysiological relationship between obesity and COPD is paradoxical and elusive. We aim to explore their inherent associations from clinical, genetic, and animal levels.

Methods

We performed literature review and cohort analysis of patients with COPD to compare lung function, symptom, and prognosis among different weight groups. After retrieving datasets of obesity and COPD in Gene Expression Omnibus (GEO) database, we carried out differentially expressed gene analysis, functional enrichment, protein-protein interactions network, and weighted gene co-expression network analysis. Then, we acquired paraffin-embedded lung tissues of fatty acid-binding protein 4-Cre-BMPR2fl/fl conditional knockout (CKO) mice that were characterized by adipocyte-specific knockout of bone morphogenetic protein receptor 2 (BMPR2) for staining and analysis.

Results

Our cohort study reports the effect of obesity on COPD is inconsistent with previous clinical studies. Lung function of overweight group was statistically superior to that of other groups. We also found that the inflammatory factors were significantly increased hub genes, and cytokine-associated pathways were enriched in white adipose tissue of patients with obesity. Similarly, injury repair-associated genes and pathways were further enhanced in the small airways of patients with COPD. CKO mice spontaneously developed lung injury, emphysema, and pulmonary vascular remodeling, along with increased infiltration of macrophages. BMPR2-defiecient adipocytes had dysregulated expression of adipocytokines.

Conclusion

Inflammation and abnormal repair might be potential mechanisms of the pathological association between obesity and COPD. BMPR2-associated adipocyte dysfunction promoted lung inflammation and aberrant repair, in which adipocytokines might play a role and thus could be a promising therapeutic target.

Risk prediction models for antineoplastic-associated cardiotoxicity in treatment of breast cancer: A systematic review

PURPOSE

The objective of this systematic review is to assess methodology of published models to predict the risk of antineoplastic-associated cardiotoxicity in patients with breast cancer.

METHODS

We searched PubMed and Embase for studies that developed or validated a multivariable risk prediction model. Data extraction and quality assessments were performed according to the Prediction Model Risk of Bias Assessment Tool (PROBAST).

RESULTS

We identified 2,816 unique publications and included 8 eligible studies (7 new risk models and 1 validation of a risk stratification tool) that modeled risk with trastuzumab (n = 5), anthracyclines (n = 2), and anthracyclines with or without trastuzumab (n = 1). The most common final predictors were previous or concomitant chemotherapy (n = 5) and age (n = 4). Three studies incorporated measures of myocardial mechanics that may not be frequently available. Model discrimination was reported in 7 studies (range of area under the receiver operating characteristic curve, 0.56-0.88), while calibration was reported in 1 study. Internal and external validation were performed in 4 studies and 1 study, respectively. Using the PROBAST methodology, we rated the overall risk of bias as high for 7 of 8 studies and unclear for 1 study. Concerns for applicability were low for all studies.

CONCLUSION

Among 8 models to predict the risk of cardiotoxicity of antineoplastic agents for breast cancer, 7 were rated as having a high risk of bias and all had low concerns for clinical applicability. Most evaluated studies reported positive measures of model performance but did not perform external validation. Efforts to improve development and reporting of these models to facilitate their use in practice are warranted.

Machine learning in the prediction of post-stroke cognitive impairment: a systematic review and meta-analysis

Objective

Cognitive impairment is a detrimental complication of stroke that compromises the quality of life of the patients and poses a huge burden on society. Due to the lack of effective early prediction tools in clinical practice, many researchers have introduced machine learning (ML) into the prediction of post-stroke cognitive impairment (PSCI). However, the mathematical models for ML are diverse, and their accuracy remains highly contentious. Therefore, this study aimed to examine the efficiency of ML in the prediction of PSCI.

Methods

Relevant articles were retrieved from Cochrane, Embase, PubMed, and Web of Science from the inception of each database to 5 December 2022. Study quality was evaluated by PROBAST, and c-index, sensitivity, specificity, and overall accuracy of the prediction models were meta-analyzed.

Results

A total of 21 articles involving 7,822 stroke patients (2,876 with PSCI) were included. The main modeling variables comprised age, gender, education level, stroke history, stroke severity, lesion volume, lesion site, stroke subtype, white matter hyperintensity (WMH), and vascular risk factors. The prediction models used were prediction nomograms constructed based on logistic regression. The pooled c-index, sensitivity, and specificity were 0.82 (95% CI 0.77-0.87), 0.77 (95% CI 0.72-0.80), and 0.80 (95% CI 0.71-0.86) in the training set, and 0.82 (95% CI 0.77-0.87), 0.82 (95% CI 0.70-0.90), and 0.80 (95% CI 0.68-0.82) in the validation set, respectively.

Conclusion

ML is a potential tool for predicting PSCI and may be used to develop simple clinical scoring scales for subsequent clinical use.

Systematic Review Registration

https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=383476.

A pooled analysis of the risk prediction models for mortality in acute exacerbation of chronic obstructive pulmonary disease

OBJECTIVE

The prognosis for acute exacerbation of chronic obstructive pulmonary disease (AECOPD) is not optimistic, and severe AECOPD leads to an increased risk of mortality. Prediction models help distinguish between high- and low-risk groups. At present, many prediction models have been established and validated, which need to be systematically reviewed to screen out more suitable models that can be used in the clinic and provide evidence for future research.

METHODS

We searched PubMed, EMBASE, Cochrane Library and Web of Science databases for studies on risk models for AECOPD mortality from their inception to 10 April 2022. The risk of bias was assessed using the prediction model risk of bias assessment tool (PROBAST). Stata software (version 16) was used to synthesize the C-statistics for each model.

RESULTS

A total of 37 studies were included. The development of risk prediction models for mortality in patients with AECOPD was described in 26 articles, in which the most common predictors were age (n = 17), dyspnea grade (n = 11), altered mental status (n = 8), pneumonia (n = 6) and blood urea nitrogen (BUN, n = 6). The remaining 11 articles only externally validated existing models. All 37 studies were evaluated at a high risk of bias using PROBAST. We performed a meta-analysis of five models included in 15 studies. DECAF (dyspnoea, eosinopenia, consolidation, acidemia and atrial fibrillation) performed well in predicting in-hospital death [C-statistic = 0.91, 95% confidence interval (CI): 0.83, 0.98] and 90-day death [C-statistic = 0.76, 95% CI: 0.69, 0.82] and CURB-65 (confusion, urea, respiratory rate, blood pressure and age) performed well in predicting 30-day death [C-statistic = 0.74, 95% CI: 0.70, 0.77].

CONCLUSIONS

This study provides information on the characteristics, performance and risk of bias of a risk model for AECOPD mortality. This pooled analysis of the present study suggests that the DECAF performs well in predicting in-hospital and 90-day deaths. Yet, external validation in different populations is still needed to prove this performance.

Learning Representations from Medical Text for Effective Diagnoses and Knowledge Discovery

Discovering knowledge and effectively predicting target events are two main goals of medical text mining. However, few models can achieve them simultaneously. In this study, we investigated the possibility of discovering knowledge and predicting diagnosis at once via raw medical text. We proposed the Enhanced Neural Topic Model (ENTM), a variant of the neural topic model, to learn interpretable representations. We introduced the auxiliary loss set to improve the effectiveness of learned representations. Then, we used learned representations to train a softmax regression model to predict target events. As each element in representations learned by the ENTM has an explicit semantic meaning, weights in softmax regression represent potential knowledge of whether an element is a significant factor in predicting diagnosis. We adopted two independent medical text datasets to evaluate our ENTM model. Results indicate that our model performed better than the latest pretrained neural language models. Meanwhile, analysis of model parameters indicates that our model has the potential discover knowledge from data.Clinical relevance- This work provides a model that can effectively predict patient diagnosis and has the potential to discover knowledge from medical text.

Systematic metareview of prediction studies demonstrates stable trends in bias and low PROBAST inter-rater agreement

OBJECTIVES

To (1) explore trends of risk of bias (ROB) in prediction research over time following key methodological publications, using the Prediction model Risk Of Bias ASsessment Tool (PROBAST) and (2) assess the inter-rater agreement of the PROBAST.

STUDY DESIGN AND SETTING

PubMed and Web of Science were searched for reviews with extractable PROBAST scores on domain and signaling question (SQ) level. ROB trends were visually correlated with yearly citations of key publications. Inter-rater agreement was assessed using Cohen's Kappa.

RESULTS

One hundred and thirty nine systematic reviews were included, of which 85 reviews (containing 2,477 single studies) on domain level and 54 reviews (containing 2,458 single studies) on SQ level. High ROB was prevalent, especially in the Analysis domain, and overall trends of ROB remained relatively stable over time. The inter-rater agreement was low, both on domain (Kappa 0.04-0.26) and SQ level (Kappa -0.14 to 0.49).

CONCLUSION

Prediction model studies are at high ROB and time trends in ROB as assessed with the PROBAST remain relatively stable. These results might be explained by key publications having no influence on ROB or recency of key publications. Moreover, the trend may suffer from the low inter-rater agreement and ceiling effect of the PROBAST. The inter-rater agreement could potentially be improved by altering the PROBAST or providing training on how to apply the PROBAST.

The Lung Transplant Candidate, Indications, Timing, and Selection Criteria

Lung transplantation can be lifesaving for patients with advanced lung disease. Demographics are evolving with recipients now sicker but determining candidacy remains predicated on one's underlying lung disease prognosis, along with the likelihood of posttransplant success. Determining optimal timing can be challenging, and most programs favor initiating the process early and proactively to allow time for patient education, informed decision-making, and preparation. A comprehensive, multidisciplinary evaluation is used to elucidate disease progrnosis and identify risk factors for poor posttransplant outcomes. Candidacy criteria vary significantly by center, and close communication between referring and transplant providers is necessary to improve access to transplant and outcomes.

Development of the ESEx index: a tool for predicting risk of recurrent severe COPD exacerbations

Background

In chronic obstructive pulmonary disease (COPD), multiple recurrent severe exacerbations that require hospitalization can occur. These events are strongly associated with death and other clinical complications.

Objectives

We aimed to develop a prognostic model that could identify patients with COPD that are at risk of multiple recurrent severe exacerbations within 3 years.

Design

Prospective cohort.

Methods

The derivation cohort comprised patients with stable, moderate-to-severe COPD. Multivariable logistic regression analyses were performed to develop the final model. Based on regression coefficients, a simplified index (ESEx) was established. Both, model and index, were assessed for predictive performance by measuring discrimination and calibration.

Results

Over 3 years, 16.4% of patients with COPD experienced at least three severe recurrent exacerbations. The prognostic model showed good discrimination of high-risk patients, based on three characteristics: the number of severe exacerbations in the previous year, performance in the five-repetition sit-to-stand test, and in the 6-minute-walk test. The ESEx index provided good level of discrimination [areas under the receiver operating characteristic curve (AUCs): 0.913].

Conclusions

The ESEx index showed good internal validation for the identification of patients at risk of three recurrent severe COPD exacerbations within 3 years. These tools could be used to identify patients who require early interventions and motivate patients to improve physical performance to prevent recurrent exacerbations.

The Relationship between Exertional Desaturation and Pulmonary Function, Exercise Capacity, or Medical Costs in Chronic Obstructive Pulmonary Disease Patients

Background and Objectives: Exertional desaturation (ED) is common and is associated with poorer clinical outcomes in chronic obstructive pulmonary disease (COPD). The age, dyspnea, airflow obstruction (ADO) and body mass index, airflow obstruction, dyspnea, and exercise (BODE) indexes are used to predict the prognosis of COPD patients. This study aimed to investigate the relationship between these indexes, pulmonary function, medical costs, and ED in COPD patients. Materials and Methods: Data were collected from the electronic database of the Kaohsiung Chang Gung Memorial Hospital. This retrospective study included 396 patients categorized as either ED (n = 231) or non-ED (n = 165). Variables (including age, smoking history, body mass index (BMI), pulmonary function test, maximum inspiratory pressure (MIP) and maximum expiratory pressure (MEP), six minutes walking test distance (6MWD), SpO₂, COPD Assessment Test (CAT) score, ADO index, BODE index, Charlson comorbidity index (CCI), and medical costs) were compared between the two groups, and their correlations were assessed. ED was defined as SpO₂ less than 90% or SpO₂ decrease of more than 4% compared to baseline levels during 6MWT. Results: A significant statistical difference was found regarding a lower score of the ADO index and the BODE index (both p < 0.001), better pulmonary function (forced expiratory volume in the first second (FEV1), p < 0.001; FEV1/ forced vital capacity (FVC), p < 0.001; diffusion capacity of the lung for carbon monoxide (DLCO), p < 0.001), and higher minimal oxygen saturation (p < 0.001) in non-ED COPD patients. No difference was found in the distance of the 6MWT (p = 0.825) and respiratory muscle strength (MIP; MEP, p = 0.86; 0.751). However, the adjusted multivariate logistic regression analysis showed that only SpO₂ (minimal) had a significant difference between of the ED and non-ED group (p < 0.001). There was either no difference in the medical expenses between ED and non-ED COPD patients. Conclusions: SpO₂ (minimal) during the 6MWT is the independent factor for ED. ED is related to BODE and ADO indices, but is not related to medical expense.

Real-World Practice of Gastric Cancer Prevention and Screening Calls for Practical Prediction Models

INTRODUCTION

Some gastric cancer prediction models have been published. Still, the value of these models for application in real-world practice remains unclear. We aim to summarize and appraise modeling studies for gastric cancer risk prediction and identify potential barriers to real-world use.

METHODS

This systematic review included studies that developed or validated gastric cancer prediction models in the general population.

RESULTS

A total of 4,223 studies were screened. We included 18 development studies for diagnostic models, 10 for prognostic models, and 1 external validation study. Diagnostic models commonly included biomarkers, such as Helicobacter pylori infection indicator, pepsinogen, hormone, and microRNA. Age, sex, smoking, body mass index, and family history of gastric cancer were frequently used in prognostic models. Most of the models were not validated. Only 25% of models evaluated the calibration. All studies had a high risk of bias, but over half had acceptable applicability. Besides, most studies failed to clearly report the application scenarios of prediction models.

DISCUSSION

Most gastric cancer prediction models showed common shortcomings in methods, validation, and reports. Model developers should further minimize the risk of bias, improve models' applicability, and report targeting application scenarios to promote real-world use.

Predictive models for the incidence of Parkinson's disease: systematic review and critical appraisal

Numerous predictive models for Parkinson's disease (PD) incidence have been published recently. However, the model performance and methodological quality of those available models are yet needed to be summarized and assessed systematically. In this systematic review, we systematically reviewed the published predictive models for PD incidence and assessed their risk of bias and applicability. Three international databases were searched. Cohort or nested case-control studies that aimed to develop or validate a predictive model for PD incidence were considered eligible. The Prediction model Risk Of Bias ASsessment Tool (PROBAST) was used for risk of bias and applicability assessment. Ten studies covering 10 predictive models were included. Among them, four studies focused on model development, covering eight models, while the remaining six studies focused on model external validation, covering two models. The discrimination of the eight new development models was generally poor, with only one model reported C index > 0.70. Four out of the six external validation studies showed excellent or outstanding discrimination. All included studies had high risk of bias. Three predictive models (the International Parkinson and Movement Disorder Society [MDS] prodromal PD criteria, the model developed by Karabayir et al. and models validated by Faust et al.) are recommended for clinical application by considering model performance and resource-demanding. In conclusion, the performance and methodological quality of most of the identified predictive models for PD incidence were unsatisfactory. The MDS prodromal PD criteria, model developed by Karabayir et al. and model validated by Faust et al. may be considered for clinical use.

Prediction models for depression risk among older adults: systematic review and critical appraisal

OBJECTIVE

To provide an overview of prediction models for the risk of major depressive disorder (MDD) among older adults.

METHODS

We conducted a systematic review combined with a meta-analysis and critical appraisal of published studies on existing geriatric depression risk models.

RESULTS

The systematic search screened 23,378 titles and abstracts; 14 studies including 20 prediction models were included. A total of 16 predictors were selected in the final model at least twice. Age, physical health, and cognitive function were the most common predictors. Only one model was externally validated, two models were presented with a complete equation, and five models examined the calibration. We found substantial heterogeneity in predictor and outcome definitions across models; important methodological information was often missing. All models were rated at high or unclear risk of bias, primarily due to methodological limitations. The pooled C-statistics of 12 prediction models was 0.83 (95%CI=0.77-0.89).

CONCLUSION

The usefulness of all models remains unclear due to several methodological limitations. Future studies should focus on methodological quality and external validation of depression risk prediction models.

Prediction Models for Prognosis of Hypoglycemia in Patients with Diabetes: A Systematic Review and Meta-Analysis

OBJECTIVE

To systematically summarize the reported prediction models for hypoglycemia in patients with diabetes, compare their performance, and evaluate their applicability in clinical practice.

METHODS

We selected studies according to the PRISMA, appraised studies according to the Prediction model Risk of Bias Assessment Tool (PROBAST), and extracted and synthesized the data according to the CHARMS. The databases of PubMed, Web of Science, Embase, and Cochrane Library were searched from inception to 31 October 2021 using a systematic review approach to capture all eligible studies developing and/or validating a prognostic prediction model for hypoglycemia in patients with diabetes. The risk bias and clinical applicability were assessed using the PROBAST. The meta-analysis of the performance of the prediction models were also conducted. The protocol of this study was recorded in PROSPERO (CRD42022309852).

RESULTS

Sixteen studies with 22 models met the eligible criteria. The predictors with the high frequency of occurrence among all models were age, HbA1c, history of hypoglycemia, and insulin use. A meta-analysis of C-statistic was performed for 21 prediction models, and the summary C-statistic and its 95% confidence interval and prediction interval were 0.7699 (0.7299-0.8098), 0.7699 (0.5862-0.9536), respectively. Heterogeneity exists between different hypoglycemia prediction models (τ² was 0.00734≠0).

CONCLUSIONS

The existing predictive models are not recommended for widespread clinical use. A high-quality hypoglycemia screening tool should be developed in future studies.

Frailty and Mortality Risk in COPD: A Cohort Study Comparing the Fried Frailty Phenotype and Short Physical Performance Battery

Background

Identifying frailty in people with chronic obstructive pulmonary disease (COPD) is deemed important, yet comparative characteristics of the most commonly used frailty measures in COPD are unknown. This study aimed to compare how the Fried Frailty Phenotype (FFP) and Short Physical Performance Battery (SPPB) characterise frailty in people with stable COPD, including prevalence of and overlap in identification of frailty, disease and health characteristics of those identified as living with frailty, and predictive value in relation to survival time.

Methods

Cohort study of people with stable COPD attending outpatient clinics. Agreement between frailty classifications was described using Cohen's Kappa. Disease and health characteristics of frail versus not frail participants were compared using t-, Mann-Whitney U and Chi-Square tests. Predictive value for mortality was examined with multivariable Cox regression.

Results

Of 714 participants, 421 (59%) were male, mean age 69.9 years (SD 9.7), mean survival time 2270 days (95% CI 2185-2355). Similar proportions were identified as frail using the FFP (26.2%) and SPPB (23.7%) measures; classifications as frail or not frail matched in 572 (80.1%) cases, showing moderate agreement (Kappa = 0.469, SE = 0.038, p < 0.001). Discrepancies seemed driven by FFP exhaustion and weight loss criteria and the SPPB balance component. People with frailty by either measure had worse exercise capacity, health-related quality of life, breathlessness, depression and dependence in activities of daily living. In multivariable analysis controlling for the Age Dyspnoea Obstruction index, sex, BMI, comorbidities and exercise capacity, both the FFP and SPPB had predictive value in relation to mortality (FFP aHR = 1.31 [95% CI 1.03-1.66]; SPPB aHR = 1.29 [95% CI 0.99-1.68]).

Conclusion

In stable COPD, both the FFP and SPPB identify similar proportions of people living with/without frailty, the majority with matching classifications. Both measures can identify individuals with multidimensional health challenges and increased mortality risk and provide additional information alongside established prognostic variables.

External Validation and User Experiences of the ProPal-COPD Tool to Identify the Palliative Phase in COPD

Background

Difficulty predicting prognosis is a major barrier to timely palliative care provision for patients with COPD. The ProPal-COPD tool, combining six clinical indicators and the Surprise Question (SQ), aims to predict 1-year mortality as a proxy for palliative care needs. It appeared to be a promising tool for healthcare providers to identify patients with COPD who could benefit from palliative care.

Objective

To externally validate the ProPal-COPD tool and to assess user experiences.

Methods

Patients admitted with an acute exacerbation COPD were recruited across 10 hospitals. Demographics, clinical characteristics and survival status were collected. Sensitivity, specificity, positive and negative predictive values of the tool using two cut-off values were calculated. Also, predictive properties of the SQ were calculated. In monitoring meetings and interviews, healthcare providers shared their experiences with the tool. Transcripts were deductively coded using six user experience domains: Acceptability, Satisfaction, Credibility, Usability, User-reported adherence and Perceived impact.

Results

A total of 523 patients with COPD were included between May 2019 and August 2020, of whom 100 (19.1%) died within 12 months. The ProPal-COPD tool had an AUC of 0.68 and a low sensitivity (55%) and moderate specificity (74%) for predicting 1-year all-cause mortality. Using a lower cut-off value, sensitivity was higher (74%), but specificity lower (46%). Sensitivity and specificity of the SQ were 56% and 73%, respectively (AUC 0.65). However, healthcare providers generally appreciated using the tool because it increased awareness of the palliative phase and provided a shared understanding of prognosis, although they considered its outcome not always correct.

Conclusion

The accuracy of the ProPal-COPD tool to predict 1-year mortality is limited, although screening patients with its indicators increases healthcare providers' awareness of palliative care needs and encourages them to timely initiate appropriate care.

Validation of Clinical COPD Phenotypes for Prognosis of Long-Term Mortality in Swedish and Dutch Cohorts

Chronic obstructive pulmonary disease (COPD) is a heterogeneous disease with variable mortality risk. The aim of our investigation was to validate a simple clinical algorithm for long-term mortality previously proposed by Burgel et al. in 2017. Subjects with COPD from two cohorts, the Swedish PRAXIS study (n = 784, mean age (standard deviation (SD)) 64.0 years (7.5), 42% males) and the Rotterdam Study (n = 735, mean age (SD) 72 years (9.2), 57% males), were included. Five clinical clusters were derived from baseline data on age, body mass index, dyspnoea grade, pulmonary function and comorbidity (cardiovascular disease/diabetes). Cox models were used to study associations with 9-year mortality. The distribution of clinical clusters (1-5) was 29%/45%/8%/6%/12% in the PRAXIS study and 23%/26%/36%/0%/15% in the Rotterdam Study. The cumulative proportion of deaths at the 9-year follow-up was highest in clusters 1 (65%) and 4 (72%), and lowest in cluster 5 (10%) in the PRAXIS study. In the Rotterdam Study, cluster 1 (44%) had the highest cumulative mortality and cluster 5 (5%) the lowest. Compared with cluster 5, the meta-analysed age- and sex-adjusted hazard ratio (95% confidence interval) for cluster 1 was 6.37 (3.94-10.32) and those for clusters 2 and 3 were 2.61 (1.58-4.32) and 3.06 (1.82-5.13), respectively. Burgel's clinical clusters can be used to predict long-term mortality risk. Clusters 1 and 4 are associated with the poorest prognosis, cluster 5 with the best prognosis and clusters 2 and 3 with intermediate prognosis in two independent cohorts from Sweden and the Netherlands.

Value of genomics- and radiomics-based machine learning models in the identification of breast cancer molecular subtypes: a systematic review and meta-analysis

Background

In the era of precision therapy, early classification of breast cancer (BRCA) molecular subtypes has clinical significance for disease management and prognosis. We explored the accuracy of machine learning (ML) models for early classification of BRCA molecular subtypes through a systematic review of the literature currently available.

Methods

We retrieved relevant studies published in PubMed, EMBASE, Cochrane, and Web of Science until 15 April 2022. A prediction model risk of bias assessment tool (PROBAST) was applied for the assessment of risk of bias of a genomics-based ML model, and the Radiomics Quality Score (RQS) was simultaneously used to evaluate the quality of this radiomics-based ML model. A random effects model was adopted to analyze the predictive accuracy of genomics-based ML and radiomics-based ML for Luminal A, Luminal B, Basal-like or triple-negative breast cancer (TNBC), and human epidermal growth factor receptor 2 (HER2). The PROSPERO of our study was prospectively registered (CRD42022333611).

Results

Of the 38 studies were selected for analysis, 14 ML models were based on gene-transcriptomic, with only 4 external validations; and 43 ML models were based on radiomics, with only 14 external validations. Meta-analysis results showed that c-statistic values of the ML based on radiomics for the identification of BRCA molecular subtypes Luminal A, Luminal B, Basal-like or TNBC, and HER2 were 0.76 [95% confidence interval (CI): 0.60-0.96], 0.78 (95% CI: 0.69-0.87), 0.89 (95% CI: 0.83-0.91), and 0.83 (95% CI: 0.81-0.86), respectively. The c-statistic values of ML based on the gene-transcriptomic analysis cohort for the identification of the previously described BRCA molecular subtypes were 0.96 (95% CI: 0.93-0.99), 0.96 (95% CI: 0.93-0.99), 0.98 (95% CI: 0.95-1.00), and 0.97 (95% CI: 0.96-0.98) respectively. Additionally, the sensitivity of the ML model based on radiomics for each molecular subtype ranged from 0.79 to 0.85, while the sensitivity of the ML model based on gene-transcriptomic was between 0.92 and 0.99.

Conclusions

Both radiomics and gene transcriptomics produced ideal effects on BRCA molecular subtype prediction. Compared with radiomics, gene transcriptomics yielded better prediction results, but radiomics was simpler and more convenient from a clinical point of view.

ACCEPT 2·0: Recalibrating and externally validating the Acute COPD exacerbation prediction tool (ACCEPT)

BACKGROUND

The Acute Chronic Obstructive Pulmonary Disease (COPD) Exacerbation Prediction Tool (ACCEPT) was developed for individualised prediction of COPD exacerbations. ACCEPT was well calibrated overall and had a high discriminatory power, but overestimated risk among individuals without recent exacerbations. The objectives of this study were to 1) fine-tune ACCEPT to make better predictions for individuals with a negative exacerbation history, 2) develop more parsimonious models, and 3) externally validate the models in a new dataset.

METHODS

We recalibrated ACCEPT using data from the Evaluation of COPD Longitudinally to Identify Predictive Surrogate End-points (ECLIPSE, a three-year observational study, 1,803 patients, 2,117 exacerbations) study by applying non-parametric regression splines to the predicted rates. We developed three reduced versions of ACCEPT by removing symptom score and/or baseline medications as predictors. We examined the discrimination, calibration, and net benefit of ACCEPT 2·0 in the placebo arm of the Towards a Revolution in COPD Health (TORCH, a three-year randomised clinical trial of inhaled therapies in COPD, 1,091 patients, 1,064 exacerbations) study. The primary outcome for prediction was the occurrence of ≥2 moderate or ≥1 severe exacerbation in the next 12 months; the secondary outcomes were prediction of the occurrence of any moderate/severe exacerbation or any severe exacerbation.

FINDINGS

ACCEPT 2·0 had an area-under-the-curve (AUC) of 0·76 for predicting the primary outcome. Exacerbation history alone (current standard of care) had an AUC of 0·68. The model was well calibrated in patients with positive or negative exacerbation histories. Changes in AUC in reduced versions were minimal for the primary outcome as well as for predicting the occurrence of any moderate/severe exacerbations (ΔAUC<0·011), but more substantial for predicting the occurrence of any severe exacerbations (ΔAUC<0·020). All versions of ACCEPT 2·0 provided positive net benefit over the use of exacerbation history alone for some range of thresholds.

INTERPRETATION

ACCEPT 2·0 showed good calibration regardless of exacerbation history, and predicts exacerbation risk better than current standard of care for a range of thresholds. Future studies need to investigate the utility of exacerbation prediction in various subgroups of patients.

FUNDING

This study was funded by a team grant from the Canadian Institutes of Health Research (PHT 178432).

Machine learning for lymph node metastasis prediction of in patients with gastric cancer: A systematic review and meta-analysis

Objective

To evaluate the diagnostic performance of machine learning (ML) in predicting lymph node metastasis (LNM) in patients with gastric cancer (GC) and to identify predictors applicable to the models.

Methods

PubMed, EMBASE, Web of Science, and Cochrane Library were searched from inception to March 16, 2022. The pooled c-index and accuracy were used to assess the diagnostic accuracy. Subgroup analysis was performed based on ML types. Meta-analyses were performed using random-effect models. Risk of bias assessment was conducted using PROBAST tool.

Results

A total of 41 studies (56182 patients) were included, and 33 of the studies divided the participants into a training set and a test set, while the rest of the studies only had a training set. The c-index of ML for LNM prediction in training set and test set was 0.837 [95%CI (0.814, 0.859)] and 0.811 [95%CI (0.785-0.838)], respectively. The pooled accuracy was 0.781 [(95%CI (0.756-0.805)] in training set and 0.753 [95%CI (0.721-0.783)] in test set. Subgroup analysis for different ML algorithms and staging of GC showed no significant difference. In contrast, in the subgroup analysis for predictors, in the training set, the model that included radiomics had better accuracy than the model with only clinical predictors (F = 3.546, p = 0.037). Additionally, cancer size, depth of cancer invasion and histological differentiation were the three most commonly used features in models built for prediction.

Conclusion

ML has shown to be of excellent diagnostic performance in predicting the LNM of GC. One of the models covering radiomics and its ML algorithms showed good accuracy for the risk of LNM in GC. However, the results revealed some methodological limitations in the development process. Future studies should focus on refining and improving existing models to improve the accuracy of LNM prediction.

Systematic Review Registration

https://www.crd.york.ac.uk/PROSPERO/, identifier CRD42022320752

How to conduct a systematic review and meta-analysis of prognostic model studies

BACKGROUND

Prognostic models are typically developed to estimate the risk that an individual in a particular health state will develop a particular health outcome, to support (shared) decision making. Systematic reviews of prognostic model studies can help identify prognostic models that need to further be validated or are ready to be implemented in healthcare.

OBJECTIVES

To provide a step-by-step guidance on how to conduct and read a systematic review of prognostic model studies and to provide an overview of methodology and guidance available for every step of the review progress.

SOURCES

Published, peer-reviewed guidance articles.

CONTENT

We describe the following steps for conducting a systematic review of prognosis studies: 1) Developing the review question using the Population, Index model, Comparator model, Outcome(s), Timing, Setting format, 2) Searching and selection of articles, 3) Data extraction using the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) checklist, 4) Quality and risk of bias assessment using the Prediction model Risk Of Bias ASsessment (PROBAST) tool, 5) Analysing data and undertaking quantitative meta-analysis, and 6) Presenting summary of findings, interpreting results, and drawing conclusions. Guidance for each step is described and illustrated using a case study on prognostic models for patients with COVID-19.

IMPLICATIONS

Guidance for conducting a systematic review of prognosis studies is available, but the implications of these reviews for clinical practice and further research highly depend on complete reporting of primary studies.

Machine Learning Prediction of Progression in Forced Expiratory Volume in 1 Second in the COPDGene® Study

BACKGROUND

The heterogeneous nature of chronic obstructive pulmonary disease (COPD) complicates the identification of the predictors of disease progression. We aimed to improve the prediction of disease progression in COPD by using machine learning and incorporating a rich dataset of phenotypic features.

METHODS

We included 4496 smokers with available data from their enrollment and 5-year follow-up visits in the COPD Genetic Epidemiology (COPDGene^®) study. We constructed linear regression (LR) and supervised random forest models to predict 5-year progression in forced expiratory in 1 second (FEV₁) from 46 baseline features. Using cross-validation, we randomly partitioned participants into training and testing samples. We also validated the results in the COPDGene 10-year follow-up visit.

RESULTS

Predicting the change in FEV₁ over time is more challenging than simply predicting the future absolute FEV₁ level. For random forest, R-squared was 0.15 and the area under the receiver operator characteristic (ROC) curves for the prediction of participants in the top quartile of observed progression was 0.71 (testing) and respectively, 0.10 and 0.70 (validation). Random forest provided slightly better performance than LR. The accuracy was best for Global initiative for chronic Obstructive Lung Disease (GOLD) grades 1-2 participants, and it was harder to achieve accurate prediction in advanced stages of the disease. Predictive variables differed in their relative importance as well as for the predictions by GOLD.

CONCLUSION

Random forest, along with deep phenotyping, predicts FEV₁ progression with reasonable accuracy. There is significant room for improvement in future models. This prediction model facilitates the identification of smokers at increased risk for rapid disease progression. Such findings may be useful in the selection of patient populations for targeted clinical trials.

Methodological Challenges in Predicting Periprosthetic Joint Infection Treatment Outcomes: A Narrative Review

The management of periprosthetic joint infection (PJI) generally requires both surgical intervention and targeted antimicrobial therapy. Decisions regarding surgical management–whether it be irrigation and debridement, one-stage revision, or two-stage revision–must take into consideration an array of factors. These include the timing and duration of symptoms, clinical characteristics of the patient, and antimicrobial susceptibilities of the microorganism(s) involved. Moreover, decisions relating to surgical management must consider clinical factors associated with the health of the patient, alongside the patient's preferences. These decisions are further complicated by concerns beyond mere eradication of the infection, such as the level of improvement in quality of life related to management strategies. To better understand the probability of successful surgical treatment of a PJI, several predictive tools have been developed over the past decade. This narrative review provides an overview of available clinical prediction models that aim to guide treatment decisions for patients with periprosthetic joint infection, and highlights key challenges to reliably implementing these tools in clinical practice.

Reproducibility of prediction models in health services research

The field of health services research studies the health care system by examining outcomes relevant to patients and clinicians but also health economists and policy makers. Such outcomes often include health care spending, and utilization of care services. Building accurate prediction models using reproducible research practices for health services research is important for evidence-based decision making. Several systematic reviews have summarized prediction models for outcomes relevant to health services research, but these systematic reviews do not present a thorough assessment of reproducibility and research quality of the prediction modelling studies. In the present commentary, we discuss how recent advances in prediction modelling in other medical fields can be applied to health services research. We also describe the current status of prediction modelling in health services research, and we summarize available methodological guidance for the development, update, external validation and systematic appraisal of prediction models.

Deep Learning Prediction of Survival in Patients with Chronic Obstructive Pulmonary Disease Using Chest Radiographs

Background Preexisting indexes for predicting the prognosis of chronic obstructive pulmonary disease (COPD) do not use radiologic information and are impractical because they involve complex history assessments or exercise tests. Purpose To develop and to validate a deep learning-based survival prediction model in patients with COPD (DLSP) using chest radiographs, in addition to other clinical factors. Materials and Methods In this retrospective study, data from patients with COPD who underwent postbronchodilator spirometry and chest radiography from 2011-2015 were collected and split into training (n = 3475), validation (n = 435), and internal test (n = 315) data sets. The algorithm for predicting survival from chest radiographs was trained (hereafter, DLSP_CXR), and then age, body mass index, and forced expiratory volume in 1 second (FEV₁) were integrated within the model (hereafter, DLSP_integ). For external test, three independent cohorts were collected (n = 394, 416, and 337). The discrimination performance of DLSP_CXR was evaluated by using time-dependent area under the receiver operating characteristic curves (TD AUCs) at 5-year survival. Goodness of fit was assessed by using the Hosmer-Lemeshow test. Using one external test data set, DLSP_integ was compared with four COPD-specific clinical indexes: BODE, ADO, COPD Assessment Test (CAT), and St George's Respiratory Questionnaire (SGRQ). Results DLSP_CXR had a higher performance at predicting 5-year survival than FEV₁ in two of the three external test cohorts (TD AUC: 0.73 vs 0.63 [P = .004]; 0.67 vs 0.60 [P = .01]; 0.76 vs 0.77 [P = .91]). DLSP_CXR demonstrated good calibration in all cohorts. The DLSP_integ model showed no differences in TD AUC compared with BODE (0.87 vs 0.80; P = .34), ADO (0.86 vs 0.89; P = .51), and SGRQ (0.86 vs 0.70; P = .09), and showed higher TD AUC than CAT (0.93 vs 0.55; P < .001). Conclusion A deep learning model using chest radiographs was capable of predicting survival in patients with chronic obstructive pulmonary disease. © RSNA, 2022 Online supplemental material is available for this article.

Does poor methodological quality of prediction modeling studies translate to poor model performance? An illustration in traumatic brain injury

BACKGROUND

Prediction modeling studies often have methodological limitations, which may compromise model performance in new patients and settings. We aimed to examine the relation between methodological quality of model development studies and their performance at external validation.

METHODS

We systematically searched for externally validated multivariable prediction models that predict functional outcome following moderate or severe traumatic brain injury. Risk of bias and applicability of development studies was assessed with the Prediction model Risk Of Bias Assessment Tool (PROBAST). Each model was rated for its presentation with sufficient detail to be used in practice. Model performance was described in terms of discrimination (AUC), and calibration. Delta AUC (dAUC) was calculated to quantify the percentage change in discrimination between development and validation for all models. Generalized estimation equations (GEE) were used to examine the relation between methodological quality and dAUC while controlling for clustering.

RESULTS

We included 54 publications, presenting ten development studies of 18 prediction models, and 52 external validation studies, including 245 unique validations. Two development studies (four models) were found to have low risk of bias (RoB). The other eight publications (14 models) showed high or unclear RoB. The median dAUC was positive in low RoB models (dAUC 8%, [IQR - 4% to 21%]) and negative in high RoB models (dAUC - 18%, [IQR - 43% to 2%]). The GEE showed a larger average negative change in discrimination for high RoB models (- 32% (95% CI: - 48 to - 15) and unclear RoB models (- 13% (95% CI: - 16 to - 10)) compared to that seen in low RoB models.

CONCLUSION

Lower methodological quality at model development associates with poorer model performance at external validation. Our findings emphasize the importance of adherence to methodological principles and reporting guidelines in prediction modeling studies.