Search filters for finding prognostic and diagnostic prediction studies in Medline to enhance systematic reviews

Predictive models of Alzheimer's disease dementia risk in older adults with mild cognitive impairment: a systematic review and critical appraisal

BACKGROUND

Mild cognitive impairment has received widespread attention as a high-risk population for Alzheimer's disease, and many studies have developed or validated predictive models to assess it. However, the performance of the model development remains unknown.

OBJECTIVE

The objective of this review was to provide an overview of prediction models for the risk of Alzheimer's disease dementia in older adults with mild cognitive impairment.

METHOD

PubMed, EMBASE, Web of Science, and MEDLINE were systematically searched up to October 19, 2023. We included cohort studies in which risk prediction models for Alzheimer's disease dementia in older adults with mild cognitive impairment were developed or validated. The Predictive Model Risk of Bias Assessment Tool (PROBAST) was employed to assess model bias and applicability. Random-effects models combined model AUCs and calculated (approximate) 95% prediction intervals for estimations. Heterogeneity across studies was evaluated using the I2 statistic, and subgroup analyses were conducted to investigate sources of heterogeneity. Additionally, funnel plot analysis was utilized to identify publication bias.

RESULTS

The analysis included 16 studies involving 9290 participants. Frequency analysis of predictors showed that 14 appeared at least twice and more, with age, functional activities questionnaire, and Mini-mental State Examination scores of cognitive functioning being the most common predictors. From the studies, only two models were externally validated. Eleven studies ultimately used machine learning, and four used traditional modelling methods. However, we found that in many of the studies, there were problems with insufficient sample sizes, missing important methodological information, lack of model presentation, and all of the models were rated as having a high or unclear risk of bias. The average AUC of the 15 best-developed predictive models was 0.87 (95% CI: 0.83, 0.90).

DISCUSSION

Most published predictive modelling studies are deficient in rigour, resulting in a high risk of bias. Upcoming research should concentrate on enhancing methodological rigour and conducting external validation of models predicting Alzheimer's disease dementia. We also emphasize the importance of following the scientific method and transparent reporting to improve the accuracy, generalizability and reproducibility of study results.

REGISTRATION

This systematic review was registered in PROSPERO (Registration ID: CRD42023468780).

Externally validated prediction models for pre-eclampsia: systematic review and meta-analysis

OBJECTIVE

This systematic review and meta-analysis aimed to evaluate the performance of existing externally validated prediction models for pre-eclampsia (PE) (specifically, any-onset, early-onset, late-onset and preterm PE).

METHODS

A systematic search was conducted in five databases (MEDLINE, EMBASE, Emcare, CINAHL and Maternity & Infant Care Database) and using Google Scholar/reference search to identify studies based on the Population, Index prediction model, Comparator, Outcome, Timing and Setting (PICOTS) approach until 20 May 2023. We extracted data using the CHARMS checklist and appraised the risk of bias using the PROBAST tool. A meta-analysis of discrimination and calibration performance was conducted when appropriate.

RESULTS

Twenty-three studies reported 52 externally validated prediction models for PE (one preterm, 20 any-onset, 17 early-onset and 14 late-onset PE models). No model had the same set of predictors. Fifteen any-onset PE models were validated externally once, two were validated twice and three were validated three times, while the Fetal Medicine Foundation (FMF) competing-risks model for preterm PE prediction was validated widely in 16 different settings. The most common predictors were maternal characteristics (prepregnancy body mass index, prior PE, family history of PE, chronic medical conditions and ethnicity) and biomarkers (uterine artery pulsatility index and pregnancy-associated plasma protein-A). The FMF model for preterm PE (triple test plus maternal factors) had the best performance, with a pooled area under the receiver-operating-characteristics curve (AUC) of 0.90 (95% prediction interval (PI), 0.76-0.96), and was well calibrated. The other models generally had poor-to-good discrimination performance (median AUC, 0.66 (range, 0.53-0.77)) and were overfitted on external validation. Apart from the FMF model, only two models that were validated multiple times for any-onset PE prediction, which were based on maternal characteristics only, produced reasonable pooled AUCs of 0.71 (95% PI, 0.66-0.76) and 0.73 (95% PI, 0.55-0.86).

CONCLUSIONS

Existing externally validated prediction models for any-, early- and late-onset PE have limited discrimination and calibration performance, and include inconsistent input variables. The triple-test FMF model had outstanding discrimination performance in predicting preterm PE in numerous settings, but the inclusion of specialized biomarkers may limit feasibility and implementation outside of high-resource settings. © 2023 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of International Society of Ultrasound in Obstetrics and Gynecology.

A systematic review of cardiac surgery clinical prediction models that include intra-operative variables

BACKGROUND

Most cardiac surgery clinical prediction models (CPMs) are developed using pre-operative variables to predict post-operative outcomes. Some CPMs are developed with intra-operative variables, but none are widely used. The objective of this systematic review was to identify CPMs with intra-operative variables that predict short-term outcomes following adult cardiac surgery.

METHODS

Ovid MEDLINE and EMBASE databases were searched from inception to December 2022, for studies developing a CPM with at least one intra-operative variable. Data were extracted using a critical appraisal framework and bias assessment tool. Model performance was analysed using discrimination and calibration measures.

RESULTS

A total of 24 models were identified. Frequent predicted outcomes were acute kidney injury (9/24 studies) and peri-operative mortality (6/24 studies). Frequent pre-operative variables were age (18/24 studies) and creatinine/eGFR (18/24 studies). Common intra-operative variables were cardiopulmonary bypass time (16/24 studies) and transfusion (13/24 studies). Model discrimination was acceptable for all internally validated models (AUC 0.69-0.91). Calibration was poor (15/24 studies) or unreported (8/24 studies). Most CPMs were at a high or indeterminate risk of bias (23/24 models). The added value of intra-operative variables was assessed in six studies with statistically significantly improved discrimination demonstrated in two.

CONCLUSION

Weak reporting and methodological limitations may restrict wider applicability and adoption of existing CPMs that include intra-operative variables. There is some evidence that CPM discrimination is improved with the addition of intra-operative variables. Further work is required to understand the role of intra-operative CPMs in the management of cardiac surgery patients.

Predictive factors of clot propagation in patients with superficial venous thrombosis towards deep venous thrombosis and pulmonary embolism: a systematic review and meta-analysis

OBJECTIVE

A subset of patients with superficial venous thrombosis (SVT) experiences clot propagation towards deep venous thrombosis (DVT) and/or pulmonary embolism (PE). The aim of this systematic review is to identify all clinically relevant cross-sectional and prognostic factors for predicting thrombotic complications in patients with SVT.

DESIGN

Systematic review.

DATA SOURCES

PubMed/MEDLINE and Embase were systematically searched until 3 March 2023.

ELIGIBILITY CRITERIA

Original research studies with patients with SVT, DVT and/or PE as the outcome and presenting cross-sectional or prognostic predictive factors.

DATA EXTRACTION AND SYNTHESIS OF RESULTS

The CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling (CHARMS) checklist for prognostic factor studies was used for systematic extraction of study characteristics. Per identified predictive factor, relevant estimates of univariable and multivariable predictor-outcome associations were extracted, such as ORs and HRs. Estimates of association for the most frequently reported predictors were summarised in forest plots, and meta-analyses with heterogeneity were presented. The Quality in Prognosis Studies (QUIPS) tool was used for risk of bias assessment and Grading of Recommendations, Assessment, Development and Evaluations (GRADE) for assessing the certainty of evidence.

RESULTS

Twenty-two studies were included (n=10 111 patients). The most reported predictive factors were high age, male sex, history of venous thromboembolism (VTE), absence of varicose veins and cancer. Pooled effect estimates were heterogenous and ranged from OR 3.12 (95% CI 1.75 to 5.59) for the cross-sectional predictor cancer to OR 0.92 (95% CI 0.56 to 1.53) for the prognostic predictor high age. The level of evidence was rated very low to low. Most studies were scored high or moderate risk of bias.

CONCLUSIONS

Although the pooled estimates of the predictors high age, male sex, history of VTE, cancer and absence of varicose veins showed predictive potential in isolation, variability in study designs, lack of multivariable adjustment and high risk of bias prevent firm conclusions. High-quality, multivariable studies are necessary to be able to identify individual SVT risk profiles.

PROSPERO REGISTRATION NUMBER

CRD42021262819.

[Comparison of different prediction models for the occurrence of nausea and vomiting in the postoperative phase : A systematic qualitative comparison based on prospectively defined quality indicators]

BACKGROUND

Various prognostic prediction models exist for evaluating the risk of nausea and vomiting in the postoperative period (PONV). So far, no systematic comparison of these prognostic scores is available.

METHOD

A systematic literature search was carried out in seven medical databases to find publications on prognostic PONV models. Identified scores were assessed against prospectively defined quality criteria, including generalizability, validation and clinical relevance of the models.

RESULTS

The literature search revealed 62 relevant publications with a total of 81,834 patients which could be assigned to 8 prognostic models. The simplified Apfel score performed best, primarily because it was extensively validated. The Van den Bosch score and Sinclair score tied for second place. The simplified Koivuranta score was in third place.

CONCLUSION

The qualitative analysis highlights the strengths and weaknesses of each prediction system based on predetermined standardized quality criteria.

Factors associated with concerns about falling and activity restriction in older adults after hip fracture: a mixed-methods systematic review

PURPOSE

To investigate factors contributing to concerns about falling and activity restriction in the community among older adults who had a hip fracture.

METHODS

A mixed method systematic review with a convergent segregated approach. We searched Medline, Embase, PsycInfo, PEDRo, CINAHL and the Cochrane library. Results were synthesised narratively considering physical, psychological, environmental, care, and social factors and presented in tables. Critical appraisal was completed in duplicate.

RESULTS

We included 19 studies (9 qualitative, 9 observational, 1 mixed methods) representing 1480 individuals and 23 factors related to concerns about falling and activity restriction. Physical factors included falls history, comorbidities, balance, strength, mobility and functionality. Psychological factors included anxiety and neuroticism scores, perceived confidence in/control over rehabilitation and abilities, and negative/positive affect about the orthopaedic trauma, pre-fracture abilities and future needs. Environmental factors included accessibility in the home, outdoors and with transport. Social and care factors related to the presence or absence of formal and informal networks, which reduced concerns and promoted activity by providing feedback, advice, encouragement, and practical support.

CONCLUSION

These findings highlight that to improve concerns about falling and activity restriction after hip fracture, it is important to: improve physical and functional abilities; boost self-confidence; promote positive affect; involve relatives and carers; increase access to clinicians, and; enhance accessibility of the home, outdoors and transport. Most factors were reported on by a small number of studies of varying quality and require replication in future research.

Prognostic factors and prediction models for hospitalisation and all-cause mortality in adults presenting to primary care with a lower respiratory tract infection: a systematic review

OBJECTIVE

To identify and synthesise relevant existing prognostic factors (PF) and prediction models (PM) for hospitalisation and all-cause mortality within 90 days in primary care patients with acute lower respiratory tract infections (LRTI).

DESIGN

Systematic review.

METHODS

Systematic searches of MEDLINE, Embase and the Cochrane Library were performed. All PF and PM studies on the risk of hospitalisation or all-cause mortality within 90 days in adult primary care LRTI patients were included. The risk of bias was assessed using the Quality in Prognostic Studies tool and Prediction Model Risk Of Bias Assessment Tool tools for PF and PM studies, respectively. The results of included PF and PM studies were descriptively summarised.

RESULTS

Of 2799 unique records identified, 16 were included: 9 PF studies, 6 PM studies and 1 combination of both. The risk of bias was judged high for all studies, mainly due to limitations in the analysis domain. Based on reported multivariable associations in PF studies, increasing age, sex, current smoking, diabetes, a history of stroke, cancer or heart failure, previous hospitalisation, influenza vaccination (negative association), current use of systemic corticosteroids, recent antibiotic use, respiratory rate ≥25/min and diagnosis of pneumonia were identified as most promising candidate predictors. One newly developed PM was externally validated (c statistic 0.74, 95% CI 0.71 to 0.78) whereas the previously hospital-derived CRB-65 was externally validated in primary care in five studies (c statistic ranging from 0.72 (95% CI 0.63 to 0.81) to 0.79 (95% CI 0.65 to 0.92)). None of the PM studies reported measures of model calibration.

CONCLUSIONS

Implementation of existing models for individualised risk prediction of 90-day hospitalisation or mortality in primary care LRTI patients in everyday practice is hampered by incomplete assessment of model performance. The identified candidate predictors provide useful information for clinicians and warrant consideration when developing or updating PMs using state-of-the-art development and validation techniques.

PROSPERO REGISTRATION NUMBER

CRD42022341233.

Multivariable prediction models for fetal macrosomia and large for gestational age: A systematic review

BACKGROUND

The identification of large for gestational age (LGA) and macrosomic fetuses is essential for counselling and managing these pregnancies.

OBJECTIVES

To systematically review the literature for multivariable prediction models for LGA and macrosomia, assessing the performance, quality and applicability of the included model in clinical practice.

SEARCH STRATEGY

MEDLINE, EMBASE and Cochrane Library were searched until June 2022.

SELECTION CRITERIA

We included observational and experimental studies reporting the development and/or validation of any multivariable prediction model for fetal macrosomia and/or LGA. We excluded studies that used a single variable or did not evaluate model performance.

DATA COLLECTION AND ANALYSIS

Data were extracted using the Checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies checklist. The model performance measures discrimination, calibration and validation were extracted. The quality and completion of reporting within each study was assessed by its adherence to the TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) checklist. The risk of bias and applicability were measured using PROBAST (Prediction model Risk Of Bias Assessment Tool).

MAIN RESULTS

A total of 8442 citations were identified, with 58 included in the analysis: 32/58 (55.2%) developed, 21/58 (36.2%) developed and internally validated and 2/58 (3.4%) developed and externally validated a model. Only three studies externally validated pre-existing models. Macrosomia and LGA were differentially defined by many studies. In total, 111 multivariable prediction models were developed using 112 different variables. Model discrimination was wide ranging area under the receiver operating characteristics curve (AUROC 0.56-0.96) and few studies reported calibration (11/58, 19.0%). Only 5/58 (8.6%) studies had a low risk of bias.

CONCLUSIONS

There are currently no multivariable prediction models for macrosomia/LGA that are ready for clinical implementation.

A literature review of quality assessment and applicability to HTA of risk prediction models of coronary heart disease in patients with diabetes

This literature review had two objectives: to identify models for predicting the risk of coronary heart diseases in patients with diabetes (DM); and to assess model quality in terms of risk of bias (RoB) and applicability for the purpose of health technology assessment (HTA). We undertook a targeted review of journal articles published in English, Dutch, Chinese, or Spanish in 5 databases from 1st January 2016 to 18th December 2022, and searched three systematic reviews for the models published after 2012. We used PROBAST (Prediction model Risk Of Bias Assessment Tool) to assess RoB, and used findings from Betts et al. 2019, which summarized recommendations and criticisms of HTA agencies on cardiovascular risk prediction models, to assess model applicability for the purpose of HTA. As a result, 71 % and 67 % models reporting C-index showed good discrimination abilities (C-index >= 0.7). Of the 26 model studies and 30 models identified, only one model study showed low RoB in all domains, and no model was fully applicable for HTA. Since the major cause of high RoB is inappropriate use of analysis method, we advise clinicians to carefully examine the model performance declared by model developers, and to trust a model if all PROBAST domains except analysis show low RoB and at least one validation study conducted in the same setting (e.g. country) is available. Moreover, since general model applicability is not informative for HTA, novel adapted tools may need to be developed.

Investigation of statistical methods used in prognostic prediction models for obstetric care: A 10 year-span cross-sectional study

INTRODUCTION

Obstetric care is a highly active area in the development and application of prognostic prediction models. The development and validation of these models often require the utilization of advanced statistical techniques. However, failure to adhere to rigorous methodological standards could greatly undermine the reliability and trustworthiness of the resultant models. Consequently, the aim of our study was to examine the current statistical practices employed in obstetric care and offer recommendations to enhance the utilization of statistical methods in the development of prognostic prediction models.

MATERIAL AND METHODS

We conducted a cross-sectional survey using a sample of studies developing or validating prognostic prediction models for obstetric care published in a 10-year span (2011-2020). A structured questionnaire was developed to investigate the statistical issues in five domains, including model derivation (predictor selection and algorithm development), model validation (internal and external), model performance, model presentation, and risk threshold setting. On the ground of survey results and existing guidelines, a list of recommendations for statistical methods in prognostic models was developed.

RESULTS

A total of 112 eligible studies were included, with 107 reporting model development and five exclusively reporting external validation. During model development, 58.9% of the studies did not include any form of validation. Of these, 46.4% used stepwise regression in a crude manner for predictor selection, while two-thirds made decisions on retaining or dropping candidate predictors solely based on p-values. Additionally, 26.2% transformed continuous predictors into categorical variables, and 80.4% did not consider nonlinear relationships between predictors and outcomes. Surprisingly, 94.4% of the studies did not examine the correlation between predictors. Moreover, 47.1% of the studies did not compare population characteristics between the development and external validation datasets, and only one-fifth evaluated both discrimination and calibration. Furthermore, 53.6% of the studies did not clearly present the model, and less than half established a risk threshold to define risk categories. In light of these findings, 10 recommendations were formulated to promote the appropriate use of statistical methods.

CONCLUSIONS

The use of statistical methods is not yet optimal. Ten recommendations were offered to assist the statistical methods of prognostic prediction models in obstetric care.

Multivariable Prediction Models for Traumatic Spinal Cord Injury: A Systematic Review

Background

Traumatic spinal cord injuries (TSCI) greatly affect the lives of patients and their families. Prognostication may improve treatment strategies, health care resource allocation, and counseling. Multivariable clinical prediction models (CPMs) for prognosis are tools that can estimate an absolute risk or probability that an outcome will occur.

Objectives

We sought to systematically review the existing literature on CPMs for TSCI and critically examine the predictor selection methods used.

Methods

We searched MEDLINE, PubMed, Embase, Scopus, and IEEE for English peer-reviewed studies and relevant references that developed multivariable CPMs to prognosticate patient-centered outcomes in adults with TSCI. Using narrative synthesis, we summarized the characteristics of the included studies and their CPMs, focusing on the predictor selection process.

Results

We screened 663 titles and abstracts; of these, 21 full-text studies (2009-2020) consisting of 33 distinct CPMs were included. The data analysis domain was most commonly at a high risk of bias when assessed for methodological quality. Model presentation formats were inconsistently included with published CPMs; only two studies followed established guidelines for transparent reporting of multivariable prediction models. Authors frequently cited previous literature for their initial selection of predictors, and stepwise selection was the most frequent predictor selection method during modelling.

Conclusion

Prediction modelling studies for TSCI serve clinicians who counsel patients, researchers aiming to risk-stratify participants for clinical trials, and patients coping with their injury. Poor methodological rigor in data analysis, inconsistent transparent reporting, and a lack of model presentation formats are vital areas for improvement in TSCI CPM research.

Risk Prediction Models for Long-Term Survival after Cardiac Surgery: A Systematic Review

BACKGROUND

The reporting of alternative postoperative measures of quality after cardiac surgery is becoming increasingly important as in-hospital mortality rates continue to decline. This study aims to systematically review and assess risk models designed to predict long-term outcomes after cardiac surgery.

METHODS

The MEDLINE and Embase databases were searched for articles published between 1990 and 2020. Studies developing or validating risk prediction models for long-term outcomes after cardiac surgery were included. Data were extracted using checklists for critical appraisal and systematic review of prediction modeling studies.

RESULTS

Eleven studies were identified for inclusion in the review, of which nine studies described the development of long-term risk prediction models after cardiac surgery and two were external validation studies. A total of 70 predictors were included across the nine models. The most frequently used predictors were age (n = 9), peripheral vascular disease (n = 8), renal disease (n = 8), and pulmonary disease (n = 8). Despite all models demonstrating acceptable performance on internal validation, only two models underwent external validation, both of which performed poorly.

CONCLUSION

Nine risk prediction models predicting long-term mortality after cardiac surgery have been identified in this review. Statistical issues with model development, limited inclusion of outcomes beyond 5 years of follow-up, and a lack of external validation studies means that none of the models identified can be recommended for use in contemporary cardiac surgery. Further work is needed either to successfully externally validate existing models or to develop new models. Newly developed models should aim to use standardized long-term specific reproducible outcome measures.

Prognostic models for clinical outcomes in patients with venous leg ulcers: A systematic review

OBJECTIVE

The purpose of this review was to identify prognostic models for clinical application in patients with venous leg ulcers (VLUs).

METHODS

Literature searches were conducted in Embase, Medline, Cochrane, and CINAHL databases from inception to December 22, 2021. Eligible studies reported prognostic models aimed at developing, validating, and adjusting multivariable prognostic models that include multiple prognostic factors combined, and that predicted clinical outcomes. Methodological quality was assessed using the CHARMS checklist and PROBAST short form questionnaire.

RESULTS

Thirteen studies were identified, of which three were validation studies of previously published models, four reported derivation and validation of models, and the remainder reported derivation models only. There was substantial heterogeneity in the model characteristics, including 11 studies focused on wound healing outcomes reporting 91 different predictors. Three studies shared similar predicted outcomes, follow-up timepoint and used a Cox proportional hazards model. However, these models reported different predictor selection methods and different predictors and it was therefore not feasible to summarize performance, such as discriminative ability.

CONCLUSIONS

There are no standout risk prediction models in the literature with promising clinical application for patients with VLUs. Future research should focus on developing and validating high-performing models in wider VLU populations.

Clinical prediction models for aneurysmal subarachnoid hemorrhage: a systematic review update

BACKGROUND

A systematic review of clinical prediction models for aneurysmal subarachnoid hemorrhage (aSAH) reported in 2011 noted that clinical prediction models for aSAH were developed using poor methods and were not externally validated. This study aimed to update the above review to guide the future development of predictive models in aSAH.

METHODS

We systematically searched Embase and MEDLINE databases (January 2010 to February 2022) for articles that reported the development of a clinical prediction model to predict functional outcomes in aSAH. Our reviews are based on the items included in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement (PRISMA) checklist, and on data abstracted from each study in accord with the Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) 2014 checklist. Bias and applicability were assessed using the Prediction model Risk Of Bias Assessment Tool (PROBAST).

RESULTS

We reviewed data on 30 466 patients contributing to 29 prediction models abstracted from 22 studies identified from an initial search of 7858 studies. Most models were developed using logistic regression (n=20) or machine learning (n=9) with prognostic variables selected through a range of methods. Age (n=13), World Federation of Neurological Surgeons (WFNS) grade (n=11), hypertension (n=6), aneurysm size (n=5), Fisher grade (n=12), Hunt and Hess score (n=5), and Glasgow Coma Scale (n=8) were the variables most frequently included in the reported models. External validation was performed in only four studies. All but one model had a high or unclear risk of bias due to poor performance or lack of validation.

CONCLUSION

Externally validated models for the prediction of functional outcome in aSAH patients have now become available. However, most of them still have a high risk of bias.

Postprandial plasma GLP-1 levels are elevated in individuals with postprandial hypoglycaemia following Roux-en-Y gastric bypass - a systematic review

BACKGROUND AND AIMS

Bariatric surgery is the most effective treatment in individuals with obesity to achieve remission of type 2 diabetes. Post-bariatric surgery hypoglycaemia occurs frequently, and management remains suboptimal, because of a poor understanding of the underlying pathophysiology. The glucoregulatory hormone responses to nutrients in individuals with and without post-bariatric surgery hypoglycaemia have not been systematically examined.

MATERIALS AND METHODS

The study protocol was prospectively registered with PROSPERO. PubMed, EMBASE, Web of Science and the Cochrane databases were searched for publications between January 1990 and November 2021 using MeSH terms related to post-bariatric surgery hypoglycaemia. Studies were included if they evaluated individuals with post-bariatric surgery hypoglycaemia and included measurements of plasma glucagon-like peptide-1 (GLP-1), glucose-dependent insulinotropic polypeptide (GIP), insulin, C-peptide and/or glucagon concentrations following an ingested nutrient load. Glycated haemoglobin (HbA1c) was also evaluated. A random-effects meta-analysis was performed, and Hedges' g (standardised mean difference) and 95% confidence intervals were reported for all outcomes where sufficient studies were available. The τ2 estimate and I2 statistic were used as tests for heterogeneity and a funnel plot with the Egger regression-based test was used to evaluate for publication bias.

RESULTS

From 377 identified publications, 12 were included in the analysis. In all 12 studies, the type of bariatric surgery was Roux-en-Y gastric bypass (RYGB). Comparing individuals with and without post-bariatric surgery hypoglycaemia following an ingested nutrient load, the standardised mean difference in peak GLP-1 was 0.57 (95% CI, 0.32, 0.82), peak GIP 0.05 (-0.26, 0.36), peak insulin 0.84 (0.44, 1.23), peak C-peptide 0.69 (0.28, 1.1) and peak glucagon 0.05 (-0.26, 0.36). HbA1c was less in individuals with hypoglycaemia - 0.40 (-0.67, -0.12). There was no evidence of substantial heterogeneity in any outcome except for peak insulin: τ2 = 0.2, I2 = 54.3. No publication bias was evident.

CONCLUSION

Following RYGB, postprandial peak plasma GLP-1, insulin and C-peptide concentrations are greater in individuals with post-bariatric surgery hypoglycaemia, while HbA1c is less. These observations support the concept that antagonism of GLP-1 would prove beneficial in the management of individuals with hypoglycaemia following RYGB.PROSPERO Registration Number: CRD42021287515.

Diagnostic clinical prediction rules for categorising low back pain: A systematic review

BACKGROUND

Low back pain (LBP) is a common complex condition, where specific diagnoses are hard to identify. Diagnostic clinical prediction rules (CPRs) are known to improve clinical decision-making. A review of LBP diagnostic-CPRs by Haskins et al. (2015) identified six diagnostic-CPRs in derivation phases of development, with one tool ready for implementation. Recent progress on these tools is unknown. Therefore, this review aimed to investigate developments in LBP diagnostic-CPRs and evaluate their readiness for implementation.

METHODS

A systematic review was performed on five databases (Medline, Amed, Cochrane Library, PsycInfo, and CINAHL) combined with hand-searching and citation-tracking to identify eligible studies. Study and tool quality were appraised for risk of bias (Quality Assessment of Diagnostic Accuracy Studies-2), methodological quality (checklist using accepted CPR methodological standards), and CPR tool appraisal (GRade and ASsess Predictive).

RESULTS

Of 5021 studies screened, 11 diagnostic-CPRs were identified. Of the six previously known, three have been externally validated but not yet undergone impact analysis. Five new tools have been identified since Haskin et al. (2015); all are still in derivation stages. The most validated diagnostic-CPRs include the Lumbar-Spinal-Stenosis-Self-Administered-Self-Reported-History-Questionnaire and Diagnosis-Support-Tool-to-Identify-Lumbar-Spinal-Stenosis, and the StEP-tool which differentiates radicular from axial-LBP.

CONCLUSIONS

This updated review of LBP diagnostic CPRs found five new tools, all in the early stages of development. Three previously known tools have now been externally validated but should be used with caution until impact evaluation studies are undertaken. Future funding should focus on externally validating and assessing the impact of existing CPRs on clinical decision-making.

Prognostic models predicting transition to psychotic disorder using blood-based biomarkers: a systematic review and critical appraisal

Accumulating evidence suggests individuals with psychotic disorder show abnormalities in metabolic and inflammatory processes. Recently, several studies have employed blood-based predictors in models predicting transition to psychotic disorder in risk-enriched populations. A systematic review of the performance and methodology of prognostic models using blood-based biomarkers in the prediction of psychotic disorder from risk-enriched populations is warranted. Databases (PubMed, EMBASE and PsycINFO) were searched for eligible texts from 1998 to 15/05/2023, which detailed model development or validation studies. The checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) was used to guide data extraction from eligible texts and the Prediction Model Risk of Bias Assessment Tool (PROBAST) was used to assess the risk of bias and applicability of the studies. A narrative synthesis of the included studies was performed. Seventeen eligible studies were identified: 16 eligible model development studies and one eligible model validation study. A wide range of biomarkers were assessed, including nucleic acids, proteins, metabolites, and lipids. The range of C-index (area under the curve) estimates reported for the models was 0.67-1.00. No studies assessed model calibration. According to PROBAST criteria, all studies were at high risk of bias in the analysis domain. While a wide range of potentially predictive biomarkers were identified in the included studies, most studies did not account for overfitting in model performance estimates, no studies assessed calibration, and all models were at high risk of bias according to PROBAST criteria. External validation of the models is needed to provide more accurate estimates of their performance. Future studies which follow the latest available methodological and reporting guidelines and adopt strategies to accommodate required sample sizes for model development or validation will clarify the value of including blood-based biomarkers in models predicting psychosis.

Prognostic prediction models for clinical outcomes in patients diagnosed with visceral leishmaniasis: protocol for a systematic review

INTRODUCTION

Visceral leishmaniasis (VL) is a neglected tropical disease responsible for many thousands of preventable deaths each year. Symptomatic patients often struggle to access effective treatment, without which death is the norm. Risk prediction tools support clinical teams and policymakers in identifying high-risk patients who could benefit from more intensive management pathways. Investigators interested in using their clinical data for prognostic research should first identify currently available models that are candidates for validation and possible updating. Addressing these needs, we aim to identify, summarise and appraise the available models predicting clinical outcomes in VL patients.

METHODS AND ANALYSIS

We will include studies that have developed, validated or updated prognostic models predicting future clinical outcomes in patients diagnosed with VL. Systematic reviews and meta-analyses that include eligible studies are also considered for review. Conference abstracts and educational theses are excluded. Data extraction, appraisal and reporting will follow current methodological guidelines. Ovid Embase; Ovid MEDLINE; the Web of Science Core Collection, SciELO and LILACS are searched from database inception to 1 March 2023 using terms developed for the identification of prediction models, and with no language restriction. Screening, data extraction and risk of bias assessment will be performed in duplicate with discordance resolved by a third independent reviewer. Risk of bias will be assessed using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). Tables and figures will compare and contrast key model information, including source data, participants, model development and performance measures, and risk of bias. We will consider the strengths, limitations and clinical applicability of the identified models.

ETHICS AND DISSEMINATION

Ethics approval is not required for this review. The systematic review and all accompanying data will be submitted to an open-access journal. Findings will also be disseminated through the research group's website (www.iddo.org/research-themes/visceral-leishmaniasis) and social media channels.

PROSPERO REGISTRATION NUMBER

CRD42023417226.

Prognostic prediction models for oropharyngeal squamous cell carcinoma (OPSCC): a protocol for systematic review, critical appraisal and meta-analysis

INTRODUCTION

Oropharyngeal squamous cell carcinoma (OPSCC) is increasingly prevalent and has significantly heterogeneous risks of survival for diagnosed individuals due to the inter-related risk factors. Precise prediction of the risk of survival for an individual patient with OPSCC presents a useful adjunct to therapeutic decision-making regarding the management of OPSCC. The aim of this systematic review, critical appraisal and meta-analysis is to assess prognostic prediction models for OPSCC and lay a foundation for future research programmes to develop and validate prognostic prediction models for OPSCC.

METHODS AND ANALYSIS

This protocol will follow the Preferred Reporting Items for Systematic Review and Meta-Analyses Protocol statement. Based on predefined criteria, electronic databases including MEDLINE, Embase, Web of Science, the Cochrane Library and China National Knowledge Infrastructure (CNKI) will be searched for relevant studies without language restrictions from inception of databases to present. This study will systematically review published prognostic prediction models for survival outcomes in patients with OPSCC, describe their characteristics, compare performance and assess risk of bias and real-world clinical utility. Selection of eligible studies, data extraction and critical appraisal will be conducted independently by two reviewers. A third reviewer will resolve any disagreements. Included studies will be systematically summarised using appropriate tools designed for prognostic prediction modelling studies. Risk of bias and quality of studies will be assessed using the Prediction Model Risk of Bias Assessment Tool and the Transparent Reporting of a multivariable prediction model for individual prognosis or diagnosis. Performance measures of these models will be pooled and analysed with meta-analyses if feasible.

ETHICS AND DISSEMINATION

This review will be conducted completely based on published data, so approval from an ethics committee or written consent is not required. The results will be disseminated through a peer-reviewed publication.

PROSPERO REGISTRATION NUMBER

CRD42023400272.

The performance of FIT-based and other risk prediction models for colorectal neoplasia in symptomatic patients: a systematic review

Background

Colorectal cancer (CRC) incidence and mortality are increasing internationally. Endoscopy services are under significant pressure with many overwhelmed. Faecal immunochemical testing (FIT) has been advocated to identify a high-risk population of symptomatic patients requiring definitive investigation by colonoscopy. Combining FIT with other factors in a risk prediction model could further improve performance in identifying those requiring investigation most urgently. We systematically reviewed performance of models predicting risk of CRC and/or advanced colorectal polyps (ACP) in symptomatic patients, with a particular focus on those models including FIT.

Methods

The review protocol was published on PROSPERO (CRD42022314710). Searches were conducted from database inception to April 2023 in MEDLINE, EMBASE, Cochrane libraries, SCOPUS and CINAHL. Risk of bias of each study was assessed using The Prediction study Risk Of Bias Assessment Tool. A narrative synthesis based on the guidelines for Synthesis Without Meta-Analysis was performed due to study heterogeneity.

Findings

We included 62 studies; 23 included FIT (n = 22) or guaiac Faecal Occult Blood Testing (n = 1) combined with one or more other variables. Twenty-one studies were conducted solely in primary care. Generally, prediction models including FIT consistently had good discriminatory ability for CRC/ACP (i.e. AUC >0.8) and performed better than models without FIT although some models without FIT also performed well. However, many studies did not present calibration and internal and external validation were limited. Two studies were rated as low risk of bias; neither model included FIT.

Interpretation

Risk prediction models, including and not including FIT, show promise for identifying those most at risk of colorectal neoplasia. Substantial limitations in evidence remain, including heterogeneity, high risk of bias, and lack of external validation. Further evaluation in studies adhering to gold standard methodology, in appropriate populations, is required before widespread adoption in clinical practice.

Funding

National Institute for Health and Care Research (NIHR) [Health Technology Assessment Programme (HTA) Programme (Project number 133852).

Predictive accuracy of risk prediction models for recurrence, metastasis and survival for early-stage cutaneous melanoma: a systematic review

OBJECTIVES

To identify prognostic models for melanoma survival, recurrence and metastasis among American Joint Committee on Cancer stage I and II patients postsurgery; and evaluate model performance, including overall survival (OS) prediction.

DESIGN

Systematic review and narrative synthesis.

DATA SOURCES

Searched MEDLINE, Embase, CINAHL, Cochrane Library, Science Citation Index and grey literature sources including cancer and guideline websites from 2000 to September 2021.

ELIGIBILITY CRITERIA

Included studies on risk prediction models for stage I and II melanoma in adults ≥18 years. Outcomes included OS, recurrence, metastases and model performance. No language or country of publication restrictions were applied.

DATA EXTRACTION AND SYNTHESIS

Two pairs of reviewers independently screened studies, extracted data and assessed the risk of bias using the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies checklist and the Prediction study Risk of Bias Assessment Tool. Heterogeneous predictors prevented statistical synthesis.

RESULTS

From 28 967 records, 15 studies reporting 20 models were included; 8 (stage I), 2 (stage II), 7 (stages I-II) and 7 (stages not reported), but were clearly applicable to early stages. Clinicopathological predictors per model ranged from 3-10. The most common were: ulceration, Breslow thickness/depth, sociodemographic status and site. Where reported, discriminatory values were ≥0.7. Calibration measures showed good matches between predicted and observed rates. None of the studies assessed clinical usefulness of the models. Risk of bias was high in eight models, unclear in nine and low in three. Seven models were internally and externally cross-validated, six models were externally validated and eight models were internally validated.

CONCLUSIONS

All models are effective in their predictive performance, however the low quality of the evidence raises concern as to whether current follow-up recommendations following surgical treatment is adequate. Future models should incorporate biomarkers for improved accuracy.

PROSPERO REGISTRATION NUMBER

CRD42018086784.

Prognostic models for predicting clinical disease progression, worsening and activity in people with multiple sclerosis

BACKGROUND

Multiple sclerosis (MS) is a chronic inflammatory disease of the central nervous system that affects millions of people worldwide. The disease course varies greatly across individuals and many disease-modifying treatments with different safety and efficacy profiles have been developed recently. Prognostic models evaluated and shown to be valid in different settings have the potential to support people with MS and their physicians during the decision-making process for treatment or disease/life management, allow stratified and more precise interpretation of interventional trials, and provide insights into disease mechanisms. Many researchers have turned to prognostic models to help predict clinical outcomes in people with MS; however, to our knowledge, no widely accepted prognostic model for MS is being used in clinical practice yet.

OBJECTIVES

To identify and summarise multivariable prognostic models, and their validation studies for quantifying the risk of clinical disease progression, worsening, and activity in adults with MS.

SEARCH METHODS

We searched MEDLINE, Embase, and the Cochrane Database of Systematic Reviews from January 1996 until July 2021. We also screened the reference lists of included studies and relevant reviews, and references citing the included studies.

SELECTION CRITERIA

We included all statistically developed multivariable prognostic models aiming to predict clinical disease progression, worsening, and activity, as measured by disability, relapse, conversion to definite MS, conversion to progressive MS, or a composite of these in adult individuals with MS. We also included any studies evaluating the performance of (i.e. validating) these models. There were no restrictions based on language, data source, timing of prognostication, or timing of outcome.

DATA COLLECTION AND ANALYSIS

Pairs of review authors independently screened titles/abstracts and full texts, extracted data using a piloted form based on the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS), assessed risk of bias using the Prediction Model Risk Of Bias Assessment Tool (PROBAST), and assessed reporting deficiencies based on the checklist items in Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD). The characteristics of the included models and their validations are described narratively. We planned to meta-analyse the discrimination and calibration of models with at least three external validations outside the model development study but no model met this criterion. We summarised between-study heterogeneity narratively but again could not perform the planned meta-regression.

MAIN RESULTS

We included 57 studies, from which we identified 75 model developments, 15 external validations corresponding to only 12 (16%) of the models, and six author-reported validations. Only two models were externally validated multiple times. None of the identified external validations were performed by researchers independent of those that developed the model. The outcome was related to disease progression in 39 (41%), relapses in 8 (8%), conversion to definite MS in 17 (18%), and conversion to progressive MS in 27 (28%) of the 96 models or validations. The disease and treatment-related characteristics of included participants, and definitions of considered predictors and outcome, were highly heterogeneous amongst the studies. Based on the publication year, we observed an increase in the percent of participants on treatment, diversification of the diagnostic criteria used, an increase in consideration of biomarkers or treatment as predictors, and increased use of machine learning methods over time. Usability and reproducibility All identified models contained at least one predictor requiring the skills of a medical specialist for measurement or assessment. Most of the models (44; 59%) contained predictors that require specialist equipment likely to be absent from primary care or standard hospital settings. Over half (52%) of the developed models were not accompanied by model coefficients, tools, or instructions, which hinders their application, independent validation or reproduction. The data used in model developments were made publicly available or reported to be available on request only in a few studies (two and six, respectively). Risk of bias We rated all but one of the model developments or validations as having high overall risk of bias. The main reason for this was the statistical methods used for the development or evaluation of prognostic models; we rated all but two of the included model developments or validations as having high risk of bias in the analysis domain. None of the model developments that were externally validated or these models' external validations had low risk of bias. There were concerns related to applicability of the models to our research question in over one-third (38%) of the models or their validations. Reporting deficiencies Reporting was poor overall and there was no observable increase in the quality of reporting over time. The items that were unclearly reported or not reported at all for most of the included models or validations were related to sample size justification, blinding of outcome assessors, details of the full model or how to obtain predictions from it, amount of missing data, and treatments received by the participants. Reporting of preferred model performance measures of discrimination and calibration was suboptimal.

AUTHORS' CONCLUSIONS

The current evidence is not sufficient for recommending the use of any of the published prognostic prediction models for people with MS in clinical routine today due to lack of independent external validations. The MS prognostic research community should adhere to the current reporting and methodological guidelines and conduct many more state-of-the-art external validation studies for the existing or newly developed models.

Evidence of questionable research practices in clinical prediction models

BACKGROUND

Clinical prediction models are widely used in health and medical research. The area under the receiver operating characteristic curve (AUC) is a frequently used estimate to describe the discriminatory ability of a clinical prediction model. The AUC is often interpreted relative to thresholds, with "good" or "excellent" models defined at 0.7, 0.8 or 0.9. These thresholds may create targets that result in "hacking", where researchers are motivated to re-analyse their data until they achieve a "good" result.

METHODS

We extracted AUC values from PubMed abstracts to look for evidence of hacking. We used histograms of the AUC values in bins of size 0.01 and compared the observed distribution to a smooth distribution from a spline.

RESULTS

The distribution of 306,888 AUC values showed clear excesses above the thresholds of 0.7, 0.8 and 0.9 and shortfalls below the thresholds.

CONCLUSIONS

The AUCs for some models are over-inflated, which risks exposing patients to sub-optimal clinical decision-making. Greater modelling transparency is needed, including published protocols, and data and code sharing.

The Prognostic Utility of Anticholinergic Burden Scales: An Integrative Review and Gap Analysis

BACKGROUND

Anticholinergic drugs are commonly prescribed, especially to older adults. Anticholinergic burden scales (ABS) have been used to evaluate the cumulative effects of multiple anticholinergics. However, studies have shown inconsistent results regarding the association between anticholinergic burden assessed with ABS and adverse clinical outcomes such as cognitive impairment, functional decline, and frailty. This review aims to identify gaps in research on the development, validation, and evaluation of ABS, and provide recommendations for future studies.

METHOD

A comprehensive search of five databases (MEDLINE, Embase, PsychInfo, CINAHL, CENTRAL) was conducted for relevant studies published from inception until 25 May 2023. Two reviewers screened for eligibility and assessed the quality of studies using different tools based on the study design and stage of the review framework. Research evidence was evaluated, and gaps were identified and grouped into evidence, knowledge, and methodological gaps, using evidence tables to summarize data.

RESULTS

Several evidence, knowledge, and methodological gaps in existing development, validation, and evaluation studies of ABS were identified. There is no universally accepted scale, and there is a need to define a clinically relevant threshold for measuring total anticholinergic burden. The current evidence has limitations, underrepresenting low- and middle-income countries, younger individuals, and populations with cognitive disabilities. The impact of anticholinergic burden on frailty is also understudied. Existing evaluation studies provide limited evidence on the benefit of reducing anticholinergic burden on clinical outcomes or the safety of anticholinergic deprescribing. There is also uncertainty regarding optimal reduction, clinically significant anticholinergic burden thresholds, and cost effectiveness.

CONCLUSIONS

Future research recommendations to bridge knowledge gaps include developing a risk assessment framework, refining ABS scales, establishing a standardized consensus scale, and creating a longitudinal measure of cumulative anticholinergic risk. Strategies to minimize bias, consider frailty, and promote multidisciplinary and multinational collaborations are also necessary to improve patient outcomes.

Prognostic factors associated with failure of total elbow replacement: a protocol for a systematic review

INTRODUCTION

Total elbow replacement (TER) has higher failure rates requiring revision surgery compared with the replacement of other joints. Understanding the factors associated with failure is essential for informed decision-making between patients and clinicians, and for reducing the failure rate. This review aims to identify, describe and appraise the literature examining prognostic factors for failure of TER.

METHODS AND ANALYSIS

This systematic review will be conducted and reported in line with the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols guidelines. Electronic literature searches will be conducted using Medline, EMBASE, PubMed and Cochrane. The search strategy will be broad, including a combination of subject headings (MESH) and free text search. This search will be supplemented with a screening of reference lists of the included studies and relevant reviews. Two independent reviewers will screen all search results in two stages (title and abstract, and full text) based on the Population, Index prognostic factor, Comparator prognostic factor, Outcome, Time and Setting criteria. The types of evidence included will be randomised trials, non-randomised trials, prospective and retrospective cohort studies, registry studies and case-control studies. If the literature lacks enough studies, then case series with 50 or more TERs will be considered for inclusion. Data extraction and risk of bias assessment for included studies will be performed by two independent reviewers using the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies for Prognostic Factors and Quality In Prognostic Studies tools.Meta-analyses of prognostic estimates for each factor will be undertaken for studies that are deemed to be sufficiently robust and comparable. Several challenges are likely to arise due to heterogeneity between studies, therefore, subgroup and sensitivity analyses will be performed to account for the differences between studies. Heterogeneity will be assessed using Q and I2 statistics. If I2>40% then pooled estimates will not be reported. When quantitative synthesis is not possible, a narrative synthesis will be undertaken. The quality of the evidence for each prognostic factor will be assessed using the Grades of Recommendation Assessment, Development and Evaluation tool.

PROSPERO REGISTRATION NUMBER

CRD42023384756.

Prognostic models for predicting postoperative recurrence in Crohn's disease: a systematic review and critical appraisal

Background and Aims

Prophylaxis of postoperative recurrence is an intractable problem for clinicians and patients with Crohn's disease. Prognostic models are effective tools for patient stratification and personalised management. This systematic review aimed to provide an overview and critically appraise the existing models for predicting postoperative recurrence of Crohn's disease.

Methods

Systematic retrieval was performed using PubMed and Web of Science in January 2022. Original articles on prognostic models for predicting postoperative recurrence of Crohn's disease were included in the analysis. The risk of bias was assessed using the Prediction Model Risk of Bias Assessment (PROBAST) tool. This study was registered with the International Prospective Register of Systematic Reviews (PROSPERO; number CRD42022311737).

Results

In total, 1948 articles were screened, of which 15 were ultimately considered. Twelve studies developed 15 new prognostic models for Crohn's disease and the other three validated the performance of three existing models. Seven models utilised regression algorithms, six utilised scoring indices, and five utilised machine learning. The area under the receiver operating characteristic curve of the models ranged from 0.51 to 0.97. Six models showed good discrimination, with an area under the receiver operating characteristic curve of >0.80. All models were determined to have a high risk of bias in modelling or analysis, while they were at low risk of applicability concerns.

Conclusions

Prognostic models have great potential for facilitating the assessment of postoperative recurrence risk in patients with Crohn's disease. Existing prognostic models require further validation regarding their reliability and applicability.

Systematic review registration

https://www.crd.york.ac.uk/PROSPERO/, identifier CRD42022311737.

Prognostic Value of Coronary Computed Tomography Angiography in Coronary Artery Bypass Graft Patients Systematic Review and Meta-Analysis

We sought to assess the prognostic value of coronary computed tomographic angiography (CCTA) in patients with coronary artery bypass graft (CABG) by meta-analysis. MEDLINE, Embase, Cochrane Central Register of Controlled Trials, and Scopus were searched for relevant original articles published up to July 2021. CCTA prognostic studies enrolling patients with CABG were screened and included if outcomes included all-cause mortality or major adverse cardiac events. Maximally adjusted hazard ratios (HRs) were extracted for CCTA-derived prognostic factors. HRs were log-transformed and pooled across studies using the DerSimonian-Laird random-effects model and statistical heterogeneity was assessed using the I² statistic. Of 1,576 screened articles, 4 retrospective studies fulfilled all inclusion criteria. Collectively, a total of 1,809 patients with CABG underwent CCTA (mean [SD] age 67.0 [8.5] years across 3 studies, 81.5% male across 4 studies). Coronary artery disease severity and revascularization were categorized using 2 models: unprotected coronary territories and coronary artery protection score. The pooled HRs from the random-effects models using the most highly adjusted study estimate were 3.64 (95% confidence interval 2.48 to 5.34, I² = 57.8%, p <0.001; 4 studies) and 4.85 (95% confidence interval 3.17 to 7.43, I² = 39.9%, p <0.001; 2 studies) for unprotected coronary territories and coronary artery protection score, respectively. In conclusion, in a limited number of studies, CCTA is an independent predictor of adverse events in patients with CABG. Larger studies using uniform models and endpoints are needed.

Outcome prediction models incorporating clinical variables for Head and Neck Squamous cell Carcinoma: A systematic review of methodological conduct and risk of bias

Multiple outcome prediction models have been developed for Head and Neck Squamous Cell Carcinoma (HNSCC). This systematic review aimed to identify HNSCC outcome prediction model studies, assess their methodological quality and identify those with potential utility for clinical practice. Inclusion criteria were mucosal HNSCC prognostic prediction model studies (development or validation) incorporating clinically available variables accessible at time of treatment decision making and predicting tumour-related outcomes. Eligible publications were identified from PubMed and Embase. Methodological quality and risk of bias were assessed using the checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS) and prediction model risk of bias assessment tool (PROBAST). Eligible publications were categorised by study type for reporting. 64 eligible publications were identified; 55 reported model development, 37 external validations, with 28 reporting both. CHARMS checklist items relating to participants, predictors, outcomes, handling of missing data, and some model development and evaluation procedures were generally well-reported. Less well-reported were measures accounting for model overfitting and model performance measures, especially model calibration. Full model information was poorly reported (3/55 model developments), specifically model intercept, baseline survival or full model code. Most publications (54/55 model developments, 28/37 external validations) were found to have high risk of bias, predominantly due to methodological issues in the PROBAST analysis domain. The identified methodological issues may affect prediction model accuracy in heterogeneous populations. Independent external validation studies in the local population and demonstration of clinical impact are essential for the clinical implementation of outcome prediction models.

What proportion of clinical prediction models make it to clinical practice? Protocol for a two-track follow-up study of prediction model development publications

INTRODUCTION

It is known that only a limited proportion of developed clinical prediction models (CPMs) are implemented and/or used in clinical practice. This may result in a large amount of research waste, even when considering that some CPMs may demonstrate poor performance. Cross-sectional estimates of the numbers of CPMs that have been developed, validated, evaluated for impact or utilized in practice, have been made in specific medical fields, but studies across multiple fields and studies following up the fate of CPMs are lacking.

METHODS AND ANALYSIS

We have conducted a systematic search for prediction model studies published between January 1995 and December 2020 using the Pubmed and Embase databases, applying a validated search strategy. Taking random samples for every calendar year, abstracts and articles were screened until a target of 100 CPM development studies were identified. Next, we will perform a forward citation search of the resulting CPM development article cohort to identify articles on external validation, impact assessment or implementation of those CPMs. We will also invite the authors of the development studies to complete an online survey to track implementation and clinical utilization of the CPMs.We will conduct a descriptive synthesis of the included studies, using data from the forward citation search and online survey to quantify the proportion of developed models that are validated, assessed for their impact, implemented and/or used in patient care. We will conduct time-to-event analysis using Kaplan-Meier plots.

ETHICS AND DISSEMINATION

No patient data are involved in the research. Most information will be extracted from published articles. We request written informed consent from the survey respondents. Results will be disseminated through publication in a peer-reviewed journal and presented at international conferences. OSF REGISTRATION: (https://osf.io/nj8s9).

Prognostic models of quality of life after total knee replacement: A systematic review

OBJECTIVE

To systematically review and critically appraise prognostic models for quality of life (QOL) in patients with total knee replacement (TKA).

METHODS

Subjects were TKA recipients recruited from inpatient postoperative settings. Searches were made on June 2022 and updated on April 2023. Databases included PubMed.gov, CINAHL, The Cochrane Library, Web of Science. Two authors performed all review stages independently. Risk of bias assessments on participants, predictors, outcomes and analysis methods followed the Prediction study Risk Of Bias ASsessment Tool (PROBAST).

RESULTS

After screening 2204 studies, 9 were eligible for inclusion. Twelve prognostic models were reported, of which 10 models were developed from data without validation and 2 were both developed and validated. The most frequently applied predictor was the pre-TKA QOL score. Discriminatory measures were reported for 9 (75.0%) models with areas under the curve values of 0.66-0.95. All models showed a high risk of bias, mostly due to limitations in statistical methods and outcome assessments.

CONCLUSION

Several prognostic models have been developed for QOL in patients with TKA, but all models show a high risk of bias and are unreliable in clinical practice. Future, prognostic models overcoming the risk of bias identified in this study are needed.

Deep Learning to Refine the Identification of High-Quality Clinical Research Articles from the Biomedical Literature: Performance Evaluation

BACKGROUND

Identifying practice-ready evidence-based journal articles in medicine is a challenge due to the sheer volume of biomedical research publications. Newer approaches to support evidence discovery apply deep learning techniques to improve the efficiency and accuracy of classifying sound evidence.

OBJECTIVE

To determine how well deep learning models using variants of Bidirectional Encoder Representations from Transformers (BERT) identify high-quality evidence with high clinical relevance from the biomedical literature for consideration in clinical practice.

METHODS

We fine-tuned variations of BERT models (BERT_BASE, BioBERT, BlueBERT, and PubMedBERT) and compared their performance in classifying articles based on methodological quality criteria. The dataset used for fine-tuning models included titles and abstracts of >160,000 PubMed records from 2012-2020 that were of interest to human health which had been manually labeled based on meeting established critical appraisal criteria for methodological rigor. The data was randomly divided into 80:10:10 sets for training, validating, and testing. In addition to using the full unbalanced set, the training data was randomly undersampled into four balanced datasets to assess performance and select the best performing model. For each of the four sets, one model that maintained sensitivity (recall) at ≥99% was selected and were ensembled. The best performing model was evaluated in a prospective, blinded test and applied to an established reference standard, the Clinical Hedges dataset.

RESULTS

In training, three of the four selected best performing models were trained using BioBERT_BASE. The ensembled model did not boost performance compared with the best individual model. Hence a solo BioBERT-based model (named DL-PLUS) was selected for further testing as it was computationally more efficient. The model had high recall (>99%) and 60% to 77% specificity in a prospective evaluation conducted with blinded research associates and saved >60% of the work required to identify high quality articles.

CONCLUSIONS

Deep learning using pretrained language models and a large dataset of classified articles produced models with improved specificity while maintaining >99% recall. The resulting DL-PLUS model identifies high-quality, clinically relevant articles from PubMed at the time of publication. The model improves the efficiency of a literature surveillance program, which allows for faster dissemination of appraised research.

Barriers and facilitators of weight bearing after hip fracture surgery among older adults. A scoping review

PURPOSE

This scoping review aimed to synthesise the available evidence on barriers and facilitators of weight bearing after hip fracture surgery in older adults.

METHODS

Published (Cochrane Central, MEDLINE, EMBASE, CINAHL, and PEDro) and unpublished (Global Health, EThOS, WorldCat dissertation and thesis, ClinicalTrials.gov , OpenAIRE, DART-Europe) evidence was electronically searched from database inception to 29 March 2022. Barriers and facilitators of weight bearing were extracted and synthesised into patient, process (non-surgical), process (surgical), and structure-related barriers/facilitators using a narrative review approach.

RESULTS

In total, 5594 were identified from the primary search strategy, 1314 duplicates were removed, 3769 were excluded on title and abstract screening, and 442 were excluded on full-text screening. In total, 69 studies (all from published literature sources) detailing 47 barriers and/or facilitators of weight bearing were included. Of barriers/facilitators identified, 27 were patient-, 8 non-surgical process-, 8 surgical process-, and 4 structure-related. Patient facilitators included anticoagulant, home discharge, and aid at discharge. Barriers included preoperative dementia and delirium, postoperative delirium, pressure sores, indoor falls, ventilator dependence, haematocrit < 36%, systemic sepsis, and acute renal failure. Non-surgical process facilitators included early surgery, early mobilisation, complete medical co-management, in-hospital rehabilitation, and patient-recorded nurses' notes. Barriers included increased operative time and standardised hip fracture care. Surgical process facilitators favoured intramedullary fixations and arthroplasty over extramedullary fixation. Structure facilitators favoured more recent years and different healthcare systems. Barriers included pre-holiday surgery and admissions in the first quarter of the year.

CONCLUSION

Most patient/surgery-related barriers/facilitators may inform future risk stratification. Future research should examine additional process/structure barriers and facilitators amenable to intervention. Furthermore, patient barriers/facilitators need to be investigated by replicating the studies identified and augmenting them with more specific details on weight bearing outcomes.

Clinical First-Trimester Prediction Models for Gestational Diabetes Mellitus: A Systematic Review and Meta-Analysis

BACKGROUND

Gestational diabetes mellitus (GDM) is a common pregnancy complication that negatively impacts the health of both the mother and child. Early prediction of the risk of GDM may permit prompt and effective interventions. This systematic review and meta-analysis aimed to summarize the study characteristics, methodological quality, and model performance of first-trimester prediction model studies for GDM.

METHODS

Five electronic databases, one clinical trial register, and gray literature were searched from the inception date to March 19, 2022. Studies developing or validating a first-trimester prediction model for GDM were included. Two reviewers independently extracted data according to an established checklist and assessed the risk of bias by the Prediction Model Risk of Bias Assessment Tool (PROBAST). We used a random-effects model to perform a quantitative meta-analysis of the predictive power of models that were externally validated at least three times.

RESULTS

We identified 43 model development studies, six model development and external validation studies, and five external validation-only studies. Body mass index, maternal age, and fasting plasma glucose were the most commonly included predictors across all models. Multiple estimates of performance measures were available for eight of the models. Summary estimates range from 0.68 to 0.78 (I² ranged from 0% to 97%).

CONCLUSION

Most studies were assessed as having a high overall risk of bias. Only eight prediction models for GDM have been externally validated at least three times. Future research needs to focus on updating and externally validating existing models.

Performance of HIV Infection Prediction Models in Men Who Have Sex with Men: A Systematic Review and Meta-Analysis

Effective ways to identify and predict men who have sex with men (MSM) at substantial risk for HIV is a global priority. HIV risk assessment tools can improve individual risk awareness and subsequent health-seeking actions. We sought to identify and characterize the performance of HIV infection risk prediction models in MSM through systematic review and meta-analysis. PubMed, Embase, and The Cochrane Library were searched. Eighteen HIV infection risk assessment models with a total of 151,422 participants and 3643 HIV cases were identified, eight of which have been externally validated by at least one study (HIRI-MSM, Menza Score, SDET Score, Li Model, DHRS, Amsterdam Score, SexPro model, and UMRSS). The number of predictor variables in each model ranged from three to 12, age, the number of male sexual partners, unprotected receptive anal intercourse, recreational drug usage (amphetamines, poppers), and sexually transmitted infections were critical scoring variables. All eight externally validated models performed well in terms of discrimination, with the pooled area under the receiver operating characteristic curve (AUC) ranging from 0.62 (95%CI: 0.51 to 0.73, SDET Score) to 0.83 (95%CI: 0.48 to 0.99, Amsterdam Score). Calibration performance was only reported in 10 studies (35.7%, 10/28). The HIV infection risk prediction models showed moderate-to-good discrimination performance. Validation of prediction models across different geographic and ethnic environments is needed to ensure their real-world application.

Area under the curve-optimized synthesis of prediction models from a meta-analytical perspective

The number of clinical prediction models sharing the same prediction task has increased in the medical literature. However, evidence synthesis methodologies that use the results of these prediction models have not been sufficiently studied, particularly in the context of meta-analysis settings where only summary statistics are available. In particular, we consider the following situation: we want to predict an outcome Y, that is not included in our current data, while the covariate data are fully available. In addition, the summary statistics from prior studies, which share the same prediction task (i.e., the prediction of Y), are available. This study introduces a new method for synthesizing the summary results of binary prediction models reported in the prior studies using a linear predictor under a distributional assumption between the current and prior studies. The method provides an integrated predictor combining all predictors reported in the prior studies with weights. The vector of the weights is designed to achieve the hypothetical improvement of area under the receiver operating characteristic curve (AUC) on the current available data under a practical situation where there are different sets of covariates in the prior studies. We observe a counterintuitive aspect in typical situations where a part of weight components in the proposed method becomes negative. It implies that flipping the sign of the prediction results reported in each individual study would improve the overall prediction performance. Finally, numerical and real-world data analysis were conducted and showed that our method outperformed conventional methods in terms of AUC.

Performance and usability of pre-operative prediction models for 30-day peri-operative mortality risk: a systematic review

Estimating pre-operative mortality risk may inform clinical decision-making for peri-operative care. However, pre-operative mortality risk prediction models are rarely implemented in routine clinical practice. High predictive accuracy and clinical usability are essential for acceptance and clinical implementation. In this systematic review, we identified and appraised prediction models for 30-day postoperative mortality in non-cardiac surgical cohorts. PubMed and Embase were searched up to December 2022 for studies investigating pre-operative prediction models for 30-day mortality. We assessed predictive performance in terms of discrimination and calibration. Risk of bias was evaluated using a tool to assess the risk of bias and applicability of prediction model studies. To further inform potential adoption, we also assessed clinical usability for selected models. In all, 15 studies evaluating 10 prediction models were included. Discrimination ranged from a c-statistic of 0.82 (MySurgeryRisk) to 0.96 (extreme gradient boosting machine learning model). Calibration was reported in only six studies. Model performance was highest for the surgical outcome risk tool (SORT) and its external validations. Clinical usability was highest for the surgical risk pre-operative assessment system. The SORT and risk quantification index also scored high on clinical usability. We found unclear or high risk of bias in the development of all models. The SORT showed the best combination of predictive performance and clinical usability and has been externally validated in several heterogeneous cohorts. To improve clinical uptake, full integration of reliable models with sufficient face validity within the electronic health record is imperative.

Regression-based prognostic models for functional independence after postacute brain injury rehabilitation are not transportable: a systematic review

BACKGROUND AND OBJECTIVES

To identify and summarize validated multivariable prognostic models for the Functional Independence Measure® (FIM®) at discharge from post-acute inpatient rehabilitation in adults with acquired brain injury (ABI).

METHODS

This review was conducted based on the recommendations of the Cochrane Prognosis Methods Group and adheres to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Three databases were systematically searched in May 2021 and updated in April 2022. Main inclusion criteria were: a) adult patients with ABI, b) validated multivariable prognostic model, c) time of prognostication within 1-week of admission to post-acute rehabilitation, and d) outcome was the FIM® at discharge from post-acute rehabilitation.

RESULTS

The search yielded 3,169 unique articles. Three articles fulfilled the inclusion criteria, accounting for n = 6 internally and n = 2 externally validated prognostic models. Discrimination was estimated as an area under the curve between 0.76 and 0.89. Calibration was deemed to be assessed insufficiently. The included models were judged to be of high risk of bias.

CONCLUSION

Current prognostic models for the FIM® in post-acute rehabilitation for patients with ABI lack the methodological rigor to support clinical use outside the development setting. Future studies addressing functional independence should ensure appropriate model validation and conform to uniform reporting standards for prognosis research.

The prognostic value of elevated neutrophil-lymphocyte ratio for cardiac surgery-associated acute kidney injury: A systematic review and meta-analysis

BACKGROUND

Patients undergoing cardiac surgery are at significant risk of developing postoperative acute kidney injury (AKI). Neutrophil-lymphocyte ratio (NLR) is a widely available inflammatory biomarker which may be of prognostic value in this setting.

METHODS

We conducted a systematic review and meta-analysis of studies reporting associations between perioperative NLR with postoperative AKI. We searched Medline, Embase and the Cochrane Library, without language restriction, from inception to May 2022 for relevant studies. We meta-analysed the reported odds ratios (ORs) with 95% confidence intervals (CIs) for both elevated preoperative and postoperative NLR with risk of postoperative AKI and need for renal replacement therapy (RRT). We conducted a meta-regression to explore inter-study statistical heterogeneity.

RESULTS

Twelve studies involving 10,724 participants undergoing cardiac surgery were included, with eight studies being deemed at high risk of bias using PROBAST modelling. We found statistically significant associations between elevated preoperative NLR and postoperative AKI (OR 1.45, 95% CI 1.18-1.77), as well as postoperative need for RRT (OR 2.37, 95% CI 1.50-3.72). Postoperative NLR measurements were not of prognostic significance.

CONCLUSIONS

Elevated preoperative NLR is a reliable inflammatory biomarker for predicting AKI following cardiac surgery.

Prognosis of adults and children following a first unprovoked seizure

BACKGROUND

Epilepsy is clinically defined as two or more unprovoked epileptic seizures more than 24 hours apart. Given that, a diagnosis of epilepsy can be associated with significant morbidity and mortality, it is imperative that clinicians (and people with seizures and their relatives) have access to accurate and reliable prognostic estimates, to guide clinical practice on the risks of developing further unprovoked seizures (and by definition, a diagnosis of epilepsy) following single unprovoked epileptic seizure.

OBJECTIVES

1. To provide an accurate estimate of the proportion of individuals going on to have further unprovoked seizures at subsequent time points following a single unprovoked epileptic seizure (or cluster of epileptic seizures within a 24-hour period, or a first episode of status epilepticus), of any seizure type (overall prognosis). 2. To evaluate the mortality rate following a first unprovoked epileptic seizure.

SEARCH METHODS

We searched the following databases on 19 September 2019 and again on 30 March 2021, with no language restrictions. The Cochrane Register of Studies (CRS Web), MEDLINE Ovid (1946 to March 29, 2021), SCOPUS (1823 onwards), ClinicalTrials.gov, the World Health Organization (WHO) International Clinical Trials Registry Platform (ICTRP). CRS Web includes randomized or quasi-randomized, controlled trials from PubMed, Embase, ClinicalTrials.gov, the World Health Organization International Clinical Trials Registry Platform (ICTRP), the Cochrane Central Register of Controlled Trials (CENTRAL), and the Specialized Registers of Cochrane Review Groups including Epilepsy. In MEDLINE (Ovid) the coverage end date always lags a few days behind the search date.

SELECTION CRITERIA

We included studies, both retrospective and prospective, of all age groups (except those in the neonatal period (< 1 month of age)), of people with a single unprovoked seizure, followed up for a minimum of six months, with no upper limit of follow-up, with the study end point being seizure recurrence, death, or loss to follow-up. To be included, studies must have included at least 30 participants. We excluded studies that involved people with seizures that occur as a result of an acute precipitant or provoking factor, or in close temporal proximity to an acute neurological insult, since these are not considered epileptic in aetiology (acute symptomatic seizures). We also excluded people with situational seizures, such as febrile convulsions.

DATA COLLECTION AND ANALYSIS

Two review authors conducted the initial screening of titles and abstracts identified through the electronic searches, and removed non-relevant articles. We obtained the full-text articles of all remaining potentially relevant studies, or those whose relevance could not be determined from the abstract alone and two authors independently assessed for eligibility. All disagreements were resolved through discussion with no need to defer to a third review author. We extracted data from included studies using a data extraction form based on the checklist for critical appraisal and data extraction for systematicreviews of prediction modelling studies (CHARMS). Two review authors then appraised the included studies, using a standardised approach based on the quality in prognostic studies (QUIPS) tool, which was adapted for overall prognosis (seizure recurrence). We conducted a meta-analysis using Review Manager 2014, with a random-effects generic inverse variance meta-analysis model, which accounted for any between-study heterogeneity in the prognostic effect. We then summarised the meta-analysis by the pooled estimate (the average prognostic factor effect), its 95% confidence interval (CI), the estimates of I² and Tau² (heterogeneity), and a 95% prediction interval for the prognostic effect in a single population at three various time points, 6 months, 12 months and 24 months. Subgroup analysis was performed according to the ages of the cohorts included; studies involving all ages, studies that recruited adult only and those that were purely paediatric.

MAIN RESULTS

Fifty-eight studies (involving 54 cohorts), with a total of 12,160 participants (median 147, range 31 to 1443), met the inclusion criteria for the review. Of the 58 studies, 26 studies were paediatric studies, 16 were adult and the remaining 16 studies were a combination of paediatric and adult populations. Most included studies had a cohort study design with two case-control studies and one nested case-control study. Thirty-two studies (29 cohorts) reported a prospective longitudinal design whilst 15 studies had a retrospective design whilst the remaining studies were randomised controlled trials. Nine of the studies included presented mortality data following a first unprovoked seizure. For a mortality study to be included, a proportional mortality ratio (PMR) or a standardised mortality ratio (SMR) had to be given at a specific time point following a first unprovoked seizure. To be included in the meta-analysis a study had to present clear seizure recurrence data at 6 months, 12 months or 24 months. Forty-six studies were included in the meta-analysis, of which 23 were paediatric, 13 were adult, and 10 were a combination of paediatric and adult populations. A meta-analysis was performed at three time points; six months, one year and two years for all ages combined, paediatric and adult studies, respectively. We found an estimated overall seizure recurrence of all included studies at six months of 27% (95% CI 24% to 31%), 36% (95% CI 33% to 40%) at one year and 43% (95% CI 37% to 44%) at two years, with slightly lower estimates for adult subgroup analysis and slightly higher estimates for paediatric subgroup analysis. It was not possible to provide a summary estimate of the risk of seizure recurrence beyond these time points as most of the included studies were of short follow-up and too few studies presented recurrence rates at a single time point beyond two years. The evidence presented was found to be of moderate certainty.

AUTHORS' CONCLUSIONS

Despite the limitations of the data (moderate-certainty of evidence), mainly relating to clinical and methodological heterogeneity we have provided summary estimates for the likely risk of seizure recurrence at six months, one year and two years for both children and adults. This provides information that is likely to be useful for the clinician counselling patients (or their parents) on the probable risk of further seizures in the short-term whilst acknowledging the paucity of long-term recurrence data, particularly beyond 10 years.

Prognostic models for outcome prediction following in-hospital cardiac arrest using pre-arrest factors: a systematic review, meta-analysis and critical appraisal

BACKGROUND

Several prediction models of survival after in-hospital cardiac arrest (IHCA) have been published, but no overview of model performance and external validation exists. We performed a systematic review of the available prognostic models for outcome prediction of attempted resuscitation for IHCA using pre-arrest factors to enhance clinical decision-making through improved outcome prediction.

METHODS

This systematic review followed the CHARMS and PRISMA guidelines. Medline, Embase, Web of Science were searched up to October 2021. Studies developing, updating or validating a prediction model with pre-arrest factors for any potential clinical outcome of attempted resuscitation for IHCA were included. Studies were appraised critically according to the PROBAST checklist. A random-effects meta-analysis was performed to pool AUROC values of externally validated models.

RESULTS

Out of 2678 initial articles screened, 33 studies were included in this systematic review: 16 model development studies, 5 model updating studies and 12 model validation studies. The most frequently included pre-arrest factors included age, functional status, (metastatic) malignancy, heart disease, cerebrovascular events, respiratory, renal or hepatic insufficiency, hypotension and sepsis. Only six of the developed models have been independently validated in external populations. The GO-FAR score showed the best performance with a pooled AUROC of 0.78 (95% CI 0.69-0.85), versus 0.59 (95%CI 0.50-0.68) for the PAM and 0.62 (95% CI 0.49-0.74) for the PAR.

CONCLUSIONS

Several prognostic models for clinical outcome after attempted resuscitation for IHCA have been published. Most have a moderate risk of bias and have not been validated externally. The GO-FAR score showed the most acceptable performance. Future research should focus on updating existing models for use in clinical settings, specifically pre-arrest counselling. Systematic review registration PROSPERO CRD42021269235. Registered 21 July 2021.

Clinical prediction tools for identifying antimicrobial-resistant organism (ARO) carriage on hospital admissions: a systematic review

BACKGROUND

Increasing prevalence of antimicrobial-resistant organisms (AROs) is a growing economic and healthcare challenge. Increasing utilization of electronic medical record (EMR) systems and improvements in computation and analytical techniques afford an opportunity to reduce the spread of AROs through the development of clinical prediction tools to identify ARO carriers on admission to hospital.

AIM

To identify existing clinical prediction tools for meticillin-resistant Staphylococcus aureus (MRSA) and carbapenemase-producing organisms (CPOs), their predictive performance, and risk factors utilized in these tools.

METHODS

The CHARMS checklist was followed. Medline, EMBASE, Cochrane SR, CRD databases (DARE, NHS EED), CINAHL and Web of Science were searched from database inception to 26^th July 2021. Full-text articles were assessed independently, and quality assessment was conducted using the Prediction Model Risk of Bias Assessment Tool.

FINDINGS

In total, 3809 abstracts were identified and 22 studies were included. Among these studies, risk score models were the most common prediction tool (N=16). Previous admission, recent antibiotic exposure, age and sex were the most common risk factors for ARO carriage. Prediction tools were commonly evaluated on sensitivity and specificity with ranges of 15-100% and 46-98.6%, respectively, for MRSA, and 30-81.3% and 79.8-99.9%, respectively, for CPOs.

CONCLUSION

There is no gold standard ARO prediction tool. However, high-performance clinical prediction tools and identification of key risk factors for the early detection of AROs exist. Risk score models are easier to use and interpret; however, with recent improvements in machine learning techniques, highly robust models can be developed with data stored in an EMR.

Bronchopulmonary dysplasia prediction models: a systematic review and meta-analysis with validation

Prediction models could identify infants at the greatest risk of bronchopulmonary dysplasia (BPD) and allow targeted preventative strategies. We performed a systematic review and meta-analysis with external validation of identified models. Studies using predictors available before day 14 of life to predict BPD in very preterm infants were included. Two reviewers assessed 7628 studies for eligibility. Meta-analysis of externally validated models was followed by validation using 62,864 very preterm infants in England and Wales. A total of 64 studies using 53 prediction models were included totalling 274,407 infants (range 32-156,587/study). In all, 35 (55%) studies predated 2010; 39 (61%) were single-centre studies. A total of 97% of studies had a high risk of bias, especially in the analysis domain. Following meta-analysis of 22 BPD and 11 BPD/death composite externally validated models, Laughon's day one model was the most promising in predicting BPD and death (C-statistic 0.76 (95% CI 0.70-0.81) and good calibration). Six models were externally validated in our cohort with C-statistics between 0.70 and 0.90 but with poor calibration. Few BPD prediction models were developed with contemporary populations, underwent external validation, or had calibration and impact analyses. Contemporary, validated, and dynamic prediction models are needed for targeted preventative strategies. IMPACT: This review aims to provide a comprehensive assessment of all BPD prediction models developed to address the uncertainty of which model is sufficiently valid and generalisable for use in clinical practice and research. Published BPD prediction models are mostly outdated, single centre and lack external validation. Laughon's 2011 model is the most promising but more robust models, using contemporary data with external validation are needed to support better treatments.

Prediction models for depression risk among older adults: systematic review and critical appraisal

OBJECTIVE

To provide an overview of prediction models for the risk of major depressive disorder (MDD) among older adults.

METHODS

We conducted a systematic review combined with a meta-analysis and critical appraisal of published studies on existing geriatric depression risk models.

RESULTS

The systematic search screened 23,378 titles and abstracts; 14 studies including 20 prediction models were included. A total of 16 predictors were selected in the final model at least twice. Age, physical health, and cognitive function were the most common predictors. Only one model was externally validated, two models were presented with a complete equation, and five models examined the calibration. We found substantial heterogeneity in predictor and outcome definitions across models; important methodological information was often missing. All models were rated at high or unclear risk of bias, primarily due to methodological limitations. The pooled C-statistics of 12 prediction models was 0.83 (95%CI=0.77-0.89).

CONCLUSION

The usefulness of all models remains unclear due to several methodological limitations. Future studies should focus on methodological quality and external validation of depression risk prediction models.

The majority of 922 prediction models supporting breast cancer decision-making are at high risk of bias

OBJECTIVES

To systematically review the currently available prediction models that may support treatment decision-making in breast cancer.

STUDY DESIGN AND SETTING

Literature was systematically searched to identify studies reporting on development of prediction models aiming to support breast cancer treatment decision-making, published between January 2010 and December 2020. Quality and risk of bias were assessed using the Prediction model Risk Of Bias (ROB) Assessment Tool (PROBAST).

RESULTS

After screening 20,460 studies, 534 studies were included, reporting on 922 models. The 922 models predicted: mortality (n = 417 45%), recurrence (n = 217, 24%), lymph node involvement (n = 141, 15%), adverse events (n = 58, 6%), treatment response (n = 56, 6%), or other outcomes (n = 33, 4%). In total, 285 models (31%) lacked a complete description of the final model and could not be applied to new patients. Most models (n = 878, 95%) were considered to contain high ROB.

CONCLUSION

A substantial overlap in predictor variables and outcomes between the models was observed. Most models were not reported according to established reporting guidelines or showed methodological flaws during the development and/or validation of the model. Further development of prediction models with thorough quality and validity assessment is an essential first step for future clinical application.

Prediction Models to Estimate the Future Risk of Osteoarthritis in the General Population: A Systematic Review

OBJECTIVE

To evaluate the performance and applicability of multivariable prediction models for osteoarthritis (OA).

METHODS

This was a systematic review and narrative synthesis using 3 databases (EMBASE, PubMed, and Web of Science) from inception to December 2021. We included general population longitudinal studies reporting derivation, comparison, or validation of multivariable models to predict individual risk of OA incidence, defined by recognized clinical or imaging criteria. We excluded studies reporting prevalent OA and joint arthroplasty outcome. Paired reviewers independently performed article selection, data extraction, and risk-of-bias assessment. Model performance, calibration, and retained predictors were summarized.

RESULTS

A total of 26 studies were included, reporting 31 final multivariable prediction models for incident knee (23), hip (4), hand (3) and any-site OA (1), with a median of 121.5 (range 27-12,803) outcome events, a median prediction horizon of 8 years (range 2-41), and a median of 6 predictors (range 3-24). Age, body mass index, previous injury, and occupational exposures were among the most commonly included predictors. Model discrimination after validation was generally acceptable to excellent (area under the curve = 0.70-0.85). Either internal or external validation processes were used in most models, although the risk of bias was often judged to be high with limited applicability to mass application in diverse populations.

CONCLUSION

Despite growing interest in multivariable prediction models for incident OA, focus remains predominantly on the knee, with reliance on data from a small pool of appropriate cohort data sets, and concerns over general population applicability.

Clinical prediction models assessing response to radiotherapy for rectal cancer: protocol for a systematic review

BACKGROUND

Rectal cancer has a high prevalence. The standard of care for management of localised disease involves major surgery and/or chemoradiotherapy, but these modalities are sometimes associated with mortality and morbidity. The notion of 'watch and wait' has therefore emerged and offers an organ-sparing approach to patients after administering a less invasive initial treatment, such as X-ray brachytherapy (Papillon technique). It is thus important to evaluate how likely patients are to respond to such therapies, to develop patient-tailored treatment pathways. We propose a systematic review to identify published clinical prediction models of the response of rectal cancer to treatment that includes radiotherapy and here present our protocol.

METHODS

Included studies will develop multivariable clinical prediction models which assess response to treatment and overall survival of adult patients who have been diagnosed with any stage of rectal cancer and have received radiotherapy treatment with curative intent. Cohort studies and randomised controlled trials will be included. The primary outcome will be the occurrence of salvage surgery at 1 year after treatment. Secondary outcomes include salavage surgery at at any reported time point, the predictive accuracy of models, the quality of the developed models and the feasibility of using the model in clinical practice. Ovid MEDLINE, PubMed, Cochrane Library, EMBASE and CINAHL will be searched from inception to 24 February 2022. Keywords and phrases related to rectal cancer, radiotherapy and prediction models will be used. Studies will be selected once the deduplication, title, abstract and full-text screening process have been completed by two independent reviewers. The PRISMA-P checklist will be followed. A third reviewer will resolve any disagreement. The data extraction form will be pilot-tested using a representative 5% sample of the studies reviewed. The CHARMS checklist will be implemented. Risk of bias in each study will be assessed using the PROBAST tool. A narrative synthesis will be performed and if sufficient data are identified, meta-analysis will be undertaken as described in Debray et al. DISCUSSION: This systematic review will identify factors that predict response to the treatment protocol. Any gaps for potential development of new clinical prediction models will be highlighted.

TRIAL REGISTRATION

CRD42022277704.

Defining the optimum strategy for identifying adults and children with coeliac disease: systematic review and economic modelling

BACKGROUND

Coeliac disease is an autoimmune disorder triggered by ingesting gluten. It affects approximately 1% of the UK population, but only one in three people is thought to have a diagnosis. Untreated coeliac disease may lead to malnutrition, anaemia, osteoporosis and lymphoma.

OBJECTIVES

The objectives were to define at-risk groups and determine the cost-effectiveness of active case-finding strategies in primary care.

DESIGN

(1) Systematic review of the accuracy of potential diagnostic indicators for coeliac disease. (2) Routine data analysis to develop prediction models for identification of people who may benefit from testing for coeliac disease. (3) Systematic review of the accuracy of diagnostic tests for coeliac disease. (4) Systematic review of the accuracy of genetic tests for coeliac disease (literature search conducted in April 2021). (5) Online survey to identify diagnostic thresholds for testing, starting treatment and referral for biopsy. (6) Economic modelling to identify the cost-effectiveness of different active case-finding strategies, informed by the findings from previous objectives.

DATA SOURCES

For the first systematic review, the following databases were searched from 1997 to April 2021: MEDLINE® (National Library of Medicine, Bethesda, MD, USA), Embase® (Elsevier, Amsterdam, the Netherlands), Cochrane Library, Web of Science™ (Clarivate™, Philadelphia, PA, USA), the World Health Organization International Clinical Trials Registry Platform ( WHO ICTRP ) and the National Institutes of Health Clinical Trials database. For the second systematic review, the following databases were searched from January 1990 to August 2020: MEDLINE, Embase, Cochrane Library, Web of Science, Kleijnen Systematic Reviews ( KSR ) Evidence, WHO ICTRP and the National Institutes of Health Clinical Trials database. For prediction model development, Clinical Practice Research Datalink GOLD, Clinical Practice Research Datalink Aurum and a subcohort of the Avon Longitudinal Study of Parents and Children were used; for estimates for the economic models, Clinical Practice Research Datalink Aurum was used.

REVIEW METHODS

For review 1, cohort and case-control studies reporting on a diagnostic indicator in a population with and a population without coeliac disease were eligible. For review 2, diagnostic cohort studies including patients presenting with coeliac disease symptoms who were tested with serological tests for coeliac disease and underwent a duodenal biopsy as reference standard were eligible. In both reviews, risk of bias was assessed using the quality assessment of diagnostic accuracy studies 2 tool. Bivariate random-effects meta-analyses were fitted, in which binomial likelihoods for the numbers of true positives and true negatives were assumed.

RESULTS

People with dermatitis herpetiformis, a family history of coeliac disease, migraine, anaemia, type 1 diabetes, osteoporosis or chronic liver disease are 1.5-2 times more likely than the general population to have coeliac disease; individual gastrointestinal symptoms were not useful for identifying coeliac disease. For children, women and men, prediction models included 24, 24 and 21 indicators of coeliac disease, respectively. The models showed good discrimination between patients with and patients without coeliac disease, but performed less well when externally validated. Serological tests were found to have good diagnostic accuracy for coeliac disease. Immunoglobulin A tissue transglutaminase had the highest sensitivity and endomysial antibody the highest specificity. There was little improvement when tests were used in combination. Survey respondents (n = 472) wanted to be 66% certain of the diagnosis from a blood test before starting a gluten-free diet if symptomatic, and 90% certain if asymptomatic. Cost-effectiveness analyses found that, among adults, and using serological testing alone, immunoglobulin A tissue transglutaminase was most cost-effective at a 1% pre-test probability (equivalent to population screening). Strategies using immunoglobulin A endomysial antibody plus human leucocyte antigen or human leucocyte antigen plus immunoglobulin A tissue transglutaminase with any pre-test probability had similar cost-effectiveness results, which were also similar to the cost-effectiveness results of immunoglobulin A tissue transglutaminase at a 1% pre-test probability. The most practical alternative for implementation within the NHS is likely to be a combination of human leucocyte antigen and immunoglobulin A tissue transglutaminase testing among those with a pre-test probability above 1.5%. Among children, the most cost-effective strategy was a 10% pre-test probability with human leucocyte antigen plus immunoglobulin A tissue transglutaminase, but there was uncertainty around the most cost-effective pre-test probability. There was substantial uncertainty in economic model results, which means that there would be great value in conducting further research.

LIMITATIONS

The interpretation of meta-analyses was limited by the substantial heterogeneity between the included studies, and most included studies were judged to be at high risk of bias. The main limitations of the prediction models were that we were restricted to diagnostic indicators that were recorded by general practitioners and that, because coeliac disease is underdiagnosed, it is also under-reported in health-care data. The cost-effectiveness model is a simplification of coeliac disease and modelled an average cohort rather than individuals. Evidence was weak on the probability of routine coeliac disease diagnosis, the accuracy of serological and genetic tests and the utility of a gluten-free diet.

CONCLUSIONS

Population screening with immunoglobulin A tissue transglutaminase (1% pre-test probability) and of immunoglobulin A endomysial antibody followed by human leucocyte antigen testing or human leucocyte antigen testing followed by immunoglobulin A tissue transglutaminase with any pre-test probability appear to have similar cost-effectiveness results. As decisions to implement population screening cannot be made based on our economic analysis alone, and given the practical challenges of identifying patients with higher pre-test probabilities, we recommend that human leucocyte antigen combined with immunoglobulin A tissue transglutaminase testing should be considered for adults with at least a 1.5% pre-test probability of coeliac disease, equivalent to having at least one predictor. A more targeted strategy of 10% pre-test probability is recommended for children (e.g. children with anaemia).

FUTURE WORK

Future work should consider whether or not population-based screening for coeliac disease could meet the UK National Screening Committee criteria and whether or not it necessitates a long-term randomised controlled trial of screening strategies. Large prospective cohort studies in which all participants receive accurate tests for coeliac disease are needed.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42019115506 and CRD42020170766.

FUNDING

This project was funded by the National Institute for Health and Care Research ( NIHR ) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 26, No. 44. See the NIHR Journals Library website for further project information.

Prediction models for the prediction of unplanned hospital admissions in community-dwelling older adults: A systematic review

Background

Identification of community-dwelling older adults at risk of unplanned hospitalizations is of importance to facilitate preventive interventions. Our objective was to review and appraise the methodological quality and predictive performance of prediction models for predicting unplanned hospitalizations in community-dwelling older adults

Methods and findings

We searched MEDLINE, EMBASE and CINAHL from August 2013 to January 2021. Additionally, we checked references of the identified articles for the inclusion of relevant publications and added studies from two previous reviews that fulfilled the eligibility criteria. We included prospective and retrospective studies with any follow-up period that recruited adults aged 65 and over and developed a prediction model predicting unplanned hospitalizations. We included models with at least one (internal or external) validation cohort. The models had to be intended to be used in a primary care setting. Two authors independently assessed studies for inclusion and undertook data extraction following recommendations of the CHARMS checklist, while quality assessment was performed using the PROBAST tool. A total of 19 studies met the inclusion criteria. Prediction horizon ranged from 4.5 months to 4 years. Most frequently included variables were specific medical diagnoses (n = 11), previous hospital admission (n = 11), age (n = 11), and sex or gender (n = 8). Predictive performance in terms of area under the curve ranged from 0.61 to 0.78. Models developed to predict potentially preventable hospitalizations tended to have better predictive performance than models predicting hospitalizations in general. Overall, risk of bias was high, predominantly in the analysis domain.

Conclusions

Models developed to predict preventable hospitalizations tended to have better predictive performance than models to predict all-cause hospitalizations. There is however substantial room for improvement on the reporting and analysis of studies. We recommend better adherence to the TRIPOD guidelines.