1
|
Hunt A, Bonnett L, Heron J, Lawton M, Clayton G, Smith G, Norman J, Kenny L, Lawlor D, Merriel A. Systematic Review of Clinical Prediction Models for the Risk of Emergency Caesarean Births. BJOG 2024. [PMID: 39256942 DOI: 10.1111/1471-0528.17948] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2024] [Revised: 08/07/2024] [Accepted: 08/10/2024] [Indexed: 09/12/2024]
Abstract
BACKGROUND Globally, caesarean births (CB), including emergency caesareans births (EmCB), are rising. It is estimated that nearly a third of all births will be CB by 2030. OBJECTIVES Identify and summarise the results from studies developing and validating prognostic multivariable models predicting the risk of EmCBs. Ultimately understanding the accuracy of their development, and whether they are operationalised for use in routine clinical practice. SEARCH STRATEGY Studies were identified using databases: MEDLINE, CINAHL, Cochrane Central and Scopus with a search strategy tailored to models predicting EmCBs. SELECTION CRITERIA Prospective studies developing and validating clinical prediction models, with two or more covariates, to predict risk of EmCB. DATA COLLECTION AND ANALYSIS Data were extracted onto a proforma using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). RESULTS In total, 8083 studies resulted in 56 unique prediction modelling studies and seven validating studies, with a total of 121 different predictors. Frequently occurring predictors included maternal height, maternal age, parity, BMI and gestational age. PROBAST highlighted 33 studies with low overall bias, and these all internally validated their model. Thirteen studies externally validated; only eight of these were graded an overall low risk of bias. Six models offered applications that could be readily used, but only one provided enough time to offer a planned caesarean birth (pCB). These well-refined models have not been recalibrated since development. Only one model, developed in a relatively low-risk population, with data collected a decade ago, remains useful at 36 weeks for arranging a pCB. CONCLUSION To improve personalised clinical conversations, there is a pressing need for a model that accurately predicts the timely risk of an EmCB for women across diverse clinical backgrounds. TRIAL REGISTRATION PROSPERO registration number: CRD42023384439.
Collapse
Affiliation(s)
- Alexandra Hunt
- Department of Health Data Science, The University of Liverpool, Liverpool, UK
| | - Laura Bonnett
- Department of Health Data Science, The University of Liverpool, Liverpool, UK
| | - Jon Heron
- Bristol Medical School, The University of Bristol, Bristol, UK
| | - Michael Lawton
- Bristol Population Health Science Institute, The University of Bristol, Bristol, UK
| | - Gemma Clayton
- Bristol Medical School, The University of Bristol, Bristol, UK
| | - Gordon Smith
- Department of Obstetrics and Gynaecology, The University of Cambridge, Cambridge, UK
- The Rosie Hospital, Cambridge, UK
| | - Jane Norman
- The University of Nottingham, Nottingham, UK
| | - Louise Kenny
- Department of Women's and Children's Health, Faculty of Health and Life Sciences, The University of Liverpool, Liverpool, UK
| | - Deborah Lawlor
- Bristol Medical School, The University of Bristol, Bristol, UK
| | - Abi Merriel
- Centre for Women's Health Research, Department of Women's and Children's Health, University of Liverpool, Liverpool, UK
- Liverpool Women's Hospital, Liverpool, UK
| |
Collapse
|
2
|
Wyrwa JM, Hoffberg AS, Stearns-Yoder KA, Lantagne AC, Kinney AR, Reis DJ, Brenner LA. Predicting Recovery After Concussion in Pediatric Patients: A Meta-Analysis. Pediatrics 2024; 154:e2023065431. [PMID: 39183674 DOI: 10.1542/peds.2023-065431] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/14/2023] [Revised: 06/20/2024] [Accepted: 06/21/2024] [Indexed: 08/27/2024] Open
Abstract
CONTEXT Prognostic prediction models (PPMs) can help clinicians predict outcomes. OBJECTIVE To critically examine peer-reviewed PPMs predicting delayed recovery among pediatric patients with concussion. DATA SOURCES Ovid Medline, Embase, Ovid PsycInfo, Web of Science Core Collection, Cumulative Index to Nursing and Allied Health Literature, Cochrane Library, Google Scholar. STUDY SELECTION The study had to report a PPM for pediatric patients to be used within 28 days of injury to estimate risk of delayed recovery at 28 days to 1 year postinjury. Studies had to have at least 30 participants. DATA EXTRACTION The Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies checklist was completed. RESULTS Six studies of 13 PPMs were included. These studies primarily reflected male patients in late childhood or early adolescence presenting to an emergency department meeting the Concussion in Sport Group concussion criteria. No study authors used the same outcome definition nor evaluated the clinical utility of a model. All studies demonstrated high risk of bias. Quality of evidence was best for the Predicting and Preventing Postconcussive Problems in Pediatrics (5P) clinical risk score. LIMITATIONS No formal PPM Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) process exists. CONCLUSIONS The 5P clinical risk score may be considered for clinical use. Rigorous external validations, particularly in other settings, are needed. The remaining PPMs require external validation. Lack of consensus regarding delayed recovery criteria limits these PPMs.
Collapse
Affiliation(s)
- Jordan M Wyrwa
- Departments of Physical Medicine & Rehabilitation
- Children's Hospital Colorado, Aurora, Colorado
| | - Adam S Hoffberg
- VA Rocky Mountain Mental Illness Research, Education, and Clinical Center for Suicide Prevention, Aurora, Colorado
| | - Kelly A Stearns-Yoder
- Departments of Physical Medicine & Rehabilitation
- VA Rocky Mountain Mental Illness Research, Education, and Clinical Center for Suicide Prevention, Aurora, Colorado
| | - Ann C Lantagne
- Departments of Physical Medicine & Rehabilitation
- Children's Hospital Colorado, Aurora, Colorado
| | - Adam R Kinney
- Departments of Physical Medicine & Rehabilitation
- VA Rocky Mountain Mental Illness Research, Education, and Clinical Center for Suicide Prevention, Aurora, Colorado
| | - Daniel J Reis
- Psychiatry
- VA Rocky Mountain Mental Illness Research, Education, and Clinical Center for Suicide Prevention, Aurora, Colorado
| | - Lisa A Brenner
- Departments of Physical Medicine & Rehabilitation
- Psychiatry
- Neurology, University of Colorado, Anschutz Medical Campus, Aurora, Colorado
| |
Collapse
|
3
|
Groeneveld NS, Bijlsma MW, van Zeggeren IE, Staal SL, Tanck MWT, van de Beek D, Brouwer MC. Diagnostic prediction models for bacterial meningitis in children with a suspected central nervous system infection: a systematic review and prospective validation study. BMJ Open 2024; 14:e081172. [PMID: 39117411 DOI: 10.1136/bmjopen-2023-081172] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 08/10/2024] Open
Abstract
OBJECTIVES Diagnostic prediction models exist to assess the probability of bacterial meningitis (BM) in paediatric patients with suspected meningitis. To evaluate the diagnostic accuracy of these models in a broad population of children suspected of a central nervous system (CNS) infection, we performed external validation. METHODS We performed a systematic literature review in Medline to identify articles on the development, refinement or validation of a prediction model for BM, and validated these models in a prospective cohort of children aged 0-18 years old suspected of a CNS infection. PRIMARY AND SECONDARY OUTCOME MEASURES We calculated sensitivity, specificity, predictive values, the area under the receiver operating characteristic curve (AUC) and evaluated calibration of the models for diagnosis of BM. RESULTS In total, 23 prediction models were validated in a cohort of 450 patients suspected of a CNS infection included between 2012 and 2015. In 75 patients (17%), the final diagnosis was a CNS infection including 30 with BM (7%). AUCs ranged from 0.69 to 0.94 (median 0.83, interquartile range [IQR] 0.79-0.87) overall, from 0.74 to 0.96 (median 0.89, IQR 0.82-0.92) in children aged ≥28 days and from 0.58 to 0.91 (median 0.79, IQR 0.75-0.82) in neonates. CONCLUSIONS Prediction models show good to excellent test characteristics for excluding BM in children and can be of help in the diagnostic workup of paediatric patients with a suspected CNS infection, but cannot replace a thorough history, physical examination and ancillary testing.
Collapse
Affiliation(s)
- Nina S Groeneveld
- Department of Neurology, Amsterdam UMC Location AMC, Amsterdam, The Netherlands
| | - Merijn W Bijlsma
- Department of Pediatrics, Amsterdam UMC Location AMC, Amsterdam, The Netherlands
| | | | - Steven L Staal
- Department of Neurology, Amsterdam UMC Location AMC, Amsterdam, The Netherlands
| | - Michael W T Tanck
- Department of Epidemiology and Data Science, Amsterdam UMC-Locatie AMC, Amsterdam, The Netherlands
| | | | - Matthijs C Brouwer
- Department of Neurology, Amsterdam UMC Location AMC, Amsterdam, The Netherlands
| |
Collapse
|
4
|
Watson V, Smith CT, Bonnett LJ. Systematic review of methods used in prediction models with recurrent event data. Diagn Progn Res 2024; 8:13. [PMID: 39103900 DOI: 10.1186/s41512-024-00173-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/05/2024] [Accepted: 06/13/2024] [Indexed: 08/07/2024] Open
Abstract
BACKGROUND Patients who suffer from chronic conditions or diseases are susceptible to experiencing repeated events of the same type (e.g. seizures), termed 'recurrent events'. Prediction models can be used to predict the risk of recurrence so that intervention or management can be tailored accordingly, but statistical methodology can vary. The objective of this systematic review was to identify and describe statistical approaches that have been applied for the development and validation of multivariable prediction models with recurrent event data. A secondary objective was to informally assess the characteristics and quality of analysis approaches used in the development and validation of prediction models of recurrent event data. METHODS Searches were run in MEDLINE using a search strategy in 2019 which included index terms and phrases related to recurrent events and prediction models. For studies to be included in the review they must have developed or validated a multivariable clinical prediction model for recurrent event outcome data, specifically modelling the recurrent events and the timing between them. The statistical analysis methods used to analyse the recurrent event data in the clinical prediction model were extracted to answer the primary aim of the systematic review. In addition, items such as the event rate as well as any discrimination and calibration statistics that were used to assess the model performance were extracted for the secondary aim of the review. RESULTS A total of 855 publications were identified using the developed search strategy and 301 of these are included in our systematic review. The Andersen-Gill method was identified as the most commonly applied method in the analysis of recurrent events, which was used in 152 (50.5%) studies. This was closely followed by frailty models which were used in 116 (38.5%) included studies. Of the 301 included studies, only 75 (24.9%) internally validated their model(s) and three (1.0%) validated their model(s) in an external dataset. CONCLUSIONS This review identified a variety of methods which are used in practice when developing or validating prediction models for recurrent events. The variability of the approaches identified is cause for concern as it indicates possible immaturity in the field and highlights the need for more methodological research to bring greater consistency in approach of recurrent event analysis. Further work is required to ensure publications report all required information and use robust statistical methods for model development and validation. PROSPERO REGISTRATION CRD42019116031.
Collapse
Affiliation(s)
- Victoria Watson
- Department of Health Data Sciences, University of Liverpool, Liverpool, UK.
| | - Catrin Tudur Smith
- Department of Health Data Sciences, University of Liverpool, Liverpool, UK
| | - Laura J Bonnett
- Department of Health Data Sciences, University of Liverpool, Liverpool, UK
| |
Collapse
|
5
|
Liu X, Liu X, Jin C, Luo Y, Yang L, Ning X, Zhuo C, Xiao F. Prediction models for diagnosis and prognosis of the colonization or infection of multidrug-resistant organisms in adults: a systematic review, critical appraisal, and meta-analysis. Clin Microbiol Infect 2024:S1198-743X(24)00316-1. [PMID: 38992430 DOI: 10.1016/j.cmi.2024.07.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2024] [Revised: 05/02/2024] [Accepted: 07/04/2024] [Indexed: 07/13/2024]
Abstract
BACKGROUND Prediction models help to target patients at risk of multidrug-resistant organism (MDRO) colonization or infection and could serve as tools informing clinical practices to prevent MDRO transmission and inappropriate empiric antibiotic therapy. However, there is limited evidence to identify which among the available models are of low risk of bias and suitable for clinical application. OBJECTIVES To identify, describe, appraise, and summarise the performance of all prognostic and diagnostic models developed or validated for predicting MDRO colonization or infection. DATA SOURCES Six electronic literature databases and clinical registration databases were searched until April 2022. STUDY ELIGIBILITY CRITERIA Development and validation studies of any multivariable prognostic and diagnostic models to predict MDRO colonization or infection in adults. PARTICIPANTS Adults (≥ 18 years old) without MDRO colonization or infection (in prognostic models) or with unknown or suspected MDRO colonization or infection (in diagnostic models). ASSESSMENT OF RISK OF BIAS The Prediction Model Risk of Bias Assessment Tool was used to assess the risk of bias. Evidence certainty was assessed using the Grading of Recommendations Assessment, Development, and Evaluation approach. METHODS OF DATA SYNTHESIS Meta-analyses were conducted to summarize the discrimination and calibration of the models' external validations conducted in at least two non-overlapping datasets. RESULTS We included 162 models (108 studies) developed for diagnosing (n = 135) and predicting (n = 27) MDRO colonization or infection. Models exhibited a high-risk of bias, especially in statistical analysis. High-frequency predictors were age, recent invasive procedures, antibiotic usage, and prior hospitalization. Less than 25% of the models underwent external validations, with only seven by independent teams. Meta-analyses for one diagnostic and two prognostic models only produced very low to low certainty of evidence. CONCLUSIONS The review comprehensively described the models for identifying patients at risk of MDRO colonization or infection. We cannot recommend which models are ready for application because of the high-risk of bias, limited validations, and low certainty of evidence from meta-analyses, indicating a clear need to improve the conducting and reporting of model development and external validation studies to facilitate clinical application.
Collapse
Affiliation(s)
- Xu Liu
- Department of Infectious Diseases, The Fifth Affiliated Hospital, Sun Yat-sen University, Zhuhai, China; Guangdong-Hong Kong-Macao University Joint Laboratory of Interventional Medicine, The Fifth Affiliated Hospital, Sun Yat-sen University, Zhuhai, China
| | - Xi Liu
- Department of Infectious Diseases, The Fifth Affiliated Hospital, Sun Yat-sen University, Zhuhai, China; Guangdong-Hong Kong-Macao University Joint Laboratory of Interventional Medicine, The Fifth Affiliated Hospital, Sun Yat-sen University, Zhuhai, China
| | - Chenyue Jin
- Department of Infectious Diseases, The Fifth Affiliated Hospital, Sun Yat-sen University, Zhuhai, China
| | - Yuting Luo
- Department of Infectious Diseases, The Fifth Affiliated Hospital, Sun Yat-sen University, Zhuhai, China; Department of Infectious Diseases, Liuzhou People's Hospital, Liuzhou, China
| | - Lianping Yang
- School of Public Health, Sun Yat-sen University, Guangzhou, China
| | - Xinjiao Ning
- Department of Infectious Diseases, The Fifth Affiliated Hospital, Sun Yat-sen University, Zhuhai, China
| | - Chao Zhuo
- State Key Laboratory of Respiratory Disease, The First Affiliated Hospital of Guangzhou Medical University, Guangzhou, China.
| | - Fei Xiao
- Department of Infectious Diseases, The Fifth Affiliated Hospital, Sun Yat-sen University, Zhuhai, China; Guangdong-Hong Kong-Macao University Joint Laboratory of Interventional Medicine, The Fifth Affiliated Hospital, Sun Yat-sen University, Zhuhai, China; Guangdong Provincial Engineering Research Center of Molecular Imaging, The Fifth Affiliated Hospital, Sun Yat-sen University, Zhuhai, China; Kashi Guangdong Institute of Science and Technology, The First People's Hospital of Kashi, Kashi, China; State Key Laboratory of Anti-Infective Drug Development, School of Pharmaceutical Sciences, Sun Yat-sen University, Guangzhou, China.
| |
Collapse
|
6
|
Wang X, Zhou S, Ye N, Li Y, Zhou P, Chen G, Hu H. Predictive models of Alzheimer's disease dementia risk in older adults with mild cognitive impairment: a systematic review and critical appraisal. BMC Geriatr 2024; 24:531. [PMID: 38898411 PMCID: PMC11188292 DOI: 10.1186/s12877-024-05044-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2023] [Accepted: 05/06/2024] [Indexed: 06/21/2024] Open
Abstract
BACKGROUND Mild cognitive impairment has received widespread attention as a high-risk population for Alzheimer's disease, and many studies have developed or validated predictive models to assess it. However, the performance of the model development remains unknown. OBJECTIVE The objective of this review was to provide an overview of prediction models for the risk of Alzheimer's disease dementia in older adults with mild cognitive impairment. METHOD PubMed, EMBASE, Web of Science, and MEDLINE were systematically searched up to October 19, 2023. We included cohort studies in which risk prediction models for Alzheimer's disease dementia in older adults with mild cognitive impairment were developed or validated. The Predictive Model Risk of Bias Assessment Tool (PROBAST) was employed to assess model bias and applicability. Random-effects models combined model AUCs and calculated (approximate) 95% prediction intervals for estimations. Heterogeneity across studies was evaluated using the I2 statistic, and subgroup analyses were conducted to investigate sources of heterogeneity. Additionally, funnel plot analysis was utilized to identify publication bias. RESULTS The analysis included 16 studies involving 9290 participants. Frequency analysis of predictors showed that 14 appeared at least twice and more, with age, functional activities questionnaire, and Mini-mental State Examination scores of cognitive functioning being the most common predictors. From the studies, only two models were externally validated. Eleven studies ultimately used machine learning, and four used traditional modelling methods. However, we found that in many of the studies, there were problems with insufficient sample sizes, missing important methodological information, lack of model presentation, and all of the models were rated as having a high or unclear risk of bias. The average AUC of the 15 best-developed predictive models was 0.87 (95% CI: 0.83, 0.90). DISCUSSION Most published predictive modelling studies are deficient in rigour, resulting in a high risk of bias. Upcoming research should concentrate on enhancing methodological rigour and conducting external validation of models predicting Alzheimer's disease dementia. We also emphasize the importance of following the scientific method and transparent reporting to improve the accuracy, generalizability and reproducibility of study results. REGISTRATION This systematic review was registered in PROSPERO (Registration ID: CRD42023468780).
Collapse
Affiliation(s)
- Xiaotong Wang
- College of Nursing, Hubei University of Chinese Medicine, Wuhan, China
| | - Shi Zhou
- College of Nursing, Hubei University of Chinese Medicine, Wuhan, China
| | - Niansi Ye
- College of Nursing, Hubei University of Chinese Medicine, Wuhan, China
| | - Yucan Li
- College of Nursing, Hubei University of Chinese Medicine, Wuhan, China
| | - Pengjun Zhou
- College of Nursing, Hubei University of Chinese Medicine, Wuhan, China
| | - Gao Chen
- College of Nursing, Hubei University of Chinese Medicine, Wuhan, China
| | - Hui Hu
- College of Nursing, Hubei University of Chinese Medicine, Wuhan, China.
- Engineering Research Center of TCM Protection Technology and New Product Development for the Elderly Brain Health, Ministry of Education, Wuhan, China.
- Hubei Shizhen Laboratory, Wuhan, China.
| |
Collapse
|
7
|
Tiruneh SA, Vu TTT, Moran LJ, Callander EJ, Allotey J, Thangaratinam S, Rolnik DL, Teede HJ, Wang R, Enticott J. Externally validated prediction models for pre-eclampsia: systematic review and meta-analysis. ULTRASOUND IN OBSTETRICS & GYNECOLOGY : THE OFFICIAL JOURNAL OF THE INTERNATIONAL SOCIETY OF ULTRASOUND IN OBSTETRICS AND GYNECOLOGY 2024; 63:592-604. [PMID: 37724649 DOI: 10.1002/uog.27490] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/12/2023] [Revised: 08/29/2023] [Accepted: 09/08/2023] [Indexed: 09/21/2023]
Abstract
OBJECTIVE This systematic review and meta-analysis aimed to evaluate the performance of existing externally validated prediction models for pre-eclampsia (PE) (specifically, any-onset, early-onset, late-onset and preterm PE). METHODS A systematic search was conducted in five databases (MEDLINE, EMBASE, Emcare, CINAHL and Maternity & Infant Care Database) and using Google Scholar/reference search to identify studies based on the Population, Index prediction model, Comparator, Outcome, Timing and Setting (PICOTS) approach until 20 May 2023. We extracted data using the CHARMS checklist and appraised the risk of bias using the PROBAST tool. A meta-analysis of discrimination and calibration performance was conducted when appropriate. RESULTS Twenty-three studies reported 52 externally validated prediction models for PE (one preterm, 20 any-onset, 17 early-onset and 14 late-onset PE models). No model had the same set of predictors. Fifteen any-onset PE models were validated externally once, two were validated twice and three were validated three times, while the Fetal Medicine Foundation (FMF) competing-risks model for preterm PE prediction was validated widely in 16 different settings. The most common predictors were maternal characteristics (prepregnancy body mass index, prior PE, family history of PE, chronic medical conditions and ethnicity) and biomarkers (uterine artery pulsatility index and pregnancy-associated plasma protein-A). The FMF model for preterm PE (triple test plus maternal factors) had the best performance, with a pooled area under the receiver-operating-characteristics curve (AUC) of 0.90 (95% prediction interval (PI), 0.76-0.96), and was well calibrated. The other models generally had poor-to-good discrimination performance (median AUC, 0.66 (range, 0.53-0.77)) and were overfitted on external validation. Apart from the FMF model, only two models that were validated multiple times for any-onset PE prediction, which were based on maternal characteristics only, produced reasonable pooled AUCs of 0.71 (95% PI, 0.66-0.76) and 0.73 (95% PI, 0.55-0.86). CONCLUSIONS Existing externally validated prediction models for any-, early- and late-onset PE have limited discrimination and calibration performance, and include inconsistent input variables. The triple-test FMF model had outstanding discrimination performance in predicting preterm PE in numerous settings, but the inclusion of specialized biomarkers may limit feasibility and implementation outside of high-resource settings. © 2023 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of International Society of Ultrasound in Obstetrics and Gynecology.
Collapse
Affiliation(s)
- S A Tiruneh
- Monash Centre for Health Research and Implementation, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, VIC, Australia
| | - T T T Vu
- Monash Centre for Health Research and Implementation, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, VIC, Australia
| | - L J Moran
- Monash Centre for Health Research and Implementation, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, VIC, Australia
| | - E J Callander
- Monash Centre for Health Research and Implementation, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, VIC, Australia
- School of Public Health, Faculty of Health, University of Technology Sydney, Sydney, NSW, Australia
| | - J Allotey
- World Health Organization (WHO) Collaborating Centre for Global Women's Health, Institute of Metabolism and Systems Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - S Thangaratinam
- World Health Organization (WHO) Collaborating Centre for Global Women's Health, Institute of Metabolism and Systems Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
- Birmingham Women's and Children's NHS Foundation Trust, Birmingham, UK
| | - D L Rolnik
- Department of Obstetrics and Gynaecology, Monash University, Clayton, VIC, Australia
| | - H J Teede
- Monash Centre for Health Research and Implementation, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, VIC, Australia
| | - R Wang
- Department of Obstetrics and Gynaecology, Monash University, Clayton, VIC, Australia
| | - J Enticott
- Monash Centre for Health Research and Implementation, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, VIC, Australia
| |
Collapse
|
8
|
Jones C, Taylor M, Sperrin M, Grant SW. A systematic review of cardiac surgery clinical prediction models that include intra-operative variables. Perfusion 2024:2676591241237758. [PMID: 38649154 DOI: 10.1177/02676591241237758] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/25/2024]
Abstract
BACKGROUND Most cardiac surgery clinical prediction models (CPMs) are developed using pre-operative variables to predict post-operative outcomes. Some CPMs are developed with intra-operative variables, but none are widely used. The objective of this systematic review was to identify CPMs with intra-operative variables that predict short-term outcomes following adult cardiac surgery. METHODS Ovid MEDLINE and EMBASE databases were searched from inception to December 2022, for studies developing a CPM with at least one intra-operative variable. Data were extracted using a critical appraisal framework and bias assessment tool. Model performance was analysed using discrimination and calibration measures. RESULTS A total of 24 models were identified. Frequent predicted outcomes were acute kidney injury (9/24 studies) and peri-operative mortality (6/24 studies). Frequent pre-operative variables were age (18/24 studies) and creatinine/eGFR (18/24 studies). Common intra-operative variables were cardiopulmonary bypass time (16/24 studies) and transfusion (13/24 studies). Model discrimination was acceptable for all internally validated models (AUC 0.69-0.91). Calibration was poor (15/24 studies) or unreported (8/24 studies). Most CPMs were at a high or indeterminate risk of bias (23/24 models). The added value of intra-operative variables was assessed in six studies with statistically significantly improved discrimination demonstrated in two. CONCLUSION Weak reporting and methodological limitations may restrict wider applicability and adoption of existing CPMs that include intra-operative variables. There is some evidence that CPM discrimination is improved with the addition of intra-operative variables. Further work is required to understand the role of intra-operative CPMs in the management of cardiac surgery patients.
Collapse
Affiliation(s)
- Ceri Jones
- Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, Manchester Academic Health Science Centre, University of Manchester, Manchester, UK
- Department of Clinical Perfusion, University Hospital Southampton NHS Foundation Trust, Southampton General Hospital, Southampton, UK
| | - Marcus Taylor
- Department of Cardiothoracic Surgery, Manchester University Hospital Foundation Trust, Wythenshawe Hospital, , Manchester, UK
| | - Matthew Sperrin
- Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, Manchester Academic Health Science Centre, University of Manchester, Manchester, UK
| | - Stuart W Grant
- Division of Cardiovascular Sciences, ERC, Manchester University Hospitals Foundation Trust, University of Manchester, Manchester, UK
- South Tees Academic Cardiovascular Unit, South Tees Hospitals NHS Foundation Trust, Middlesbrough, UK
| |
Collapse
|
9
|
van Royen FSA, van Smeden M, van Doorn S, Rutten FH, Geersing GJ. Predictive factors of clot propagation in patients with superficial venous thrombosis towards deep venous thrombosis and pulmonary embolism: a systematic review and meta-analysis. BMJ Open 2024; 14:e074818. [PMID: 38626964 PMCID: PMC11029256 DOI: 10.1136/bmjopen-2023-074818] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/18/2023] [Accepted: 04/02/2024] [Indexed: 04/19/2024] Open
Abstract
OBJECTIVE A subset of patients with superficial venous thrombosis (SVT) experiences clot propagation towards deep venous thrombosis (DVT) and/or pulmonary embolism (PE). The aim of this systematic review is to identify all clinically relevant cross-sectional and prognostic factors for predicting thrombotic complications in patients with SVT. DESIGN Systematic review. DATA SOURCES PubMed/MEDLINE and Embase were systematically searched until 3 March 2023. ELIGIBILITY CRITERIA Original research studies with patients with SVT, DVT and/or PE as the outcome and presenting cross-sectional or prognostic predictive factors. DATA EXTRACTION AND SYNTHESIS OF RESULTS The CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling (CHARMS) checklist for prognostic factor studies was used for systematic extraction of study characteristics. Per identified predictive factor, relevant estimates of univariable and multivariable predictor-outcome associations were extracted, such as ORs and HRs. Estimates of association for the most frequently reported predictors were summarised in forest plots, and meta-analyses with heterogeneity were presented. The Quality in Prognosis Studies (QUIPS) tool was used for risk of bias assessment and Grading of Recommendations, Assessment, Development and Evaluations (GRADE) for assessing the certainty of evidence. RESULTS Twenty-two studies were included (n=10 111 patients). The most reported predictive factors were high age, male sex, history of venous thromboembolism (VTE), absence of varicose veins and cancer. Pooled effect estimates were heterogenous and ranged from OR 3.12 (95% CI 1.75 to 5.59) for the cross-sectional predictor cancer to OR 0.92 (95% CI 0.56 to 1.53) for the prognostic predictor high age. The level of evidence was rated very low to low. Most studies were scored high or moderate risk of bias. CONCLUSIONS Although the pooled estimates of the predictors high age, male sex, history of VTE, cancer and absence of varicose veins showed predictive potential in isolation, variability in study designs, lack of multivariable adjustment and high risk of bias prevent firm conclusions. High-quality, multivariable studies are necessary to be able to identify individual SVT risk profiles. PROSPERO REGISTRATION NUMBER CRD42021262819.
Collapse
Affiliation(s)
- Florien Sophie-Anne van Royen
- Julius Center for Health Sciences and Primary Care, Department of General Practice and Nursing Science, University Medical Centre Utrecht, Utrecht, The Netherlands
| | - Maarten van Smeden
- Julius Center for Health Sciences and Primary Care, Department of Epidemiology and Health Economics, University Medical Centre Utrecht, Utrecht, The Netherlands
| | - Sander van Doorn
- Julius Center for Health Sciences and Primary Care, Department of General Practice and Nursing Science, University Medical Centre Utrecht, Utrecht, The Netherlands
| | - Frans H Rutten
- Julius Center for Health Sciences and Primary Care, Department of General Practice and Nursing Science, University Medical Centre Utrecht, Utrecht, The Netherlands
| | - Geert-Jan Geersing
- Julius Center for Health Sciences and Primary Care, Department of General Practice and Nursing Science, University Medical Centre Utrecht, Utrecht, The Netherlands
| |
Collapse
|
10
|
Backmund T, Bohlender T, Gaik C, Koch T, Kranke P, Nardi-Hiebl S, Vojnar B, Eberhart LHJ. [Comparison of different prediction models for the occurrence of nausea and vomiting in the postoperative phase : A systematic qualitative comparison based on prospectively defined quality indicators]. DIE ANAESTHESIOLOGIE 2024; 73:251-262. [PMID: 38319326 DOI: 10.1007/s00101-024-01386-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 11/23/2023] [Indexed: 02/07/2024]
Abstract
BACKGROUND Various prognostic prediction models exist for evaluating the risk of nausea and vomiting in the postoperative period (PONV). So far, no systematic comparison of these prognostic scores is available. METHOD A systematic literature search was carried out in seven medical databases to find publications on prognostic PONV models. Identified scores were assessed against prospectively defined quality criteria, including generalizability, validation and clinical relevance of the models. RESULTS The literature search revealed 62 relevant publications with a total of 81,834 patients which could be assigned to 8 prognostic models. The simplified Apfel score performed best, primarily because it was extensively validated. The Van den Bosch score and Sinclair score tied for second place. The simplified Koivuranta score was in third place. CONCLUSION The qualitative analysis highlights the strengths and weaknesses of each prediction system based on predetermined standardized quality criteria.
Collapse
Affiliation(s)
- T Backmund
- Klinik für Anästhesie und Intensivtherapie, Philipps Universität Marburg, Baldinger Straße, 35043 Marburg, Deutschland.
| | - T Bohlender
- Klinik für Anästhesie und Intensivtherapie, Philipps Universität Marburg, Baldinger Straße, 35043 Marburg, Deutschland
| | - C Gaik
- Klinik für Anästhesie und Intensivtherapie, Philipps Universität Marburg, Baldinger Straße, 35043 Marburg, Deutschland
| | - T Koch
- Klinik für Anästhesie und Intensivtherapie, Philipps Universität Marburg, Baldinger Straße, 35043 Marburg, Deutschland
| | - P Kranke
- Klinik und Poliklinik für Anästhesiologie, Intensivmedizin, Notfallmedizin und Schmerztherapie, Universitätsklinikum Würzburg, Oberdürrbacher Straße 6, 97080 Würzburg, Deutschland
| | - S Nardi-Hiebl
- Klinik für Anästhesie und Intensivtherapie, Philipps Universität Marburg, Baldinger Straße, 35043 Marburg, Deutschland
| | - B Vojnar
- Klinik für Anästhesie und Intensivtherapie, Philipps Universität Marburg, Baldinger Straße, 35043 Marburg, Deutschland
| | - L H J Eberhart
- Klinik für Anästhesie und Intensivtherapie, Philipps Universität Marburg, Baldinger Straße, 35043 Marburg, Deutschland
| |
Collapse
|
11
|
Guerra S, Ellmers T, Turabi R, Law M, Chauhan A, Milton-Cole R, Godfrey E, Sheehan KJ. Factors associated with concerns about falling and activity restriction in older adults after hip fracture: a mixed-methods systematic review. Eur Geriatr Med 2024; 15:305-332. [PMID: 38418713 PMCID: PMC10997732 DOI: 10.1007/s41999-024-00936-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2023] [Accepted: 01/02/2024] [Indexed: 03/02/2024]
Abstract
PURPOSE To investigate factors contributing to concerns about falling and activity restriction in the community among older adults who had a hip fracture. METHODS A mixed method systematic review with a convergent segregated approach. We searched Medline, Embase, PsycInfo, PEDRo, CINAHL and the Cochrane library. Results were synthesised narratively considering physical, psychological, environmental, care, and social factors and presented in tables. Critical appraisal was completed in duplicate. RESULTS We included 19 studies (9 qualitative, 9 observational, 1 mixed methods) representing 1480 individuals and 23 factors related to concerns about falling and activity restriction. Physical factors included falls history, comorbidities, balance, strength, mobility and functionality. Psychological factors included anxiety and neuroticism scores, perceived confidence in/control over rehabilitation and abilities, and negative/positive affect about the orthopaedic trauma, pre-fracture abilities and future needs. Environmental factors included accessibility in the home, outdoors and with transport. Social and care factors related to the presence or absence of formal and informal networks, which reduced concerns and promoted activity by providing feedback, advice, encouragement, and practical support. CONCLUSION These findings highlight that to improve concerns about falling and activity restriction after hip fracture, it is important to: improve physical and functional abilities; boost self-confidence; promote positive affect; involve relatives and carers; increase access to clinicians, and; enhance accessibility of the home, outdoors and transport. Most factors were reported on by a small number of studies of varying quality and require replication in future research.
Collapse
Affiliation(s)
- Stefanny Guerra
- Department of Population Health Sciences, School of Life Course and Population Sciences, Kings College London, London, UK.
- Bone and Joint Health, Blizard Institute, Queen Mary University of London, London, UK.
| | - Toby Ellmers
- Department of Brain Sciences, Faculty of Medicine, Imperial College London, London, UK
| | - Ruqayyah Turabi
- Department of Population Health Sciences, School of Life Course and Population Sciences, Kings College London, London, UK
| | - Magda Law
- Department of Population Health Sciences, School of Life Course and Population Sciences, Kings College London, London, UK
| | - Aishwarya Chauhan
- Department of Population Health Sciences, School of Life Course and Population Sciences, Kings College London, London, UK
| | - Rhian Milton-Cole
- Department of Population Health Sciences, School of Life Course and Population Sciences, Kings College London, London, UK
| | - Emma Godfrey
- Department of Population Health Sciences, School of Life Course and Population Sciences, Kings College London, London, UK
| | - Katie J Sheehan
- Department of Population Health Sciences, School of Life Course and Population Sciences, Kings College London, London, UK
- Bone and Joint Health, Blizard Institute, Queen Mary University of London, London, UK
| |
Collapse
|
12
|
Rijk MH, Platteel TN, van den Berg TMC, Geersing GJ, Little P, Rutten FH, van Smeden M, Venekamp RP. Prognostic factors and prediction models for hospitalisation and all-cause mortality in adults presenting to primary care with a lower respiratory tract infection: a systematic review. BMJ Open 2024; 14:e075475. [PMID: 38521534 PMCID: PMC10961536 DOI: 10.1136/bmjopen-2023-075475] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/10/2023] [Accepted: 03/12/2024] [Indexed: 03/25/2024] Open
Abstract
OBJECTIVE To identify and synthesise relevant existing prognostic factors (PF) and prediction models (PM) for hospitalisation and all-cause mortality within 90 days in primary care patients with acute lower respiratory tract infections (LRTI). DESIGN Systematic review. METHODS Systematic searches of MEDLINE, Embase and the Cochrane Library were performed. All PF and PM studies on the risk of hospitalisation or all-cause mortality within 90 days in adult primary care LRTI patients were included. The risk of bias was assessed using the Quality in Prognostic Studies tool and Prediction Model Risk Of Bias Assessment Tool tools for PF and PM studies, respectively. The results of included PF and PM studies were descriptively summarised. RESULTS Of 2799 unique records identified, 16 were included: 9 PF studies, 6 PM studies and 1 combination of both. The risk of bias was judged high for all studies, mainly due to limitations in the analysis domain. Based on reported multivariable associations in PF studies, increasing age, sex, current smoking, diabetes, a history of stroke, cancer or heart failure, previous hospitalisation, influenza vaccination (negative association), current use of systemic corticosteroids, recent antibiotic use, respiratory rate ≥25/min and diagnosis of pneumonia were identified as most promising candidate predictors. One newly developed PM was externally validated (c statistic 0.74, 95% CI 0.71 to 0.78) whereas the previously hospital-derived CRB-65 was externally validated in primary care in five studies (c statistic ranging from 0.72 (95% CI 0.63 to 0.81) to 0.79 (95% CI 0.65 to 0.92)). None of the PM studies reported measures of model calibration. CONCLUSIONS Implementation of existing models for individualised risk prediction of 90-day hospitalisation or mortality in primary care LRTI patients in everyday practice is hampered by incomplete assessment of model performance. The identified candidate predictors provide useful information for clinicians and warrant consideration when developing or updating PMs using state-of-the-art development and validation techniques. PROSPERO REGISTRATION NUMBER CRD42022341233.
Collapse
Affiliation(s)
- Merijn H Rijk
- Department of General Practice, Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht, The Netherlands
| | - Tamara N Platteel
- Department of General Practice, Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht, The Netherlands
| | - Teun M C van den Berg
- Department of General Practice, Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht, The Netherlands
| | - Geert-Jan Geersing
- Department of General Practice, Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht, The Netherlands
| | - Paul Little
- Primary Care and Population Science, University of Southampton, Southampton, UK
| | - Frans H Rutten
- Department of General Practice, Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht, The Netherlands
| | - Maarten van Smeden
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Roderick P Venekamp
- Department of General Practice, Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht, The Netherlands
| |
Collapse
|
13
|
Ewington L, Black N, Leeson C, Al Wattar BH, Quenby S. Multivariable prediction models for fetal macrosomia and large for gestational age: A systematic review. BJOG 2024. [PMID: 38465451 DOI: 10.1111/1471-0528.17802] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2023] [Revised: 02/08/2024] [Accepted: 02/22/2024] [Indexed: 03/12/2024]
Abstract
BACKGROUND The identification of large for gestational age (LGA) and macrosomic fetuses is essential for counselling and managing these pregnancies. OBJECTIVES To systematically review the literature for multivariable prediction models for LGA and macrosomia, assessing the performance, quality and applicability of the included model in clinical practice. SEARCH STRATEGY MEDLINE, EMBASE and Cochrane Library were searched until June 2022. SELECTION CRITERIA We included observational and experimental studies reporting the development and/or validation of any multivariable prediction model for fetal macrosomia and/or LGA. We excluded studies that used a single variable or did not evaluate model performance. DATA COLLECTION AND ANALYSIS Data were extracted using the Checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies checklist. The model performance measures discrimination, calibration and validation were extracted. The quality and completion of reporting within each study was assessed by its adherence to the TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) checklist. The risk of bias and applicability were measured using PROBAST (Prediction model Risk Of Bias Assessment Tool). MAIN RESULTS A total of 8442 citations were identified, with 58 included in the analysis: 32/58 (55.2%) developed, 21/58 (36.2%) developed and internally validated and 2/58 (3.4%) developed and externally validated a model. Only three studies externally validated pre-existing models. Macrosomia and LGA were differentially defined by many studies. In total, 111 multivariable prediction models were developed using 112 different variables. Model discrimination was wide ranging area under the receiver operating characteristics curve (AUROC 0.56-0.96) and few studies reported calibration (11/58, 19.0%). Only 5/58 (8.6%) studies had a low risk of bias. CONCLUSIONS There are currently no multivariable prediction models for macrosomia/LGA that are ready for clinical implementation.
Collapse
Affiliation(s)
- Lauren Ewington
- Division of Biomedical Sciences, University of Warwick, Coventry, UK
- University Hospitals Coventry and Warwickshire, Coventry, UK
| | - Naomi Black
- Division of Biomedical Sciences, University of Warwick, Coventry, UK
- University Hospitals Coventry and Warwickshire, Coventry, UK
| | - Charlotte Leeson
- Division of Biomedical Sciences, University of Warwick, Coventry, UK
- University Hospitals Coventry and Warwickshire, Coventry, UK
| | - Bassel H Al Wattar
- Beginnings Assisted Conception Unit, Epsom and St Helier University Hospitals, London, UK
- Comprehensive Clinical Trials Unit, Institute for Clinical Trials and Methodology, University College London, London, UK
| | - Siobhan Quenby
- Division of Biomedical Sciences, University of Warwick, Coventry, UK
- University Hospitals Coventry and Warwickshire, Coventry, UK
| |
Collapse
|
14
|
Jiu L, Wang J, Javier Somolinos-Simón F, Tapia-Galisteo J, García-Sáez G, Hernando M, Li X, Vreman RA, Mantel-Teeuwisse AK, Goettsch WG. A literature review of quality assessment and applicability to HTA of risk prediction models of coronary heart disease in patients with diabetes. Diabetes Res Clin Pract 2024; 209:111574. [PMID: 38346592 DOI: 10.1016/j.diabres.2024.111574] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/15/2023] [Revised: 01/17/2024] [Accepted: 02/06/2024] [Indexed: 02/23/2024]
Abstract
This literature review had two objectives: to identify models for predicting the risk of coronary heart diseases in patients with diabetes (DM); and to assess model quality in terms of risk of bias (RoB) and applicability for the purpose of health technology assessment (HTA). We undertook a targeted review of journal articles published in English, Dutch, Chinese, or Spanish in 5 databases from 1st January 2016 to 18th December 2022, and searched three systematic reviews for the models published after 2012. We used PROBAST (Prediction model Risk Of Bias Assessment Tool) to assess RoB, and used findings from Betts et al. 2019, which summarized recommendations and criticisms of HTA agencies on cardiovascular risk prediction models, to assess model applicability for the purpose of HTA. As a result, 71 % and 67 % models reporting C-index showed good discrimination abilities (C-index >= 0.7). Of the 26 model studies and 30 models identified, only one model study showed low RoB in all domains, and no model was fully applicable for HTA. Since the major cause of high RoB is inappropriate use of analysis method, we advise clinicians to carefully examine the model performance declared by model developers, and to trust a model if all PROBAST domains except analysis show low RoB and at least one validation study conducted in the same setting (e.g. country) is available. Moreover, since general model applicability is not informative for HTA, novel adapted tools may need to be developed.
Collapse
Affiliation(s)
- Li Jiu
- Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Universiteitsweg 99, 3584 CG Utrecht, Netherlands
| | - Junfeng Wang
- Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Universiteitsweg 99, 3584 CG Utrecht, Netherlands
| | - Francisco Javier Somolinos-Simón
- Bioengineering and Telemedicine Group, Centro de Tecnología Biomédica, ETSI de Telecomunicación, Universidad Politécnica de Madrid, Parque Científico y Tecnológico de la UPM, Crta. M40, Km. 38, 28223 Pozuelo de Alarcón, Madrid, Spain
| | - Jose Tapia-Galisteo
- Bioengineering and Telemedicine Group, Centro de Tecnología Biomédica, ETSI de Telecomunicación, Universidad Politécnica de Madrid, Parque Científico y Tecnológico de la UPM, Crta. M40, Km. 38, 28223 Pozuelo de Alarcón, Madrid, Spain; CIBER-BBN: Networking Research Centre for Bioengineering, Biomaterials and Nanomedicine, Parque Científico y Tecnológico de la UPM, Crta. M40, Km. 38, 28223 Pozuelo de Alarcón, Madrid, Spain
| | - Gema García-Sáez
- Bioengineering and Telemedicine Group, Centro de Tecnología Biomédica, ETSI de Telecomunicación, Universidad Politécnica de Madrid, Parque Científico y Tecnológico de la UPM, Crta. M40, Km. 38, 28223 Pozuelo de Alarcón, Madrid, Spain; CIBER-BBN: Networking Research Centre for Bioengineering, Biomaterials and Nanomedicine, Parque Científico y Tecnológico de la UPM, Crta. M40, Km. 38, 28223 Pozuelo de Alarcón, Madrid, Spain
| | - Mariaelena Hernando
- Bioengineering and Telemedicine Group, Centro de Tecnología Biomédica, ETSI de Telecomunicación, Universidad Politécnica de Madrid, Parque Científico y Tecnológico de la UPM, Crta. M40, Km. 38, 28223 Pozuelo de Alarcón, Madrid, Spain; CIBER-BBN: Networking Research Centre for Bioengineering, Biomaterials and Nanomedicine, Parque Científico y Tecnológico de la UPM, Crta. M40, Km. 38, 28223 Pozuelo de Alarcón, Madrid, Spain
| | - Xinyu Li
- Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Universiteitsweg 99, 3584 CG Utrecht, Netherlands; University of Groningen, Faculty of Science and Engineering, Groningen Research Institute of Pharmacy, Broerstraat 5, 9712 CP Groningen, the Netherlands
| | - Rick A Vreman
- Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Universiteitsweg 99, 3584 CG Utrecht, Netherlands; National Health Care Institute (ZIN), Diemen, Willem Dudokhof 1, 1112 ZA Diemen, Netherlands
| | - Aukje K Mantel-Teeuwisse
- Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Universiteitsweg 99, 3584 CG Utrecht, Netherlands
| | - Wim G Goettsch
- Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Universiteitsweg 99, 3584 CG Utrecht, Netherlands; National Health Care Institute (ZIN), Diemen, Willem Dudokhof 1, 1112 ZA Diemen, Netherlands.
| |
Collapse
|
15
|
Tan J, Liu C, Yang M, Xiong Y, Huang S, Qi Y, Chen M, Thabane L, Liu X, He L, Sun X. Investigation of statistical methods used in prognostic prediction models for obstetric care: A 10 year-span cross-sectional study. Acta Obstet Gynecol Scand 2024; 103:611-620. [PMID: 38140844 PMCID: PMC10867372 DOI: 10.1111/aogs.14757] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2023] [Revised: 11/06/2023] [Accepted: 12/06/2023] [Indexed: 12/24/2023]
Abstract
INTRODUCTION Obstetric care is a highly active area in the development and application of prognostic prediction models. The development and validation of these models often require the utilization of advanced statistical techniques. However, failure to adhere to rigorous methodological standards could greatly undermine the reliability and trustworthiness of the resultant models. Consequently, the aim of our study was to examine the current statistical practices employed in obstetric care and offer recommendations to enhance the utilization of statistical methods in the development of prognostic prediction models. MATERIAL AND METHODS We conducted a cross-sectional survey using a sample of studies developing or validating prognostic prediction models for obstetric care published in a 10-year span (2011-2020). A structured questionnaire was developed to investigate the statistical issues in five domains, including model derivation (predictor selection and algorithm development), model validation (internal and external), model performance, model presentation, and risk threshold setting. On the ground of survey results and existing guidelines, a list of recommendations for statistical methods in prognostic models was developed. RESULTS A total of 112 eligible studies were included, with 107 reporting model development and five exclusively reporting external validation. During model development, 58.9% of the studies did not include any form of validation. Of these, 46.4% used stepwise regression in a crude manner for predictor selection, while two-thirds made decisions on retaining or dropping candidate predictors solely based on p-values. Additionally, 26.2% transformed continuous predictors into categorical variables, and 80.4% did not consider nonlinear relationships between predictors and outcomes. Surprisingly, 94.4% of the studies did not examine the correlation between predictors. Moreover, 47.1% of the studies did not compare population characteristics between the development and external validation datasets, and only one-fifth evaluated both discrimination and calibration. Furthermore, 53.6% of the studies did not clearly present the model, and less than half established a risk threshold to define risk categories. In light of these findings, 10 recommendations were formulated to promote the appropriate use of statistical methods. CONCLUSIONS The use of statistical methods is not yet optimal. Ten recommendations were offered to assist the statistical methods of prognostic prediction models in obstetric care.
Collapse
Affiliation(s)
- Jing Tan
- Chinese Evidence‐based Medicine Center, West China HospitalSichuan UniversityChengduSichuanChina
- NMPA Key Laboratory for Real World Data Research and Evaluation in HainanChengduSichuanChina
- Department of Health Research Methods, Evidence, and ImpactMcMaster UniversityHamiltonOntarioCanada
- Biostatistics UnitSt Joseph's Healthcare—HamiltonHamiltonOntarioCanada
| | - Chunrong Liu
- Chinese Evidence‐based Medicine Center, West China HospitalSichuan UniversityChengduSichuanChina
- NMPA Key Laboratory for Real World Data Research and Evaluation in HainanChengduSichuanChina
| | - Min Yang
- Department of Epidemiology and Biostatistics, West China School of Public HealthSichuan UniversityChengduChina
- Faculty of Health, Design and ArtSwinburne Technology UniversityMelbourneVictoriaAustralia
| | - Yiquan Xiong
- Chinese Evidence‐based Medicine Center, West China HospitalSichuan UniversityChengduSichuanChina
- NMPA Key Laboratory for Real World Data Research and Evaluation in HainanChengduSichuanChina
| | - Shiyao Huang
- Chinese Evidence‐based Medicine Center, West China HospitalSichuan UniversityChengduSichuanChina
- NMPA Key Laboratory for Real World Data Research and Evaluation in HainanChengduSichuanChina
| | - Yana Qi
- Chinese Evidence‐based Medicine Center, West China HospitalSichuan UniversityChengduSichuanChina
- NMPA Key Laboratory for Real World Data Research and Evaluation in HainanChengduSichuanChina
| | - Meng Chen
- Department of Obstetrics and Gynecology, and Key Laboratory of Birth Defects and Related Diseases of Women and Children (Sichuan University), Ministry of Education, West China Second University HospitalSichuan UniversityChengduSichuanChina
| | - Lehana Thabane
- Department of Health Research Methods, Evidence, and ImpactMcMaster UniversityHamiltonOntarioCanada
- Biostatistics UnitSt Joseph's Healthcare—HamiltonHamiltonOntarioCanada
| | - Xinghui Liu
- Department of Obstetrics and Gynecology, and Key Laboratory of Birth Defects and Related Diseases of Women and Children (Sichuan University), Ministry of Education, West China Second University HospitalSichuan UniversityChengduSichuanChina
| | - Lin He
- The Intelligence Library Center, Ministry of Science and Technology, Chinese Evidence‐Based Medicine Center, West China HospitalSichuan UniversityChengduSichuanChina
| | - Xin Sun
- Chinese Evidence‐based Medicine Center, West China HospitalSichuan UniversityChengduSichuanChina
- NMPA Key Laboratory for Real World Data Research and Evaluation in HainanChengduSichuanChina
| |
Collapse
|
16
|
Hakimjavadi R, Basiratzadeh S, Wai EK, Baddour N, Kingwell S, Michalowski W, Stratton A, Tsai E, Viktor H, Phan P. Multivariable Prediction Models for Traumatic Spinal Cord Injury: A Systematic Review. Top Spinal Cord Inj Rehabil 2024; 30:1-44. [PMID: 38433735 PMCID: PMC10906375 DOI: 10.46292/sci23-00010] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/05/2024]
Abstract
Background Traumatic spinal cord injuries (TSCI) greatly affect the lives of patients and their families. Prognostication may improve treatment strategies, health care resource allocation, and counseling. Multivariable clinical prediction models (CPMs) for prognosis are tools that can estimate an absolute risk or probability that an outcome will occur. Objectives We sought to systematically review the existing literature on CPMs for TSCI and critically examine the predictor selection methods used. Methods We searched MEDLINE, PubMed, Embase, Scopus, and IEEE for English peer-reviewed studies and relevant references that developed multivariable CPMs to prognosticate patient-centered outcomes in adults with TSCI. Using narrative synthesis, we summarized the characteristics of the included studies and their CPMs, focusing on the predictor selection process. Results We screened 663 titles and abstracts; of these, 21 full-text studies (2009-2020) consisting of 33 distinct CPMs were included. The data analysis domain was most commonly at a high risk of bias when assessed for methodological quality. Model presentation formats were inconsistently included with published CPMs; only two studies followed established guidelines for transparent reporting of multivariable prediction models. Authors frequently cited previous literature for their initial selection of predictors, and stepwise selection was the most frequent predictor selection method during modelling. Conclusion Prediction modelling studies for TSCI serve clinicians who counsel patients, researchers aiming to risk-stratify participants for clinical trials, and patients coping with their injury. Poor methodological rigor in data analysis, inconsistent transparent reporting, and a lack of model presentation formats are vital areas for improvement in TSCI CPM research.
Collapse
Affiliation(s)
| | | | - Eugene K. Wai
- University of Ottawa, Ottawa, Ontario, Canada
- The Ottawa Hospital, Ottawa, Ontario, Canada
| | | | - Stephen Kingwell
- University of Ottawa, Ottawa, Ontario, Canada
- The Ottawa Hospital, Ottawa, Ontario, Canada
| | | | - Alexandra Stratton
- University of Ottawa, Ottawa, Ontario, Canada
- The Ottawa Hospital, Ottawa, Ontario, Canada
| | - Eve Tsai
- University of Ottawa, Ottawa, Ontario, Canada
- The Ottawa Hospital, Ottawa, Ontario, Canada
| | | | - Philippe Phan
- University of Ottawa, Ottawa, Ontario, Canada
- The Ottawa Hospital, Ottawa, Ontario, Canada
| |
Collapse
|
17
|
Argus L, Taylor M, Ouzounian M, Venkateswaran R, Grant SW. Risk Prediction Models for Long-Term Survival after Cardiac Surgery: A Systematic Review. Thorac Cardiovasc Surg 2024; 72:29-39. [PMID: 36750201 DOI: 10.1055/s-0043-1760747] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/09/2023]
Abstract
BACKGROUND The reporting of alternative postoperative measures of quality after cardiac surgery is becoming increasingly important as in-hospital mortality rates continue to decline. This study aims to systematically review and assess risk models designed to predict long-term outcomes after cardiac surgery. METHODS The MEDLINE and Embase databases were searched for articles published between 1990 and 2020. Studies developing or validating risk prediction models for long-term outcomes after cardiac surgery were included. Data were extracted using checklists for critical appraisal and systematic review of prediction modeling studies. RESULTS Eleven studies were identified for inclusion in the review, of which nine studies described the development of long-term risk prediction models after cardiac surgery and two were external validation studies. A total of 70 predictors were included across the nine models. The most frequently used predictors were age (n = 9), peripheral vascular disease (n = 8), renal disease (n = 8), and pulmonary disease (n = 8). Despite all models demonstrating acceptable performance on internal validation, only two models underwent external validation, both of which performed poorly. CONCLUSION Nine risk prediction models predicting long-term mortality after cardiac surgery have been identified in this review. Statistical issues with model development, limited inclusion of outcomes beyond 5 years of follow-up, and a lack of external validation studies means that none of the models identified can be recommended for use in contemporary cardiac surgery. Further work is needed either to successfully externally validate existing models or to develop new models. Newly developed models should aim to use standardized long-term specific reproducible outcome measures.
Collapse
Affiliation(s)
- Leah Argus
- The University of Manchester, Manchester, United Kingdom
| | - Marcus Taylor
- Department of Cardiothoracic Surgery, Manchester University NHS Foundation Trust, Manchester, United Kingdom
| | - Maral Ouzounian
- Division of Cardiovascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, Ontario, Canada
| | - Rajamiyer Venkateswaran
- Department of Cardiothoracic Surgery, Manchester University NHS Foundation Trust, Manchester, United Kingdom
| | - Stuart W Grant
- Division of Cardiovascular Sciences, University of Manchester, Manchester, United Kingdom
- Academic Cardiovascular Unit, South Tees Hospitals NHS Foundation Trust, Middlesborough, United Kingdom
| |
Collapse
|
18
|
Veličković VM, Macmillan T, Kottner J, Crompton A, Munro I, Paine A, Savović J, Spelman T, Clark M, Smit HJ, Smola H, Webb N, Steyerberg E. Prognostic models for clinical outcomes in patients with venous leg ulcers: A systematic review. J Vasc Surg Venous Lymphat Disord 2024; 12:101673. [PMID: 37689364 DOI: 10.1016/j.jvsv.2023.06.017] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2023] [Revised: 06/02/2023] [Accepted: 06/26/2023] [Indexed: 09/11/2023]
Abstract
OBJECTIVE The purpose of this review was to identify prognostic models for clinical application in patients with venous leg ulcers (VLUs). METHODS Literature searches were conducted in Embase, Medline, Cochrane, and CINAHL databases from inception to December 22, 2021. Eligible studies reported prognostic models aimed at developing, validating, and adjusting multivariable prognostic models that include multiple prognostic factors combined, and that predicted clinical outcomes. Methodological quality was assessed using the CHARMS checklist and PROBAST short form questionnaire. RESULTS Thirteen studies were identified, of which three were validation studies of previously published models, four reported derivation and validation of models, and the remainder reported derivation models only. There was substantial heterogeneity in the model characteristics, including 11 studies focused on wound healing outcomes reporting 91 different predictors. Three studies shared similar predicted outcomes, follow-up timepoint and used a Cox proportional hazards model. However, these models reported different predictor selection methods and different predictors and it was therefore not feasible to summarize performance, such as discriminative ability. CONCLUSIONS There are no standout risk prediction models in the literature with promising clinical application for patients with VLUs. Future research should focus on developing and validating high-performing models in wider VLU populations.
Collapse
Affiliation(s)
- Vladica M Veličković
- Health Economics and Outcome Research (HEOR) Department, Hartmann Group, Heidenheim, Germany; Institute of Public Health, Medical Decision Making and Health Technology Assessment, UMIT, Hall, Tirol, Austria.
| | | | - Jan Kottner
- Charité-Universitätsmedizin Berlin, Berlin, Germany
| | | | | | - Abby Paine
- Source Health Economics, London, United Kingdom
| | - Jelena Savović
- Bristol Population Health Science Institute, Bristol, United Kingdom
| | - Tim Spelman
- Burnet Institute, Melbourne, Australia, School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia
| | - Michael Clark
- Welsh Wound Innovation Centre, Ynysmaerdy, Pontyclun, United Kingdom
| | | | - Hans Smola
- Health Economics and Outcome Research (HEOR) Department, Hartmann Group, Heidenheim, Germany; Department of Dermatology, University of Cologne, Cologne, Germany
| | - Neil Webb
- Source Health Economics, London, United Kingdom
| | | |
Collapse
|
19
|
Parekh A, Satish S, Dulhanty L, Berzuini C, Patel H. Clinical prediction models for aneurysmal subarachnoid hemorrhage: a systematic review update. J Neurointerv Surg 2023:jnis-2023-021107. [PMID: 38129109 DOI: 10.1136/jnis-2023-021107] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2023] [Accepted: 12/05/2023] [Indexed: 12/23/2023]
Abstract
BACKGROUND A systematic review of clinical prediction models for aneurysmal subarachnoid hemorrhage (aSAH) reported in 2011 noted that clinical prediction models for aSAH were developed using poor methods and were not externally validated. This study aimed to update the above review to guide the future development of predictive models in aSAH. METHODS We systematically searched Embase and MEDLINE databases (January 2010 to February 2022) for articles that reported the development of a clinical prediction model to predict functional outcomes in aSAH. Our reviews are based on the items included in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement (PRISMA) checklist, and on data abstracted from each study in accord with the Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) 2014 checklist. Bias and applicability were assessed using the Prediction model Risk Of Bias Assessment Tool (PROBAST). RESULTS We reviewed data on 30 466 patients contributing to 29 prediction models abstracted from 22 studies identified from an initial search of 7858 studies. Most models were developed using logistic regression (n=20) or machine learning (n=9) with prognostic variables selected through a range of methods. Age (n=13), World Federation of Neurological Surgeons (WFNS) grade (n=11), hypertension (n=6), aneurysm size (n=5), Fisher grade (n=12), Hunt and Hess score (n=5), and Glasgow Coma Scale (n=8) were the variables most frequently included in the reported models. External validation was performed in only four studies. All but one model had a high or unclear risk of bias due to poor performance or lack of validation. CONCLUSION Externally validated models for the prediction of functional outcome in aSAH patients have now become available. However, most of them still have a high risk of bias.
Collapse
Affiliation(s)
| | | | - Louise Dulhanty
- Salford Royal Hospital Manchester Centre for Clinical Neurosciences, Salford, UK
| | - Carlo Berzuini
- Centre for Biostatistics, The University of Manchester, Manchester, UK
| | - Hiren Patel
- Greater Manchester Neurosciences Centre, Salford Royal NHS Foundation Trust, Salford, UK
| |
Collapse
|
20
|
Jalleh RJ, Umapathysivam MM, Plummer MP, Deane A, Jones KL, Horowitz M. Postprandial plasma GLP-1 levels are elevated in individuals with postprandial hypoglycaemia following Roux-en-Y gastric bypass - a systematic review. Rev Endocr Metab Disord 2023; 24:1075-1088. [PMID: 37439960 PMCID: PMC10697890 DOI: 10.1007/s11154-023-09823-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 06/28/2023] [Indexed: 07/14/2023]
Abstract
BACKGROUND AND AIMS Bariatric surgery is the most effective treatment in individuals with obesity to achieve remission of type 2 diabetes. Post-bariatric surgery hypoglycaemia occurs frequently, and management remains suboptimal, because of a poor understanding of the underlying pathophysiology. The glucoregulatory hormone responses to nutrients in individuals with and without post-bariatric surgery hypoglycaemia have not been systematically examined. MATERIALS AND METHODS The study protocol was prospectively registered with PROSPERO. PubMed, EMBASE, Web of Science and the Cochrane databases were searched for publications between January 1990 and November 2021 using MeSH terms related to post-bariatric surgery hypoglycaemia. Studies were included if they evaluated individuals with post-bariatric surgery hypoglycaemia and included measurements of plasma glucagon-like peptide-1 (GLP-1), glucose-dependent insulinotropic polypeptide (GIP), insulin, C-peptide and/or glucagon concentrations following an ingested nutrient load. Glycated haemoglobin (HbA1c) was also evaluated. A random-effects meta-analysis was performed, and Hedges' g (standardised mean difference) and 95% confidence intervals were reported for all outcomes where sufficient studies were available. The τ2 estimate and I2 statistic were used as tests for heterogeneity and a funnel plot with the Egger regression-based test was used to evaluate for publication bias. RESULTS From 377 identified publications, 12 were included in the analysis. In all 12 studies, the type of bariatric surgery was Roux-en-Y gastric bypass (RYGB). Comparing individuals with and without post-bariatric surgery hypoglycaemia following an ingested nutrient load, the standardised mean difference in peak GLP-1 was 0.57 (95% CI, 0.32, 0.82), peak GIP 0.05 (-0.26, 0.36), peak insulin 0.84 (0.44, 1.23), peak C-peptide 0.69 (0.28, 1.1) and peak glucagon 0.05 (-0.26, 0.36). HbA1c was less in individuals with hypoglycaemia - 0.40 (-0.67, -0.12). There was no evidence of substantial heterogeneity in any outcome except for peak insulin: τ2 = 0.2, I2 = 54.3. No publication bias was evident. CONCLUSION Following RYGB, postprandial peak plasma GLP-1, insulin and C-peptide concentrations are greater in individuals with post-bariatric surgery hypoglycaemia, while HbA1c is less. These observations support the concept that antagonism of GLP-1 would prove beneficial in the management of individuals with hypoglycaemia following RYGB.PROSPERO Registration Number: CRD42021287515.
Collapse
Affiliation(s)
- Ryan Joseph Jalleh
- Adelaide Medical School, The University of Adelaide, South Australia, Australia
- Endocrine and Metabolic Unit, Royal Adelaide Hospital, South Australia, Australia
- Diabetes and Endocrine Services, Northern Adelaide Local Health Network, South Australia, Australia
| | - Mahesh Michael Umapathysivam
- Adelaide Medical School, The University of Adelaide, South Australia, Australia
- Endocrine and Metabolic Unit, Royal Adelaide Hospital, South Australia, Australia
| | - Mark Philip Plummer
- Adelaide Medical School, The University of Adelaide, South Australia, Australia
| | - Adam Deane
- Intensive Care Unit, Royal Melbourne Hospital, Parkville, VIC, Australia
| | - Karen Louise Jones
- Adelaide Medical School, The University of Adelaide, South Australia, Australia
- Endocrine and Metabolic Unit, Royal Adelaide Hospital, South Australia, Australia
| | - Michael Horowitz
- Adelaide Medical School, The University of Adelaide, South Australia, Australia.
- Endocrine and Metabolic Unit, Royal Adelaide Hospital, South Australia, Australia.
| |
Collapse
|
21
|
Hill CJ, Banerjee A, Hill J, Stapleton C. Diagnostic clinical prediction rules for categorising low back pain: A systematic review. Musculoskeletal Care 2023; 21:1482-1496. [PMID: 37807828 DOI: 10.1002/msc.1816] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2023] [Accepted: 08/31/2023] [Indexed: 10/10/2023]
Abstract
BACKGROUND Low back pain (LBP) is a common complex condition, where specific diagnoses are hard to identify. Diagnostic clinical prediction rules (CPRs) are known to improve clinical decision-making. A review of LBP diagnostic-CPRs by Haskins et al. (2015) identified six diagnostic-CPRs in derivation phases of development, with one tool ready for implementation. Recent progress on these tools is unknown. Therefore, this review aimed to investigate developments in LBP diagnostic-CPRs and evaluate their readiness for implementation. METHODS A systematic review was performed on five databases (Medline, Amed, Cochrane Library, PsycInfo, and CINAHL) combined with hand-searching and citation-tracking to identify eligible studies. Study and tool quality were appraised for risk of bias (Quality Assessment of Diagnostic Accuracy Studies-2), methodological quality (checklist using accepted CPR methodological standards), and CPR tool appraisal (GRade and ASsess Predictive). RESULTS Of 5021 studies screened, 11 diagnostic-CPRs were identified. Of the six previously known, three have been externally validated but not yet undergone impact analysis. Five new tools have been identified since Haskin et al. (2015); all are still in derivation stages. The most validated diagnostic-CPRs include the Lumbar-Spinal-Stenosis-Self-Administered-Self-Reported-History-Questionnaire and Diagnosis-Support-Tool-to-Identify-Lumbar-Spinal-Stenosis, and the StEP-tool which differentiates radicular from axial-LBP. CONCLUSIONS This updated review of LBP diagnostic CPRs found five new tools, all in the early stages of development. Three previously known tools have now been externally validated but should be used with caution until impact evaluation studies are undertaken. Future funding should focus on externally validating and assessing the impact of existing CPRs on clinical decision-making.
Collapse
|
22
|
Byrne JF, Mongan D, Murphy J, Healy C, Fӧcking M, Cannon M, Cotter DR. Prognostic models predicting transition to psychotic disorder using blood-based biomarkers: a systematic review and critical appraisal. Transl Psychiatry 2023; 13:333. [PMID: 37898606 PMCID: PMC10613280 DOI: 10.1038/s41398-023-02623-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/16/2022] [Revised: 09/15/2023] [Accepted: 10/06/2023] [Indexed: 10/30/2023] Open
Abstract
Accumulating evidence suggests individuals with psychotic disorder show abnormalities in metabolic and inflammatory processes. Recently, several studies have employed blood-based predictors in models predicting transition to psychotic disorder in risk-enriched populations. A systematic review of the performance and methodology of prognostic models using blood-based biomarkers in the prediction of psychotic disorder from risk-enriched populations is warranted. Databases (PubMed, EMBASE and PsycINFO) were searched for eligible texts from 1998 to 15/05/2023, which detailed model development or validation studies. The checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) was used to guide data extraction from eligible texts and the Prediction Model Risk of Bias Assessment Tool (PROBAST) was used to assess the risk of bias and applicability of the studies. A narrative synthesis of the included studies was performed. Seventeen eligible studies were identified: 16 eligible model development studies and one eligible model validation study. A wide range of biomarkers were assessed, including nucleic acids, proteins, metabolites, and lipids. The range of C-index (area under the curve) estimates reported for the models was 0.67-1.00. No studies assessed model calibration. According to PROBAST criteria, all studies were at high risk of bias in the analysis domain. While a wide range of potentially predictive biomarkers were identified in the included studies, most studies did not account for overfitting in model performance estimates, no studies assessed calibration, and all models were at high risk of bias according to PROBAST criteria. External validation of the models is needed to provide more accurate estimates of their performance. Future studies which follow the latest available methodological and reporting guidelines and adopt strategies to accommodate required sample sizes for model development or validation will clarify the value of including blood-based biomarkers in models predicting psychosis.
Collapse
Affiliation(s)
- Jonah F Byrne
- Department of Psychiatry, Royal College of Surgeons in Ireland, Dublin, Ireland.
- SFI FutureNeuro Research Centre, Royal College of Surgeons in Ireland, Dublin, Ireland.
| | - David Mongan
- Department of Psychiatry, Royal College of Surgeons in Ireland, Dublin, Ireland
- Centre for Public Health, Queen's University Belfast, Belfast, United Kingdom
| | - Jennifer Murphy
- Department of Psychiatry, Royal College of Surgeons in Ireland, Dublin, Ireland
| | - Colm Healy
- Department of Psychiatry, Royal College of Surgeons in Ireland, Dublin, Ireland
| | - Melanie Fӧcking
- Department of Psychiatry, Royal College of Surgeons in Ireland, Dublin, Ireland
| | - Mary Cannon
- Department of Psychiatry, Royal College of Surgeons in Ireland, Dublin, Ireland
- SFI FutureNeuro Research Centre, Royal College of Surgeons in Ireland, Dublin, Ireland
| | - David R Cotter
- Department of Psychiatry, Royal College of Surgeons in Ireland, Dublin, Ireland
- SFI FutureNeuro Research Centre, Royal College of Surgeons in Ireland, Dublin, Ireland
| |
Collapse
|
23
|
Wilson J, Chowdhury F, Hassan S, Harriss EK, Alves F, Dahal P, Stepniewska K, Guérin PJ. Prognostic prediction models for clinical outcomes in patients diagnosed with visceral leishmaniasis: protocol for a systematic review. BMJ Open 2023; 13:e075597. [PMID: 37879686 PMCID: PMC10603465 DOI: 10.1136/bmjopen-2023-075597] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/12/2023] [Accepted: 10/05/2023] [Indexed: 10/27/2023] Open
Abstract
INTRODUCTION Visceral leishmaniasis (VL) is a neglected tropical disease responsible for many thousands of preventable deaths each year. Symptomatic patients often struggle to access effective treatment, without which death is the norm. Risk prediction tools support clinical teams and policymakers in identifying high-risk patients who could benefit from more intensive management pathways. Investigators interested in using their clinical data for prognostic research should first identify currently available models that are candidates for validation and possible updating. Addressing these needs, we aim to identify, summarise and appraise the available models predicting clinical outcomes in VL patients. METHODS AND ANALYSIS We will include studies that have developed, validated or updated prognostic models predicting future clinical outcomes in patients diagnosed with VL. Systematic reviews and meta-analyses that include eligible studies are also considered for review. Conference abstracts and educational theses are excluded. Data extraction, appraisal and reporting will follow current methodological guidelines. Ovid Embase; Ovid MEDLINE; the Web of Science Core Collection, SciELO and LILACS are searched from database inception to 1 March 2023 using terms developed for the identification of prediction models, and with no language restriction. Screening, data extraction and risk of bias assessment will be performed in duplicate with discordance resolved by a third independent reviewer. Risk of bias will be assessed using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). Tables and figures will compare and contrast key model information, including source data, participants, model development and performance measures, and risk of bias. We will consider the strengths, limitations and clinical applicability of the identified models. ETHICS AND DISSEMINATION Ethics approval is not required for this review. The systematic review and all accompanying data will be submitted to an open-access journal. Findings will also be disseminated through the research group's website (www.iddo.org/research-themes/visceral-leishmaniasis) and social media channels. PROSPERO REGISTRATION NUMBER CRD42023417226.
Collapse
Affiliation(s)
- James Wilson
- Infectious Diseases Data Observatory (IDDO), University of Oxford, Oxford, UK
- Centre for Tropical Medicine and Global Health, University of Oxford, Oxford, UK
| | - Forhad Chowdhury
- Infectious Diseases Data Observatory (IDDO), University of Oxford, Oxford, UK
- Centre for Tropical Medicine and Global Health, University of Oxford, Oxford, UK
| | - Shermarke Hassan
- Infectious Diseases Data Observatory (IDDO), University of Oxford, Oxford, UK
- Centre for Tropical Medicine and Global Health, University of Oxford, Oxford, UK
| | - Elinor K Harriss
- Bodleian Health Care Libraries, University of Oxford, Oxford, UK
| | - Fabiana Alves
- Drugs for Neglected Disease Initiative, Geneva, Switzerland
| | - Prabin Dahal
- Infectious Diseases Data Observatory (IDDO), University of Oxford, Oxford, UK
- Centre for Tropical Medicine and Global Health, University of Oxford, Oxford, UK
| | - Kasia Stepniewska
- Infectious Diseases Data Observatory (IDDO), University of Oxford, Oxford, UK
- Centre for Tropical Medicine and Global Health, University of Oxford, Oxford, UK
| | - Philippe J Guérin
- Infectious Diseases Data Observatory (IDDO), University of Oxford, Oxford, UK
- Centre for Tropical Medicine and Global Health, University of Oxford, Oxford, UK
| |
Collapse
|
24
|
Lu Z, Zhou X, Fu L, Li Y, Tian T, Liu Q, Zou H. Prognostic prediction models for oropharyngeal squamous cell carcinoma (OPSCC): a protocol for systematic review, critical appraisal and meta-analysis. BMJ Open 2023; 13:e073375. [PMID: 37827742 PMCID: PMC10582955 DOI: 10.1136/bmjopen-2023-073375] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/03/2023] [Accepted: 08/25/2023] [Indexed: 10/14/2023] Open
Abstract
INTRODUCTION Oropharyngeal squamous cell carcinoma (OPSCC) is increasingly prevalent and has significantly heterogeneous risks of survival for diagnosed individuals due to the inter-related risk factors. Precise prediction of the risk of survival for an individual patient with OPSCC presents a useful adjunct to therapeutic decision-making regarding the management of OPSCC. The aim of this systematic review, critical appraisal and meta-analysis is to assess prognostic prediction models for OPSCC and lay a foundation for future research programmes to develop and validate prognostic prediction models for OPSCC. METHODS AND ANALYSIS This protocol will follow the Preferred Reporting Items for Systematic Review and Meta-Analyses Protocol statement. Based on predefined criteria, electronic databases including MEDLINE, Embase, Web of Science, the Cochrane Library and China National Knowledge Infrastructure (CNKI) will be searched for relevant studies without language restrictions from inception of databases to present. This study will systematically review published prognostic prediction models for survival outcomes in patients with OPSCC, describe their characteristics, compare performance and assess risk of bias and real-world clinical utility. Selection of eligible studies, data extraction and critical appraisal will be conducted independently by two reviewers. A third reviewer will resolve any disagreements. Included studies will be systematically summarised using appropriate tools designed for prognostic prediction modelling studies. Risk of bias and quality of studies will be assessed using the Prediction Model Risk of Bias Assessment Tool and the Transparent Reporting of a multivariable prediction model for individual prognosis or diagnosis. Performance measures of these models will be pooled and analysed with meta-analyses if feasible. ETHICS AND DISSEMINATION This review will be conducted completely based on published data, so approval from an ethics committee or written consent is not required. The results will be disseminated through a peer-reviewed publication. PROSPERO REGISTRATION NUMBER CRD42023400272.
Collapse
Affiliation(s)
- Zhen Lu
- School of Public Health (Shenzhen), Sun Yat-sen University, Shenzhen, Guangdong, China
| | - Xinyi Zhou
- School of Public Health (Shenzhen), Sun Yat-sen University, Shenzhen, Guangdong, China
| | - Leiwen Fu
- School of Public Health (Shenzhen), Sun Yat-sen University, Shenzhen, Guangdong, China
| | - Yuwei Li
- School of Public Health (Shenzhen), Sun Yat-sen University, Shenzhen, Guangdong, China
| | - Tian Tian
- School of Public Health (Shenzhen), Sun Yat-sen University, Shenzhen, Guangdong, China
| | - Qi Liu
- School of Public Health (Shenzhen), Sun Yat-sen University, Shenzhen, Guangdong, China
| | - Huachun Zou
- School of Public Health, Fudan University, Shanghai, China
- Fujian Maternity and Child Health Hospital, College of Clinical Medicine for Obstetrics and Gynecology and Pediatrics, Fujian Medical University, Fuzhou, Fujian, China
- School of Public Health, Southwest Medical University, Luzhou, Sichuan, China
- Kirby Institute, University of New South Wales, Sydney, New South Wales, Australia
| |
Collapse
|
25
|
Hampton JS, Kenny RP, Rees CJ, Hamilton W, Eastaugh C, Richmond C, Sharp L. The performance of FIT-based and other risk prediction models for colorectal neoplasia in symptomatic patients: a systematic review. EClinicalMedicine 2023; 64:102204. [PMID: 37781155 PMCID: PMC10541467 DOI: 10.1016/j.eclinm.2023.102204] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/22/2023] [Revised: 08/21/2023] [Accepted: 08/28/2023] [Indexed: 10/03/2023] Open
Abstract
Background Colorectal cancer (CRC) incidence and mortality are increasing internationally. Endoscopy services are under significant pressure with many overwhelmed. Faecal immunochemical testing (FIT) has been advocated to identify a high-risk population of symptomatic patients requiring definitive investigation by colonoscopy. Combining FIT with other factors in a risk prediction model could further improve performance in identifying those requiring investigation most urgently. We systematically reviewed performance of models predicting risk of CRC and/or advanced colorectal polyps (ACP) in symptomatic patients, with a particular focus on those models including FIT. Methods The review protocol was published on PROSPERO (CRD42022314710). Searches were conducted from database inception to April 2023 in MEDLINE, EMBASE, Cochrane libraries, SCOPUS and CINAHL. Risk of bias of each study was assessed using The Prediction study Risk Of Bias Assessment Tool. A narrative synthesis based on the guidelines for Synthesis Without Meta-Analysis was performed due to study heterogeneity. Findings We included 62 studies; 23 included FIT (n = 22) or guaiac Faecal Occult Blood Testing (n = 1) combined with one or more other variables. Twenty-one studies were conducted solely in primary care. Generally, prediction models including FIT consistently had good discriminatory ability for CRC/ACP (i.e. AUC >0.8) and performed better than models without FIT although some models without FIT also performed well. However, many studies did not present calibration and internal and external validation were limited. Two studies were rated as low risk of bias; neither model included FIT. Interpretation Risk prediction models, including and not including FIT, show promise for identifying those most at risk of colorectal neoplasia. Substantial limitations in evidence remain, including heterogeneity, high risk of bias, and lack of external validation. Further evaluation in studies adhering to gold standard methodology, in appropriate populations, is required before widespread adoption in clinical practice. Funding National Institute for Health and Care Research (NIHR) [Health Technology Assessment Programme (HTA) Programme (Project number 133852).
Collapse
Affiliation(s)
- James S. Hampton
- Population Health Sciences Institute, Newcastle University, United Kingdom
- Department of Gastroenterology, South Tyneside and Sunderland NHS Foundation Trust, United Kingdom
| | - Ryan P.W. Kenny
- Evidence Synthesis Group, The Catalyst, Population Health Sciences Institute, Newcastle University, United Kingdom
- National Institute for Health and Care Research Innovation Observatory, The Catalyst, Newcastle University, United Kingdom
| | - Colin J. Rees
- Population Health Sciences Institute, Newcastle University, United Kingdom
- Department of Gastroenterology, South Tyneside and Sunderland NHS Foundation Trust, United Kingdom
| | - William Hamilton
- College of Medicine and Health, University of Exeter, United Kingdom
| | - Claire Eastaugh
- Evidence Synthesis Group, The Catalyst, Population Health Sciences Institute, Newcastle University, United Kingdom
- National Institute for Health and Care Research Innovation Observatory, The Catalyst, Newcastle University, United Kingdom
| | - Catherine Richmond
- Evidence Synthesis Group, The Catalyst, Population Health Sciences Institute, Newcastle University, United Kingdom
- National Institute for Health and Care Research Innovation Observatory, The Catalyst, Newcastle University, United Kingdom
| | - Linda Sharp
- Population Health Sciences Institute, Newcastle University, United Kingdom
| | - COLOFIT Research Team
- Population Health Sciences Institute, Newcastle University, United Kingdom
- Department of Gastroenterology, South Tyneside and Sunderland NHS Foundation Trust, United Kingdom
- Evidence Synthesis Group, The Catalyst, Population Health Sciences Institute, Newcastle University, United Kingdom
- National Institute for Health and Care Research Innovation Observatory, The Catalyst, Newcastle University, United Kingdom
- College of Medicine and Health, University of Exeter, United Kingdom
| |
Collapse
|
26
|
Kunonga TP, Kenny RPW, Astin M, Bryant A, Kontogiannis V, Coughlan D, Richmond C, Eastaugh CH, Beyer FR, Pearson F, Craig D, Lovat P, Vale L, Ellis R. Predictive accuracy of risk prediction models for recurrence, metastasis and survival for early-stage cutaneous melanoma: a systematic review. BMJ Open 2023; 13:e073306. [PMID: 37770261 PMCID: PMC10546114 DOI: 10.1136/bmjopen-2023-073306] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/02/2023] [Accepted: 09/03/2023] [Indexed: 09/30/2023] Open
Abstract
OBJECTIVES To identify prognostic models for melanoma survival, recurrence and metastasis among American Joint Committee on Cancer stage I and II patients postsurgery; and evaluate model performance, including overall survival (OS) prediction. DESIGN Systematic review and narrative synthesis. DATA SOURCES Searched MEDLINE, Embase, CINAHL, Cochrane Library, Science Citation Index and grey literature sources including cancer and guideline websites from 2000 to September 2021. ELIGIBILITY CRITERIA Included studies on risk prediction models for stage I and II melanoma in adults ≥18 years. Outcomes included OS, recurrence, metastases and model performance. No language or country of publication restrictions were applied. DATA EXTRACTION AND SYNTHESIS Two pairs of reviewers independently screened studies, extracted data and assessed the risk of bias using the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies checklist and the Prediction study Risk of Bias Assessment Tool. Heterogeneous predictors prevented statistical synthesis. RESULTS From 28 967 records, 15 studies reporting 20 models were included; 8 (stage I), 2 (stage II), 7 (stages I-II) and 7 (stages not reported), but were clearly applicable to early stages. Clinicopathological predictors per model ranged from 3-10. The most common were: ulceration, Breslow thickness/depth, sociodemographic status and site. Where reported, discriminatory values were ≥0.7. Calibration measures showed good matches between predicted and observed rates. None of the studies assessed clinical usefulness of the models. Risk of bias was high in eight models, unclear in nine and low in three. Seven models were internally and externally cross-validated, six models were externally validated and eight models were internally validated. CONCLUSIONS All models are effective in their predictive performance, however the low quality of the evidence raises concern as to whether current follow-up recommendations following surgical treatment is adequate. Future models should incorporate biomarkers for improved accuracy. PROSPERO REGISTRATION NUMBER CRD42018086784.
Collapse
Affiliation(s)
- Tafadzwa Patience Kunonga
- Evidence Synthesis Group, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
- NIHR Innovation Observatory, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
| | - R P W Kenny
- Evidence Synthesis Group, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
- NIHR Innovation Observatory, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
| | - Margaret Astin
- Evidence Synthesis Group, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
| | - Andrew Bryant
- Biostatistics Research Group, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
| | - Vasileios Kontogiannis
- Health Economics Group, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
| | - Diarmuid Coughlan
- Health Economics Group, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
| | - Catherine Richmond
- Evidence Synthesis Group, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
- NIHR Innovation Observatory, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
| | - Claire H Eastaugh
- Evidence Synthesis Group, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
- NIHR Innovation Observatory, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
| | - Fiona R Beyer
- Evidence Synthesis Group, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
- NIHR Innovation Observatory, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
| | - Fiona Pearson
- Evidence Synthesis Group, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
- NIHR Innovation Observatory, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
| | - Dawn Craig
- Evidence Synthesis Group, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
- NIHR Innovation Observatory, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
- Health Economics Group, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
| | - Penny Lovat
- Dermatological Sciences, Translation and Clinical Research Institute, Newcastle University, Newcastle upon Tyne, UK
- AMLo Bisciences, The Biosphere, Newcastle Helix, Newcastle upon Tyne, UK
| | - Luke Vale
- Health Economics Group, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
| | - Robert Ellis
- Dermatological Sciences, Translation and Clinical Research Institute, Newcastle University, Newcastle upon Tyne, UK
- AMLo Bisciences, The Biosphere, Newcastle Helix, Newcastle upon Tyne, UK
- Department of Dermatology, South Tees Hospitals NHS FT, Middlesbrough, UK
| |
Collapse
|
27
|
Reeve K, On BI, Havla J, Burns J, Gosteli-Peter MA, Alabsawi A, Alayash Z, Götschi A, Seibold H, Mansmann U, Held U. Prognostic models for predicting clinical disease progression, worsening and activity in people with multiple sclerosis. Cochrane Database Syst Rev 2023; 9:CD013606. [PMID: 37681561 PMCID: PMC10486189 DOI: 10.1002/14651858.cd013606.pub2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 09/09/2023]
Abstract
BACKGROUND Multiple sclerosis (MS) is a chronic inflammatory disease of the central nervous system that affects millions of people worldwide. The disease course varies greatly across individuals and many disease-modifying treatments with different safety and efficacy profiles have been developed recently. Prognostic models evaluated and shown to be valid in different settings have the potential to support people with MS and their physicians during the decision-making process for treatment or disease/life management, allow stratified and more precise interpretation of interventional trials, and provide insights into disease mechanisms. Many researchers have turned to prognostic models to help predict clinical outcomes in people with MS; however, to our knowledge, no widely accepted prognostic model for MS is being used in clinical practice yet. OBJECTIVES To identify and summarise multivariable prognostic models, and their validation studies for quantifying the risk of clinical disease progression, worsening, and activity in adults with MS. SEARCH METHODS We searched MEDLINE, Embase, and the Cochrane Database of Systematic Reviews from January 1996 until July 2021. We also screened the reference lists of included studies and relevant reviews, and references citing the included studies. SELECTION CRITERIA We included all statistically developed multivariable prognostic models aiming to predict clinical disease progression, worsening, and activity, as measured by disability, relapse, conversion to definite MS, conversion to progressive MS, or a composite of these in adult individuals with MS. We also included any studies evaluating the performance of (i.e. validating) these models. There were no restrictions based on language, data source, timing of prognostication, or timing of outcome. DATA COLLECTION AND ANALYSIS Pairs of review authors independently screened titles/abstracts and full texts, extracted data using a piloted form based on the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS), assessed risk of bias using the Prediction Model Risk Of Bias Assessment Tool (PROBAST), and assessed reporting deficiencies based on the checklist items in Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD). The characteristics of the included models and their validations are described narratively. We planned to meta-analyse the discrimination and calibration of models with at least three external validations outside the model development study but no model met this criterion. We summarised between-study heterogeneity narratively but again could not perform the planned meta-regression. MAIN RESULTS We included 57 studies, from which we identified 75 model developments, 15 external validations corresponding to only 12 (16%) of the models, and six author-reported validations. Only two models were externally validated multiple times. None of the identified external validations were performed by researchers independent of those that developed the model. The outcome was related to disease progression in 39 (41%), relapses in 8 (8%), conversion to definite MS in 17 (18%), and conversion to progressive MS in 27 (28%) of the 96 models or validations. The disease and treatment-related characteristics of included participants, and definitions of considered predictors and outcome, were highly heterogeneous amongst the studies. Based on the publication year, we observed an increase in the percent of participants on treatment, diversification of the diagnostic criteria used, an increase in consideration of biomarkers or treatment as predictors, and increased use of machine learning methods over time. Usability and reproducibility All identified models contained at least one predictor requiring the skills of a medical specialist for measurement or assessment. Most of the models (44; 59%) contained predictors that require specialist equipment likely to be absent from primary care or standard hospital settings. Over half (52%) of the developed models were not accompanied by model coefficients, tools, or instructions, which hinders their application, independent validation or reproduction. The data used in model developments were made publicly available or reported to be available on request only in a few studies (two and six, respectively). Risk of bias We rated all but one of the model developments or validations as having high overall risk of bias. The main reason for this was the statistical methods used for the development or evaluation of prognostic models; we rated all but two of the included model developments or validations as having high risk of bias in the analysis domain. None of the model developments that were externally validated or these models' external validations had low risk of bias. There were concerns related to applicability of the models to our research question in over one-third (38%) of the models or their validations. Reporting deficiencies Reporting was poor overall and there was no observable increase in the quality of reporting over time. The items that were unclearly reported or not reported at all for most of the included models or validations were related to sample size justification, blinding of outcome assessors, details of the full model or how to obtain predictions from it, amount of missing data, and treatments received by the participants. Reporting of preferred model performance measures of discrimination and calibration was suboptimal. AUTHORS' CONCLUSIONS The current evidence is not sufficient for recommending the use of any of the published prognostic prediction models for people with MS in clinical routine today due to lack of independent external validations. The MS prognostic research community should adhere to the current reporting and methodological guidelines and conduct many more state-of-the-art external validation studies for the existing or newly developed models.
Collapse
Affiliation(s)
- Kelly Reeve
- Epidemiology, Biostatistics and Prevention Institute, University of Zürich, Zurich, Switzerland
| | - Begum Irmak On
- Institute for Medical Information Processing, Biometry and Epidemiology, Ludwig-Maximilians-Universität München, Munich, Germany
| | - Joachim Havla
- lnstitute of Clinical Neuroimmunology, Ludwig-Maximilians-Universität München, Munich, Germany
| | - Jacob Burns
- Institute for Medical Information Processing, Biometry and Epidemiology, Ludwig-Maximilians-Universität München, Munich, Germany
- Pettenkofer School of Public Health, Munich, Germany
| | | | - Albraa Alabsawi
- Institute for Medical Information Processing, Biometry and Epidemiology, Ludwig-Maximilians-Universität München, Munich, Germany
| | - Zoheir Alayash
- Institute for Medical Information Processing, Biometry and Epidemiology, Ludwig-Maximilians-Universität München, Munich, Germany
- Institute of Health Services Research in Dentistry, University of Münster, Muenster, Germany
| | - Andrea Götschi
- Epidemiology, Biostatistics and Prevention Institute, University of Zürich, Zurich, Switzerland
| | | | - Ulrich Mansmann
- Institute for Medical Information Processing, Biometry and Epidemiology, Ludwig-Maximilians-Universität München, Munich, Germany
- Pettenkofer School of Public Health, Munich, Germany
| | - Ulrike Held
- Epidemiology, Biostatistics and Prevention Institute, University of Zürich, Zurich, Switzerland
| |
Collapse
|
28
|
White N, Parsons R, Collins G, Barnett A. Evidence of questionable research practices in clinical prediction models. BMC Med 2023; 21:339. [PMID: 37667344 PMCID: PMC10478406 DOI: 10.1186/s12916-023-03048-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/15/2023] [Accepted: 08/24/2023] [Indexed: 09/06/2023] Open
Abstract
BACKGROUND Clinical prediction models are widely used in health and medical research. The area under the receiver operating characteristic curve (AUC) is a frequently used estimate to describe the discriminatory ability of a clinical prediction model. The AUC is often interpreted relative to thresholds, with "good" or "excellent" models defined at 0.7, 0.8 or 0.9. These thresholds may create targets that result in "hacking", where researchers are motivated to re-analyse their data until they achieve a "good" result. METHODS We extracted AUC values from PubMed abstracts to look for evidence of hacking. We used histograms of the AUC values in bins of size 0.01 and compared the observed distribution to a smooth distribution from a spline. RESULTS The distribution of 306,888 AUC values showed clear excesses above the thresholds of 0.7, 0.8 and 0.9 and shortfalls below the thresholds. CONCLUSIONS The AUCs for some models are over-inflated, which risks exposing patients to sub-optimal clinical decision-making. Greater modelling transparency is needed, including published protocols, and data and code sharing.
Collapse
Affiliation(s)
- Nicole White
- Australian Centre for Health Services Innovation and Centre for Healthcare Transformation, School of Public Health and Social Work, Faculty of Health, Queensland University of Technology, Kelvin Grove, Queensland, Australia
| | - Rex Parsons
- Australian Centre for Health Services Innovation and Centre for Healthcare Transformation, School of Public Health and Social Work, Faculty of Health, Queensland University of Technology, Kelvin Grove, Queensland, Australia
| | - Gary Collins
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Adrian Barnett
- Australian Centre for Health Services Innovation and Centre for Healthcare Transformation, School of Public Health and Social Work, Faculty of Health, Queensland University of Technology, Kelvin Grove, Queensland, Australia.
| |
Collapse
|
29
|
Michael HU, Enechukwu O, Brouillette MJ, Tamblyn R, Fellows LK, Mayo NE. The Prognostic Utility of Anticholinergic Burden Scales: An Integrative Review and Gap Analysis. Drugs Aging 2023; 40:763-783. [PMID: 37462902 DOI: 10.1007/s40266-023-01050-4] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/29/2023] [Indexed: 08/25/2023]
Abstract
BACKGROUND Anticholinergic drugs are commonly prescribed, especially to older adults. Anticholinergic burden scales (ABS) have been used to evaluate the cumulative effects of multiple anticholinergics. However, studies have shown inconsistent results regarding the association between anticholinergic burden assessed with ABS and adverse clinical outcomes such as cognitive impairment, functional decline, and frailty. This review aims to identify gaps in research on the development, validation, and evaluation of ABS, and provide recommendations for future studies. METHOD A comprehensive search of five databases (MEDLINE, Embase, PsychInfo, CINAHL, CENTRAL) was conducted for relevant studies published from inception until 25 May 2023. Two reviewers screened for eligibility and assessed the quality of studies using different tools based on the study design and stage of the review framework. Research evidence was evaluated, and gaps were identified and grouped into evidence, knowledge, and methodological gaps, using evidence tables to summarize data. RESULTS Several evidence, knowledge, and methodological gaps in existing development, validation, and evaluation studies of ABS were identified. There is no universally accepted scale, and there is a need to define a clinically relevant threshold for measuring total anticholinergic burden. The current evidence has limitations, underrepresenting low- and middle-income countries, younger individuals, and populations with cognitive disabilities. The impact of anticholinergic burden on frailty is also understudied. Existing evaluation studies provide limited evidence on the benefit of reducing anticholinergic burden on clinical outcomes or the safety of anticholinergic deprescribing. There is also uncertainty regarding optimal reduction, clinically significant anticholinergic burden thresholds, and cost effectiveness. CONCLUSIONS Future research recommendations to bridge knowledge gaps include developing a risk assessment framework, refining ABS scales, establishing a standardized consensus scale, and creating a longitudinal measure of cumulative anticholinergic risk. Strategies to minimize bias, consider frailty, and promote multidisciplinary and multinational collaborations are also necessary to improve patient outcomes.
Collapse
Affiliation(s)
- Henry Ukachukwu Michael
- Division of Experimental Medicine, McGill University, Montreal, QC, Canada.
- Centre for Outcomes Research & Evaluation, Research Institute of McGill University Health Centre (RI-MUHC), 5252 de Maisonneuve, 2B:43, Montréal, QC, H4A 3S5, Canada.
| | | | - Marie-Josée Brouillette
- Department of Psychiatry, Faculty of Medicine, McGill University, Montreal, QC, Canada
- Chronic Viral Illness Service, McGill University Health Centre (MUHC), Montreal, QC, Canada
- Infectious Diseases and Immunity in Global Health Program, MUHC-RI, Montreal, QC, Canada
| | - Robyn Tamblyn
- Division of Experimental Medicine, McGill University, Montreal, QC, Canada
- Department of Epidemiology, Biostatistics and Occupational Health, McGill University, Montreal, QC, Canada
| | - Lesley K Fellows
- Department of Neurology and Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, QC, Canada
| | - Nancy E Mayo
- Division of Experimental Medicine, McGill University, Montreal, QC, Canada
- Centre for Outcomes Research & Evaluation, Research Institute of McGill University Health Centre (RI-MUHC), 5252 de Maisonneuve, 2B:43, Montréal, QC, H4A 3S5, Canada
- School of Physical and Occupational Therapy, Faculty of Medicine and Health Sciences, McGill University, Montreal, QC, Canada
| |
Collapse
|
30
|
Hamoodi Z, Gehringer CK, Bull LM, Hughes T, Kearsley-Fleet L, Sergeant JC, Watts AC. Prognostic factors associated with failure of total elbow replacement: a protocol for a systematic review. BMJ Open 2023; 13:e071705. [PMID: 37648384 PMCID: PMC10471856 DOI: 10.1136/bmjopen-2023-071705] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/17/2023] [Accepted: 08/08/2023] [Indexed: 09/01/2023] Open
Abstract
INTRODUCTION Total elbow replacement (TER) has higher failure rates requiring revision surgery compared with the replacement of other joints. Understanding the factors associated with failure is essential for informed decision-making between patients and clinicians, and for reducing the failure rate. This review aims to identify, describe and appraise the literature examining prognostic factors for failure of TER. METHODS AND ANALYSIS This systematic review will be conducted and reported in line with the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols guidelines. Electronic literature searches will be conducted using Medline, EMBASE, PubMed and Cochrane. The search strategy will be broad, including a combination of subject headings (MESH) and free text search. This search will be supplemented with a screening of reference lists of the included studies and relevant reviews. Two independent reviewers will screen all search results in two stages (title and abstract, and full text) based on the Population, Index prognostic factor, Comparator prognostic factor, Outcome, Time and Setting criteria. The types of evidence included will be randomised trials, non-randomised trials, prospective and retrospective cohort studies, registry studies and case-control studies. If the literature lacks enough studies, then case series with 50 or more TERs will be considered for inclusion. Data extraction and risk of bias assessment for included studies will be performed by two independent reviewers using the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies for Prognostic Factors and Quality In Prognostic Studies tools.Meta-analyses of prognostic estimates for each factor will be undertaken for studies that are deemed to be sufficiently robust and comparable. Several challenges are likely to arise due to heterogeneity between studies, therefore, subgroup and sensitivity analyses will be performed to account for the differences between studies. Heterogeneity will be assessed using Q and I2 statistics. If I2>40% then pooled estimates will not be reported. When quantitative synthesis is not possible, a narrative synthesis will be undertaken. The quality of the evidence for each prognostic factor will be assessed using the Grades of Recommendation Assessment, Development and Evaluation tool. PROSPERO REGISTRATION NUMBER CRD42023384756.
Collapse
Affiliation(s)
- Zaid Hamoodi
- Centre for Epidemiology Versus Arthritis, Centre for Musculoskeletal Research, University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
- Upper Limb Unit, Wrightington Wigan and Leigh NHS Foundation Trust, Wigan, UK
| | - Celina K Gehringer
- Centre for Epidemiology Versus Arthritis, Centre for Musculoskeletal Research, University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
- Centre for Biostatistics, School of Health Sciences, Faculty of Biology, Medicine and Health, Manchester Academic Health Science Centre, The University of Manchester, Manchester, UK
| | - Lucy M Bull
- Technology Department, Health Navigator Ltd, London, UK
| | - Tom Hughes
- Department of Health Professions, Manchester Metropolitan University, Manchester, UK
| | - Lianne Kearsley-Fleet
- Centre for Epidemiology Versus Arthritis, Centre for Musculoskeletal Research, University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | - Jamie C Sergeant
- Centre for Epidemiology Versus Arthritis, Centre for Musculoskeletal Research, University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
- Centre for Biostatistics, School of Health Sciences, Faculty of Biology, Medicine and Health, Manchester Academic Health Science Centre, The University of Manchester, Manchester, UK
| | - Adam C Watts
- Upper Limb Unit, Wrightington Wigan and Leigh NHS Foundation Trust, Wigan, UK
| |
Collapse
|
31
|
Chen R, Zheng J, Li C, Chen Q, Zeng Z, Li L, Chen M, Zhang S. Prognostic models for predicting postoperative recurrence in Crohn's disease: a systematic review and critical appraisal. Front Immunol 2023; 14:1215116. [PMID: 37457731 PMCID: PMC10349525 DOI: 10.3389/fimmu.2023.1215116] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2023] [Accepted: 06/21/2023] [Indexed: 07/18/2023] Open
Abstract
Background and Aims Prophylaxis of postoperative recurrence is an intractable problem for clinicians and patients with Crohn's disease. Prognostic models are effective tools for patient stratification and personalised management. This systematic review aimed to provide an overview and critically appraise the existing models for predicting postoperative recurrence of Crohn's disease. Methods Systematic retrieval was performed using PubMed and Web of Science in January 2022. Original articles on prognostic models for predicting postoperative recurrence of Crohn's disease were included in the analysis. The risk of bias was assessed using the Prediction Model Risk of Bias Assessment (PROBAST) tool. This study was registered with the International Prospective Register of Systematic Reviews (PROSPERO; number CRD42022311737). Results In total, 1948 articles were screened, of which 15 were ultimately considered. Twelve studies developed 15 new prognostic models for Crohn's disease and the other three validated the performance of three existing models. Seven models utilised regression algorithms, six utilised scoring indices, and five utilised machine learning. The area under the receiver operating characteristic curve of the models ranged from 0.51 to 0.97. Six models showed good discrimination, with an area under the receiver operating characteristic curve of >0.80. All models were determined to have a high risk of bias in modelling or analysis, while they were at low risk of applicability concerns. Conclusions Prognostic models have great potential for facilitating the assessment of postoperative recurrence risk in patients with Crohn's disease. Existing prognostic models require further validation regarding their reliability and applicability. Systematic review registration https://www.crd.york.ac.uk/PROSPERO/, identifier CRD42022311737.
Collapse
Affiliation(s)
- Rirong Chen
- Department of Gastroenterology, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Jieqi Zheng
- Department of Clinical Medicine, Zhongshan School of Medicine, Sun Yat-Sen University, Guangzhou, China
| | - Chao Li
- Department of Clinical Medicine, Zhongshan School of Medicine, Sun Yat-Sen University, Guangzhou, China
| | - Qia Chen
- Department of Clinical Medicine, Zhongshan School of Medicine, Sun Yat-Sen University, Guangzhou, China
| | - Zhirong Zeng
- Department of Gastroenterology, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Li Li
- Department of Gastroenterology, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Minhu Chen
- Department of Gastroenterology, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Shenghong Zhang
- Department of Gastroenterology, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| |
Collapse
|
32
|
Hakimjavadi R, DiRienzo L, Rattanawong P, Ayoub C, Visintini SM, Small GR, Chow B. Prognostic Value of Coronary Computed Tomography Angiography in Coronary Artery Bypass Graft Patients Systematic Review and Meta-Analysis. Am J Cardiol 2023; 201:107-115. [PMID: 37354866 DOI: 10.1016/j.amjcard.2023.05.048] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/22/2023] [Revised: 04/28/2023] [Accepted: 05/27/2023] [Indexed: 06/26/2023]
Abstract
We sought to assess the prognostic value of coronary computed tomographic angiography (CCTA) in patients with coronary artery bypass graft (CABG) by meta-analysis. MEDLINE, Embase, Cochrane Central Register of Controlled Trials, and Scopus were searched for relevant original articles published up to July 2021. CCTA prognostic studies enrolling patients with CABG were screened and included if outcomes included all-cause mortality or major adverse cardiac events. Maximally adjusted hazard ratios (HRs) were extracted for CCTA-derived prognostic factors. HRs were log-transformed and pooled across studies using the DerSimonian-Laird random-effects model and statistical heterogeneity was assessed using the I2 statistic. Of 1,576 screened articles, 4 retrospective studies fulfilled all inclusion criteria. Collectively, a total of 1,809 patients with CABG underwent CCTA (mean [SD] age 67.0 [8.5] years across 3 studies, 81.5% male across 4 studies). Coronary artery disease severity and revascularization were categorized using 2 models: unprotected coronary territories and coronary artery protection score. The pooled HRs from the random-effects models using the most highly adjusted study estimate were 3.64 (95% confidence interval 2.48 to 5.34, I2 = 57.8%, p <0.001; 4 studies) and 4.85 (95% confidence interval 3.17 to 7.43, I2 = 39.9%, p <0.001; 2 studies) for unprotected coronary territories and coronary artery protection score, respectively. In conclusion, in a limited number of studies, CCTA is an independent predictor of adverse events in patients with CABG. Larger studies using uniform models and endpoints are needed.
Collapse
Affiliation(s)
| | - Lucas DiRienzo
- Division of Cardiology, University of Ottawa Heart Institute, Canada
| | | | - Chadi Ayoub
- Department of Cardiovascular Medicine, Mayo Clinic, Scottsdale, Arizona
| | - Sarah M Visintini
- Division of Cardiology, University of Ottawa Heart Institute, Canada
| | - Gary R Small
- Division of Cardiology, University of Ottawa Heart Institute, Canada
| | - Benjamin Chow
- Division of Cardiology, University of Ottawa Heart Institute, Canada; Department of Radiology, University of Ottawa, Canada.
| |
Collapse
|
33
|
Aly F, Hansen CR, Al Mouiee D, Sundaresan P, Haidar A, Vinod S, Holloway L. Outcome prediction models incorporating clinical variables for Head and Neck Squamous cell Carcinoma: A systematic review of methodological conduct and risk of bias. Radiother Oncol 2023; 183:109629. [PMID: 36934895 DOI: 10.1016/j.radonc.2023.109629] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2022] [Revised: 02/20/2023] [Accepted: 03/10/2023] [Indexed: 03/19/2023]
Abstract
Multiple outcome prediction models have been developed for Head and Neck Squamous Cell Carcinoma (HNSCC). This systematic review aimed to identify HNSCC outcome prediction model studies, assess their methodological quality and identify those with potential utility for clinical practice. Inclusion criteria were mucosal HNSCC prognostic prediction model studies (development or validation) incorporating clinically available variables accessible at time of treatment decision making and predicting tumour-related outcomes. Eligible publications were identified from PubMed and Embase. Methodological quality and risk of bias were assessed using the checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS) and prediction model risk of bias assessment tool (PROBAST). Eligible publications were categorised by study type for reporting. 64 eligible publications were identified; 55 reported model development, 37 external validations, with 28 reporting both. CHARMS checklist items relating to participants, predictors, outcomes, handling of missing data, and some model development and evaluation procedures were generally well-reported. Less well-reported were measures accounting for model overfitting and model performance measures, especially model calibration. Full model information was poorly reported (3/55 model developments), specifically model intercept, baseline survival or full model code. Most publications (54/55 model developments, 28/37 external validations) were found to have high risk of bias, predominantly due to methodological issues in the PROBAST analysis domain. The identified methodological issues may affect prediction model accuracy in heterogeneous populations. Independent external validation studies in the local population and demonstration of clinical impact are essential for the clinical implementation of outcome prediction models.
Collapse
Affiliation(s)
- Farhannah Aly
- Ingham Institute for Applied Medical Research, Sydney, Australia; Southwest Sydney Clinical Campus, University of New South Wales, Sydney, Australia; Liverpool and Macarthur Cancer Therapy Centres, Sydney, Australia.
| | - Christian Rønn Hansen
- Laboratory of Radiation Physics, Odense University Hospital, Odense, Denmark; Department of Clinical Research, University of Southern Denmark, Odense, Denmark; Danish Centre for Particle Therapy, Aarhus University Hospital, Denmark; Institute of Medical Physics, School of Physics, University of Sydney, Sydney, Australia
| | - Daniel Al Mouiee
- Ingham Institute for Applied Medical Research, Sydney, Australia; Southwest Sydney Clinical Campus, University of New South Wales, Sydney, Australia; Liverpool and Macarthur Cancer Therapy Centres, Sydney, Australia
| | - Purnima Sundaresan
- Sydney West Radiation Oncology Network, Western Sydney Local Health District, Sydney, Australia; Sydney Medical School, The University of Sydney, Sydney, Australia
| | - Ali Haidar
- Ingham Institute for Applied Medical Research, Sydney, Australia; Southwest Sydney Clinical Campus, University of New South Wales, Sydney, Australia
| | - Shalini Vinod
- Southwest Sydney Clinical Campus, University of New South Wales, Sydney, Australia; Liverpool and Macarthur Cancer Therapy Centres, Sydney, Australia
| | - Lois Holloway
- Ingham Institute for Applied Medical Research, Sydney, Australia; Southwest Sydney Clinical Campus, University of New South Wales, Sydney, Australia; Liverpool and Macarthur Cancer Therapy Centres, Sydney, Australia; Institute of Medical Physics, School of Physics, University of Sydney, Sydney, Australia
| |
Collapse
|
34
|
Arshi B, Wynants L, Rijnhart E, Reeve K, Cowley LE, Smits LJ. What proportion of clinical prediction models make it to clinical practice? Protocol for a two-track follow-up study of prediction model development publications. BMJ Open 2023; 13:e073174. [PMID: 37197813 DOI: 10.1136/bmjopen-2023-073174] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 05/19/2023] Open
Abstract
INTRODUCTION It is known that only a limited proportion of developed clinical prediction models (CPMs) are implemented and/or used in clinical practice. This may result in a large amount of research waste, even when considering that some CPMs may demonstrate poor performance. Cross-sectional estimates of the numbers of CPMs that have been developed, validated, evaluated for impact or utilized in practice, have been made in specific medical fields, but studies across multiple fields and studies following up the fate of CPMs are lacking. METHODS AND ANALYSIS We have conducted a systematic search for prediction model studies published between January 1995 and December 2020 using the Pubmed and Embase databases, applying a validated search strategy. Taking random samples for every calendar year, abstracts and articles were screened until a target of 100 CPM development studies were identified. Next, we will perform a forward citation search of the resulting CPM development article cohort to identify articles on external validation, impact assessment or implementation of those CPMs. We will also invite the authors of the development studies to complete an online survey to track implementation and clinical utilization of the CPMs.We will conduct a descriptive synthesis of the included studies, using data from the forward citation search and online survey to quantify the proportion of developed models that are validated, assessed for their impact, implemented and/or used in patient care. We will conduct time-to-event analysis using Kaplan-Meier plots. ETHICS AND DISSEMINATION No patient data are involved in the research. Most information will be extracted from published articles. We request written informed consent from the survey respondents. Results will be disseminated through publication in a peer-reviewed journal and presented at international conferences. OSF REGISTRATION: (https://osf.io/nj8s9).
Collapse
Affiliation(s)
- Banafsheh Arshi
- Department of Epidemiology, CAPHRI School for Public Health and Primary Care, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands
| | - Laure Wynants
- Department of Epidemiology, CAPHRI School for Public Health and Primary Care, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands
- Department of Development and Regeneration, KU Leuven, Leuven, Belgium
| | - Eline Rijnhart
- Department of Epidemiology, CAPHRI School for Public Health and Primary Care, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands
| | - Kelly Reeve
- Department of Epidemiology, Biostatistics and Prevention Institute, Department of Biostatistics, University of Zurich, Hirschengraben 84, CH-8001 Zurich, Switzerland
| | | | - Luc J Smits
- Department of Epidemiology, CAPHRI School for Public Health and Primary Care, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands
| |
Collapse
|
35
|
Ohyama Y, Iwamura T, Hoshino T, Miyata K. Prognostic models of quality of life after total knee replacement: A systematic review. Physiother Theory Pract 2023:1-12. [PMID: 37162481 DOI: 10.1080/09593985.2023.2211716] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/11/2023]
Abstract
OBJECTIVE To systematically review and critically appraise prognostic models for quality of life (QOL) in patients with total knee replacement (TKA). METHODS Subjects were TKA recipients recruited from inpatient postoperative settings. Searches were made on June 2022 and updated on April 2023. Databases included PubMed.gov, CINAHL, The Cochrane Library, Web of Science. Two authors performed all review stages independently. Risk of bias assessments on participants, predictors, outcomes and analysis methods followed the Prediction study Risk Of Bias ASsessment Tool (PROBAST). RESULTS After screening 2204 studies, 9 were eligible for inclusion. Twelve prognostic models were reported, of which 10 models were developed from data without validation and 2 were both developed and validated. The most frequently applied predictor was the pre-TKA QOL score. Discriminatory measures were reported for 9 (75.0%) models with areas under the curve values of 0.66-0.95. All models showed a high risk of bias, mostly due to limitations in statistical methods and outcome assessments. CONCLUSION Several prognostic models have been developed for QOL in patients with TKA, but all models show a high risk of bias and are unreliable in clinical practice. Future, prognostic models overcoming the risk of bias identified in this study are needed.
Collapse
Affiliation(s)
- Yuki Ohyama
- Department of Rehabilitation, Hidaka Rehabilitation Hospital, Takasaki, Japan
| | - Taiki Iwamura
- Department of Rehabilitation, Azumabashi Orthopedics, Tokyo, Japan
| | - Taichi Hoshino
- Department of Rehabilitation, Gunma Chuo Hospital, Maebashi, Gunma, Japan
| | - Kazuhiro Miyata
- Department of Physical Therapy, Ibaraki Prefectural University of Health Science, Ibaraki, Japan
| |
Collapse
|
36
|
Lokker C, Bagheri E, Abdelkader W, Parrish R, Afzal M, Navarro T, Cotoi C, Germini F, Linkins L, Brian Haynes R, Chu L, Iorio A. Deep Learning to Refine the Identification of High-Quality Clinical Research Articles from the Biomedical Literature: Performance Evaluation. J Biomed Inform 2023; 142:104384. [PMID: 37164244 DOI: 10.1016/j.jbi.2023.104384] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2022] [Revised: 04/24/2023] [Accepted: 05/03/2023] [Indexed: 05/12/2023]
Abstract
BACKGROUND Identifying practice-ready evidence-based journal articles in medicine is a challenge due to the sheer volume of biomedical research publications. Newer approaches to support evidence discovery apply deep learning techniques to improve the efficiency and accuracy of classifying sound evidence. OBJECTIVE To determine how well deep learning models using variants of Bidirectional Encoder Representations from Transformers (BERT) identify high-quality evidence with high clinical relevance from the biomedical literature for consideration in clinical practice. METHODS We fine-tuned variations of BERT models (BERTBASE, BioBERT, BlueBERT, and PubMedBERT) and compared their performance in classifying articles based on methodological quality criteria. The dataset used for fine-tuning models included titles and abstracts of >160,000 PubMed records from 2012-2020 that were of interest to human health which had been manually labeled based on meeting established critical appraisal criteria for methodological rigor. The data was randomly divided into 80:10:10 sets for training, validating, and testing. In addition to using the full unbalanced set, the training data was randomly undersampled into four balanced datasets to assess performance and select the best performing model. For each of the four sets, one model that maintained sensitivity (recall) at ≥99% was selected and were ensembled. The best performing model was evaluated in a prospective, blinded test and applied to an established reference standard, the Clinical Hedges dataset. RESULTS In training, three of the four selected best performing models were trained using BioBERTBASE. The ensembled model did not boost performance compared with the best individual model. Hence a solo BioBERT-based model (named DL-PLUS) was selected for further testing as it was computationally more efficient. The model had high recall (>99%) and 60% to 77% specificity in a prospective evaluation conducted with blinded research associates and saved >60% of the work required to identify high quality articles. CONCLUSIONS Deep learning using pretrained language models and a large dataset of classified articles produced models with improved specificity while maintaining >99% recall. The resulting DL-PLUS model identifies high-quality, clinically relevant articles from PubMed at the time of publication. The model improves the efficiency of a literature surveillance program, which allows for faster dissemination of appraised research.
Collapse
Affiliation(s)
- Cynthia Lokker
- Health Information Research Unit, Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada.
| | - Elham Bagheri
- Health Information Research Unit, Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada
| | - Wael Abdelkader
- Health Information Research Unit, Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada
| | - Rick Parrish
- Health Information Research Unit, Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada
| | - Muhammad Afzal
- Department of Computing, Birmingham City University, Birmingham, UK
| | - Tamara Navarro
- Health Information Research Unit, Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada
| | - Chris Cotoi
- Health Information Research Unit, Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada
| | - Federico Germini
- Health Information Research Unit, Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Department of Medicine, McMaster University, Hamilton, Ontario, Canada
| | - Lori Linkins
- Department of Medicine, McMaster University, Hamilton, Ontario, Canada
| | - R Brian Haynes
- Health Information Research Unit, Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Department of Medicine, McMaster University, Hamilton, Ontario, Canada
| | - Lingyang Chu
- Department of Computing and Software, McMaster University, Hamilton, Ontario, Canada
| | - Alfonso Iorio
- Health Information Research Unit, Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Department of Medicine, McMaster University, Hamilton, Ontario, Canada
| |
Collapse
|
37
|
Snell KIE, Levis B, Damen JAA, Dhiman P, Debray TPA, Hooft L, Reitsma JB, Moons KGM, Collins GS, Riley RD. Transparent reporting of multivariable prediction models for individual prognosis or diagnosis: checklist for systematic reviews and meta-analyses (TRIPOD-SRMA). BMJ 2023; 381:e073538. [PMID: 37137496 PMCID: PMC10155050 DOI: 10.1136/bmj-2022-073538] [Citation(s) in RCA: 23] [Impact Index Per Article: 23.0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 02/22/2023] [Indexed: 05/05/2023]
Affiliation(s)
- Kym I E Snell
- Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham B15 2TT, UK
| | - Brooke Levis
- Centre for Prognosis Research, School of Medicine, Keele University, Keele, UK
| | - Johanna A A Damen
- Cochrane Netherlands, Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Paula Dhiman
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
- NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Thomas P A Debray
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Lotty Hooft
- Cochrane Netherlands, Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Johannes B Reitsma
- Cochrane Netherlands, Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Karel G M Moons
- Cochrane Netherlands, Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Gary S Collins
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
- NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Richard D Riley
- Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham B15 2TT, UK
| |
Collapse
|
38
|
Turabi RY, Wyatt D, Guerra S, O'Connell MDL, Khatun T, Sageer SA, Alhazmi A, Sheehan KJ. Barriers and facilitators of weight bearing after hip fracture surgery among older adults. A scoping review. Osteoporos Int 2023:10.1007/s00198-023-06735-5. [PMID: 37016146 DOI: 10.1007/s00198-023-06735-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/13/2022] [Accepted: 03/24/2023] [Indexed: 04/06/2023]
Abstract
PURPOSE This scoping review aimed to synthesise the available evidence on barriers and facilitators of weight bearing after hip fracture surgery in older adults. METHODS Published (Cochrane Central, MEDLINE, EMBASE, CINAHL, and PEDro) and unpublished (Global Health, EThOS, WorldCat dissertation and thesis, ClinicalTrials.gov , OpenAIRE, DART-Europe) evidence was electronically searched from database inception to 29 March 2022. Barriers and facilitators of weight bearing were extracted and synthesised into patient, process (non-surgical), process (surgical), and structure-related barriers/facilitators using a narrative review approach. RESULTS In total, 5594 were identified from the primary search strategy, 1314 duplicates were removed, 3769 were excluded on title and abstract screening, and 442 were excluded on full-text screening. In total, 69 studies (all from published literature sources) detailing 47 barriers and/or facilitators of weight bearing were included. Of barriers/facilitators identified, 27 were patient-, 8 non-surgical process-, 8 surgical process-, and 4 structure-related. Patient facilitators included anticoagulant, home discharge, and aid at discharge. Barriers included preoperative dementia and delirium, postoperative delirium, pressure sores, indoor falls, ventilator dependence, haematocrit < 36%, systemic sepsis, and acute renal failure. Non-surgical process facilitators included early surgery, early mobilisation, complete medical co-management, in-hospital rehabilitation, and patient-recorded nurses' notes. Barriers included increased operative time and standardised hip fracture care. Surgical process facilitators favoured intramedullary fixations and arthroplasty over extramedullary fixation. Structure facilitators favoured more recent years and different healthcare systems. Barriers included pre-holiday surgery and admissions in the first quarter of the year. CONCLUSION Most patient/surgery-related barriers/facilitators may inform future risk stratification. Future research should examine additional process/structure barriers and facilitators amenable to intervention. Furthermore, patient barriers/facilitators need to be investigated by replicating the studies identified and augmenting them with more specific details on weight bearing outcomes.
Collapse
Affiliation(s)
- R Y Turabi
- Department of Population Health Sciences, School of Life Course and Population Sciences, King's College London, London, UK.
- Department of Physical Therapy, Applied Medical Sciences, Jazan University, Jazan, Saudi Arabia.
| | - D Wyatt
- Department of Population Health Sciences, School of Life Course and Population Sciences, King's College London, London, UK
| | - S Guerra
- Department of Population Health Sciences, School of Life Course and Population Sciences, King's College London, London, UK
| | - M D L O'Connell
- Department of Population Health Sciences, School of Life Course and Population Sciences, King's College London, London, UK
| | - T Khatun
- Centre for Implementation Science, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK
| | - S A Sageer
- Department of Orthopaedic, Relief Hospital and Trauma Centre, Kerala, India
| | - A Alhazmi
- Department of Orthopaedic, King Fahad Central Hospital, Jazan, Saudi Arabia
| | - K J Sheehan
- Department of Population Health Sciences, School of Life Course and Population Sciences, King's College London, London, UK
| |
Collapse
|
39
|
Huang QF, Hu YC, Wang CK, Huang J, Shen MD, Ren LH. Clinical First-Trimester Prediction Models for Gestational Diabetes Mellitus: A Systematic Review and Meta-Analysis. Biol Res Nurs 2023; 25:185-197. [PMID: 36218132 DOI: 10.1177/10998004221131993] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
BACKGROUND Gestational diabetes mellitus (GDM) is a common pregnancy complication that negatively impacts the health of both the mother and child. Early prediction of the risk of GDM may permit prompt and effective interventions. This systematic review and meta-analysis aimed to summarize the study characteristics, methodological quality, and model performance of first-trimester prediction model studies for GDM. METHODS Five electronic databases, one clinical trial register, and gray literature were searched from the inception date to March 19, 2022. Studies developing or validating a first-trimester prediction model for GDM were included. Two reviewers independently extracted data according to an established checklist and assessed the risk of bias by the Prediction Model Risk of Bias Assessment Tool (PROBAST). We used a random-effects model to perform a quantitative meta-analysis of the predictive power of models that were externally validated at least three times. RESULTS We identified 43 model development studies, six model development and external validation studies, and five external validation-only studies. Body mass index, maternal age, and fasting plasma glucose were the most commonly included predictors across all models. Multiple estimates of performance measures were available for eight of the models. Summary estimates range from 0.68 to 0.78 (I2 ranged from 0% to 97%). CONCLUSION Most studies were assessed as having a high overall risk of bias. Only eight prediction models for GDM have been externally validated at least three times. Future research needs to focus on updating and externally validating existing models.
Collapse
Affiliation(s)
- Qi-Fang Huang
- School of Nursing, 33133Peking University, Beijing, China
| | - Yin-Chu Hu
- School of Nursing, 33133Peking University, Beijing, China
| | - Chong-Kun Wang
- School of Nursing, 33133Peking University, Beijing, China
| | - Jing Huang
- Florence Nightingale School of Nursing, 4616King's College London, London, UK
| | - Mei-Di Shen
- School of Nursing, 33133Peking University, Beijing, China
| | - Li-Hua Ren
- School of Nursing, 33133Peking University, Beijing, China
| |
Collapse
|
40
|
Luo Q, Luo Y, Cui T, Li T. Performance of HIV Infection Prediction Models in Men Who Have Sex with Men: A Systematic Review and Meta-Analysis. ARCHIVES OF SEXUAL BEHAVIOR 2023:10.1007/s10508-023-02574-x. [PMID: 36884160 DOI: 10.1007/s10508-023-02574-x] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/02/2022] [Revised: 01/29/2023] [Accepted: 02/22/2023] [Indexed: 06/18/2023]
Abstract
Effective ways to identify and predict men who have sex with men (MSM) at substantial risk for HIV is a global priority. HIV risk assessment tools can improve individual risk awareness and subsequent health-seeking actions. We sought to identify and characterize the performance of HIV infection risk prediction models in MSM through systematic review and meta-analysis. PubMed, Embase, and The Cochrane Library were searched. Eighteen HIV infection risk assessment models with a total of 151,422 participants and 3643 HIV cases were identified, eight of which have been externally validated by at least one study (HIRI-MSM, Menza Score, SDET Score, Li Model, DHRS, Amsterdam Score, SexPro model, and UMRSS). The number of predictor variables in each model ranged from three to 12, age, the number of male sexual partners, unprotected receptive anal intercourse, recreational drug usage (amphetamines, poppers), and sexually transmitted infections were critical scoring variables. All eight externally validated models performed well in terms of discrimination, with the pooled area under the receiver operating characteristic curve (AUC) ranging from 0.62 (95%CI: 0.51 to 0.73, SDET Score) to 0.83 (95%CI: 0.48 to 0.99, Amsterdam Score). Calibration performance was only reported in 10 studies (35.7%, 10/28). The HIV infection risk prediction models showed moderate-to-good discrimination performance. Validation of prediction models across different geographic and ethnic environments is needed to ensure their real-world application.
Collapse
Affiliation(s)
- Qianqian Luo
- School of Nursing, Binzhou Medical University, 346 Guanhai Road, Laishan District, Yantai, 264003, China.
| | - Yongchuan Luo
- Yantai Affiliated Hospital of Binzhou Medical University, Yantai, China
| | - Tianyu Cui
- School of Nursing, Binzhou Medical University, 346 Guanhai Road, Laishan District, Yantai, 264003, China
| | - Tianying Li
- School of Nursing, Binzhou Medical University, 346 Guanhai Road, Laishan District, Yantai, 264003, China
| |
Collapse
|
41
|
Yoneoka D, Omae K, Henmi M, Eguchi S. Area under the curve-optimized synthesis of prediction models from a meta-analytical perspective. Res Synth Methods 2023; 14:234-246. [PMID: 36424356 DOI: 10.1002/jrsm.1612] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2021] [Revised: 08/31/2022] [Accepted: 11/07/2022] [Indexed: 11/27/2022]
Abstract
The number of clinical prediction models sharing the same prediction task has increased in the medical literature. However, evidence synthesis methodologies that use the results of these prediction models have not been sufficiently studied, particularly in the context of meta-analysis settings where only summary statistics are available. In particular, we consider the following situation: we want to predict an outcome Y, that is not included in our current data, while the covariate data are fully available. In addition, the summary statistics from prior studies, which share the same prediction task (i.e., the prediction of Y), are available. This study introduces a new method for synthesizing the summary results of binary prediction models reported in the prior studies using a linear predictor under a distributional assumption between the current and prior studies. The method provides an integrated predictor combining all predictors reported in the prior studies with weights. The vector of the weights is designed to achieve the hypothetical improvement of area under the receiver operating characteristic curve (AUC) on the current available data under a practical situation where there are different sets of covariates in the prior studies. We observe a counterintuitive aspect in typical situations where a part of weight components in the proposed method becomes negative. It implies that flipping the sign of the prediction results reported in each individual study would improve the overall prediction performance. Finally, numerical and real-world data analysis were conducted and showed that our method outperformed conventional methods in terms of AUC.
Collapse
Affiliation(s)
- Daisuke Yoneoka
- Infectious Disease Surveillance Center, National Institute of Infectious Diseases, Tokyo, Japan
| | - Katsuhiro Omae
- Department of Data Science, National Cerebral and Cardiovascular Center, Osaka, Japan
| | | | - Shinto Eguchi
- The Institute of Statistical Mathematics, Tokyo, Japan
| |
Collapse
|
42
|
Vernooij JEM, Koning NJ, Geurts JW, Holewijn S, Preckel B, Kalkman CJ, Vernooij LM. Performance and usability of pre-operative prediction models for 30-day peri-operative mortality risk: a systematic review. Anaesthesia 2023; 78:607-619. [PMID: 36823388 DOI: 10.1111/anae.15988] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/06/2023] [Indexed: 02/25/2023]
Abstract
Estimating pre-operative mortality risk may inform clinical decision-making for peri-operative care. However, pre-operative mortality risk prediction models are rarely implemented in routine clinical practice. High predictive accuracy and clinical usability are essential for acceptance and clinical implementation. In this systematic review, we identified and appraised prediction models for 30-day postoperative mortality in non-cardiac surgical cohorts. PubMed and Embase were searched up to December 2022 for studies investigating pre-operative prediction models for 30-day mortality. We assessed predictive performance in terms of discrimination and calibration. Risk of bias was evaluated using a tool to assess the risk of bias and applicability of prediction model studies. To further inform potential adoption, we also assessed clinical usability for selected models. In all, 15 studies evaluating 10 prediction models were included. Discrimination ranged from a c-statistic of 0.82 (MySurgeryRisk) to 0.96 (extreme gradient boosting machine learning model). Calibration was reported in only six studies. Model performance was highest for the surgical outcome risk tool (SORT) and its external validations. Clinical usability was highest for the surgical risk pre-operative assessment system. The SORT and risk quantification index also scored high on clinical usability. We found unclear or high risk of bias in the development of all models. The SORT showed the best combination of predictive performance and clinical usability and has been externally validated in several heterogeneous cohorts. To improve clinical uptake, full integration of reliable models with sufficient face validity within the electronic health record is imperative.
Collapse
Affiliation(s)
- J E M Vernooij
- Department of Anaesthesia, Rijnstate Hospital, the Netherlands
| | - N J Koning
- Department of Anaesthesia, Rijnstate Hospital, the Netherlands
| | - J W Geurts
- Department of Anaesthesia, Rijnstate Hospital, the Netherlands
| | - S Holewijn
- Department of Vascular Surgery, Rijnstate Hospital, the Netherlands
| | - B Preckel
- Department of Anaesthesia, Amsterdam UMC, Amsterdam, the Netherlands
| | - C J Kalkman
- University Medical Centre, Utrecht, the Netherlands
| | - L M Vernooij
- Department of Anaesthesia, University Medical Centre Utrecht, the Netherlands
| |
Collapse
|
43
|
Pommerich UM, Stubbs PW, Eggertsen PP, Fabricius J, Nielsen JF. Regression-based prognostic models for functional independence after postacute brain injury rehabilitation are not transportable: a systematic review. J Clin Epidemiol 2023; 156:53-65. [PMID: 36764467 DOI: 10.1016/j.jclinepi.2023.02.009] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Revised: 01/30/2023] [Accepted: 02/02/2023] [Indexed: 02/11/2023]
Abstract
BACKGROUND AND OBJECTIVES To identify and summarize validated multivariable prognostic models for the Functional Independence Measure® (FIM®) at discharge from post-acute inpatient rehabilitation in adults with acquired brain injury (ABI). METHODS This review was conducted based on the recommendations of the Cochrane Prognosis Methods Group and adheres to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Three databases were systematically searched in May 2021 and updated in April 2022. Main inclusion criteria were: a) adult patients with ABI, b) validated multivariable prognostic model, c) time of prognostication within 1-week of admission to post-acute rehabilitation, and d) outcome was the FIM® at discharge from post-acute rehabilitation. RESULTS The search yielded 3,169 unique articles. Three articles fulfilled the inclusion criteria, accounting for n = 6 internally and n = 2 externally validated prognostic models. Discrimination was estimated as an area under the curve between 0.76 and 0.89. Calibration was deemed to be assessed insufficiently. The included models were judged to be of high risk of bias. CONCLUSION Current prognostic models for the FIM® in post-acute rehabilitation for patients with ABI lack the methodological rigor to support clinical use outside the development setting. Future studies addressing functional independence should ensure appropriate model validation and conform to uniform reporting standards for prognosis research.
Collapse
Affiliation(s)
- Uwe M Pommerich
- Hammel Neurorehabilitation Centre and University Research Clinic, Department of Clinical Medicine, Aarhus University, Hammel, Denmark.
| | - Peter W Stubbs
- Discipline of Physiotherapy, Graduate School of Health, University of Technology Sydney, Ultimo 2007, Australia
| | - Peter Preben Eggertsen
- Hammel Neurorehabilitation Centre and University Research Clinic, Department of Clinical Medicine, Aarhus University, Hammel, Denmark
| | - Jesper Fabricius
- Hammel Neurorehabilitation Centre and University Research Clinic, Department of Clinical Medicine, Aarhus University, Hammel, Denmark
| | - Jørgen Feldbæk Nielsen
- Hammel Neurorehabilitation Centre and University Research Clinic, Department of Clinical Medicine, Aarhus University, Hammel, Denmark
| |
Collapse
|
44
|
Wheatley J, Liu Z, Loth J, Plummer MP, Penny-Dimri JC, Segal R, Smith J, Perry LA. The prognostic value of elevated neutrophil-lymphocyte ratio for cardiac surgery-associated acute kidney injury: A systematic review and meta-analysis. Acta Anaesthesiol Scand 2023; 67:131-141. [PMID: 36367845 PMCID: PMC10099461 DOI: 10.1111/aas.14170] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/09/2022] [Revised: 10/07/2022] [Accepted: 11/03/2022] [Indexed: 11/13/2022]
Abstract
BACKGROUND Patients undergoing cardiac surgery are at significant risk of developing postoperative acute kidney injury (AKI). Neutrophil-lymphocyte ratio (NLR) is a widely available inflammatory biomarker which may be of prognostic value in this setting. METHODS We conducted a systematic review and meta-analysis of studies reporting associations between perioperative NLR with postoperative AKI. We searched Medline, Embase and the Cochrane Library, without language restriction, from inception to May 2022 for relevant studies. We meta-analysed the reported odds ratios (ORs) with 95% confidence intervals (CIs) for both elevated preoperative and postoperative NLR with risk of postoperative AKI and need for renal replacement therapy (RRT). We conducted a meta-regression to explore inter-study statistical heterogeneity. RESULTS Twelve studies involving 10,724 participants undergoing cardiac surgery were included, with eight studies being deemed at high risk of bias using PROBAST modelling. We found statistically significant associations between elevated preoperative NLR and postoperative AKI (OR 1.45, 95% CI 1.18-1.77), as well as postoperative need for RRT (OR 2.37, 95% CI 1.50-3.72). Postoperative NLR measurements were not of prognostic significance. CONCLUSIONS Elevated preoperative NLR is a reliable inflammatory biomarker for predicting AKI following cardiac surgery.
Collapse
Affiliation(s)
- Joseph Wheatley
- Department of Anaesthesia, Perioperative Medicine and Pain Medicine, Peter MacCallum Cancer Centre, Melbourne, Victoria, Australia.,Department of Intensive Care Medicine, Royal Melbourne Hospital, Melbourne, Victoria, Australia
| | - Zhengyang Liu
- Department of Anaesthesia, Royal Melbourne Hospital, Melbourne, Victoria, Australia.,Department of Critical Care, Melbourne Medical School, Faculty of Medicine, Dentistry and Health Sciences, University of Melbourne, Melbourne, Victoria, Australia
| | - Joel Loth
- Department of Anaesthesia, Royal Melbourne Hospital, Melbourne, Victoria, Australia
| | - Mark P Plummer
- Department of Critical Care, Melbourne Medical School, Faculty of Medicine, Dentistry and Health Sciences, University of Melbourne, Melbourne, Victoria, Australia.,Department of Intensive Care Medicine, Royal Adelaide Hospital, Adelaide, South Australia, Australia
| | - Jahan C Penny-Dimri
- Department of Surgery (School of Clinical Sciences at Monash Health), Monash University, Melbourne, Victoria, Australia.,Department of Cardiothoracic Surgery, Monash Health, Melbourne, Victoria, Australia
| | - Reny Segal
- Department of Anaesthesia, Royal Melbourne Hospital, Melbourne, Victoria, Australia.,Department of Critical Care, Melbourne Medical School, Faculty of Medicine, Dentistry and Health Sciences, University of Melbourne, Melbourne, Victoria, Australia
| | - Julian Smith
- Department of Surgery (School of Clinical Sciences at Monash Health), Monash University, Melbourne, Victoria, Australia.,Department of Cardiothoracic Surgery, Monash Health, Melbourne, Victoria, Australia
| | - Luke A Perry
- Department of Anaesthesia, Royal Melbourne Hospital, Melbourne, Victoria, Australia.,Department of Critical Care, Melbourne Medical School, Faculty of Medicine, Dentistry and Health Sciences, University of Melbourne, Melbourne, Victoria, Australia
| |
Collapse
|
45
|
Neligan A, Adan G, Nevitt SJ, Pullen A, Sander JW, Bonnett L, Marson AG. Prognosis of adults and children following a first unprovoked seizure. Cochrane Database Syst Rev 2023; 1:CD013847. [PMID: 36688481 PMCID: PMC9869434 DOI: 10.1002/14651858.cd013847.pub2] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/24/2023]
Abstract
BACKGROUND Epilepsy is clinically defined as two or more unprovoked epileptic seizures more than 24 hours apart. Given that, a diagnosis of epilepsy can be associated with significant morbidity and mortality, it is imperative that clinicians (and people with seizures and their relatives) have access to accurate and reliable prognostic estimates, to guide clinical practice on the risks of developing further unprovoked seizures (and by definition, a diagnosis of epilepsy) following single unprovoked epileptic seizure. OBJECTIVES 1. To provide an accurate estimate of the proportion of individuals going on to have further unprovoked seizures at subsequent time points following a single unprovoked epileptic seizure (or cluster of epileptic seizures within a 24-hour period, or a first episode of status epilepticus), of any seizure type (overall prognosis). 2. To evaluate the mortality rate following a first unprovoked epileptic seizure. SEARCH METHODS We searched the following databases on 19 September 2019 and again on 30 March 2021, with no language restrictions. The Cochrane Register of Studies (CRS Web), MEDLINE Ovid (1946 to March 29, 2021), SCOPUS (1823 onwards), ClinicalTrials.gov, the World Health Organization (WHO) International Clinical Trials Registry Platform (ICTRP). CRS Web includes randomized or quasi-randomized, controlled trials from PubMed, Embase, ClinicalTrials.gov, the World Health Organization International Clinical Trials Registry Platform (ICTRP), the Cochrane Central Register of Controlled Trials (CENTRAL), and the Specialized Registers of Cochrane Review Groups including Epilepsy. In MEDLINE (Ovid) the coverage end date always lags a few days behind the search date. SELECTION CRITERIA We included studies, both retrospective and prospective, of all age groups (except those in the neonatal period (< 1 month of age)), of people with a single unprovoked seizure, followed up for a minimum of six months, with no upper limit of follow-up, with the study end point being seizure recurrence, death, or loss to follow-up. To be included, studies must have included at least 30 participants. We excluded studies that involved people with seizures that occur as a result of an acute precipitant or provoking factor, or in close temporal proximity to an acute neurological insult, since these are not considered epileptic in aetiology (acute symptomatic seizures). We also excluded people with situational seizures, such as febrile convulsions. DATA COLLECTION AND ANALYSIS Two review authors conducted the initial screening of titles and abstracts identified through the electronic searches, and removed non-relevant articles. We obtained the full-text articles of all remaining potentially relevant studies, or those whose relevance could not be determined from the abstract alone and two authors independently assessed for eligibility. All disagreements were resolved through discussion with no need to defer to a third review author. We extracted data from included studies using a data extraction form based on the checklist for critical appraisal and data extraction for systematicreviews of prediction modelling studies (CHARMS). Two review authors then appraised the included studies, using a standardised approach based on the quality in prognostic studies (QUIPS) tool, which was adapted for overall prognosis (seizure recurrence). We conducted a meta-analysis using Review Manager 2014, with a random-effects generic inverse variance meta-analysis model, which accounted for any between-study heterogeneity in the prognostic effect. We then summarised the meta-analysis by the pooled estimate (the average prognostic factor effect), its 95% confidence interval (CI), the estimates of I² and Tau² (heterogeneity), and a 95% prediction interval for the prognostic effect in a single population at three various time points, 6 months, 12 months and 24 months. Subgroup analysis was performed according to the ages of the cohorts included; studies involving all ages, studies that recruited adult only and those that were purely paediatric. MAIN RESULTS Fifty-eight studies (involving 54 cohorts), with a total of 12,160 participants (median 147, range 31 to 1443), met the inclusion criteria for the review. Of the 58 studies, 26 studies were paediatric studies, 16 were adult and the remaining 16 studies were a combination of paediatric and adult populations. Most included studies had a cohort study design with two case-control studies and one nested case-control study. Thirty-two studies (29 cohorts) reported a prospective longitudinal design whilst 15 studies had a retrospective design whilst the remaining studies were randomised controlled trials. Nine of the studies included presented mortality data following a first unprovoked seizure. For a mortality study to be included, a proportional mortality ratio (PMR) or a standardised mortality ratio (SMR) had to be given at a specific time point following a first unprovoked seizure. To be included in the meta-analysis a study had to present clear seizure recurrence data at 6 months, 12 months or 24 months. Forty-six studies were included in the meta-analysis, of which 23 were paediatric, 13 were adult, and 10 were a combination of paediatric and adult populations. A meta-analysis was performed at three time points; six months, one year and two years for all ages combined, paediatric and adult studies, respectively. We found an estimated overall seizure recurrence of all included studies at six months of 27% (95% CI 24% to 31%), 36% (95% CI 33% to 40%) at one year and 43% (95% CI 37% to 44%) at two years, with slightly lower estimates for adult subgroup analysis and slightly higher estimates for paediatric subgroup analysis. It was not possible to provide a summary estimate of the risk of seizure recurrence beyond these time points as most of the included studies were of short follow-up and too few studies presented recurrence rates at a single time point beyond two years. The evidence presented was found to be of moderate certainty. AUTHORS' CONCLUSIONS Despite the limitations of the data (moderate-certainty of evidence), mainly relating to clinical and methodological heterogeneity we have provided summary estimates for the likely risk of seizure recurrence at six months, one year and two years for both children and adults. This provides information that is likely to be useful for the clinician counselling patients (or their parents) on the probable risk of further seizures in the short-term whilst acknowledging the paucity of long-term recurrence data, particularly beyond 10 years.
Collapse
Affiliation(s)
- Aidan Neligan
- Homerton University Hospital, NHS Foundation Trust, London, UK
- Department of Clinical and Experimental Epilepsy, UCL Queen Square Institute of Neurology, London, UK
| | - Guleed Adan
- Department of Pharmacology and Therapeutics, Institute of Systems, Molecular and Integrative Biology, University of Liverpool, Liverpool, UK
- The Walton Centre NHS Foundation Trust, Liverpool, UK
| | - Sarah J Nevitt
- Department of Health Data Science, University of Liverpool, Liverpool, UK
| | | | - Josemir W Sander
- Department of Clinical and Experimental Epilepsy, UCL Queen Square Institute of Neurology, London, UK
- National Hospital for Neurology and Neurosurgery, London, UK
| | - Laura Bonnett
- Department of Health Data Science, University of Liverpool, Liverpool, UK
| | - Anthony G Marson
- Department of Pharmacology and Therapeutics, Institute of Systems, Molecular and Integrative Biology, University of Liverpool, Liverpool, UK
- The Walton Centre NHS Foundation Trust, Liverpool, UK
- Liverpool Health Partners, Liverpool, UK
| |
Collapse
|
46
|
Grandbois van Ravenhorst C, Schluep M, Endeman H, Stolker RJ, Hoeks SE. Prognostic models for outcome prediction following in-hospital cardiac arrest using pre-arrest factors: a systematic review, meta-analysis and critical appraisal. Crit Care 2023; 27:32. [PMID: 36670450 PMCID: PMC9862512 DOI: 10.1186/s13054-023-04306-y] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2022] [Accepted: 01/06/2023] [Indexed: 01/22/2023] Open
Abstract
BACKGROUND Several prediction models of survival after in-hospital cardiac arrest (IHCA) have been published, but no overview of model performance and external validation exists. We performed a systematic review of the available prognostic models for outcome prediction of attempted resuscitation for IHCA using pre-arrest factors to enhance clinical decision-making through improved outcome prediction. METHODS This systematic review followed the CHARMS and PRISMA guidelines. Medline, Embase, Web of Science were searched up to October 2021. Studies developing, updating or validating a prediction model with pre-arrest factors for any potential clinical outcome of attempted resuscitation for IHCA were included. Studies were appraised critically according to the PROBAST checklist. A random-effects meta-analysis was performed to pool AUROC values of externally validated models. RESULTS Out of 2678 initial articles screened, 33 studies were included in this systematic review: 16 model development studies, 5 model updating studies and 12 model validation studies. The most frequently included pre-arrest factors included age, functional status, (metastatic) malignancy, heart disease, cerebrovascular events, respiratory, renal or hepatic insufficiency, hypotension and sepsis. Only six of the developed models have been independently validated in external populations. The GO-FAR score showed the best performance with a pooled AUROC of 0.78 (95% CI 0.69-0.85), versus 0.59 (95%CI 0.50-0.68) for the PAM and 0.62 (95% CI 0.49-0.74) for the PAR. CONCLUSIONS Several prognostic models for clinical outcome after attempted resuscitation for IHCA have been published. Most have a moderate risk of bias and have not been validated externally. The GO-FAR score showed the most acceptable performance. Future research should focus on updating existing models for use in clinical settings, specifically pre-arrest counselling. Systematic review registration PROSPERO CRD42021269235. Registered 21 July 2021.
Collapse
Affiliation(s)
- Casey Grandbois van Ravenhorst
- grid.5645.2000000040459992XDepartment of Anaesthesia, Erasmus University Medical Centre, Room Na-1718, P.O. Box 2040, 3000 CA Rotterdam, The Netherlands
| | - Marc Schluep
- grid.5645.2000000040459992XDepartment of Anaesthesia, Erasmus University Medical Centre, Room Na-1718, P.O. Box 2040, 3000 CA Rotterdam, The Netherlands
| | - Henrik Endeman
- grid.5645.2000000040459992XDepartment of Intensive Care Medicine, Erasmus University Medical Centre, Rotterdam, The Netherlands
| | - Robert-Jan Stolker
- grid.5645.2000000040459992XDepartment of Anaesthesia, Erasmus University Medical Centre, Room Na-1718, P.O. Box 2040, 3000 CA Rotterdam, The Netherlands
| | - Sanne Elisabeth Hoeks
- grid.5645.2000000040459992XDepartment of Anaesthesia, Erasmus University Medical Centre, Room Na-1718, P.O. Box 2040, 3000 CA Rotterdam, The Netherlands
| |
Collapse
|
47
|
Clinical prediction tools for identifying antimicrobial-resistant organism (ARO) carriage on hospital admissions: a systematic review. J Hosp Infect 2023; 134:11-26. [PMID: 36657490 DOI: 10.1016/j.jhin.2023.01.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2022] [Revised: 12/20/2022] [Accepted: 01/09/2023] [Indexed: 01/18/2023]
Abstract
BACKGROUND Increasing prevalence of antimicrobial-resistant organisms (AROs) is a growing economic and healthcare challenge. Increasing utilization of electronic medical record (EMR) systems and improvements in computation and analytical techniques afford an opportunity to reduce the spread of AROs through the development of clinical prediction tools to identify ARO carriers on admission to hospital. AIM To identify existing clinical prediction tools for meticillin-resistant Staphylococcus aureus (MRSA) and carbapenemase-producing organisms (CPOs), their predictive performance, and risk factors utilized in these tools. METHODS The CHARMS checklist was followed. Medline, EMBASE, Cochrane SR, CRD databases (DARE, NHS EED), CINAHL and Web of Science were searched from database inception to 26th July 2021. Full-text articles were assessed independently, and quality assessment was conducted using the Prediction Model Risk of Bias Assessment Tool. FINDINGS In total, 3809 abstracts were identified and 22 studies were included. Among these studies, risk score models were the most common prediction tool (N=16). Previous admission, recent antibiotic exposure, age and sex were the most common risk factors for ARO carriage. Prediction tools were commonly evaluated on sensitivity and specificity with ranges of 15-100% and 46-98.6%, respectively, for MRSA, and 30-81.3% and 79.8-99.9%, respectively, for CPOs. CONCLUSION There is no gold standard ARO prediction tool. However, high-performance clinical prediction tools and identification of key risk factors for the early detection of AROs exist. Risk score models are easier to use and interpret; however, with recent improvements in machine learning techniques, highly robust models can be developed with data stored in an EMR.
Collapse
|
48
|
Bronchopulmonary dysplasia prediction models: a systematic review and meta-analysis with validation. Pediatr Res 2023:10.1038/s41390-022-02451-8. [PMID: 36624282 DOI: 10.1038/s41390-022-02451-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/20/2022] [Revised: 12/06/2022] [Accepted: 12/14/2022] [Indexed: 01/11/2023]
Abstract
Prediction models could identify infants at the greatest risk of bronchopulmonary dysplasia (BPD) and allow targeted preventative strategies. We performed a systematic review and meta-analysis with external validation of identified models. Studies using predictors available before day 14 of life to predict BPD in very preterm infants were included. Two reviewers assessed 7628 studies for eligibility. Meta-analysis of externally validated models was followed by validation using 62,864 very preterm infants in England and Wales. A total of 64 studies using 53 prediction models were included totalling 274,407 infants (range 32-156,587/study). In all, 35 (55%) studies predated 2010; 39 (61%) were single-centre studies. A total of 97% of studies had a high risk of bias, especially in the analysis domain. Following meta-analysis of 22 BPD and 11 BPD/death composite externally validated models, Laughon's day one model was the most promising in predicting BPD and death (C-statistic 0.76 (95% CI 0.70-0.81) and good calibration). Six models were externally validated in our cohort with C-statistics between 0.70 and 0.90 but with poor calibration. Few BPD prediction models were developed with contemporary populations, underwent external validation, or had calibration and impact analyses. Contemporary, validated, and dynamic prediction models are needed for targeted preventative strategies. IMPACT: This review aims to provide a comprehensive assessment of all BPD prediction models developed to address the uncertainty of which model is sufficiently valid and generalisable for use in clinical practice and research. Published BPD prediction models are mostly outdated, single centre and lack external validation. Laughon's 2011 model is the most promising but more robust models, using contemporary data with external validation are needed to support better treatments.
Collapse
|
49
|
Tan J, Ma C, Zhu C, Wang Y, Zou X, Li H, Li J, He Y, Wu C. Prediction models for depression risk among older adults: systematic review and critical appraisal. Ageing Res Rev 2023; 83:101803. [PMID: 36410622 DOI: 10.1016/j.arr.2022.101803] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2022] [Revised: 11/15/2022] [Accepted: 11/16/2022] [Indexed: 11/23/2022]
Abstract
OBJECTIVE To provide an overview of prediction models for the risk of major depressive disorder (MDD) among older adults. METHODS We conducted a systematic review combined with a meta-analysis and critical appraisal of published studies on existing geriatric depression risk models. RESULTS The systematic search screened 23,378 titles and abstracts; 14 studies including 20 prediction models were included. A total of 16 predictors were selected in the final model at least twice. Age, physical health, and cognitive function were the most common predictors. Only one model was externally validated, two models were presented with a complete equation, and five models examined the calibration. We found substantial heterogeneity in predictor and outcome definitions across models; important methodological information was often missing. All models were rated at high or unclear risk of bias, primarily due to methodological limitations. The pooled C-statistics of 12 prediction models was 0.83 (95%CI=0.77-0.89). CONCLUSION The usefulness of all models remains unclear due to several methodological limitations. Future studies should focus on methodological quality and external validation of depression risk prediction models.
Collapse
Affiliation(s)
- Jie Tan
- Global Health Research Center, Duke Kunshan University, Kunshan, Jiangsu, China; School of Public Health, Wuhan University, Wuhan, Hubei, China
| | - Chenxinan Ma
- Global Health Research Center, Duke Kunshan University, Kunshan, Jiangsu, China
| | - Chonglin Zhu
- College of Pharmacy, Southwest Medical University, Luzhou, Sichuang, China
| | - Yin Wang
- College of Management Science, Chengdu University of Technology, Chengdu, Sichuan, China
| | - Xiaoshuang Zou
- College of Basic Medicine Science, Shenyang Medical College, Shenyang, Liaoning, China
| | - Han Li
- School of Public Health, Zunyi Medical University, Zunyi, Guizhou, China
| | - Jiarun Li
- School of Basic Medicine, Guizhou Medical University, Guiyang, Guizhou, China
| | - Yanxuan He
- School of Kinesiology, Shanghai University of Sport, Shanghai, China
| | - Chenkai Wu
- Global Health Research Center, Duke Kunshan University, Kunshan, Jiangsu, China.
| |
Collapse
|
50
|
Hueting TA, van Maaren MC, Hendriks MP, Koffijberg H, Siesling S. The majority of 922 prediction models supporting breast cancer decision-making are at high risk of bias. J Clin Epidemiol 2022; 152:238-247. [PMID: 36633901 DOI: 10.1016/j.jclinepi.2022.10.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2022] [Revised: 09/25/2022] [Accepted: 10/20/2022] [Indexed: 11/23/2022]
Abstract
OBJECTIVES To systematically review the currently available prediction models that may support treatment decision-making in breast cancer. STUDY DESIGN AND SETTING Literature was systematically searched to identify studies reporting on development of prediction models aiming to support breast cancer treatment decision-making, published between January 2010 and December 2020. Quality and risk of bias were assessed using the Prediction model Risk Of Bias (ROB) Assessment Tool (PROBAST). RESULTS After screening 20,460 studies, 534 studies were included, reporting on 922 models. The 922 models predicted: mortality (n = 417 45%), recurrence (n = 217, 24%), lymph node involvement (n = 141, 15%), adverse events (n = 58, 6%), treatment response (n = 56, 6%), or other outcomes (n = 33, 4%). In total, 285 models (31%) lacked a complete description of the final model and could not be applied to new patients. Most models (n = 878, 95%) were considered to contain high ROB. CONCLUSION A substantial overlap in predictor variables and outcomes between the models was observed. Most models were not reported according to established reporting guidelines or showed methodological flaws during the development and/or validation of the model. Further development of prediction models with thorough quality and validity assessment is an essential first step for future clinical application.
Collapse
Affiliation(s)
- Tom A Hueting
- Department of Health Technology & Services Research, Technical Medical Centre, University of Twente, Enschede, The Netherlands
| | - Marissa C van Maaren
- Department of Health Technology & Services Research, Technical Medical Centre, University of Twente, Enschede, The Netherlands; Department of Research and Development, Netherlands Comprehensive Cancer Organisation (IKNL), Utrecht, The Netherlands
| | - Mathijs P Hendriks
- Department of Health Technology & Services Research, Technical Medical Centre, University of Twente, Enschede, The Netherlands; Department of Research and Development, Netherlands Comprehensive Cancer Organisation (IKNL), Utrecht, The Netherlands; Department of Medical Oncology, Northwest Clinics, Alkmaar, The Netherlands
| | - Hendrik Koffijberg
- Department of Health Technology & Services Research, Technical Medical Centre, University of Twente, Enschede, The Netherlands
| | - Sabine Siesling
- Department of Health Technology & Services Research, Technical Medical Centre, University of Twente, Enschede, The Netherlands; Department of Research and Development, Netherlands Comprehensive Cancer Organisation (IKNL), Utrecht, The Netherlands.
| |
Collapse
|