1
|
Archer L, Peat G, Snell KIE, Hill JC, Dunn KM, Foster NE, Bishop A, van der Windt D, Wynne-Jones G. Musculoskeletal Health and Work: Development and Internal-External Cross-Validation of a Model to Predict Risk of Work Absence and Presenteeism in People Seeking Primary Healthcare. JOURNAL OF OCCUPATIONAL REHABILITATION 2024:10.1007/s10926-024-10223-w. [PMID: 38963652 DOI: 10.1007/s10926-024-10223-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 06/22/2024] [Indexed: 07/05/2024]
Abstract
PURPOSE To develop and validate prediction models for the risk of future work absence and level of presenteeism, in adults seeking primary healthcare with musculoskeletal disorders (MSD). METHODS Six studies from the West-Midlands/Northwest regions of England, recruiting adults consulting primary care with MSD were included for model development and internal-external cross-validation (IECV). The primary outcome was any work absence within 6 months of their consultation. Secondary outcomes included 6-month presenteeism and 12-month work absence. Ten candidate predictors were included: age; sex; multisite pain; baseline pain score; pain duration; job type; anxiety/depression; comorbidities; absence in the previous 6 months; and baseline presenteeism. RESULTS For the 6-month absence model, 2179 participants (215 absences) were available across five studies. Calibration was promising, although varied across studies, with a pooled calibration slope of 0.93 (95% CI: 0.41-1.46) on IECV. On average, the model discriminated well between those with work absence within 6 months, and those without (IECV-pooled C-statistic 0.76, 95% CI: 0.66-0.86). The 6-month presenteeism model, while well calibrated on average, showed some individual-level variation in predictive accuracy, and the 12-month absence model was poorly calibrated due to the small available size for model development. CONCLUSIONS The developed models predict 6-month work absence and presenteeism with reasonable accuracy, on average, in adults consulting with MSD. The model to predict 12-month absence was poorly calibrated and is not yet ready for use in practice. This information may support shared decision-making and targeting occupational health interventions at those with a higher risk of absence or presenteeism in the 6 months following consultation. Further external validation is needed before the models' use can be recommended or their impact on patients can be fully assessed.
Collapse
Affiliation(s)
- Lucinda Archer
- School of Medicine, Keele University, Staffordshire, ST5 5BG, UK
- Institute of Applied Health Research, University of Birmingham, Edgbaston, Birmingham, B15 2TT, UK
- National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK
| | - George Peat
- School of Medicine, Keele University, Staffordshire, ST5 5BG, UK
- Centre for Applied Health and Social Care Research (CARe), Sheffield Hallam University, Sheffield, S10 2BP, UK
| | - Kym I E Snell
- Institute of Applied Health Research, University of Birmingham, Edgbaston, Birmingham, B15 2TT, UK
- National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK
| | - Jonathan C Hill
- School of Medicine, Keele University, Staffordshire, ST5 5BG, UK
| | - Kate M Dunn
- School of Medicine, Keele University, Staffordshire, ST5 5BG, UK
| | - Nadine E Foster
- School of Medicine, Keele University, Staffordshire, ST5 5BG, UK
- Surgical Treatment and Rehabilitation Service (STARS) Education and Research Alliance, The University of Queensland and Metro North Hospital and Health Service, St Lucia, QLD, Australia
| | - Annette Bishop
- School of Medicine, Keele University, Staffordshire, ST5 5BG, UK
| | | | | |
Collapse
|
2
|
Nguyen TL, Trompet S, Brodersen JB, Hoogland J, Debray TPA, Sattar N, Jukema JW, Westendorp RGJ. The potential benefit of statin prescription based on prediction of treatment responsiveness in older individuals: an application to the PROSPER randomized controlled trial. Eur J Prev Cardiol 2024; 31:945-953. [PMID: 38085032 PMCID: PMC11144465 DOI: 10.1093/eurjpc/zwad383] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/08/2022] [Revised: 03/03/2023] [Accepted: 12/06/2023] [Indexed: 06/04/2024]
Abstract
AIMS Clinical guidelines often recommend treating individuals based on their cardiovascular risk. We revisit this paradigm and quantify the efficacy of three treatment strategies: (i) overall prescription, i.e. treatment to all individuals sharing the eligibility criteria of a trial; (ii) risk-stratified prescription, i.e. treatment only to those at an elevated outcome risk; and (iii) prescription based on predicted treatment responsiveness. METHODS AND RESULTS We reanalysed the PROSPER randomized controlled trial, which included individuals aged 70-82 years with a history of, or risk factors for, vascular diseases. We conducted the derivation and internal-external validation of a model predicting treatment responsiveness. We compared with placebo (n = 2913): (i) pravastatin (n = 2891); (ii) pravastatin in the presence of previous vascular diseases and placebo in the absence thereof (n = 2925); and (iii) pravastatin in the presence of a favourable prediction of treatment response and placebo in the absence thereof (n = 2890). We found an absolute difference in primary outcome events composed of coronary death, non-fatal myocardial infarction, and fatal or non-fatal stroke, per 10 000 person-years equal to: -78 events (95% CI, -144 to -12) when prescribing pravastatin to all participants; -66 events (95% CI, -114 to -18) when treating only individuals with an elevated vascular risk; and -103 events (95% CI, -162 to -44) when restricting pravastatin to individuals with a favourable prediction of treatment response. CONCLUSION Pravastatin prescription based on predicted responsiveness may have an encouraging potential for cardiovascular prevention. Further external validation of our results and clinical experiments are needed. TRIAL REGISTRATION ISRCTN40976937.
Collapse
Affiliation(s)
- Tri-Long Nguyen
- Section of Epidemiology, Department of Public Health, University of Copenhagen, Øster Farimagsgade 5, DK-1356 Copenhagen K, Denmark
| | - Stella Trompet
- Department of Cardiology, Leiden University Medical Centre, Leiden, The Netherlands
- Departments of Gerontology and Geriatrics, Leiden University Medical Centre, Leiden, The Netherlands
| | - John B Brodersen
- Centre of General Practice, Department of Public Health, University of Copenhagen, Copenhagen, Denmark
- Primary Health Care Research Unit, Region Zealand, Denmark
| | - Jeroen Hoogland
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
- Department of Epidemiology and Data Science, Amsterdam University Medical Centers, Amsterdam, The Netherlands
| | - Thomas P A Debray
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
- Smart Data Analysis and Statistics B.V., Utrecht, The Netherlands
| | - Naveed Sattar
- School of Cardiovascular & Metabolic Health, British Heart Foundation Centre of Research Excellence for Heart Failure Prevention and Treatment, University of Glasgow, Glasgow, United Kingdom
| | - J Wouter Jukema
- Department of Cardiology, Leiden University Medical Centre, Leiden, The Netherlands
- Durrer Center for Cardiovascular Research, Netherlands Heart Institute, Utrecht, The Netherlands
| | - Rudi G J Westendorp
- Section of Epidemiology, Department of Public Health, University of Copenhagen, Øster Farimagsgade 5, DK-1356 Copenhagen K, Denmark
- Center for Healthy Ageing, University of Copenhagen, Copenhagen, Denmark
| |
Collapse
|
3
|
Bouvier F, Chaimani A, Peyrot E, Gueyffier F, Grenet G, Porcher R. Estimating individualized treatment effects using an individual participant data meta-analysis. BMC Med Res Methodol 2024; 24:74. [PMID: 38528447 DOI: 10.1186/s12874-024-02202-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2023] [Accepted: 03/15/2024] [Indexed: 03/27/2024] Open
Abstract
BACKGROUND One key aspect of personalized medicine is to identify individuals who benefit from an intervention. Some approaches have been developed to estimate individualized treatment effects (ITE) with a single randomized control trial (RCT) or observational data, but they are often underpowered for the ITE estimation. Using individual participant data meta-analyses (IPD-MA) might solve this problem. Few studies have investigated how to develop risk prediction models with IPD-MA, and it remains unclear how to combine those methods with approaches used for ITE estimation. In this article, we compared different approaches using both simulated and real data with binary and time-to-event outcomes to estimate the individualized treatment effects from an IPD-MA in a one-stage approach. METHODS We compared five one-stage models: naive model (NA), random intercept (RI), stratified intercept (SI), rank-1 (R1), and fully stratified (FS), built with two different strategies, the S-learner and the T-learner constructed with a Monte Carlo simulation study in which we explored different scenarios with a binary or a time-to-event outcome. To evaluate the performance of the models, we used the c-statistic for benefit, the calibration of predictions, and the mean squared error. The different models were also used on the INDANA IPD-MA, comparing an anti-hypertensive treatment to no treatment or placebo ( N = 40 237 , 836 events). RESULTS Simulation results showed that using the S-learner led to better ITE estimation performances for both binary and time-to-event outcomes. None of the risk models stand out and had significantly better results. For the INDANA dataset with a binary outcome, the naive and the random intercept models had the best performances. CONCLUSIONS For the choice of the strategy, using interactions with treatment (the S-learner) is preferable. For the choice of the method, no approach is better than the other.
Collapse
Affiliation(s)
- Florie Bouvier
- Université Paris Cité and Université Sorbonne Paris Nord, Inserm, INRAE, Center for Research in Epidemiology and StatisticS (CRESS), Paris, France.
| | - Anna Chaimani
- Université Paris Cité and Université Sorbonne Paris Nord, Inserm, INRAE, Center for Research in Epidemiology and StatisticS (CRESS), Paris, France
- Cochrane France, Paris, France
| | - Etienne Peyrot
- Université Paris Cité and Université Sorbonne Paris Nord, Inserm, INRAE, Center for Research in Epidemiology and StatisticS (CRESS), Paris, France
| | - François Gueyffier
- Laboratoire de Biométrie et Biologie Evolutive UMR 5558, CNRS, Université Lyon 1, Université de Lyon, Villeurbanne, France
| | - Guillaume Grenet
- Laboratoire de Biométrie et Biologie Evolutive UMR 5558, CNRS, Université Lyon 1, Université de Lyon, Villeurbanne, France
| | - Raphaël Porcher
- Université Paris Cité and Université Sorbonne Paris Nord, Inserm, INRAE, Center for Research in Epidemiology and StatisticS (CRESS), Paris, France
- Centre d'Épidémiologie Clinique, AP-HP, Hôtel-Dieu, Paris, France
| |
Collapse
|
4
|
Riley RD, Archer L, Snell KIE, Ensor J, Dhiman P, Martin GP, Bonnett LJ, Collins GS. Evaluation of clinical prediction models (part 2): how to undertake an external validation study. BMJ 2024; 384:e074820. [PMID: 38224968 PMCID: PMC10788734 DOI: 10.1136/bmj-2023-074820] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 09/13/2023] [Indexed: 01/17/2024]
Affiliation(s)
- Richard D Riley
- Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham B15 2TT, UK
- National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK
| | - Lucinda Archer
- Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham B15 2TT, UK
- National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK
| | - Kym I E Snell
- Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham B15 2TT, UK
- National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK
| | - Joie Ensor
- Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham B15 2TT, UK
- National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK
| | - Paula Dhiman
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Glen P Martin
- Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | - Laura J Bonnett
- Department of Biostatistics, University of Liverpool, Liverpool, UK
| | - Gary S Collins
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| |
Collapse
|
5
|
Collins GS, Dhiman P, Ma J, Schlussel MM, Archer L, Van Calster B, Harrell FE, Martin GP, Moons KGM, van Smeden M, Sperrin M, Bullock GS, Riley RD. Evaluation of clinical prediction models (part 1): from development to external validation. BMJ 2024; 384:e074819. [PMID: 38191193 PMCID: PMC10772854 DOI: 10.1136/bmj-2023-074819] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 09/04/2023] [Indexed: 01/10/2024]
Affiliation(s)
- Gary S Collins
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK
| | - Paula Dhiman
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK
| | - Jie Ma
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK
| | - Michael M Schlussel
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK
| | - Lucinda Archer
- Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
- National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, UK
| | - Ben Van Calster
- KU Leuven, Department of Development and Regeneration, Leuven, Belgium
- Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, Netherlands
- EPI-Centre, KU Leuven, Belgium
| | - Frank E Harrell
- Department of Biostatistics, Vanderbilt University, Nashville, TN, USA
| | - Glen P Martin
- Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | - Karel G M Moons
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Maarten van Smeden
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Matthew Sperrin
- Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | - Garrett S Bullock
- Department of Orthopaedic Surgery, Wake Forest School of Medicine, Winston-Salem, NC, USA
- Centre for Sport, Exercise and Osteoarthritis Research Versus Arthritis, University of Oxford, Oxford, UK
| | - Richard D Riley
- Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
- National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, UK
| |
Collapse
|
6
|
Flinterman LE, González-González AI, Seils L, Bes J, Ballester M, Bañeres J, Dan S, Domagala A, Dubas-Jakóbczyk K, Likic R, Kroezen M, Batenburg R. Characteristics of Medical Deserts and Approaches to Mitigate Their Health Workforce Issues: A Scoping Review of Empirical Studies in Western Countries. Int J Health Policy Manag 2023; 12:7454. [PMID: 38618823 PMCID: PMC10590222 DOI: 10.34172/ijhpm.2023.7454] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2022] [Accepted: 05/30/2023] [Indexed: 04/16/2024] Open
Abstract
BACKGROUND Medical deserts are considered a problematic issue for many Western countries which try to employ multitude of policies and initiatives to achieve a better distribution of their health workforce (HWF). The aim of this study was to systematically map research and provide an overview of definitions, characteristics, contributing factors and approaches to mitigate medical deserts within the European Union (EU)-funded project "ROUTE-HWF" (a Roadmap OUT of mEdical deserts into supportive Health WorkForce initiatives and policies). METHODS We performed a scoping review to identify knowledge clusters/research gaps in the field of medical deserts focusing on HWF issues. Six databases were searched till June 2021. Studies reporting primary research from Western countries on definitions, characteristics, contributing factors, and approaches were included. Two independent reviewers assessed studies for eligibility, extracted data and clustered studies according to the four defined outcomes. RESULTS Two-hundred and forty studies were included (n=116, 48% Australia/New Zealand; n=105, 44% North America; n=20, 8% Europe). All used observational designs except for five quasi-experimental studies. Studies provided definitions (n=171, 71%), characteristics (n=95, 40%), contributing factors (n=112, 47%), and approaches to mitigate medical deserts (n=87, 36%). Most medical deserts were defined by the density of the population in an area. Contributing factors to HWF issues in medical deserts consisted in work-related (n=55, 23%) and lifestyle-related factors (n=33, 14%) of the HWF as well as sociodemographic characteristics (n=79, 33%). Approaches to mitigate them focused on training adapted to the scope of rural practice (n=67, 28%), HWF distribution (n=3, 1%), support/infrastructure (n=8, 3%) and innovative models of care (n=7, 3%). CONCLUSION Our study provides the first scoping review that presents and categorizes definitions, characteristics, contributing factors, and approaches to mitigate HWF issues in medical deserts. We identified gaps such as the scarcity of longitudinal studies to investigate the impact of factors contributing to medical deserts, and interventional studies to evaluate the effectiveness of approaches to mitigate HWF issues.
Collapse
Affiliation(s)
- Linda E. Flinterman
- Health Workforce and Organization Studies, Netherlands Institute for Health Services Research (NIVEL), Utrecht, The Netherlands
| | | | - Laura Seils
- Avedis Donabedian Research Institute – UAB, Madrid, Spain
| | - Julia Bes
- Health Workforce and Organization Studies, Netherlands Institute for Health Services Research (NIVEL), Utrecht, The Netherlands
| | | | | | - Sorin Dan
- Innovation and Entrepreneurship InnoLab, University of Vaasa, Vaasa, Finland
| | - Alicja Domagala
- Department of Health Policy and Management, Institute of Public Health, Jagiellonian University, Krakow, Poland
| | - Katarzyna Dubas-Jakóbczyk
- Department of Health Economics and Social Security, Institute of Public Health, Jagiellonian University, Krakow, Poland
| | - Robert Likic
- School of Medicine, University of Zagreb, Zagreb, Croatia
| | - Marieke Kroezen
- Trimbos Institute, Netherlands Institute of Mental Health and Addiction, Utrecht, The Netherlands
| | - Ronald Batenburg
- Health Workforce and Organization Studies, Netherlands Institute for Health Services Research (NIVEL), Utrecht, The Netherlands
- Department of Sociology, Radboud University, Nijmegen, The Netherlands
| |
Collapse
|
7
|
Debray TPA, Collins GS, Riley RD, Snell KIE, Van Calster B, Reitsma JB, Moons KGM. Transparent reporting of multivariable prediction models developed or validated using clustered data (TRIPOD-Cluster): explanation and elaboration. BMJ 2023; 380:e071058. [PMID: 36750236 PMCID: PMC9903176 DOI: 10.1136/bmj-2022-071058] [Citation(s) in RCA: 11] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 10/07/2022] [Indexed: 02/09/2023]
Affiliation(s)
- Thomas P A Debray
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
- Cochrane Netherlands, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | - Gary S Collins
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Botnar Research Centre, University of Oxford, Oxford, UK
- National Institute for Health and Care Research Oxford Biomedical Research Centre, John Radcliffe Hospital, Oxford, UK
| | - Richard D Riley
- Centre for Prognosis Research, School of Medicine, Keele University, Keele, UK
| | - Kym I E Snell
- Centre for Prognosis Research, School of Medicine, Keele University, Keele, UK
| | - Ben Van Calster
- Department of Development and Regeneration, KU Leuven, Leuven, Belgium
- EPI-centre, KU Leuven, Leuven, Belgium
- Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, Netherlands
| | - Johannes B Reitsma
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
- Cochrane Netherlands, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | - Karel G M Moons
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
- Cochrane Netherlands, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| |
Collapse
|
8
|
McLernon DJ, Giardiello D, Van Calster B, Wynants L, van Geloven N, van Smeden M, Therneau T, Steyerberg EW. Assessing Performance and Clinical Usefulness in Prediction Models With Survival Outcomes: Practical Guidance for Cox Proportional Hazards Models. Ann Intern Med 2023; 176:105-114. [PMID: 36571841 DOI: 10.7326/m22-0844] [Citation(s) in RCA: 32] [Impact Index Per Article: 32.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/27/2022] Open
Abstract
Risk prediction models need thorough validation to assess their performance. Validation of models for survival outcomes poses challenges due to the censoring of observations and the varying time horizon at which predictions can be made. This article describes measures to evaluate predictions and the potential improvement in decision making from survival models based on Cox proportional hazards regression. As a motivating case study, the authors consider the prediction of the composite outcome of recurrence or death (the "event") in patients with breast cancer after surgery. They developed a simple Cox regression model with 3 predictors, as in the Nottingham Prognostic Index, in 2982 women (1275 events over 5 years of follow-up) and externally validated this model in 686 women (285 events over 5 years). Improvement in performance was assessed after the addition of progesterone receptor as a prognostic biomarker. The model predictions can be evaluated across the full range of observed follow-up times or for the event occurring by the end of a fixed time horizon of interest. The authors first discuss recommended statistical measures that evaluate model performance in terms of discrimination, calibration, or overall performance. Further, they evaluate the potential clinical utility of the model to support clinical decision making according to a net benefit measure. They provide SAS and R code to illustrate internal and external validation. The authors recommend the proposed set of performance measures for transparent reporting of the validity of predictions from survival models.
Collapse
Affiliation(s)
- David J McLernon
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, United Kingdom (D.J.M.)
| | - Daniele Giardiello
- Netherlands Cancer Institute, Amsterdam, the Netherlands, Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, the Netherlands, and Institute of Biomedicine, Eurac Research, Affiliated Institute of the University of Lübeck, Bolzano, Italy (D.G.)
| | - Ben Van Calster
- Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, the Netherlands, and Department of Development and Regeneration, Katholieke Universiteit Leuven, Leuven, Belgium (B.V.)
| | - Laure Wynants
- School for Public Health and Primary Care, Maastricht University, Maastricht, the Netherlands (L.W.)
| | - Nan van Geloven
- Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, the Netherlands (N.V., E.W.S.)
| | - Maarten van Smeden
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands (M.V.)
| | - Terry Therneau
- Department of Quantitative Health Sciences, Mayo Clinic, Rochester, Minnesota (T.T.)
| | - Ewout W Steyerberg
- Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, the Netherlands (N.V., E.W.S.)
| |
Collapse
|
9
|
Chen Q, Wang Y, Liu Y, Xi B. ESRRG, ATP4A, and ATP4B as Diagnostic Biomarkers for Gastric Cancer: A Bioinformatic Analysis Based on Machine Learning. Front Physiol 2022; 13:905523. [PMID: 35812327 PMCID: PMC9262247 DOI: 10.3389/fphys.2022.905523] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2022] [Accepted: 05/10/2022] [Indexed: 11/13/2022] Open
Abstract
Based on multiple bioinformatics methods and machine learning techniques, this study was designed to explore potential hub genes of gastric cancer with a diagnostic value. The novel biomarkers were detected through multiple databases of gastric cancer–related genes. The NCBI Gene Expression Omnibus (GEO) database was used to obtain gene expression files. Three hub genes (ESRRG, ATP4A, and ATP4B) were detected through a combination of weighted gene co-expression network analysis (WGCNA), gene–gene interaction network analysis, and supervised feature selection method. GEPIA2 was used to verify the differences in the expression levels of the hub genes in normal and cancer tissues in the RNA-seq levels of Genotype-Tissue Expression (GTEx) and The Cancer Genome Atlas (TCGA) databases. The objectivity of potential hub genes was also verified by immunohistochemistry in the Human Protein Atlas (HPA) database and transcription factor–hub gene regulatory network. Machine learning (ML) methods including data pre-processing, model selection and cross-validation, and performance evaluation were examined on the hub-gene expression profiles in five Gene Expression Omnibus datasets and verified on a GEO external validation (EV) dataset. Six supervised learning models (support vector machine, random forest, k-nearest neighbors, neural network, decision tree, and eXtreme Gradient Boosting) and one semi-supervised learning model (label spreading) were established to evaluate the diagnostic value of biomarkers. Among the six supervised models, the support vector machine (SVM) algorithm was the most effective one according to calculated performance metrics, including 0.93 and 0.99 area under the curve (AUC) scores on the test and external validation datasets, respectively. Furthermore, the semi-supervised model could also successfully learn and predict sample types, achieving a 0.986 AUC score on the EV dataset, even when 10% samples in the five GEO datasets were labeled. In conclusion, three hub genes (ATP4A, ATP4B, and ESRRG) closely related to gastric cancer were mined, based on which the ML diagnostic model of gastric cancer was conducted.
Collapse
Affiliation(s)
- Qiu Chen
- Medical College, Yangzhou University, Yangzhou, China
| | - Yu Wang
- College of Physics Science and Technology, Yangzhou University, Yangzhou, China
| | - Yongjun Liu
- College of Physics Science and Technology, Yangzhou University, Yangzhou, China
| | - Bin Xi
- College of Physics Science and Technology, Yangzhou University, Yangzhou, China
- *Correspondence: Bin Xi,
| |
Collapse
|
10
|
Yang S, Han Y, Yu C, Guo Y, Pang Y, Sun D, Pei P, Yang L, Chen Y, Du H, Wang H, Massa MS, Bennett D, Clarke R, Chen J, Chen Z, Lv J, Li L. Development of a Model to Predict 10-Year Risk of Ischemic and Hemorrhagic Stroke and Ischemic Heart Disease Using the China Kadoorie Biobank. Neurology 2022; 98:e2307-e2317. [PMID: 35410902 PMCID: PMC9202526 DOI: 10.1212/wnl.0000000000200139] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2021] [Accepted: 01/18/2022] [Indexed: 11/21/2022] Open
Abstract
BACKGROUND AND OBJECTIVES Contemporary cardiovascular disease (CVD) risk prediction models are rarely applied in routine clinical practice in China due to substantial regional differences in absolute risks of major CVD types within China. Moreover, the inclusion of blood lipids in most risk prediction models also limits their use in the Chinese population. We developed 10-year CVD risk prediction models excluding blood lipids that may be applicable to diverse regions of China. METHODS We derived sex-specific models separately for ischemic heart disease (IHD), ischemic stroke (IS), and hemorrhagic stroke (HS) in addition to total CVD in the China Kadoorie Biobank. Participants were age 30-79 years without CVD at baseline. Predictors included age, systolic and diastolic blood pressure, use of blood pressure-lowering treatment, current daily smoking, diabetes, and waist circumference. Total CVD risks were combined in terms of conditional probability using the predicted risks of 3 submodels. Risk models were recalibrated in each region by 2 methods (practical and ideal) and risk prediction was estimated before and after recalibration. RESULTS Model derivation involved 489,596 individuals, including 45,947 IHD, 43,647 IS, and 11,168 HS cases during 11 years of follow-up. In women, the Harrell C was 0.732 (95% CI 0.706-0.758), 0.759 (0.738-0.779), and 0.803 (0.778-0.827) for IHD, IS, and HS, respectively. The Harrell C for total CVD was 0.734 (0.732-0.736), 0.754 (0.752-0.756), and 0.774 (0.772-0.776) for models before recalibration, after practical recalibration, and after ideal recalibration. The calibration performances improved after recalibration, with models after ideal recalibration showing the best model performances. The results for men were comparable to those for women. DISCUSSION Our CVD risk prediction models yielded good discrimination of IHD and stroke subtypes in addition to total CVD without including blood lipids. Flexible recalibration of our models for different regions could enable more widespread use using resident health records covering the overall Chinese population. CLASSIFICATION OF EVIDENCE This study provides Class I evidence that a prediction model incorporating accessible clinical variables predicts 10-year risk of IHD, IS, and HS in the Chinese population age 30-79 years.
Collapse
Affiliation(s)
- Songchun Yang
- From the Department of Epidemiology & Biostatistics (S.Y., Y.H., C.Y., Y.P., D.S., J.L., L.L.), School of Public Health, Peking University; Peking University Center for Public Health and Epidemic Preparedness & Response (C.Y., J.L., L.L.); Fuwai Hospital Chinese Academy of Medical Sciences (Y.G.); Chinese Academy of Medical Sciences (P.P.), Beijing, China; Medical Research Council Population Health Research Unit at the University of Oxford (L.Y., Y.C., H.D.); Clinical Trial Service Unit & Epidemiological Studies Unit (L.Y., Y.C., H.D., M.S.M., D.B., R.C., Z.C.), Nuffield Department of Population Health, University of Oxford, UK; NCDs Prevention and Control Department (H.W.), Zhejiang CDC, Hangzhou; China National Center for Food Safety Risk Assessment (J.C.); and Key Laboratory of Molecular Cardiovascular Sciences (Peking University) (J.L.), Ministry of Education, Beijing, China
| | - Yuting Han
- From the Department of Epidemiology & Biostatistics (S.Y., Y.H., C.Y., Y.P., D.S., J.L., L.L.), School of Public Health, Peking University; Peking University Center for Public Health and Epidemic Preparedness & Response (C.Y., J.L., L.L.); Fuwai Hospital Chinese Academy of Medical Sciences (Y.G.); Chinese Academy of Medical Sciences (P.P.), Beijing, China; Medical Research Council Population Health Research Unit at the University of Oxford (L.Y., Y.C., H.D.); Clinical Trial Service Unit & Epidemiological Studies Unit (L.Y., Y.C., H.D., M.S.M., D.B., R.C., Z.C.), Nuffield Department of Population Health, University of Oxford, UK; NCDs Prevention and Control Department (H.W.), Zhejiang CDC, Hangzhou; China National Center for Food Safety Risk Assessment (J.C.); and Key Laboratory of Molecular Cardiovascular Sciences (Peking University) (J.L.), Ministry of Education, Beijing, China
| | - Canqing Yu
- From the Department of Epidemiology & Biostatistics (S.Y., Y.H., C.Y., Y.P., D.S., J.L., L.L.), School of Public Health, Peking University; Peking University Center for Public Health and Epidemic Preparedness & Response (C.Y., J.L., L.L.); Fuwai Hospital Chinese Academy of Medical Sciences (Y.G.); Chinese Academy of Medical Sciences (P.P.), Beijing, China; Medical Research Council Population Health Research Unit at the University of Oxford (L.Y., Y.C., H.D.); Clinical Trial Service Unit & Epidemiological Studies Unit (L.Y., Y.C., H.D., M.S.M., D.B., R.C., Z.C.), Nuffield Department of Population Health, University of Oxford, UK; NCDs Prevention and Control Department (H.W.), Zhejiang CDC, Hangzhou; China National Center for Food Safety Risk Assessment (J.C.); and Key Laboratory of Molecular Cardiovascular Sciences (Peking University) (J.L.), Ministry of Education, Beijing, China
| | - Yu Guo
- From the Department of Epidemiology & Biostatistics (S.Y., Y.H., C.Y., Y.P., D.S., J.L., L.L.), School of Public Health, Peking University; Peking University Center for Public Health and Epidemic Preparedness & Response (C.Y., J.L., L.L.); Fuwai Hospital Chinese Academy of Medical Sciences (Y.G.); Chinese Academy of Medical Sciences (P.P.), Beijing, China; Medical Research Council Population Health Research Unit at the University of Oxford (L.Y., Y.C., H.D.); Clinical Trial Service Unit & Epidemiological Studies Unit (L.Y., Y.C., H.D., M.S.M., D.B., R.C., Z.C.), Nuffield Department of Population Health, University of Oxford, UK; NCDs Prevention and Control Department (H.W.), Zhejiang CDC, Hangzhou; China National Center for Food Safety Risk Assessment (J.C.); and Key Laboratory of Molecular Cardiovascular Sciences (Peking University) (J.L.), Ministry of Education, Beijing, China
| | - Yuanjie Pang
- From the Department of Epidemiology & Biostatistics (S.Y., Y.H., C.Y., Y.P., D.S., J.L., L.L.), School of Public Health, Peking University; Peking University Center for Public Health and Epidemic Preparedness & Response (C.Y., J.L., L.L.); Fuwai Hospital Chinese Academy of Medical Sciences (Y.G.); Chinese Academy of Medical Sciences (P.P.), Beijing, China; Medical Research Council Population Health Research Unit at the University of Oxford (L.Y., Y.C., H.D.); Clinical Trial Service Unit & Epidemiological Studies Unit (L.Y., Y.C., H.D., M.S.M., D.B., R.C., Z.C.), Nuffield Department of Population Health, University of Oxford, UK; NCDs Prevention and Control Department (H.W.), Zhejiang CDC, Hangzhou; China National Center for Food Safety Risk Assessment (J.C.); and Key Laboratory of Molecular Cardiovascular Sciences (Peking University) (J.L.), Ministry of Education, Beijing, China
| | - Dianjianyi Sun
- From the Department of Epidemiology & Biostatistics (S.Y., Y.H., C.Y., Y.P., D.S., J.L., L.L.), School of Public Health, Peking University; Peking University Center for Public Health and Epidemic Preparedness & Response (C.Y., J.L., L.L.); Fuwai Hospital Chinese Academy of Medical Sciences (Y.G.); Chinese Academy of Medical Sciences (P.P.), Beijing, China; Medical Research Council Population Health Research Unit at the University of Oxford (L.Y., Y.C., H.D.); Clinical Trial Service Unit & Epidemiological Studies Unit (L.Y., Y.C., H.D., M.S.M., D.B., R.C., Z.C.), Nuffield Department of Population Health, University of Oxford, UK; NCDs Prevention and Control Department (H.W.), Zhejiang CDC, Hangzhou; China National Center for Food Safety Risk Assessment (J.C.); and Key Laboratory of Molecular Cardiovascular Sciences (Peking University) (J.L.), Ministry of Education, Beijing, China
| | - Pei Pei
- From the Department of Epidemiology & Biostatistics (S.Y., Y.H., C.Y., Y.P., D.S., J.L., L.L.), School of Public Health, Peking University; Peking University Center for Public Health and Epidemic Preparedness & Response (C.Y., J.L., L.L.); Fuwai Hospital Chinese Academy of Medical Sciences (Y.G.); Chinese Academy of Medical Sciences (P.P.), Beijing, China; Medical Research Council Population Health Research Unit at the University of Oxford (L.Y., Y.C., H.D.); Clinical Trial Service Unit & Epidemiological Studies Unit (L.Y., Y.C., H.D., M.S.M., D.B., R.C., Z.C.), Nuffield Department of Population Health, University of Oxford, UK; NCDs Prevention and Control Department (H.W.), Zhejiang CDC, Hangzhou; China National Center for Food Safety Risk Assessment (J.C.); and Key Laboratory of Molecular Cardiovascular Sciences (Peking University) (J.L.), Ministry of Education, Beijing, China
| | - Ling Yang
- From the Department of Epidemiology & Biostatistics (S.Y., Y.H., C.Y., Y.P., D.S., J.L., L.L.), School of Public Health, Peking University; Peking University Center for Public Health and Epidemic Preparedness & Response (C.Y., J.L., L.L.); Fuwai Hospital Chinese Academy of Medical Sciences (Y.G.); Chinese Academy of Medical Sciences (P.P.), Beijing, China; Medical Research Council Population Health Research Unit at the University of Oxford (L.Y., Y.C., H.D.); Clinical Trial Service Unit & Epidemiological Studies Unit (L.Y., Y.C., H.D., M.S.M., D.B., R.C., Z.C.), Nuffield Department of Population Health, University of Oxford, UK; NCDs Prevention and Control Department (H.W.), Zhejiang CDC, Hangzhou; China National Center for Food Safety Risk Assessment (J.C.); and Key Laboratory of Molecular Cardiovascular Sciences (Peking University) (J.L.), Ministry of Education, Beijing, China
| | - Yiping Chen
- From the Department of Epidemiology & Biostatistics (S.Y., Y.H., C.Y., Y.P., D.S., J.L., L.L.), School of Public Health, Peking University; Peking University Center for Public Health and Epidemic Preparedness & Response (C.Y., J.L., L.L.); Fuwai Hospital Chinese Academy of Medical Sciences (Y.G.); Chinese Academy of Medical Sciences (P.P.), Beijing, China; Medical Research Council Population Health Research Unit at the University of Oxford (L.Y., Y.C., H.D.); Clinical Trial Service Unit & Epidemiological Studies Unit (L.Y., Y.C., H.D., M.S.M., D.B., R.C., Z.C.), Nuffield Department of Population Health, University of Oxford, UK; NCDs Prevention and Control Department (H.W.), Zhejiang CDC, Hangzhou; China National Center for Food Safety Risk Assessment (J.C.); and Key Laboratory of Molecular Cardiovascular Sciences (Peking University) (J.L.), Ministry of Education, Beijing, China
| | - Huaidong Du
- From the Department of Epidemiology & Biostatistics (S.Y., Y.H., C.Y., Y.P., D.S., J.L., L.L.), School of Public Health, Peking University; Peking University Center for Public Health and Epidemic Preparedness & Response (C.Y., J.L., L.L.); Fuwai Hospital Chinese Academy of Medical Sciences (Y.G.); Chinese Academy of Medical Sciences (P.P.), Beijing, China; Medical Research Council Population Health Research Unit at the University of Oxford (L.Y., Y.C., H.D.); Clinical Trial Service Unit & Epidemiological Studies Unit (L.Y., Y.C., H.D., M.S.M., D.B., R.C., Z.C.), Nuffield Department of Population Health, University of Oxford, UK; NCDs Prevention and Control Department (H.W.), Zhejiang CDC, Hangzhou; China National Center for Food Safety Risk Assessment (J.C.); and Key Laboratory of Molecular Cardiovascular Sciences (Peking University) (J.L.), Ministry of Education, Beijing, China
| | - Hao Wang
- From the Department of Epidemiology & Biostatistics (S.Y., Y.H., C.Y., Y.P., D.S., J.L., L.L.), School of Public Health, Peking University; Peking University Center for Public Health and Epidemic Preparedness & Response (C.Y., J.L., L.L.); Fuwai Hospital Chinese Academy of Medical Sciences (Y.G.); Chinese Academy of Medical Sciences (P.P.), Beijing, China; Medical Research Council Population Health Research Unit at the University of Oxford (L.Y., Y.C., H.D.); Clinical Trial Service Unit & Epidemiological Studies Unit (L.Y., Y.C., H.D., M.S.M., D.B., R.C., Z.C.), Nuffield Department of Population Health, University of Oxford, UK; NCDs Prevention and Control Department (H.W.), Zhejiang CDC, Hangzhou; China National Center for Food Safety Risk Assessment (J.C.); and Key Laboratory of Molecular Cardiovascular Sciences (Peking University) (J.L.), Ministry of Education, Beijing, China
| | - M Sofia Massa
- From the Department of Epidemiology & Biostatistics (S.Y., Y.H., C.Y., Y.P., D.S., J.L., L.L.), School of Public Health, Peking University; Peking University Center for Public Health and Epidemic Preparedness & Response (C.Y., J.L., L.L.); Fuwai Hospital Chinese Academy of Medical Sciences (Y.G.); Chinese Academy of Medical Sciences (P.P.), Beijing, China; Medical Research Council Population Health Research Unit at the University of Oxford (L.Y., Y.C., H.D.); Clinical Trial Service Unit & Epidemiological Studies Unit (L.Y., Y.C., H.D., M.S.M., D.B., R.C., Z.C.), Nuffield Department of Population Health, University of Oxford, UK; NCDs Prevention and Control Department (H.W.), Zhejiang CDC, Hangzhou; China National Center for Food Safety Risk Assessment (J.C.); and Key Laboratory of Molecular Cardiovascular Sciences (Peking University) (J.L.), Ministry of Education, Beijing, China
| | - Derrick Bennett
- From the Department of Epidemiology & Biostatistics (S.Y., Y.H., C.Y., Y.P., D.S., J.L., L.L.), School of Public Health, Peking University; Peking University Center for Public Health and Epidemic Preparedness & Response (C.Y., J.L., L.L.); Fuwai Hospital Chinese Academy of Medical Sciences (Y.G.); Chinese Academy of Medical Sciences (P.P.), Beijing, China; Medical Research Council Population Health Research Unit at the University of Oxford (L.Y., Y.C., H.D.); Clinical Trial Service Unit & Epidemiological Studies Unit (L.Y., Y.C., H.D., M.S.M., D.B., R.C., Z.C.), Nuffield Department of Population Health, University of Oxford, UK; NCDs Prevention and Control Department (H.W.), Zhejiang CDC, Hangzhou; China National Center for Food Safety Risk Assessment (J.C.); and Key Laboratory of Molecular Cardiovascular Sciences (Peking University) (J.L.), Ministry of Education, Beijing, China
| | - Robert Clarke
- From the Department of Epidemiology & Biostatistics (S.Y., Y.H., C.Y., Y.P., D.S., J.L., L.L.), School of Public Health, Peking University; Peking University Center for Public Health and Epidemic Preparedness & Response (C.Y., J.L., L.L.); Fuwai Hospital Chinese Academy of Medical Sciences (Y.G.); Chinese Academy of Medical Sciences (P.P.), Beijing, China; Medical Research Council Population Health Research Unit at the University of Oxford (L.Y., Y.C., H.D.); Clinical Trial Service Unit & Epidemiological Studies Unit (L.Y., Y.C., H.D., M.S.M., D.B., R.C., Z.C.), Nuffield Department of Population Health, University of Oxford, UK; NCDs Prevention and Control Department (H.W.), Zhejiang CDC, Hangzhou; China National Center for Food Safety Risk Assessment (J.C.); and Key Laboratory of Molecular Cardiovascular Sciences (Peking University) (J.L.), Ministry of Education, Beijing, China
| | - Junshi Chen
- From the Department of Epidemiology & Biostatistics (S.Y., Y.H., C.Y., Y.P., D.S., J.L., L.L.), School of Public Health, Peking University; Peking University Center for Public Health and Epidemic Preparedness & Response (C.Y., J.L., L.L.); Fuwai Hospital Chinese Academy of Medical Sciences (Y.G.); Chinese Academy of Medical Sciences (P.P.), Beijing, China; Medical Research Council Population Health Research Unit at the University of Oxford (L.Y., Y.C., H.D.); Clinical Trial Service Unit & Epidemiological Studies Unit (L.Y., Y.C., H.D., M.S.M., D.B., R.C., Z.C.), Nuffield Department of Population Health, University of Oxford, UK; NCDs Prevention and Control Department (H.W.), Zhejiang CDC, Hangzhou; China National Center for Food Safety Risk Assessment (J.C.); and Key Laboratory of Molecular Cardiovascular Sciences (Peking University) (J.L.), Ministry of Education, Beijing, China
| | - Zhengming Chen
- From the Department of Epidemiology & Biostatistics (S.Y., Y.H., C.Y., Y.P., D.S., J.L., L.L.), School of Public Health, Peking University; Peking University Center for Public Health and Epidemic Preparedness & Response (C.Y., J.L., L.L.); Fuwai Hospital Chinese Academy of Medical Sciences (Y.G.); Chinese Academy of Medical Sciences (P.P.), Beijing, China; Medical Research Council Population Health Research Unit at the University of Oxford (L.Y., Y.C., H.D.); Clinical Trial Service Unit & Epidemiological Studies Unit (L.Y., Y.C., H.D., M.S.M., D.B., R.C., Z.C.), Nuffield Department of Population Health, University of Oxford, UK; NCDs Prevention and Control Department (H.W.), Zhejiang CDC, Hangzhou; China National Center for Food Safety Risk Assessment (J.C.); and Key Laboratory of Molecular Cardiovascular Sciences (Peking University) (J.L.), Ministry of Education, Beijing, China
| | - Jun Lv
- From the Department of Epidemiology & Biostatistics (S.Y., Y.H., C.Y., Y.P., D.S., J.L., L.L.), School of Public Health, Peking University; Peking University Center for Public Health and Epidemic Preparedness & Response (C.Y., J.L., L.L.); Fuwai Hospital Chinese Academy of Medical Sciences (Y.G.); Chinese Academy of Medical Sciences (P.P.), Beijing, China; Medical Research Council Population Health Research Unit at the University of Oxford (L.Y., Y.C., H.D.); Clinical Trial Service Unit & Epidemiological Studies Unit (L.Y., Y.C., H.D., M.S.M., D.B., R.C., Z.C.), Nuffield Department of Population Health, University of Oxford, UK; NCDs Prevention and Control Department (H.W.), Zhejiang CDC, Hangzhou; China National Center for Food Safety Risk Assessment (J.C.); and Key Laboratory of Molecular Cardiovascular Sciences (Peking University) (J.L.), Ministry of Education, Beijing, China
| | - Liming Li
- From the Department of Epidemiology & Biostatistics (S.Y., Y.H., C.Y., Y.P., D.S., J.L., L.L.), School of Public Health, Peking University; Peking University Center for Public Health and Epidemic Preparedness & Response (C.Y., J.L., L.L.); Fuwai Hospital Chinese Academy of Medical Sciences (Y.G.); Chinese Academy of Medical Sciences (P.P.), Beijing, China; Medical Research Council Population Health Research Unit at the University of Oxford (L.Y., Y.C., H.D.); Clinical Trial Service Unit & Epidemiological Studies Unit (L.Y., Y.C., H.D., M.S.M., D.B., R.C., Z.C.), Nuffield Department of Population Health, University of Oxford, UK; NCDs Prevention and Control Department (H.W.), Zhejiang CDC, Hangzhou; China National Center for Food Safety Risk Assessment (J.C.); and Key Laboratory of Molecular Cardiovascular Sciences (Peking University) (J.L.), Ministry of Education, Beijing, China
| |
Collapse
|
11
|
Maile HP, Li JPO, Fortune MD, Royston P, Leucci MT, Moghul I, Szabo A, Balaskas K, Allan BD, Hardcastle AJ, Hysi P, Pontikos N, Tuft SJ, Gore DM. Personalized Model to Predict Keratoconus Progression From Demographic, Topographic, and Genetic Data. Am J Ophthalmol 2022; 240:321-329. [PMID: 35469790 DOI: 10.1016/j.ajo.2022.04.004] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2022] [Revised: 04/02/2022] [Accepted: 04/13/2022] [Indexed: 11/26/2022]
Abstract
PURPOSE To generate a prognostic model to predict keratoconus progression to corneal crosslinking (CXL). DESIGN Retrospective cohort study. METHODS We recruited 5025 patients (9341 eyes) with early keratoconus between January 2011 and November 2020. Genetic data from 926 patients were available. We investigated both keratometry or CXL as end points for progression and used the Royston-Parmar method on the proportional hazards scale to generate a prognostic model. We calculated hazard ratios (HRs) for each significant covariate, with explained variation and discrimination, and performed internal-external cross validation by geographic regions. RESULTS After exclusions, model fitting comprised 8701 eyes, of which 3232 underwent CXL. For early keratoconus, CXL provided a more robust prognostic model than keratometric progression. The final model explained 33% of the variation in time to event: age HR (95% CI) 0.9 (0.90-0.91), maximum anterior keratometry 1.08 (1.07-1.09), and minimum corneal thickness 0.95 (0.93-0.96) as significant covariates. Single-nucleotide polymorphisms (SNPs) associated with keratoconus (n=28) did not significantly contribute to the model. The predicted time-to-event curves closely followed the observed curves during internal-external validation. Differences in discrimination between geographic regions was low, suggesting the model maintained its predictive ability. CONCLUSIONS A prognostic model to predict keratoconus progression could aid patient empowerment, triage, and service provision. Age at presentation is the most significant predictor of progression risk. Candidate SNPs associated with keratoconus do not contribute to progression risk.
Collapse
|
12
|
Tamási B, Crowther M, Puhan MA, Steyerberg EW, Hothorn T. Individual participant data meta-analysis with mixed-effects transformation models. Biostatistics 2021; 23:1083-1098. [PMID: 34969073 PMCID: PMC9566326 DOI: 10.1093/biostatistics/kxab045] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2021] [Revised: 11/03/2021] [Accepted: 11/22/2021] [Indexed: 11/23/2022] Open
Abstract
One-stage meta-analysis of individual participant data (IPD) poses several statistical and computational challenges. For time-to-event outcomes, the approach requires the estimation of complicated nonlinear mixed-effects models that are flexible enough to realistically capture the most important characteristics of the IPD. We present a model class that incorporates general normally distributed random effects into linear transformation models. We discuss extensions to model between-study heterogeneity in baseline risks and covariate effects and also relax the assumption of proportional hazards. Within the proposed framework, data with arbitrary random censoring patterns can be handled. The accompanying \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{upgreek}
\usepackage{mathrsfs}
\setlength{\oddsidemargin}{-69pt}
\begin{document}
}{}$\textsf{R}$\end{document} package tramME utilizes the Laplace approximation and automatic differentiation to perform efficient maximum likelihood estimation and inference in mixed-effects transformation models. We compare several variants of our model to predict the survival of patients with chronic obstructive pulmonary disease using a large data set of prognostic studies. Finally, a simulation study is presented that verifies the correctness of the implementation and highlights its efficiency compared to an alternative approach.
Collapse
Affiliation(s)
- Bálint Tamási
- Institut für Epidemiologie, Biostatistik und Prävention, Departement Biostatistik, Universität Zürich, Hirschengraben 84, CH-8001 Zürich, Switzerland
| | - Michael Crowther
- Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, 171 77 Stockholm, Sweden
| | - Milo Alan Puhan
- Institut für Epidemiologie, Biostatistik und Prävention, Departement Epidemiologie, Universität Zürich, Hirschengraben 84, CH-8001 Zürich, Switzerland
| | - Ewout W Steyerberg
- Department of Biomedical Data Sciences, Leiden University Medical Center, Albinusdreef 2, 2333 ZA Leiden, the Netherlands
| | - Torsten Hothorn
- Institut für Epidemiologie, Biostatistik und Prävention, Departement Biostatistik, Universität Zürich, Hirschengraben 84, CH-8001 Zürich, Switzerland
| |
Collapse
|
13
|
Verburg E, van Gils CH, van der Velden BHM, Bakker MF, Pijnappel RM, Veldhuis WB, Gilhuijs KGA. Deep Learning for Automated Triaging of 4581 Breast MRI Examinations from the DENSE Trial. Radiology 2021; 302:29-36. [PMID: 34609196 DOI: 10.1148/radiol.2021203960] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
Background Supplemental screening with MRI has proved beneficial in women with extremely dense breasts. Most MRI examinations show normal anatomic and physiologic variation that may not require radiologic review. Thus, ways to triage these normal MRI examinations to reduce radiologist workload are needed. Purpose To determine the feasibility of an automated triaging method using deep learning (DL) to dismiss the highest number of MRI examinations without lesions while still identifying malignant disease. Materials and Methods This secondary analysis of data from the Dense Tissue and Early Breast Neoplasm Screening, or DENSE, trial evaluated breast MRI examinations from the first screening round performed in eight hospitals between December 2011 and January 2016. A DL model was developed to differentiate between breasts with lesions and breasts without lesions. The model was trained to dismiss breasts with normal phenotypical variation and to triage lesions (Breast Imaging Reporting and Data System [BI-RADS] categories 2-5) using eightfold internal-external validation. The model was trained on data from seven hospitals and tested on data from the eighth hospital, alternating such that each hospital was used once as an external test set. Performance was assessed using receiver operating characteristic analysis. At 100% sensitivity for malignant disease, the fraction of examinations dismissed from radiologic review was estimated. Results A total of 4581 MRI examinations of extremely dense breasts from 4581women (mean age, 54.3 years; interquartile range, 51.5-59.8 years) were included. Of the 9162 breasts, 838 had at least one lesion (BI-RADS category 2-5, of which 77 were malignant) and 8324 had no lesions. At 100% sensitivity for malignant lesions, the DL model considered 90.7% (95% CI: 86.7, 94.7) of the MRI examinations with lesions to be nonnormal and triaged them to radiologic review. The DL model dismissed 39.7% (95% CI: 30.0, 49.4) of the MRI examinations without lesions. The DL model had an average area under the receiver operating characteristic curve of 0.83 (95% CI: 0.80, 0.85) in the differentiation between normal breast MRI examinations and MRI examinations with lesions. Conclusion Automated analysis of breast MRI examinations in women with dense breasts dismissed nearly 40% of MRI scans without lesions while not missing any cancers. ClinicalTrials.gov: NCT01315015 © RSNA, 2021 Online supplemental material is available for this article. See also the editorial by Joe in this issue.
Collapse
Affiliation(s)
- Erik Verburg
- From the Image Sciences Institute (E.V., B.H.M.v.d.V., K.G.A.G.), Julius Center for Health Sciences and Primary Care (C.H.v.G., M.F.B.), and Department of Radiology (R.M.P., W.B.V.), University Medical Center Utrecht, Utrecht University, Utrecht 3584 CX, the Netherlands
| | - Carla H van Gils
- From the Image Sciences Institute (E.V., B.H.M.v.d.V., K.G.A.G.), Julius Center for Health Sciences and Primary Care (C.H.v.G., M.F.B.), and Department of Radiology (R.M.P., W.B.V.), University Medical Center Utrecht, Utrecht University, Utrecht 3584 CX, the Netherlands
| | - Bas H M van der Velden
- From the Image Sciences Institute (E.V., B.H.M.v.d.V., K.G.A.G.), Julius Center for Health Sciences and Primary Care (C.H.v.G., M.F.B.), and Department of Radiology (R.M.P., W.B.V.), University Medical Center Utrecht, Utrecht University, Utrecht 3584 CX, the Netherlands
| | - Marije F Bakker
- From the Image Sciences Institute (E.V., B.H.M.v.d.V., K.G.A.G.), Julius Center for Health Sciences and Primary Care (C.H.v.G., M.F.B.), and Department of Radiology (R.M.P., W.B.V.), University Medical Center Utrecht, Utrecht University, Utrecht 3584 CX, the Netherlands
| | - Ruud M Pijnappel
- From the Image Sciences Institute (E.V., B.H.M.v.d.V., K.G.A.G.), Julius Center for Health Sciences and Primary Care (C.H.v.G., M.F.B.), and Department of Radiology (R.M.P., W.B.V.), University Medical Center Utrecht, Utrecht University, Utrecht 3584 CX, the Netherlands
| | - Wouter B Veldhuis
- From the Image Sciences Institute (E.V., B.H.M.v.d.V., K.G.A.G.), Julius Center for Health Sciences and Primary Care (C.H.v.G., M.F.B.), and Department of Radiology (R.M.P., W.B.V.), University Medical Center Utrecht, Utrecht University, Utrecht 3584 CX, the Netherlands
| | - Kenneth G A Gilhuijs
- From the Image Sciences Institute (E.V., B.H.M.v.d.V., K.G.A.G.), Julius Center for Health Sciences and Primary Care (C.H.v.G., M.F.B.), and Department of Radiology (R.M.P., W.B.V.), University Medical Center Utrecht, Utrecht University, Utrecht 3584 CX, the Netherlands
| |
Collapse
|
14
|
Allotey J, Snell KI, Smuk M, Hooper R, Chan CL, Ahmed A, Chappell LC, von Dadelszen P, Dodds J, Green M, Kenny L, Khalil A, Khan KS, Mol BW, Myers J, Poston L, Thilaganathan B, Staff AC, Smith GC, Ganzevoort W, Laivuori H, Odibo AO, Ramírez JA, Kingdom J, Daskalakis G, Farrar D, Baschat AA, Seed PT, Prefumo F, da Silva Costa F, Groen H, Audibert F, Masse J, Skråstad RB, Salvesen KÅ, Haavaldsen C, Nagata C, Rumbold AR, Heinonen S, Askie LM, Smits LJ, Vinter CA, Magnus PM, Eero K, Villa PM, Jenum AK, Andersen LB, Norman JE, Ohkuchi A, Eskild A, Bhattacharya S, McAuliffe FM, Galindo A, Herraiz I, Carbillon L, Klipstein-Grobusch K, Yeo S, Teede HJ, Browne JL, Moons KG, Riley RD, Thangaratinam S. Validation and development of models using clinical, biochemical and ultrasound markers for predicting pre-eclampsia: an individual participant data meta-analysis. Health Technol Assess 2021; 24:1-252. [PMID: 33336645 DOI: 10.3310/hta24720] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022] Open
Abstract
BACKGROUND Pre-eclampsia is a leading cause of maternal and perinatal mortality and morbidity. Early identification of women at risk is needed to plan management. OBJECTIVES To assess the performance of existing pre-eclampsia prediction models and to develop and validate models for pre-eclampsia using individual participant data meta-analysis. We also estimated the prognostic value of individual markers. DESIGN This was an individual participant data meta-analysis of cohort studies. SETTING Source data from secondary and tertiary care. PREDICTORS We identified predictors from systematic reviews, and prioritised for importance in an international survey. PRIMARY OUTCOMES Early-onset (delivery at < 34 weeks' gestation), late-onset (delivery at ≥ 34 weeks' gestation) and any-onset pre-eclampsia. ANALYSIS We externally validated existing prediction models in UK cohorts and reported their performance in terms of discrimination and calibration. We developed and validated 12 new models based on clinical characteristics, clinical characteristics and biochemical markers, and clinical characteristics and ultrasound markers in the first and second trimesters. We summarised the data set-specific performance of each model using a random-effects meta-analysis. Discrimination was considered promising for C-statistics of ≥ 0.7, and calibration was considered good if the slope was near 1 and calibration-in-the-large was near 0. Heterogeneity was quantified using I 2 and τ2. A decision curve analysis was undertaken to determine the clinical utility (net benefit) of the models. We reported the unadjusted prognostic value of individual predictors for pre-eclampsia as odds ratios with 95% confidence and prediction intervals. RESULTS The International Prediction of Pregnancy Complications network comprised 78 studies (3,570,993 singleton pregnancies) identified from systematic reviews of tests to predict pre-eclampsia. Twenty-four of the 131 published prediction models could be validated in 11 UK cohorts. Summary C-statistics were between 0.6 and 0.7 for most models, and calibration was generally poor owing to large between-study heterogeneity, suggesting model overfitting. The clinical utility of the models varied between showing net harm to showing minimal or no net benefit. The average discrimination for IPPIC models ranged between 0.68 and 0.83. This was highest for the second-trimester clinical characteristics and biochemical markers model to predict early-onset pre-eclampsia, and lowest for the first-trimester clinical characteristics models to predict any pre-eclampsia. Calibration performance was heterogeneous across studies. Net benefit was observed for International Prediction of Pregnancy Complications first and second-trimester clinical characteristics and clinical characteristics and biochemical markers models predicting any pre-eclampsia, when validated in singleton nulliparous women managed in the UK NHS. History of hypertension, parity, smoking, mode of conception, placental growth factor and uterine artery pulsatility index had the strongest unadjusted associations with pre-eclampsia. LIMITATIONS Variations in study population characteristics, type of predictors reported, too few events in some validation cohorts and the type of measurements contributed to heterogeneity in performance of the International Prediction of Pregnancy Complications models. Some published models were not validated because model predictors were unavailable in the individual participant data. CONCLUSION For models that could be validated, predictive performance was generally poor across data sets. Although the International Prediction of Pregnancy Complications models show good predictive performance on average, and in the singleton nulliparous population, heterogeneity in calibration performance is likely across settings. FUTURE WORK Recalibration of model parameters within populations may improve calibration performance. Additional strong predictors need to be identified to improve model performance and consistency. Validation, including examination of calibration heterogeneity, is required for the models we could not validate. STUDY REGISTRATION This study is registered as PROSPERO CRD42015029349. FUNDING This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 24, No. 72. See the NIHR Journals Library website for further project information.
Collapse
|
15
|
Meid AD, Gonzalez-Gonzalez AI, Dinh TS, Blom J, van den Akker M, Elders P, Thiem U, Küllenberg de Gaudry D, Swart KMA, Rudolf H, Bosch-Lenders D, Trampisch HJ, Meerpohl JJ, Gerlach FM, Flaig B, Kom G, Snell KIE, Perera R, Haefeli WE, Glasziou P, Muth C. Predicting hospital admissions from individual patient data (IPD): an applied example to explore key elements driving external validity. BMJ Open 2021; 11:e045572. [PMID: 34348947 PMCID: PMC8340284 DOI: 10.1136/bmjopen-2020-045572] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
Abstract
OBJECTIVE To explore factors that potentially impact external validation performance while developing and validating a prognostic model for hospital admissions (HAs) in complex older general practice patients. STUDY DESIGN AND SETTING Using individual participant data from four cluster-randomised trials conducted in the Netherlands and Germany, we used logistic regression to develop a prognostic model to predict all-cause HAs within a 6-month follow-up period. A stratified intercept was used to account for heterogeneity in baseline risk between the studies. The model was validated both internally and by using internal-external cross-validation (IECV). RESULTS Prior HAs, physical components of the health-related quality of life comorbidity index, and medication-related variables were used in the final model. While achieving moderate discriminatory performance, internal bootstrap validation revealed a pronounced risk of overfitting. The results of the IECV, in which calibration was highly variable even after accounting for between-study heterogeneity, agreed with this finding. Heterogeneity was equally reflected in differing baseline risk, predictor effects and absolute risk predictions. CONCLUSIONS Predictor effect heterogeneity and differing baseline risk can explain the limited external performance of HA prediction models. With such drivers known, model adjustments in external validation settings (eg, intercept recalibration, complete updating) can be applied more purposefully. TRIAL REGISTRATION NUMBER PROSPERO id: CRD42018088129.
Collapse
Affiliation(s)
- Andreas Daniel Meid
- Department of Clinical Pharmacology & Pharmacoepidemiology, Heidelberg University, Heidelberg, Baden-Württemberg, Germany
| | - Ana Isabel Gonzalez-Gonzalez
- Institute of General Practice, Goethe University, Frankfurt am Main, Hessen, Germany
- Red de Investigación en Servicios de Salud en Enfermedades Crónicas (REDISSEC), Madrid, Spain
| | - Truc Sophia Dinh
- Institute of General Practice, Goethe University, Frankfurt am Main, Hessen, Germany
| | - Jeanet Blom
- Department of Public Health and Primary Care, Leiden University Medical Center, Leiden, The Netherlands
| | - Marjan van den Akker
- Institute of General Practice, Goethe University, Frankfurt am Main, Hessen, Germany
- School of CAPHRI, Department of Family Medicine, Maastricht University, Maastricht, The Netherlands
| | - Petra Elders
- Department of General Practice and Elderly Care Medicine, Amsterdam UMC, Vrije Universiteit, Amstedarm Public Health Research Institute, Amsterdam, The Netherlands
| | - Ulrich Thiem
- Chair of Geriatrics and Gerontology, University Clinic Eppendorf, Hamburg, Germany
| | - Daniela Küllenberg de Gaudry
- Institute for Evidence in Medicine (for Cochrane Germany Foundation), Medical Center-University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Karin M A Swart
- Department of General Practice and Elderly Care Medicine, Amsterdam UMC, Vrije Universiteit, Amstedarm Public Health Research Institute, Amsterdam, The Netherlands
| | - Henrik Rudolf
- Department of Medical Informatics, Biometry and Epidemiology, Ruhr University Bochum, Bochum, Nordrhein-Westfalen, Germany
| | - Donna Bosch-Lenders
- School of CAPHRI, Department of Family Medicine, Maastricht University, Maastricht, The Netherlands
| | - Hans J Trampisch
- Department of Medical Informatics, Biometry and Epidemiology, Ruhr University Bochum, Bochum, Nordrhein-Westfalen, Germany
| | - Joerg J Meerpohl
- Institute for Evidence in Medicine (for Cochrane Germany Foundation), Medical Center-University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Ferdinand M Gerlach
- Institute of General Practice, Goethe University, Frankfurt am Main, Hessen, Germany
| | - Benno Flaig
- Institute of General Practice, Goethe University, Frankfurt am Main, Hessen, Germany
| | | | - Kym I E Snell
- Centre for Prognosis Research, School of Primary Care Research, Community and Social Care, Keele University, Keele, UK
| | - Rafael Perera
- Nuffield Department of Primary Care, University of Oxford, Oxford, UK
| | - Walter Emil Haefeli
- Department of Clinical Pharmacology & Pharmacoepidemiology, Heidelberg University, Heidelberg, Baden-Württemberg, Germany
| | - Paul Glasziou
- Centre for Research in Evidence-Based Practice, Bond University, Robina, Queensland, Australia
| | - Christiane Muth
- Institute of General Practice, Goethe University, Frankfurt am Main, Hessen, Germany
- Department of General Practice and Family Medicine, Medical Faculty OWL, University of Bielefeld, Bielefeld, Germany
| |
Collapse
|
16
|
O'Neill T, Hudda MT, Patel R, Liu WK, Young AM, Patel HR, Afshar M. A new prognostic model for predicting 30-day mortality in acute oncology patients. Expert Rev Anticancer Ther 2021; 21:1171-1177. [PMID: 34325618 DOI: 10.1080/14737140.2021.1945446] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Abstract
INTRODUCTION Acute oncology services (AOS) provide rapid review and expedited pathways for referral to specialist care for cancer patients. Blood tests may support AOS in providing estimates of prognosis. We aimed to develop and validate a prognostic model of 30-day mortality based on routine blood markers to inform an AOS decision to actively treat or palliate patients. METHODS AND MATERIALS Using clinical data from 752 AOS referrals, multivariable logistic regression analysis was conducted to develop a 30-day mortality prognostic model. Internal validation and then internal-external cross-validation were used to examine overfitting and generalizability of the model's predictive performance. RESULTS Urea, alkaline phosphatase, albumin and neutrophils were the strongest predictors of outcome. The model separated patients into distinct prognostic groups from the cross-validation (C Statistic: 0.70; 95% CI: 0.64-0.76). Admission year was included as a predictor in the model to improve the model calibration. CONCLUSION The developed prediction model was able to classify patients into distinct prognostic risk groups, which is clinically useful for delivering an evidence-based AOS. Collation of data from other AOS centers would allow for the development of a more generalizable prognostic model.
Collapse
Affiliation(s)
- Tess O'Neill
- Department of Medicine, Barts and the London NHS Trust, London, London, UK
| | - Mohammed T Hudda
- St George's University of London, Population Health Research Institute, London, UK
| | - Reena Patel
- Department of Medicine, St George's University of London, London, UK
| | - Wing Kin Liu
- Department of Medicine, St George's University of London, London, UK
| | - Anna-Mary Young
- Department of Medicine, St George's University of London, London, UK
| | - Hitendra Rh Patel
- Department of Urology and Endocrine Surgery,University Hospital North Norway, Tromso, Troms Norway
| | - Mehran Afshar
- Department of Medicine, St George's University of London, London, UK
| |
Collapse
|
17
|
Moriarty AS, Paton LW, Snell KIE, Riley RD, Buckman JEJ, Gilbody S, Chew-Graham CA, Ali S, Pilling S, Meader N, Phillips B, Coventry PA, Delgadillo J, Richards DA, Salisbury C, McMillan D. The development and validation of a prognostic model to PREDICT Relapse of depression in adult patients in primary care: protocol for the PREDICTR study. Diagn Progn Res 2021; 5:12. [PMID: 34215317 PMCID: PMC8254312 DOI: 10.1186/s41512-021-00101-x] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/14/2021] [Accepted: 05/19/2021] [Indexed: 12/23/2022] Open
Abstract
BACKGROUND Most patients who present with depression are treated in primary care by general practitioners (GPs). Relapse of depression is common (at least 50% of patients treated for depression will relapse after a single episode) and leads to considerable morbidity and decreased quality of life for patients. The majority of patients will relapse within 6 months, and those with a history of relapse are more likely to relapse in the future than those with no such history. GPs see a largely undifferentiated case-mix of patients, and once patients with depression reach remission, there is limited guidance to help GPs stratify patients according to risk of relapse. We aim to develop a prognostic model to predict an individual's risk of relapse within 6-8 months of entering remission. The long-term objective is to inform the clinical management of depression after the acute phase. METHODS We will develop a prognostic model using secondary analysis of individual participant data drawn from seven RCTs and one longitudinal cohort study in primary or community care settings. We will use logistic regression to predict the outcome of relapse of depression within 6-8 months. We plan to include the following established relapse predictors in the model: residual depressive symptoms, number of previous depressive episodes, co-morbid anxiety and severity of index episode. We will use a "full model" development approach, including all available predictors. Performance statistics (optimism-adjusted C-statistic, calibration-in-the-large, calibration slope) and calibration plots (with smoothed calibration curves) will be calculated. Generalisability of predictive performance will be assessed through internal-external cross-validation. Clinical utility will be explored through net benefit analysis. DISCUSSION We will derive a statistical model to predict relapse of depression in remitted depressed patients in primary care. Assuming the model has sufficient predictive performance, we outline the next steps including independent external validation and further assessment of clinical utility and impact. STUDY REGISTRATION ClinicalTrials.gov ID: NCT04666662.
Collapse
Affiliation(s)
- Andrew S Moriarty
- Department of Health Sciences, University of York, York, England.
- Hull York Medical School, University of York, York, England.
| | - Lewis W Paton
- Department of Health Sciences, University of York, York, England
| | - Kym I E Snell
- Centre for Prognosis Research, School of Medicine, Keele University, Keele, England
| | - Richard D Riley
- Centre for Prognosis Research, School of Medicine, Keele University, Keele, England
| | - Joshua E J Buckman
- Centre for Outcomes and Research Effectiveness, Research Department of Clinical, Educational and Health Psychology, University College London, London, England
- iCope - Camden and Islington Psychological Therapies Services, Camden & Islington NHS Foundation Trust, London, England
| | - Simon Gilbody
- Department of Health Sciences, University of York, York, England
- Hull York Medical School, University of York, York, England
| | | | - Shehzad Ali
- Department of Health Sciences, University of York, York, England
- Department of Epidemiology and Biostatistics, Schulich School of Medicine & Dentistry, Western University, London, ON, Canada
| | - Stephen Pilling
- Centre for Outcomes and Research Effectiveness, Research Department of Clinical, Educational and Health Psychology, University College London, London, England
- Camden & Islington NHS Foundation Trust, St Pancras Hospital, London, England
| | - Nick Meader
- Centre for Reviews and Dissemination, University of York, York, England
| | - Bob Phillips
- Centre for Reviews and Dissemination, University of York, York, England
| | - Peter A Coventry
- Department of Health Sciences, University of York, York, England
| | - Jaime Delgadillo
- Department of Psychology, University of Sheffield, Sheffield, England
| | - David A Richards
- Institute of Health Research, College of Medicine and Health, University of Exeter, Exeter, England
- Department of Health and Caring Sciences, Western Norway University of Applied Sciences, Inndalsveien 28, 5063 Bergen, Norway, USA
| | - Chris Salisbury
- Centre for Academic Primary Care, University of Bristol, Bristol, England
| | - Dean McMillan
- Department of Health Sciences, University of York, York, England
- Hull York Medical School, University of York, York, England
| |
Collapse
|
18
|
de Jong VMT, Moons KGM, Eijkemans MJC, Riley RD, Debray TPA. Developing more generalizable prediction models from pooled studies and large clustered data sets. Stat Med 2021; 40:3533-3559. [PMID: 33948970 PMCID: PMC8252590 DOI: 10.1002/sim.8981] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2019] [Revised: 02/16/2021] [Accepted: 03/22/2021] [Indexed: 12/14/2022]
Abstract
Prediction models often yield inaccurate predictions for new individuals. Large data sets from pooled studies or electronic healthcare records may alleviate this with an increased sample size and variability in sample characteristics. However, existing strategies for prediction model development generally do not account for heterogeneity in predictor‐outcome associations between different settings and populations. This limits the generalizability of developed models (even from large, combined, clustered data sets) and necessitates local revisions. We aim to develop methodology for producing prediction models that require less tailoring to different settings and populations. We adopt internal‐external cross‐validation to assess and reduce heterogeneity in models' predictive performance during the development. We propose a predictor selection algorithm that optimizes the (weighted) average performance while minimizing its variability across the hold‐out clusters (or studies). Predictors are added iteratively until the estimated generalizability is optimized. We illustrate this by developing a model for predicting the risk of atrial fibrillation and updating an existing one for diagnosing deep vein thrombosis, using individual participant data from 20 cohorts (N = 10 873) and 11 diagnostic studies (N = 10 014), respectively. Meta‐analysis of calibration and discrimination performance in each hold‐out cluster shows that trade‐offs between average and heterogeneity of performance occurred. Our methodology enables the assessment of heterogeneity of prediction model performance during model development in multiple or clustered data sets, thereby informing researchers on predictor selection to improve the generalizability to different settings and populations, and reduce the need for model tailoring. Our methodology has been implemented in the R package metamisc.
Collapse
Affiliation(s)
- Valentijn M T de Jong
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands.,Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Karel G M Moons
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands.,Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Marinus J C Eijkemans
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Richard D Riley
- Centre for Prognosis Research, School of Medicine, Keele University, Staffordshire, UK
| | - Thomas P A Debray
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands.,Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| |
Collapse
|
19
|
Best JG, Ambler G, Wilson D, Lee KJ, Lim JS, Shiozawa M, Koga M, Li L, Lovelock C, Chabriat H, Hennerici M, Wong YK, Mak HKF, Prats-Sanchez L, Martínez-Domeño A, Inamura S, Yoshifuji K, Arsava EM, Horstmann S, Purrucker J, Lam BYK, Wong A, Kim YD, Song TJ, Lemmens R, Eppinger S, Gattringer T, Uysal E, Tanriverdi Z, Bornstein NM, Ben Assayag E, Hallevi H, Molad J, Nishihara M, Tanaka J, Coutts SB, Polymeris A, Wagner B, Seiffge DJ, Lyrer P, Algra A, Kappelle LJ, Al-Shahi Salman R, Jäger HR, Lip GYH, Fischer U, El-Koussy M, Mas JL, Legrand L, Karayiannis C, Phan T, Gunkel S, Christ N, Abrigo J, Leung T, Chu W, Chappell F, Makin S, Hayden D, Williams DJ, Mess WH, Nederkoorn PJ, Barbato C, Browning S, Wiegertjes K, Tuladhar AM, Maaijwee N, Guevarra AC, Yatawara C, Mendyk AM, Delmaire C, Köhler S, van Oostenbrugge R, Zhou Y, Xu C, Hilal S, Gyanwali B, Chen C, Lou M, Staals J, Bordet R, Kandiah N, de Leeuw FE, Simister R, Hendrikse J, Kelly PJ, Wardlaw J, Soo Y, Fluri F, Srikanth V, Calvet D, Jung S, Kwa VIH, Engelter ST, Peters N, Smith EE, Hara H, Yakushiji Y, Orken DN, Fazekas F, Thijs V, Heo JH, Mok V, Veltkamp R, Ay H, Imaizumi T, Gomez-Anson B, Lau KK, Jouvent E, Rothwell PM, Toyoda K, Bae HJ, Marti-Fabregas J, Werring DJ. Development of imaging-based risk scores for prediction of intracranial haemorrhage and ischaemic stroke in patients taking antithrombotic therapy after ischaemic stroke or transient ischaemic attack: a pooled analysis of individual patient data from cohort studies. Lancet Neurol 2021; 20:294-303. [PMID: 33743239 DOI: 10.1016/s1474-4422(21)00024-7] [Citation(s) in RCA: 33] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2020] [Revised: 12/16/2020] [Accepted: 01/08/2021] [Indexed: 12/23/2022]
Abstract
BACKGROUND Balancing the risks of recurrent ischaemic stroke and intracranial haemorrhage is important for patients treated with antithrombotic therapy after ischaemic stroke or transient ischaemic attack. However, existing predictive models offer insufficient performance, particularly for assessing the risk of intracranial haemorrhage. We aimed to develop new risk scores incorporating clinical variables and cerebral microbleeds, an MRI biomarker of intracranial haemorrhage and ischaemic stroke risk. METHODS We did a pooled analysis of individual-patient data from the Microbleeds International Collaborative Network (MICON), which includes 38 hospital-based prospective cohort studies from 18 countries. All studies recruited participants with previous ischaemic stroke or transient ischaemic attack, acquired baseline MRI allowing quantification of cerebral microbleeds, and followed-up participants for ischaemic stroke and intracranial haemorrhage. Participants not taking antithrombotic drugs were excluded. We developed Cox regression models to predict the 5-year risks of intracranial haemorrhage and ischaemic stroke, selecting candidate predictors on biological relevance and simplifying models using backward elimination. We derived integer risk scores for clinical use. We assessed model performance in internal validation, adjusted for optimism using bootstrapping. The study is registered on PROSPERO, CRD42016036602. FINDINGS The included studies recruited participants between Aug 28, 2001, and Feb 4, 2018. 15 766 participants had follow-up for intracranial haemorrhage, and 15 784 for ischaemic stroke. Over a median follow-up of 2 years, 184 intracranial haemorrhages and 1048 ischaemic strokes were reported. The risk models we developed included cerebral microbleed burden and simple clinical variables. Optimism-adjusted c indices were 0·73 (95% CI 0·69-0·77) with a calibration slope of 0·94 (0·81-1·06) for the intracranial haemorrhage model and 0·63 (0·62-0·65) with a calibration slope of 0·97 (0·87-1·07) for the ischaemic stroke model. There was good agreement between predicted and observed risk for both models. INTERPRETATION The MICON risk scores, incorporating clinical variables and cerebral microbleeds, offer predictive value for the long-term risks of intracranial haemorrhage and ischaemic stroke in patients prescribed antithrombotic therapy for secondary stroke prevention; external validation is warranted. FUNDING British Heart Foundation and Stroke Association.
Collapse
Affiliation(s)
- Jonathan G Best
- UCL Stroke Research Centre, Department of Brain Repair and Rehabilitation, University College London Queen Square Institute of Neurology, London, UK
| | - Gareth Ambler
- Department of Statistical Science, University College London, Gower Street, London, UK
| | - Duncan Wilson
- UCL Stroke Research Centre, Department of Brain Repair and Rehabilitation, University College London Queen Square Institute of Neurology, London, UK; New Zealand Brain Research Institute, Christchurch, New Zealand
| | - Keon-Joo Lee
- Department of Neurology, Seoul National University Bundang Hospital, Seoul National University College of Medicine, Seongnam, South Korea
| | - Jae-Sung Lim
- Department of Neurology, Hallym Neurological Institute, Hallym University College of Medicine, Hallym University Sacred Heart Hospital, Anyang, South Korea
| | - Masayuki Shiozawa
- Department of Cerebrovascular Medicine, National Cerebral and Cardiovascular Centre, Suita, Japan
| | - Masatoshi Koga
- Department of Cerebrovascular Medicine, National Cerebral and Cardiovascular Centre, Suita, Japan
| | - Linxin Li
- Wolfson Centre for Prevention of Stroke and Dementia, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | - Caroline Lovelock
- Wolfson Centre for Prevention of Stroke and Dementia, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | - Hugues Chabriat
- Assistance Publique - Hôpitaux de Paris, Lariboisière Hospital, Department of Neurology, Paris, France; Federation Hospitalo-Universitaire NeuroVasc, Université de Paris, Paris, France; INSERM U1141, Paris, France
| | - Michael Hennerici
- Department of Neurology, Universitätsmedizin Mannheim, University of Heidelberg, Mannheim, Germany
| | - Yuen Kwun Wong
- Division of Neurology, Department of Medicine, The University of Hong Kong, Hong Kong Special Administrative Region, China
| | - Henry Ka Fung Mak
- Department of Diagnostic Radiology, The University of Hong Kong, Hong Kong Special Administrative Region, China
| | - Luis Prats-Sanchez
- Department of Neurology, Hospital de la Santa Creu i Sant Pau, Biomedical Research Institute, Barcelona, Spain
| | - Alejandro Martínez-Domeño
- Department of Neurology, Hospital de la Santa Creu i Sant Pau, Biomedical Research Institute, Barcelona, Spain
| | - Shigeru Inamura
- Department of Neurosurgery, Kushiro City General Hospital, Kushiro, Japan
| | - Kazuhisa Yoshifuji
- Department of Neurosurgery, Kushiro City General Hospital, Kushiro, Japan
| | - Ethem Murat Arsava
- A A Martinos Center for Biomedial Imaging, Department of Neurology, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston MA, USA
| | - Solveig Horstmann
- Department of Neurology, Heidelberg University Hospital, Heidelberg, Germany
| | - Jan Purrucker
- Department of Neurology, Heidelberg University Hospital, Heidelberg, Germany
| | - Bonnie Yin Ka Lam
- Therese Pei Fong Chow Research Centre for Prevention of Dementia, Gerald Choa Neuroscience Centre, Lui Che Woo Institute of Innovative Medicine, Department of Medicine and Therapeutics, The Chinese University of Hong Kong, Hong Kong Special Administrative Region, China
| | - Adrian Wong
- Therese Pei Fong Chow Research Centre for Prevention of Dementia, Gerald Choa Neuroscience Centre, Lui Che Woo Institute of Innovative Medicine, Department of Medicine and Therapeutics, The Chinese University of Hong Kong, Hong Kong Special Administrative Region, China
| | - Young Dae Kim
- Department of Neurology, Yonsei University College of Medicine, Seoul, South Korea
| | - Tae-Jin Song
- Department of Neurology, Seoul Hospital, Ewha Womans University College of Medicine, Seoul, South Korea
| | - Robin Lemmens
- Experimental Neurology, Department of Neurosciences, Katholieke Universiteit Leuven, Leuven, Belgium; Vlaams Instituut voor Biotechnologie, Center for Brain & Disease Research; Department of Neurology, University Hospitals Leuven, Leuven, Belgium
| | | | | | - Ender Uysal
- Department of Radiology, Saglık Bilimleri University, Sisli Etfal Education and Research Hospital, Istanbul, Turkey
| | - Zeynep Tanriverdi
- Department of Neurology, İzmir Katip Çelebi University Atatürk Education and Research Hospital, İzmir Turkey
| | - Natan M Bornstein
- Department of Neurology, Tel-Aviv Sourasky Medical Center, Tel-Aviv, Israel; Sackler Faculty of Medicine, Tel-Aviv University, Tel-Aviv, Israel
| | - Einor Ben Assayag
- Department of Neurology, Tel-Aviv Sourasky Medical Center, Tel-Aviv, Israel; Sackler Faculty of Medicine, Tel-Aviv University, Tel-Aviv, Israel
| | - Hen Hallevi
- Department of Neurology, Tel-Aviv Sourasky Medical Center, Tel-Aviv, Israel; Sackler Faculty of Medicine, Tel-Aviv University, Tel-Aviv, Israel
| | - Jeremy Molad
- Department of Neurology, Tel-Aviv Sourasky Medical Center, Tel-Aviv, Israel; Sackler Faculty of Medicine, Tel-Aviv University, Tel-Aviv, Israel
| | - Masashi Nishihara
- Department of Radiology, Saga University Faculty of Medicine, Saga, Japan
| | - Jun Tanaka
- Division of Neurology, Department of Internal Medicine, Saga University Faculty of Medicine, Saga, Japan
| | - Shelagh B Coutts
- Calgary Stroke Program, Department of Clinical Neurosciences, Radiology and Community Health Sciences, Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada
| | - Alexandros Polymeris
- Department of Neurology and Stroke Centre, University Hospital Basel and University of Basel, Switzerland
| | - Benjamin Wagner
- Department of Neurology and Stroke Centre, University Hospital Basel and University of Basel, Switzerland
| | - David J Seiffge
- UCL Stroke Research Centre, Department of Brain Repair and Rehabilitation, University College London Queen Square Institute of Neurology, London, UK; Department of Neurology and Stroke Centre, University Hospital Basel and University of Basel, Switzerland; Department of Neurology, University Hospital Inselspital Bern, University of Bern, Bern, Switzerland
| | - Philippe Lyrer
- Department of Neurology and Stroke Centre, University Hospital Basel and University of Basel, Switzerland
| | - Ale Algra
- Julius Centre for Health Sciences and Primary Care, Utrecht University, Utrecht, The Netherlands; Department of Neurology and Neurosurgery, Utrecht University, Utrecht, The Netherlands
| | - L Jaap Kappelle
- Department of Neurology and Neurosurgery, Utrecht University, Utrecht, The Netherlands
| | - Rustam Al-Shahi Salman
- Centre for Clinical Brain Sciences, School of Clinical Sciences, University of Edinburgh, Edinburgh, UK
| | - Hans R Jäger
- Lysholm Department of Neuroradiology and the Neuroradiological Academic Unit, Department of Brain Repair and Rehabilitation, University College London Institute of Neurology and the National Hospital for Neurology and Neurosurgery, London, UK
| | - Gregory Y H Lip
- Liverpool Centre for Cardiovascular Science, University of Liverpool, Liverpool, UK; Liverpool Heart & Chest Hospital, Liverpool, UK; Aalborg Thrombosis Research Unit, Department of Clinical Medicine, Aalborg University, Aalborg, Denmark
| | - Urs Fischer
- Department of Neurology, University Hospital Inselspital Bern, University of Bern, Bern, Switzerland
| | - Marwan El-Koussy
- Department of Diagnostic and Interventional Neuroradiology, University Hospital Inselspital Bern, University of Bern, Bern, Switzerland
| | - Jean-Louis Mas
- Department of Neurology, Sainte-Anne Hospital, Institut de Psychiatrie et Neurosciences de Paris, INSERM, Université de Paris, Paris, France
| | - Laurence Legrand
- Department of Neuroradiology, Sainte-Anne Hospital, Institut de Psychiatrie et Neurosciences de Paris, INSERM, Université de Paris, Paris, France
| | | | - Thanh Phan
- Stroke and Ageing Research Group, School of Clinical Sciences at Monash Health, Monash University, Melbourne, Australia
| | - Sarah Gunkel
- Department of Neurology, University Hospital of Würzburg, Würzburg, Germany
| | - Nicolas Christ
- Department of Neurology, University Hospital of Würzburg, Würzburg, Germany
| | - Jill Abrigo
- Department of Imaging and Interventional Radiology, Prince of Wales Hospital, The Chinese University of Hong Kong, Special Administrative Region, China
| | - Thomas Leung
- Department of Medicine and Therapeutics, Prince of Wales Hospital, The Chinese University of Hong Kong, Special Administrative Region, China
| | - Winnie Chu
- Department of Imaging and Interventional Radiology, Prince of Wales Hospital, The Chinese University of Hong Kong, Special Administrative Region, China
| | - Francesca Chappell
- Centre for Clinical Brain Sciences, Edinburgh Imaging, University of Edinburgh, Edinburgh, UK; UK Dementia Research Institute, University of Edinburgh, Edinburgh, UK
| | - Stephen Makin
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, UK
| | - Derek Hayden
- The Neurovascular Research Unit and Health Research Board, Stroke Clinical Trials Network Ireland, University College Dublin, Dublin, Ireland; Department of Medical Gerontology, Trinity College Dublin, Dublin, Ireland
| | - David J Williams
- Department of Geriatric and Stroke Medicine, Royal College of Surgeons in Ireland, University of Medicine and Health Sciences Dublin, Ireland; Department of Geriatric and Stroke Medicine, Beaumont Hospital Dublin, Ireland
| | - Werner H Mess
- Department of Clinical Neurophysiology, Maastricht University Medical Centre, Maastricht, the Netherlands
| | - Paul J Nederkoorn
- Department of Neurology, Amsterdam University Medical Centres, Netherlands
| | - Carmen Barbato
- UCL Stroke Research Centre, Department of Brain Repair and Rehabilitation, University College London Queen Square Institute of Neurology, London, UK; Comprehensive Stroke Service, University College London Hospitals NHS Trust, London, UK
| | - Simone Browning
- UCL Stroke Research Centre, Department of Brain Repair and Rehabilitation, University College London Queen Square Institute of Neurology, London, UK; Comprehensive Stroke Service, University College London Hospitals NHS Trust, London, UK
| | - Kim Wiegertjes
- Department of Neurology, Donders Institute for Brain, Cognition and Behaviour, Donders Centre for Medical Neuroscience, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Anil M Tuladhar
- Department of Neurology, Donders Institute for Brain, Cognition and Behaviour, Donders Centre for Medical Neuroscience, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Noortje Maaijwee
- Department for Neurology and Neurorehabilitation, Neurocenter, Lucerne State Hospital, Lucerne, Switzerland
| | | | | | - Anne-Marie Mendyk
- Degenerative and vascular cognitive disorders, University of Lille, INSERM, Centre Hospitalier Universitaire de Lille, Lille, France
| | - Christine Delmaire
- Degenerative and vascular cognitive disorders, University of Lille, INSERM, Centre Hospitalier Universitaire de Lille, Lille, France; Department of Radiology, Fondation A de Rothschild, Paris, France
| | - Sebastian Köhler
- Department of Psychiatry and Neuropsychology, School for Mental Health and Neuroscience, Maastricht University Medical Centre, The Netherlands
| | - Robert van Oostenbrugge
- Department of Neurology, Cardiovascular Research Institute Maastricht, Maastricht University Medical Centre, The Netherlands
| | - Ying Zhou
- Department of Neurology, The 2nd affiliated Hospital of Zhejiang University, School of Medicine, Hangzhou, China
| | - Chao Xu
- Department of Neurology, The 2nd affiliated Hospital of Zhejiang University, School of Medicine, Hangzhou, China
| | - Saima Hilal
- Memory Aging & Cognition Centre, Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| | - Bibek Gyanwali
- Memory Aging & Cognition Centre, Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| | - Christopher Chen
- Memory Aging & Cognition Centre, Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| | - Min Lou
- Department of Neurology, The 2nd affiliated Hospital of Zhejiang University, School of Medicine, Hangzhou, China
| | - Julie Staals
- Department of Neurology, Cardiovascular Research Institute Maastricht, Maastricht University Medical Centre, The Netherlands
| | - Régis Bordet
- Degenerative and vascular cognitive disorders, University of Lille, INSERM, Centre Hospitalier Universitaire de Lille, Lille, France
| | | | - Frank-Erik de Leeuw
- Department of Neurology, Donders Institute for Brain, Cognition and Behaviour, Donders Centre for Medical Neuroscience, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Robert Simister
- UCL Stroke Research Centre, Department of Brain Repair and Rehabilitation, University College London Queen Square Institute of Neurology, London, UK; Comprehensive Stroke Service, University College London Hospitals NHS Trust, London, UK
| | - Jeroen Hendrikse
- Department of Radiology, University Medical Centre Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Peter J Kelly
- The Neurovascular Research Unit and Health Research Board, Stroke Clinical Trials Network Ireland, University College Dublin, Dublin, Ireland
| | - Joanna Wardlaw
- Centre for Clinical Brain Sciences, Edinburgh Imaging, University of Edinburgh, Edinburgh, UK; UK Dementia Research Institute, University of Edinburgh, Edinburgh, UK
| | - Yannie Soo
- Department of Medicine and Therapeutics, Prince of Wales Hospital, The Chinese University of Hong Kong, Special Administrative Region, China
| | - Felix Fluri
- Department of Neurology, University Hospital of Würzburg, Würzburg, Germany
| | - Velandai Srikanth
- Peninsula Clinical School, Peninsula Health, Monash University, Melbourne, Australia
| | - David Calvet
- Department of Neurology, Sainte-Anne Hospital, Institut de Psychiatrie et Neurosciences de Paris, INSERM, Université de Paris, Paris, France
| | - Simon Jung
- Department of Neurology, University Hospital Inselspital Bern, University of Bern, Bern, Switzerland
| | - Vincent I H Kwa
- Department of Neurology, Onze Lieve Vrouwe Gasthuis, Amsterdam, The Netherlands
| | - Stefan T Engelter
- Department of Neurology and Stroke Centre, University Hospital Basel and University of Basel, Switzerland; Neurology and Neurorehabilitation, Department of Geriatric Medicine FELIX PLATTER, University of Basel, Switzerland
| | - Nils Peters
- Department of Neurology and Stroke Centre, University Hospital Basel and University of Basel, Switzerland; Neurology and Neurorehabilitation, Department of Geriatric Medicine FELIX PLATTER, University of Basel, Switzerland
| | - Eric E Smith
- Calgary Stroke Program, Department of Clinical Neurosciences, Radiology and Community Health Sciences, Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada
| | - Hideo Hara
- Division of Neurology, Department of Internal Medicine, Saga University Faculty of Medicine, Saga, Japan
| | - Yusuke Yakushiji
- Division of Neurology, Department of Internal Medicine, Saga University Faculty of Medicine, Saga, Japan; Department of Neurology, Kansai Medical University, Osaka, Japan
| | | | - Franz Fazekas
- Department of Neurology, Medical University of Graz, Graz, Austria
| | - Vincent Thijs
- Stroke Division, Florey Institute of Neuroscience and Mental Health, University of Melbourne, Heidelberg, VIC, Australia; Department of Neurology, Austin Health, Heidelberg, VIC, Australia
| | - Ji Hoe Heo
- Department of Neurology, Yonsei University College of Medicine, Seoul, South Korea
| | - Vincent Mok
- Therese Pei Fong Chow Research Centre for Prevention of Dementia, Gerald Choa Neuroscience Centre, Lui Che Woo Institute of Innovative Medicine, Department of Medicine and Therapeutics, The Chinese University of Hong Kong, Hong Kong Special Administrative Region, China
| | - Roland Veltkamp
- Department of Neurology, Heidelberg University Hospital, Heidelberg, Germany; Department of Brain Sciences, Imperial College London, London, UK
| | - Hakan Ay
- A A Martinos Center for Biomedial Imaging, Department of Neurology, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston MA, USA; Takeda, Cambridge, MA, USA
| | - Toshio Imaizumi
- Department of Neurosurgery, Kushiro City General Hospital, Kushiro, Japan
| | - Beatriz Gomez-Anson
- Unit of Neuroradiology, Hospital de la Santa Creu i Sant Pau, Biomedical Research Institute, Barcelona, Spain
| | - Kui Kai Lau
- Division of Neurology, Department of Medicine, The University of Hong Kong, Hong Kong Special Administrative Region, China
| | - Eric Jouvent
- Assistance Publique - Hôpitaux de Paris, Lariboisière Hospital, Department of Neurology, Paris, France; Federation Hospitalo-Universitaire NeuroVasc, Université de Paris, Paris, France; INSERM U1141, Paris, France
| | - Peter M Rothwell
- Wolfson Centre for Prevention of Stroke and Dementia, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | - Kazunori Toyoda
- Department of Cerebrovascular Medicine, National Cerebral and Cardiovascular Centre, Suita, Japan
| | - Hee-Joon Bae
- Department of Neurology, Seoul National University Bundang Hospital, Seoul National University College of Medicine, Seongnam, South Korea
| | - Joan Marti-Fabregas
- Department of Neurology, Hospital de la Santa Creu i Sant Pau, Biomedical Research Institute, Barcelona, Spain
| | - David J Werring
- UCL Stroke Research Centre, Department of Brain Repair and Rehabilitation, University College London Queen Square Institute of Neurology, London, UK.
| |
Collapse
|
20
|
Ensor J, Snell KIE, Debray TPA, Lambert PC, Look MP, Mamas MA, Moons KGM, Riley RD. Individual participant data meta-analysis for external validation, recalibration, and updating of a flexible parametric prognostic model. Stat Med 2021; 40:3066-3084. [PMID: 33768582 DOI: 10.1002/sim.8959] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2019] [Revised: 03/04/2021] [Accepted: 03/05/2021] [Indexed: 12/14/2022]
Abstract
Individual participant data (IPD) from multiple sources allows external validation of a prognostic model across multiple populations. Often this reveals poor calibration, potentially causing poor predictive performance in some populations. However, rather than discarding the model outright, it may be possible to modify the model to improve performance using recalibration techniques. We use IPD meta-analysis to identify the simplest method to achieve good model performance. We examine four options for recalibrating an existing time-to-event model across multiple populations: (i) shifting the baseline hazard by a constant, (ii) re-estimating the shape of the baseline hazard, (iii) adjusting the prognostic index as a whole, and (iv) adjusting individual predictor effects. For each strategy, IPD meta-analysis examines (heterogeneity in) model performance across populations. Additionally, the probability of achieving good performance in a new population can be calculated allowing ranking of recalibration methods. In an applied example, IPD meta-analysis reveals that the existing model had poor calibration in some populations, and large heterogeneity across populations. However, re-estimation of the intercept substantially improved the expected calibration in new populations, and reduced between-population heterogeneity. Comparing recalibration strategies showed that re-estimating both the magnitude and shape of the baseline hazard gave the highest predicted probability of good performance in a new population. In conclusion, IPD meta-analysis allows a prognostic model to be externally validated in multiple settings, and enables recalibration strategies to be compared and ranked to decide on the least aggressive recalibration strategy to achieve acceptable external model performance without discarding existing model information.
Collapse
Affiliation(s)
- Joie Ensor
- Centre for Prognosis Research, School of Medicine, Keele University, Keele, UK
| | - Kym I E Snell
- Centre for Prognosis Research, School of Medicine, Keele University, Keele, UK
| | - Thomas P A Debray
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands.,Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Paul C Lambert
- Biostatistics Research Group, Department of Health Sciences, University of Leicester, Centre for Medicine, Leicester, UK.,Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden
| | - Maxime P Look
- Department of Medical Oncology, Erasmus MC Cancer Institute, Erasmus University Medical Center, Rotterdam, The Netherlands
| | - Mamas A Mamas
- Centre for Prognosis Research, School of Medicine, Keele University, Keele, UK.,Department of Cardiology, Royal Stoke University Hospital, Stoke-on-Trent, UK
| | - Karel G M Moons
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands.,Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Richard D Riley
- Centre for Prognosis Research, School of Medicine, Keele University, Keele, UK
| |
Collapse
|
21
|
Chen Y, Voors AA, Jaarsma T, Lang CC, Sama IE, Akkerhuis KM, Boersma E, Hillege HL, Postmus D. A heart failure phenotype stratified model for predicting 1-year mortality in patients admitted with acute heart failure: results from an individual participant data meta-analysis of four prospective European cohorts. BMC Med 2021; 19:21. [PMID: 33499866 PMCID: PMC7839199 DOI: 10.1186/s12916-020-01894-2] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/11/2020] [Accepted: 12/21/2020] [Indexed: 12/23/2022] Open
Abstract
BACKGROUND Prognostic models developed in general cohorts with a mixture of heart failure (HF) phenotypes, though more widely applicable, are also likely to yield larger prediction errors in settings where the HF phenotypes have substantially different baseline mortality rates or different predictor-outcome associations. This study sought to use individual participant data meta-analysis to develop an HF phenotype stratified model for predicting 1-year mortality in patients admitted with acute HF. METHODS Four prospective European cohorts were used to develop an HF phenotype stratified model. Cox model with two rounds of backward elimination was used to derive the prognostic index. Weibull model was used to obtain the baseline hazard functions. The internal-external cross-validation (IECV) approach was used to evaluate the generalizability of the developed model in terms of discrimination and calibration. RESULTS 3577 acute HF patients were included, of which 2368 were classified as having HF with reduced ejection fraction (EF) (HFrEF; EF < 40%), 588 as having HF with midrange EF (HFmrEF; EF 40-49%), and 621 as having HF with preserved EF (HFpEF; EF ≥ 50%). A total of 11 readily available variables built up the prognostic index. For four of these predictor variables, namely systolic blood pressure, serum creatinine, myocardial infarction, and diabetes, the effect differed across the three HF phenotypes. With a weighted IECV-adjusted AUC of 0.79 (0.74-0.83) for HFrEF, 0.74 (0.70-0.79) for HFmrEF, and 0.74 (0.71-0.77) for HFpEF, the model showed excellent discrimination. Moreover, there was a good agreement between the average observed and predicted 1-year mortality risks, especially after recalibration of the baseline mortality risks. CONCLUSIONS Our HF phenotype stratified model showed excellent generalizability across four European cohorts and may provide a useful tool in HF phenotype-specific clinical decision-making.
Collapse
Affiliation(s)
- Yuntao Chen
- Department of Epidemiology, University of Groningen, University Medical Center Groningen, Hanzeplein 1, P.O. Box 30.001, 9700 RB, Groningen, the Netherlands.
| | - Adriaan A Voors
- Department of Cardiology, University of Groningen, University Medical Center Groningen, Groningen, the Netherlands
| | - Tiny Jaarsma
- Department of Social and Welfare Studies, Faculty of Health Sciences, Linköping University, Linköping, Sweden
| | - Chim C Lang
- Division of Molecular and Clinical Medicine, University of Dundee, Ninewells Hospital and Medical School, Dundee, UK
| | - Iziah E Sama
- Department of Cardiology, University of Groningen, University Medical Center Groningen, Groningen, the Netherlands
| | - K Martijn Akkerhuis
- Department of Cardiology, Thoraxcenter, Erasmus Medical Centre, Rotterdam, the Netherlands
| | - Eric Boersma
- Department of Cardiology, Thoraxcenter, Erasmus Medical Centre, Rotterdam, the Netherlands
| | - Hans L Hillege
- Department of Epidemiology, University of Groningen, University Medical Center Groningen, Hanzeplein 1, P.O. Box 30.001, 9700 RB, Groningen, the Netherlands
| | - Douwe Postmus
- Department of Epidemiology, University of Groningen, University Medical Center Groningen, Hanzeplein 1, P.O. Box 30.001, 9700 RB, Groningen, the Netherlands
| |
Collapse
|
22
|
Quinn J, Chung S, Murchland A, Casazza G, Costantino G, Solbiati M, Furlan R. Association Between US Physician Malpractice Claims Rates and Hospital Admission Rates Among Patients With Lower-Risk Syncope. JAMA Netw Open 2020; 3:e2025860. [PMID: 33320263 PMCID: PMC7739124 DOI: 10.1001/jamanetworkopen.2020.25860] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/04/2023] Open
Abstract
IMPORTANCE The US Government Accountability Office has changed its estimate of the annual costs of defensive medicine, largely because it has been difficult to objectively measure its impact. Evaluating the association of malpractice claims rates with hospital admission rates and the costs of admitting patients with low-risk conditions would help to document the impact of defensive medicine. Although syncope is a concerning symptom, most patients with syncope have a low risk of adverse outcomes. However, many low-risk patients are still admitted to the hospital, with associated costs of more than $2.5 billion per year in the US. OBJECTIVE To assess whether hospital admission rates after emergency department visits among patients with lower-risk syncope are associated with state-level variations in malpractice claims rates. DESIGN, SETTING, AND PARTICIPANTS This cross-sectional study of emergency department visits among patients with lower-risk syncope used deidentified data from the Clinformatics Data Mart database (Optum). Lower-risk syncope visits were defined as those with a primary diagnosis of syncope and collapse based on International Classification of Diseases, Ninth Revision, Clinical Modification code 780.2 or International Classification of Diseases, Tenth Revision, Clinical Modification code R55 that did not include another major diagnostic code for a condition requiring hospital admission (such as heart disease, cancer, or medical shock) or an inpatient hospital stay of more than 3 days. These data were linked to publicly available data from the National Practitioner Data Bank pertaining to physician malpractice claims between January 1, 2008, and December 31, 2017. The 2 data sets were linked at the state-year level. Data were analyzed from October 2, 2019, to September 12, 2020. MAIN OUTCOMES AND MEASURES The association between the rate of hospital admission after emergency department visits among patients with lower-risk syncope and the rate of physician malpractice claims was assessed at the state-year level using a state-level fixed-effects model. Standardized costs obtained from the Clinformatics Data Mart database were adjusted for inflation and expressed in 2017 US dollars using the Consumer Price Index. RESULTS Among 40 482 813 emergency department visits between 2008 and 2017, 519 724 visits (1.3%) were associated with syncope. Of those, 234 750 visits (45.2%) met the criteria for lower-risk syncope. The mean (SD) age of patients in the lower-risk cohort was 71.8 (13.5) years; 141 050 patients (60.1%) were female, and 44 115 patients (18.8%) were admitted to the hospital, representing an extra cost of $6542 per admission. The mean rate of physician malpractice claims varied from 0.27 claims per 100 000 people to 8.63 claims per 100 000 people across states and across years within states. A state-level fixed-effects regression model indicated that, for every 1 in 100 000-person increase in the physician malpractice claims rate, there was an absolute increase of 6.70% (95% CI, 4.65%-8.75%) or a relative increase of 35.6% in the hospital admission rate, which represented an additional $102 million in costs associated with this lower-risk cohort. CONCLUSIONS AND RELEVANCE In this study, increases in physician malpractice claims rates were associated with increases in hospital admission rates and substantial health care costs for patients with lower-risk syncope, and these increases are likely associated with the practice of defensive medicine.
Collapse
Affiliation(s)
- James Quinn
- Department of Emergency Medicine, Stanford University, Stanford, California
| | - Sukyung Chung
- Stanford University School of Medicine, Stanford, California
| | | | - Giovanni Casazza
- Dipartimento di Scienze Biomedichee Cliniche “L. Sacco,” Universita' degli Studi di Milano, Milano, Italy
| | - Giorgio Costantino
- Fondazione IRCCS Ca' Granda, Ospedale Maggiore Policlinico, Milano, Italy
| | - Monica Solbiati
- Fondazione IRCCS Ca' Granda, Ospedale Maggiore Policlinico, Milano, Italy
| | - Rafaello Furlan
- Department of Internal Medicine, Humanitas University, Rozzano, Italy
| |
Collapse
|
23
|
A prognostic model predicted deterioration in health-related quality of life in older patients with multimorbidity and polypharmacy. J Clin Epidemiol 2020; 130:1-12. [PMID: 33065164 DOI: 10.1016/j.jclinepi.2020.10.006] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2020] [Revised: 08/12/2020] [Accepted: 10/07/2020] [Indexed: 01/07/2023]
Abstract
OBJECTIVES To develop and validate a prognostic model to predict deterioration in health-related quality of life (dHRQoL) in older general practice patients with at least one chronic condition and one chronic prescription. STUDY DESIGN AND SETTING We used individual participant data from five cluster-randomized trials conducted in the Netherlands and Germany to predict dHRQoL, defined as a decrease in EQ-5D-3 L index score of ≥5% after 6-month follow-up in logistic regression models with stratified intercepts to account for between-study heterogeneity. The model was validated internally and by using internal-external cross-validation (IECV). RESULTS In 3,582 patients with complete data, of whom 1,046 (29.2%) showed deterioration in HRQoL, and 12/87 variables were selected that were related to single (chronic) conditions, inappropriate medication, medication underuse, functional status, well-being, and HRQoL. Bootstrap internal validation showed a C-statistic of 0.71 (0.69 to 0.72) and a calibration slope of 0.88 (0.78 to 0.98). In the IECV loop, the model provided a pooled C-statistic of 0.68 (0.65 to 0.70) and calibration-in-the-large of 0 (-0.13 to 0.13). HRQoL/functionality had the strongest prognostic value. CONCLUSION The model performed well in terms of discrimination, calibration, and generalizability and might help clinicians identify older patients at high risk of dHRQoL. REGISTRATION PROSPERO ID: CRD42018088129.
Collapse
|
24
|
Furukawa TA, Debray TPA, Akechi T, Yamada M, Kato T, Seo M, Efthimiou O. Can personalized treatment prediction improve the outcomes, compared with the group average approach, in a randomized trial? Developing and validating a multivariable prediction model in a pragmatic megatrial of acute treatment for major depression. J Affect Disord 2020; 274:690-697. [PMID: 32664003 DOI: 10.1016/j.jad.2020.05.141] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/10/2020] [Revised: 03/25/2020] [Accepted: 05/26/2020] [Indexed: 02/09/2023]
Abstract
BACKGROUND Clinical trials have traditionally been analysed at the aggregate level, assuming that the group average would be applicable to all eligible and similar patients. We re-analyzed a mega-trial of antidepressant therapy for major depression to explore whether a multivariable prediction model may lead to different treatment recommendations for individual participants. METHODS The trial compared the second-line treatment strategies of continuing sertraline, combining it with mirtazapine or switching to mirtazapine after initial failure to remit on sertraline among 1,544 patients with major depression. The outcome was the Personal Health Questionnaire-9 (PHQ-9) at week 9: the original analyses showed that both combining and switching resulted in greater reduction in PHQ-9 by 1.0 point than continuing. We considered several models of penalized regression or machine learning. RESULTS Models using support vector machines (SVMs) provided the best performance. Using SVMs, continuing sertraline was predicted to be the best treatment for 123 patients, combining for 696 patients, and switching for 725 patients. In the last two subgroups, both combining and switching were equally superior to continuing by 1.2 to 1.4 points, resulting in the same treatment recommendations as with the original aggregate data level analyses; in the first subgroup, however, switching was substantively inferior to combining (-3.1, 95%CI: -5.4 to -0.5). LIMITATIONS Stronger predictors are needed to make more precise predictions. CONCLUSIONS The multivariable prediction models led to improved recommendations for a minority of participants than the group average approach in a megatrial.
Collapse
Affiliation(s)
- Toshi A Furukawa
- Departments of Health Promotion and Human Behavior and of Clinical Epidemiology, Kyoto University Graduate School of Medicine / School of Public Health, Kyoto, Japan.
| | - Thomas P A Debray
- Julius Center for Health Sciences and Primary Care, UMC Utrecht, Utrecht University, The Netherlands.
| | - Tatsuo Akechi
- Department of Psychiatry and Cognitive-Behavioral Medicine, Nagoya City University Graduate School of Medical Sciences, Nagoya, Japan.
| | - Mitsuhiko Yamada
- Department of Neuropsychopharmacology, National Institute of Mental Health, National Center of Neurology and Psychiatry, Tokyo, Japan.
| | | | - Michael Seo
- Institute of Social and Preventive Medicine (ISPM), University of Bern, Bern, Switzerland.
| | - Orestis Efthimiou
- Institute of Social and Preventive Medicine, University of Bern, Switzerland.
| |
Collapse
|
25
|
Prelaj A, Lo Russo G, Proto C, Signorelli D, Ferrara R, Galli G, De Toma A, Randon G, Pagani F, Trevisan B, Ganzinelli M, Zilembo N, Montrone M, Longo V, Pesola F, Pizzutilo P, Del Bene G, Varesano N, Galetta D, Torri V, Garassino MC, Di Maio M, Catino A. DiM: Prognostic Score for Second- or Further-line Immunotherapy in Advanced Non–Small-Cell Lung Cancer: An External Validation. Clin Lung Cancer 2020; 21:e337-e348. [DOI: 10.1016/j.cllc.2020.01.005] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2019] [Revised: 12/21/2019] [Accepted: 01/20/2020] [Indexed: 11/16/2022]
|
26
|
Priestap F, Kao R, Martin CM. External validation of a prognostic model for intensive care unit mortality: a retrospective study using the Ontario Critical Care Information System. Can J Anaesth 2020; 67:981-991. [PMID: 32383124 PMCID: PMC7223438 DOI: 10.1007/s12630-020-01686-5] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2019] [Revised: 02/21/2020] [Accepted: 03/05/2020] [Indexed: 12/23/2022] Open
Abstract
PURPOSE To externally validate an intensive care unit (ICU) mortality prediction model that was created using the Ontario Critical Care Information System (CCIS), which includes the Multiple Organ Dysfunction Score (MODS). METHODS We applied the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) recommendations to a prospective longitudinal cohort of patients discharged between 1 July 2015 and 31 December 31 2016 from 90 adult level-3 critical care units in Ontario. We used multivariable logistic regression with measures of discrimination, calibration-in-the-large, calibration slope, and flexible calibration plots to compare prediction model performance of the entire data set and for each ICU subtype. RESULTS Among 121,201 CCIS records with ICU mortality of 11.3%, the C-statistic for the validation data set was 0.805. The C-statistic ranged from 0.775 to 0.846 among the ICU subtypes. After intercept recalibration to adjust the baseline risk, the mean predicted risk of death matched actual ICU mortality. The calibration slope was close to 1 with all CCIS data and ICU subtypes of cardiovascular and community hospitals with low ventilation rates. Calibration slopes significantly less than 1 were found for ICUs in teaching hospitals and community hospitals with high ventilation rates whereas coronary care units had a calibration slope significantly higher than 1. Calibration plots revealed over-prediction in high risk groups to a varying degree across all cohorts. CONCLUSIONS A risk prediction model primarily based on the MODS shows reproducibility and transportability after intercept recalibration. Risk adjusting models that use existing and feasible data collection can support performance measurement at the individual ICU level.
Collapse
Affiliation(s)
- Fran Priestap
- London Health Sciences Centre - Victoria Hospital, 800 Commissioner's Rd E, London, ON, Canada, N6A 5W9.
| | - Raymond Kao
- London Health Sciences Centre - Victoria Hospital, 800 Commissioner's Rd E, London, ON, Canada, N6A 5W9
- Division of Critical Care, Department of Medicine, Schulich School of Dentistry and Medicine, Western University, London, ON, Canada
| | - Claudio M Martin
- London Health Sciences Centre - Victoria Hospital, 800 Commissioner's Rd E, London, ON, Canada, N6A 5W9
- Division of Critical Care, Department of Medicine, Schulich School of Dentistry and Medicine, Western University, London, ON, Canada
- Lawson Health Research Institute, London, ON, Canada
| |
Collapse
|
27
|
Gravesteijn BY, Nieboer D, Ercole A, Lingsma HF, Nelson D, van Calster B, Steyerberg EW. Machine learning algorithms performed no better than regression models for prognostication in traumatic brain injury. J Clin Epidemiol 2020; 122:95-107. [PMID: 32201256 DOI: 10.1016/j.jclinepi.2020.03.005] [Citation(s) in RCA: 98] [Impact Index Per Article: 24.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2019] [Revised: 02/04/2020] [Accepted: 03/09/2020] [Indexed: 12/23/2022]
Abstract
OBJECTIVE We aimed to explore the added value of common machine learning (ML) algorithms for prediction of outcome for moderate and severe traumatic brain injury. STUDY DESIGN AND SETTING We performed logistic regression (LR), lasso regression, and ridge regression with key baseline predictors in the IMPACT-II database (15 studies, n = 11,022). ML algorithms included support vector machines, random forests, gradient boosting machines, and artificial neural networks and were trained using the same predictors. To assess generalizability of predictions, we performed internal, internal-external, and external validation on the recent CENTER-TBI study (patients with Glasgow Coma Scale <13, n = 1,554). Both calibration (calibration slope/intercept) and discrimination (area under the curve) was quantified. RESULTS In the IMPACT-II database, 3,332/11,022 (30%) died and 5,233(48%) had unfavorable outcome (Glasgow Outcome Scale less than 4). In the CENTER-TBI study, 348/1,554(29%) died and 651(54%) had unfavorable outcome. Discrimination and calibration varied widely between the studies and less so between the studied algorithms. The mean area under the curve was 0.82 for mortality and 0.77 for unfavorable outcomes in the CENTER-TBI study. CONCLUSION ML algorithms may not outperform traditional regression approaches in a low-dimensional setting for outcome prediction after moderate or severe traumatic brain injury. Similar to regression-based prediction models, ML algorithms should be rigorously validated to ensure applicability to new populations.
Collapse
Affiliation(s)
- Benjamin Y Gravesteijn
- Departments of Public Health, Erasmus MC - University Medical Centre Rotterdam, Postbus 2040, 3000 CA, Rotterdam, the Netherlands.
| | - Daan Nieboer
- Departments of Public Health, Erasmus MC - University Medical Centre Rotterdam, Rotterdam, the Netherlands
| | - Ari Ercole
- Division of Anaesthesia, University of Cambridge, Cambridge, United Kingdom
| | - Hester F Lingsma
- Departments of Public Health, Erasmus MC - University Medical Centre Rotterdam, Rotterdam, the Netherlands
| | - David Nelson
- Department of Physiology and Pharmacology, Section of Perioperative Medicine and Intensive Care, Karolinska Institutet, Stockholm, Sweden
| | - Ben van Calster
- Department of Development and Regeneration, KU Leuven, Belgium; Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, the Netherlands
| | - Ewout W Steyerberg
- Departments of Public Health, Erasmus MC - University Medical Centre Rotterdam, Rotterdam, the Netherlands; Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, the Netherlands
| | | |
Collapse
|
28
|
Abstract
Osteoarthritis (OA) is an extremely common musculoskeletal disease. However, current guidelines are not well suited for diagnosing patients in the early stages of disease and do not discriminate patients for whom the disease might progress rapidly. The most important hurdle in OA management is identifying and classifying patients who will benefit most from treatment. Further efforts are needed in patient subgrouping and developing prediction models. Conventional statistical modelling approaches exist; however, these models are limited in the amount of information they can adequately process. Comprehensive patient-specific prediction models need to be developed. Approaches such as data mining and machine learning should aid in the development of such models. Although a challenging task, technology is now available that should enable subgrouping of patients with OA and lead to improved clinical decision-making and precision medicine.
Collapse
|
29
|
Kaptoge S, Pennells L, De Bacquer D, Cooney MT, Kavousi M, Stevens G, Riley LM, Savin S, Khan T, Altay S, Amouyel P, Assmann G, Bell S, Ben-Shlomo Y, Berkman L, Beulens JW, Björkelund C, Blaha M, Blazer DG, Bolton T, Bonita Beaglehole R, Brenner H, Brunner EJ, Casiglia E, Chamnan P, Choi YH, Chowdry R, Coady S, Crespo CJ, Cushman M, Dagenais GR, D'Agostino Sr RB, Daimon M, Davidson KW, Engström G, Ford I, Gallacher J, Gansevoort RT, Gaziano TA, Giampaoli S, Grandits G, Grimsgaard S, Grobbee DE, Gudnason V, Guo Q, Tolonen H, Humphries S, Iso H, Jukema JW, Kauhanen J, Kengne AP, Khalili D, Koenig W, Kromhout D, Krumholz H, Lam TH, Laughlin G, Marín Ibañez A, Meade TW, Moons KGM, Nietert PJ, Ninomiya T, Nordestgaard BG, O'Donnell C, Palmieri L, Patel A, Perel P, Price JF, Providencia R, Ridker PM, Rodriguez B, Rosengren A, Roussel R, Sakurai M, Salomaa V, Sato S, Schöttker B, Shara N, Shaw JE, Shin HC, Simons LA, Sofianopoulou E, Sundström J, Völzke H, Wallace RB, Wareham NJ, Willeit P, Wood D, Wood A, Zhao D, Woodward M, Danaei G, Roth G, Mendis S, Onuma O, Varghese C, Ezzati M, Graham I, Jackson R, Danesh J, Di Angelantonio E. World Health Organization cardiovascular disease risk charts: revised models to estimate risk in 21 global regions. Lancet Glob Health 2019; 7:e1332-e1345. [PMID: 31488387 PMCID: PMC7025029 DOI: 10.1016/s2214-109x(19)30318-3] [Citation(s) in RCA: 495] [Impact Index Per Article: 99.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2019] [Revised: 05/16/2019] [Accepted: 07/10/2019] [Indexed: 01/31/2023]
Abstract
BACKGROUND To help adapt cardiovascular disease risk prediction approaches to low-income and middle-income countries, WHO has convened an effort to develop, evaluate, and illustrate revised risk models. Here, we report the derivation, validation, and illustration of the revised WHO cardiovascular disease risk prediction charts that have been adapted to the circumstances of 21 global regions. METHODS In this model revision initiative, we derived 10-year risk prediction models for fatal and non-fatal cardiovascular disease (ie, myocardial infarction and stroke) using individual participant data from the Emerging Risk Factors Collaboration. Models included information on age, smoking status, systolic blood pressure, history of diabetes, and total cholesterol. For derivation, we included participants aged 40-80 years without a known baseline history of cardiovascular disease, who were followed up until the first myocardial infarction, fatal coronary heart disease, or stroke event. We recalibrated models using age-specific and sex-specific incidences and risk factor values available from 21 global regions. For external validation, we analysed individual participant data from studies distinct from those used in model derivation. We illustrated models by analysing data on a further 123 743 individuals from surveys in 79 countries collected with the WHO STEPwise Approach to Surveillance. FINDINGS Our risk model derivation involved 376 177 individuals from 85 cohorts, and 19 333 incident cardiovascular events recorded during 10 years of follow-up. The derived risk prediction models discriminated well in external validation cohorts (19 cohorts, 1 096 061 individuals, 25 950 cardiovascular disease events), with Harrell's C indices ranging from 0·685 (95% CI 0·629-0·741) to 0·833 (0·783-0·882). For a given risk factor profile, we found substantial variation across global regions in the estimated 10-year predicted risk. For example, estimated cardiovascular disease risk for a 60-year-old male smoker without diabetes and with systolic blood pressure of 140 mm Hg and total cholesterol of 5 mmol/L ranged from 11% in Andean Latin America to 30% in central Asia. When applied to data from 79 countries (mostly low-income and middle-income countries), the proportion of individuals aged 40-64 years estimated to be at greater than 20% risk ranged from less than 1% in Uganda to more than 16% in Egypt. INTERPRETATION We have derived, calibrated, and validated new WHO risk prediction models to estimate cardiovascular disease risk in 21 Global Burden of Disease regions. The widespread use of these models could enhance the accuracy, practicability, and sustainability of efforts to reduce the burden of cardiovascular disease worldwide. FUNDING World Health Organization, British Heart Foundation (BHF), BHF Cambridge Centre for Research Excellence, UK Medical Research Council, and National Institute for Health Research.
Collapse
|
30
|
Steyerberg EW, Nieboer D, Debray TPA, van Houwelingen HC. Assessment of heterogeneity in an individual participant data meta-analysis of prediction models: An overview and illustration. Stat Med 2019; 38:4290-4309. [PMID: 31373722 PMCID: PMC6772012 DOI: 10.1002/sim.8296] [Citation(s) in RCA: 28] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/04/2017] [Revised: 03/23/2019] [Accepted: 06/06/2019] [Indexed: 02/06/2023]
Abstract
Clinical prediction models aim to provide estimates of absolute risk for a diagnostic or prognostic endpoint. Such models may be derived from data from various studies in the context of a meta‐analysis. We describe and propose approaches for assessing heterogeneity in predictor effects and predictions arising from models based on data from different sources. These methods are illustrated in a case study with patients suffering from traumatic brain injury, where we aim to predict 6‐month mortality based on individual patient data using meta‐analytic techniques (15 studies, n = 11 022 patients). The insights into various aspects of heterogeneity are important to develop better models and understand problems with the transportability of absolute risk predictions.
Collapse
Affiliation(s)
- Ewout W Steyerberg
- Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, The Netherlands.,Department of Public Health, Erasmus MC, Rotterdam, The Netherlands
| | - Daan Nieboer
- Department of Public Health, Erasmus MC, Rotterdam, The Netherlands
| | - Thomas P A Debray
- Julius Center for Health Sciences and Primary Care, Utrecht University Medical Center, Utrecht, The Netherlands.,Cochrane Netherlands, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Hans C van Houwelingen
- Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, The Netherlands
| |
Collapse
|
31
|
Hudda MT, Fewtrell MS, Haroun D, Lum S, Williams JE, Wells JCK, Riley RD, Owen CG, Cook DG, Rudnicka AR, Whincup PH, Nightingale CM. Development and validation of a prediction model for fat mass in children and adolescents: meta-analysis using individual participant data. BMJ 2019; 366:l4293. [PMID: 31340931 PMCID: PMC6650932 DOI: 10.1136/bmj.l4293] [Citation(s) in RCA: 37] [Impact Index Per Article: 7.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/23/2022]
Abstract
OBJECTIVES To develop and validate a prediction model for fat mass in children aged 4-15 years using routinely available risk factors of height, weight, and demographic information without the need for more complex forms of assessment. DESIGN Individual participant data meta-analysis. SETTING Four population based cross sectional studies and a fifth study for external validation, United Kingdom. PARTICIPANTS A pooled derivation dataset (four studies) of 2375 children and an external validation dataset of 176 children with complete data on anthropometric measurements and deuterium dilution assessments of fat mass. MAIN OUTCOME MEASURE Multivariable linear regression analysis, using backwards selection for inclusion of predictor variables and allowing non-linear relations, was used to develop a prediction model for fat-free mass (and subsequently fat mass by subtracting resulting estimates from weight) based on the four studies. Internal validation and then internal-external cross validation were used to examine overfitting and generalisability of the model's predictive performance within the four development studies; external validation followed using the fifth dataset. RESULTS Model derivation was based on a multi-ethnic population of 2375 children (47.8% boys, n=1136) aged 4-15 years. The final model containing predictor variables of height, weight, age, sex, and ethnicity had extremely high predictive ability (optimism adjusted R2: 94.8%, 95% confidence interval 94.4% to 95.2%) with excellent calibration of observed and predicted values. The internal validation showed minimal overfitting and good model generalisability, with excellent calibration and predictive performance. External validation in 176 children aged 11-12 years showed promising generalisability of the model (R2: 90.0%, 95% confidence interval 87.2% to 92.8%) with good calibration of observed and predicted fat mass (slope: 1.02, 95% confidence interval 0.97 to 1.07). The mean difference between observed and predicted fat mass was -1.29 kg (95% confidence interval -1.62 to -0.96 kg). CONCLUSION The developed model accurately predicted levels of fat mass in children aged 4-15 years. The prediction model is based on simple anthropometric measures without the need for more complex forms of assessment and could improve the accuracy of assessments for body fatness in children (compared with those provided by body mass index) for effective surveillance, prevention, and management of clinical and public health obesity.
Collapse
Affiliation(s)
- Mohammed T Hudda
- Population Health Research Institute, St George's, University of London, London SW17 0RE, UK
| | - Mary S Fewtrell
- Population, Policy and Practice Programme, UCL Great Ormond Street Institute of Child Health, London, UK
| | - Dalia Haroun
- College of Natural and Health Sciences, Department of Public Health and Nutrition, Zayed University, Dubai, UAE
| | - Sooky Lum
- Respiratory, Critical Care and Anaesthesia section of III Programme, UCL Great Ormond Street Institute of Child Health, London, UK
| | - Jane E Williams
- Population, Policy and Practice Programme, UCL Great Ormond Street Institute of Child Health, London, UK
| | - Jonathan C K Wells
- Population, Policy and Practice Programme, UCL Great Ormond Street Institute of Child Health, London, UK
| | - Richard D Riley
- Centre for Prognosis Research, Research Institute for Primary Care and Health Sciences, Keele University, Staffordshire, UK
| | - Christopher G Owen
- Population Health Research Institute, St George's, University of London, London SW17 0RE, UK
| | - Derek G Cook
- Population Health Research Institute, St George's, University of London, London SW17 0RE, UK
| | - Alicja R Rudnicka
- Population Health Research Institute, St George's, University of London, London SW17 0RE, UK
| | - Peter H Whincup
- Population Health Research Institute, St George's, University of London, London SW17 0RE, UK
| | - Claire M Nightingale
- Population Health Research Institute, St George's, University of London, London SW17 0RE, UK
| |
Collapse
|
32
|
Debray TP, de Jong VM, Moons KG, Riley RD. Evidence synthesis in prognosis research. Diagn Progn Res 2019; 3:13. [PMID: 31338426 PMCID: PMC6621956 DOI: 10.1186/s41512-019-0059-4] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/23/2018] [Accepted: 04/16/2019] [Indexed: 12/11/2022] Open
Abstract
Over the past few years, evidence synthesis has become essential to investigate and improve the generalizability of medical research findings. This strategy often involves a meta-analysis to formally summarize quantities of interest, such as relative treatment effect estimates. The use of meta-analysis methods is, however, less straightforward in prognosis research because substantial variation exists in research objectives, analysis methods and the level of reported evidence. We present a gentle overview of statistical methods that can be used to summarize data of prognostic factor and prognostic model studies. We discuss how aggregate data, individual participant data, or a combination thereof can be combined through meta-analysis methods. Recent examples are provided throughout to illustrate the various methods.
Collapse
Affiliation(s)
- Thomas P.A. Debray
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Universiteitsweg 100, Utrecht, 3584 CG The Netherlands
- Cochrane Netherlands, University Medical Center Utrecht, Universiteitsweg 100, Utrecht, 3584 CG The Netherlands
| | - Valentijn M.T. de Jong
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Universiteitsweg 100, Utrecht, 3584 CG The Netherlands
| | - Karel G.M. Moons
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Universiteitsweg 100, Utrecht, 3584 CG The Netherlands
- Cochrane Netherlands, University Medical Center Utrecht, Universiteitsweg 100, Utrecht, 3584 CG The Netherlands
| | - Richard D. Riley
- Research Institute for Primary Care & Health Sciences, Keele University, Staffordshire, ST5 5BG UK
| |
Collapse
|
33
|
Wilder-Smith A, Wei Y, de Araújo TVB, VanKerkhove M, Turchi Martelli CM, Turchi MD, Teixeira M, Tami A, Souza J, Sousa P, Soriano-Arandes A, Soria-Segarra C, Sanchez Clemente N, Rosenberger KD, Reveiz L, Prata-Barbosa A, Pomar L, Pelá Rosado LE, Perez F, Passos SD, Nogueira M, Noel TP, Moura da Silva A, Moreira ME, Morales I, Miranda Montoya MC, Miranda-Filho DDB, Maxwell L, Macpherson CNL, Low N, Lan Z, LaBeaud AD, Koopmans M, Kim C, João E, Jaenisch T, Hofer CB, Gustafson P, Gérardin P, Ganz JS, Dias ACF, Elias V, Duarte G, Debray TPA, Cafferata ML, Buekens P, Broutet N, Brickley EB, Brasil P, Brant F, Bethencourt S, Benedetti A, Avelino-Silva VL, Ximenes RADA, Alves da Cunha A, Alger J. Understanding the relation between Zika virus infection during pregnancy and adverse fetal, infant and child outcomes: a protocol for a systematic review and individual participant data meta-analysis of longitudinal studies of pregnant women and their infants and children. BMJ Open 2019; 9:e026092. [PMID: 31217315 PMCID: PMC6588966 DOI: 10.1136/bmjopen-2018-026092] [Citation(s) in RCA: 28] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/17/2018] [Revised: 02/11/2019] [Accepted: 05/09/2019] [Indexed: 12/14/2022] Open
Abstract
INTRODUCTION Zika virus (ZIKV) infection during pregnancy is a known cause of microcephaly and other congenital and developmental anomalies. In the absence of a ZIKV vaccine or prophylactics, principal investigators (PIs) and international leaders in ZIKV research have formed the ZIKV Individual Participant Data (IPD) Consortium to identify, collect and synthesise IPD from longitudinal studies of pregnant women that measure ZIKV infection during pregnancy and fetal, infant or child outcomes. METHODS AND ANALYSIS We will identify eligible studies through the ZIKV IPD Consortium membership and a systematic review and invite study PIs to participate in the IPD meta-analysis (IPD-MA). We will use the combined dataset to estimate the relative and absolute risk of congenital Zika syndrome (CZS), including microcephaly and late symptomatic congenital infections; identify and explore sources of heterogeneity in those estimates and develop and validate a risk prediction model to identify the pregnancies at the highest risk of CZS or adverse developmental outcomes. The variable accuracy of diagnostic assays and differences in exposure and outcome definitions means that included studies will have a higher level of systematic variability, a component of measurement error, than an IPD-MA of studies of an established pathogen. We will use expert testimony, existing internal and external diagnostic accuracy validation studies and laboratory external quality assessments to inform the distribution of measurement error in our models. We will apply both Bayesian and frequentist methods to directly account for these and other sources of uncertainty. ETHICS AND DISSEMINATION The IPD-MA was deemed exempt from ethical review. We will convene a group of patient advocates to evaluate the ethical implications and utility of the risk stratification tool. Findings from these analyses will be shared via national and international conferences and through publication in open access, peer-reviewed journals. TRIAL REGISTRATION NUMBER PROSPERO International prospective register of systematic reviews (CRD42017068915).
Collapse
Affiliation(s)
- Annelies Wilder-Smith
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore
| | - Yinghui Wei
- Centre for Mathematical Sciences, University of Plymouth, Plymouth, UK
| | | | - Maria VanKerkhove
- Health Emergencies Programme, Organisation mondiale de la Sante, Geneve, Switzerland
| | | | - Marília Dalva Turchi
- Institute of Tropical Pathology and Public Health, Federal University of Goias, Goiânia, Brazil
| | - Mauro Teixeira
- Department of Biochemistry and Immunology, Federal University of Minas Gerais, Belo Horizonte, Minas Gerais, Brazil
| | - Adriana Tami
- Department of Medical Microbiology, University Medical Center Groningen, Groningen, The Netherlands
| | - João Souza
- Department of Social Medicine, University of São Paulo, São Paulo, Brazil
| | - Patricia Sousa
- Reference Center for Neurodevelopment, Assistance, and Rehabilitation of Children, State Department of Health of Maranhão, Sao Luís, Brazil
| | | | | | | | - Kerstin Daniela Rosenberger
- Department of Infectious Diseases, Section Clinical Tropical Medicine, UniversitatsKlinikum Heidelberg, Heidelberg, Germany
| | - Ludovic Reveiz
- Evidence and Intelligence for Action in Health, Pan American Health Organization, Washington, District of Columbia, USA
| | - Arnaldo Prata-Barbosa
- Department of Pediatrics, D’Or Institute for Research & Education, Rio de Janeiro, Brazil
| | - Léo Pomar
- Department of Obstetrics and Gynecology, Centre Hospitalier de l’Ouest Guyanais, Saint-Laurent du Maroni, French Guiana
| | | | - Freddy Perez
- Communicable Diseases and Environmental Determinants of Health Department, Pan American Health Organization, Washington, District of Columbia, USA
| | | | - Mauricio Nogueira
- Faculdade de Medicina de Sao Jose do Rio Preto, Department of Dermatologic Diseases, São José do Rio Preto, Brazil
| | - Trevor P. Noel
- Windward Islands Research and Education Foundation, St. George’s University, True Blue Point, Grenada
| | - Antônio Moura da Silva
- Department of Public Health, Universidade Federal do Maranhão – São Luís, São Luís, Brazil
| | | | - Ivonne Morales
- Department of Infectious Diseases, Section Clinical Tropical Medicine, UniversitatsKlinikum Heidelberg, Heidelberg, Germany
| | | | | | - Lauren Maxwell
- Reproductive Health and Research, World Health Organization, Geneva, Switzerland
- Hubert Department of Global Health, Emory University, Atlanta, Georgia, USA
| | - Calum N. L. Macpherson
- Windward Islands Research and Education Foundation, St. George’s University, True Blue Point, Grenada
| | - Nicola Low
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
| | - Zhiyi Lan
- McGill University Health Centre, McGill University, Montréal, Canada
| | | | - Marion Koopmans
- Department of Virology, Erasmus Medical Center, Rotterdam, The Netherlands
| | - Caron Kim
- Department of Reproductive Health and Research, World Health Organization, Geneva, Switzerland
| | - Esaú João
- Department of Infectious Diseases, Hospital Federal dos Servidores do Estado, Rio de Janeiro, Brazil
| | - Thomas Jaenisch
- Department of Infectious Diseases, Section Clinical Tropical Medicine, UniversitatsKlinikum Heidelberg, Heidelberg, Germany
| | - Cristina Barroso Hofer
- Instituto de Puericultura e Pediatria Martagão Gesteira, Universidade Federal do Rio de Janeiro, Rio de Janeiro, Brazil
| | - Paul Gustafson
- Statistics, University of British Columbia, British Columbia, Vancouver, Canada
| | - Patrick Gérardin
- INSERM CIC1410 Clinical Epidemiology, CHU La Réunion, Saint Pierre, Réunion
- UM 134 PIMIT (CNRS 9192, INSERM U1187, IRD 249, Université de la Réunion), Universite de la Reunion, Sainte Clotilde, Réunion
| | | | - Ana Carolina Fialho Dias
- Department of Biochemistry and Immunology, Federal University of Minas Gerais, Belo Horizonte, Minas Gerais, Brazil
| | - Vanessa Elias
- Sustainable Development and Environmental Health, Pan American Health Organization, Washington, District of Columbia, USA
| | - Geraldo Duarte
- Department of Gynecology and Obstetrics, University of São Paulo, São Paulo, Brazil
| | - Thomas Paul Alfons Debray
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands
| | - María Luisa Cafferata
- Mother and Children Health Research Department, Instituto de Efectividad Clinica y Sanitaria, Buenos Aires, Argentina
| | - Pierre Buekens
- School of Public Health and Tropical Medicine, Tulane University, New Orleans, USA
| | - Nathalie Broutet
- Department of Reproductive Health and Research, World Health Organization, Geneva, Switzerland
| | - Elizabeth B. Brickley
- Department of Infectious Disease Epidemiology, London School of Hygiene and Tropical Medicine, London, UK
| | - Patrícia Brasil
- Instituto de pesquisa Clínica Evandro Chagas, Fundacao Oswaldo Cruz, Rio de Janeiro, Brazil
| | - Fátima Brant
- Department of Biochemistry and Immunology, Federal University of Minas Gerais, Belo Horizonte, Minas Gerais, Brazil
| | - Sarah Bethencourt
- Facultad de Ciencias de la Salud, Universidad de Carabobo, Valencia, Carabobo, Bolivarian Republic of Venezuela
| | - Andrea Benedetti
- Departments of Medicine and of Epidemiology, Biostatistics & Occupational Health, McGill University, Montreal, Quebec, Canada
| | - Vivian Lida Avelino-Silva
- Department of Infectious and Parasitic Diseases, Faculdade de Medicina da Universidade de Sao Paulo, São Paulo, Brazil
| | | | | | - Jackeline Alger
- Facultad de Ciencias Médicas, Universidad Nacional Autónoma de Honduras, Tegucigalpa, Honduras
| | | |
Collapse
|
34
|
Ageron FX, Gayet-Ageron A, Steyerberg E, Bouzat P, Roberts I. Prognostic model for traumatic death due to bleeding: cross-sectional international study. BMJ Open 2019; 9:e026823. [PMID: 31142526 PMCID: PMC6549712 DOI: 10.1136/bmjopen-2018-026823] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/23/2022] Open
Abstract
OBJECTIVE To develop and validate a prognostic model and a simple model to predict death due to bleeding in trauma patients. DESIGN Cross-sectional study with multivariable logistic regression using data from two large trauma cohorts. SETTING 274 hospitals from 40 countries in the Clinical Randomisation of Anti-fibrinolytic in Significant Haemorrhage (CRASH-2) trial and 24 hospitals in the Northern French Alps Trauma registry. PARTICIPANTS 13 485 trauma patients in the CRASH-2 trial and 9945 patients in the Northern French Alps Trauma registry who were admitted to hospital within 3 hours of injury. MAIN OUTCOME MEASURE In-hospital death due to bleeding within 28 days. RESULTS There were 815 (6%) deaths from bleeding in the CRASH-2 trial and 102 (1%) in the Northern French Alps Trauma registry. The full model included age, systolic blood pressure (SBP), Glasgow Coma Scale (GCS), heart rate, respiratory rate and type of injury (penetrating). The simple model included age, SBP and GCS. In a cross-validation procedure by country, discrimination and calibration were adequate (pooled C-statistic 0.85 (95% CI 0.81 to 0.88) for the full model and 0.84 (95% CI 0.80 to 0.88) for the simple model). CONCLUSION This prognostic model can identify trauma patients at risk of death due to bleeding in a wide range of settings and can support prehospital triage and trauma audit, including audit of tranexamic acid use.
Collapse
Affiliation(s)
- Francois-Xavier Ageron
- Clinical Trials Unit, London School of Hygiene & Tropical Medicine, London, UK
- Emergency Department and Northern French Alps Emergency Network, Hospital Annecy Genevois, Annecy, France
| | - Angele Gayet-Ageron
- Clinical Trials Unit, London School of Hygiene & Tropical Medicine, London, UK
- Clinical Research Center and Division of Clinical Epidemiology, Department of Health and Community Medicine, University Hospital Geneva, Geneva, Switzerland
| | - Ewout Steyerberg
- Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, Rotterdam, The Netherlands
- Department of Public Health, Erasmus MC, Rotterdam, The Netherlands
| | - Pierre Bouzat
- Grenoble Alpes Trauma Center, Pôle Anesthésie-Réanimation, CHU Grenoble Alpes, Grenoble, France
| | - Ian Roberts
- Clinical Trials Unit, London School of Hygiene & Tropical Medicine, London, UK
| |
Collapse
|
35
|
Chen WS, Tan JH, Mohamad Y, Imran R. External validation of a modified trauma and injury severity score model in major trauma injury. Injury 2019; 50:1118-1124. [PMID: 30591225 DOI: 10.1016/j.injury.2018.12.031] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/24/2018] [Revised: 12/08/2018] [Accepted: 12/21/2018] [Indexed: 02/02/2023]
Abstract
BACKGROUND The establishment of an accurate prognostic model in major trauma patients is important mainly because this group of patients will benefit the most. Clinical prediction models must be validated internally and externally on a regular basis to ensure the prediction is accurate and current. This study aims to externally validate two prediction models, the Trauma and Injury Severity Score model developed using the Major Trauma Outcome Study in North America (MTOS-TRISS model), and the NTrD-TRISS model, which is a refined MTOS-TRISS model with coefficients derived from the Malaysian National Trauma Database (NTrD), by regarding mortality as the outcome measurement. METHOD This retrospective study included patients with major trauma injuries reported to a trauma centre of Hospital Sultanah Aminah over a 6-year period from 2011 and 2017. Model validation was examined using the measures of discrimination and calibration. Discrimination was assessed using the area under the receiver operating characteristic curve (AUC) and 95% confidence interval (CI). The Hosmer-Lemeshow (H-L) goodness-of-fit test was used to examine calibration capabilities. The predictive validity of both MTOS-TRISS and NTrD-TRISS models were further evaluated by incorporating parameters such as the New Injury Severity Scale and the Injury Severity Score. RESULTS Total patients of 3788 (3434 blunt and 354 penetrating injuries) with average age of 37 years (standard deviation of 16 years) were included in this study. All MTOS-TRISS and NTrD-TRISS models examined in this study showed adequate discriminative ability with AUCs ranged from 0.86 to 0.89 for patients with blunt trauma mechanism and 0.89 to 0.99 for patients with penetrating trauma mechanism. The H-L goodness-of-fit test indicated the NTrD-TRISS model calibrated as good as the MTOS-TRISS model for patients with blunt trauma mechanism. CONCLUSION For patients with blunt trauma mechanism, both the MTOS-TRISS and NTrD-TRISS models showed good discrimination and calibration performances. Discrimination performance for the NTrD-TRISS model was revealed to be as good as the MTOS-TRISS model specifically for patients with penetrating trauma mechanism. Overall, this validation study has ascertained the discrimination and calibration performances of the NTrD-TRISS model to be as good as the MTOS-TRISS model particularly for patients with blunt trauma mechanism.
Collapse
Affiliation(s)
- W S Chen
- Department of Statistics, Data Science and Epidemiology, Swinburne University of Technology, Melbourne, Australia.
| | - J H Tan
- General Surgery Department, Hospital Sultanah Aminah, Johor Bahru, Malaysia.
| | - Y Mohamad
- General Surgery Department, Hospital Sultanah Aminah, Johor Bahru, Malaysia.
| | - R Imran
- General Surgery Department, Hospital Sultanah Aminah, Johor Bahru, Malaysia.
| |
Collapse
|
36
|
Wynants L, Kent DM, Timmerman D, Lundquist CM, Van Calster B. Untapped potential of multicenter studies: a review of cardiovascular risk prediction models revealed inappropriate analyses and wide variation in reporting. Diagn Progn Res 2019; 3:6. [PMID: 31093576 PMCID: PMC6460661 DOI: 10.1186/s41512-019-0046-9] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/13/2018] [Accepted: 01/03/2019] [Indexed: 12/17/2022] Open
Abstract
BACKGROUND Clinical prediction models are often constructed using multicenter databases. Such a data structure poses additional challenges for statistical analysis (clustered data) but offers opportunities for model generalizability to a broad range of centers. The purpose of this study was to describe properties, analysis, and reporting of multicenter studies in the Tufts PACE Clinical Prediction Model Registry and to illustrate consequences of common design and analyses choices. METHODS Fifty randomly selected studies that are included in the Tufts registry as multicenter and published after 2000 underwent full-text screening. Simulated examples illustrate some key concepts relevant to multicenter prediction research. RESULTS Multicenter studies differed widely in the number of participating centers (range 2 to 5473). Thirty-nine of 50 studies ignored the multicenter nature of data in the statistical analysis. In the others, clustering was resolved by developing the model on only one center, using mixed effects or stratified regression, or by using center-level characteristics as predictors. Twenty-three of 50 studies did not describe the clinical settings or type of centers from which data was obtained. Four of 50 studies discussed neither generalizability nor external validity of the developed model. CONCLUSIONS Regression methods and validation strategies tailored to multicenter studies are underutilized. Reporting on generalizability and potential external validity of the model lacks transparency. Hence, multicenter prediction research has untapped potential. REGISTRATION This review was not registered.
Collapse
Affiliation(s)
- L. Wynants
- Department of Development and Regeneration, KU Leuven, Herestraat 49, box 7003, 3000 Leuven, Belgium
- Department of Epidemiology, CAPHRI Care and Public Health Research Institute, Maastricht University, PO Box 9600, 6200 MD Maastricht, The Netherlands
| | - D. M. Kent
- Predictive Analytics and Comparative Effectiveness (PACE) Center, Institute for Clinical Research and Health Policy Studies, Tufts Medical Center, 800 Washington St, Box 63, Boston, MA 02111 USA
| | - D. Timmerman
- Department of Development and Regeneration, KU Leuven, Herestraat 49, box 7003, 3000 Leuven, Belgium
- Department of Obstetrics and Gynecology, University Hospitals Leuven, Herestraat 49, 3000 Leuven, Belgium
| | - C. M. Lundquist
- Predictive Analytics and Comparative Effectiveness (PACE) Center, Institute for Clinical Research and Health Policy Studies, Tufts Medical Center, 800 Washington St, Box 63, Boston, MA 02111 USA
| | - B. Van Calster
- Department of Development and Regeneration, KU Leuven, Herestraat 49, box 7003, 3000 Leuven, Belgium
- Department of Biomedical Data Sciences, Leiden University Medical Center, PO Box 9600, Leiden, 2300RC The Netherlands
| |
Collapse
|
37
|
Nguyen TL, Debray TPA. The use of prognostic scores for causal inference with general treatment regimes. Stat Med 2019; 38:2013-2029. [PMID: 30652333 PMCID: PMC6590249 DOI: 10.1002/sim.8084] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2018] [Revised: 12/03/2018] [Accepted: 12/09/2018] [Indexed: 01/29/2023]
Abstract
In nonrandomised studies, inferring causal effects requires appropriate methods for addressing confounding bias. Although it is common to adopt propensity score analysis to this purpose, prognostic score analysis has recently been proposed as an alternative strategy. While both approaches were originally introduced to estimate causal effects for binary interventions, the theory of propensity score has since been extended to the case of general treatment regimes. Indeed, many treatments are not assigned in a binary fashion and require a certain extent of dosing. Hence, researchers may often be interested in estimating treatment effects across multiple exposures. To the best of our knowledge, the prognostic score analysis has not been yet generalised to this case. In this article, we describe the theory of prognostic scores for causal inference with general treatment regimes. Our methods can be applied to compare multiple treatments using nonrandomised data, a topic of great relevance in contemporary evaluations of clinical interventions. We propose estimators for the average treatment effects in different populations of interest, the validity of which is assessed through a series of simulations. Finally, we present an illustrative case in which we estimate the effect of the delay to Aspirin administration on a composite outcome of death or dependence at 6 months in stroke patients.
Collapse
Affiliation(s)
- Tri-Long Nguyen
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands.,Section of Epidemiology, Department of Public Health, University of Copenhagen, Copenhagen K, Denmark.,Department of Pharmacy, Nîmes University Hospital Centre, Nîmes, France
| | - Thomas P A Debray
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands.,Cochrane Netherlands, University Medical Center Utrecht, Utrecht, The Netherlands.,Botnar Research Centre, University of Oxford, Oxford, UK.,Institute of Health Informatics, University College London, London, UK
| |
Collapse
|
38
|
Härmälä S, O’Brien A, Parisinos CA, Direk K, Shallcross L, Hayward A. Development and validation of a prediction model to estimate the risk of liver cirrhosis in primary care patients with abnormal liver blood test results: protocol for an electronic health record study in Clinical Practice Research Datalink. Diagn Progn Res 2019; 3:10. [PMID: 31143841 PMCID: PMC6532213 DOI: 10.1186/s41512-019-0056-7] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/19/2019] [Accepted: 03/29/2019] [Indexed: 12/13/2022] Open
Abstract
BACKGROUND Driven by alcohol consumption and obesity, the prevalence of non-viral liver disease in the UK is increasing. Due to its silent and slow nature, the progression of liver disease is currently unpredictable and challenging to monitor. The latest National Institute for Health Care Excellence cirrhosis guidelines call for a validated risk tool that would allow general practitioners to identify patients that are at high risk of developing cirrhosis. METHODS Using linked electronic health records from the Clinical Practice Research Datalink (a database of > 10 million patients in England), we aim to develop and validate a prediction model to estimate 2-, 5- and 10-year risk of cirrhosis. The model will provide individualised cirrhosis risk predictions for adult primary care patients, free from underlying liver disease or viral hepatitis infection, whose liver blood test results come back abnormal. We will externally validate the model in patients from 30 further Clinical Practice Research Datalink general practices in England. DISCUSSION The prediction model will provide estimates of cirrhosis risk in primary care patients with abnormal liver blood test results to guide referral to secondary care, to identify patients who are in serious need of preventative health interventions and to help reassure patients at low risk of cirrhosis in the long term.
Collapse
Affiliation(s)
- Suvi Härmälä
- 0000000121901201grid.83440.3bInstitute of Health Informatics, University College London, 222 Euston Road, London, NW1 2DA UK
| | - Alastair O’Brien
- 0000000121901201grid.83440.3bDivision of Medicine, University College London, Rayne Building, 5 University Street, London, WC1E 6JJ UK
| | - Constantinos A. Parisinos
- 0000000121901201grid.83440.3bInstitute of Health Informatics, University College London, 222 Euston Road, London, NW1 2DA UK
| | - Kenan Direk
- 0000000121901201grid.83440.3bInstitute of Health Informatics, University College London, 222 Euston Road, London, NW1 2DA UK
| | - Laura Shallcross
- 0000000121901201grid.83440.3bInstitute of Health Informatics, University College London, 222 Euston Road, London, NW1 2DA UK
| | - Andrew Hayward
- 0000000121901201grid.83440.3bInstitute of Epidemiology and Health Care, University College London, 1-19 Torrington Place, London, WC1E 7HB UK
| |
Collapse
|
39
|
Yu D, Jordan KP, Snell KIE, Riley RD, Bedson J, Edwards JJ, Mallen CD, Tan V, Ukachukwu V, Prieto-Alhambra D, Walker C, Peat G. Development and validation of prediction models to estimate risk of primary total hip and knee replacements using data from the UK: two prospective open cohorts using the UK Clinical Practice Research Datalink. Ann Rheum Dis 2018; 78:91-99. [PMID: 30337425 PMCID: PMC6317440 DOI: 10.1136/annrheumdis-2018-213894] [Citation(s) in RCA: 26] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2018] [Revised: 09/14/2018] [Accepted: 09/15/2018] [Indexed: 12/23/2022]
Abstract
Objectives The ability to efficiently and accurately predict future risk of primary total hip and knee replacement (THR/TKR) in earlier stages of osteoarthritis (OA) has potentially important applications. We aimed to develop and validate two models to estimate an individual’s risk of primary THR and TKR in patients newly presenting to primary care. Methods We identified two cohorts of patients aged ≥40 years newly consulting hip pain/OA and knee pain/OA in the Clinical Practice Research Datalink. Candidate predictors were identified by systematic review, novel hypothesis-free ‘Record-Wide Association Study’ with replication, and panel consensus. Cox proportional hazards models accounting for competing risk of death were applied to derive risk algorithms for THR and TKR. Internal–external cross-validation (IECV) was then applied over geographical regions to validate two models. Results 45 predictors for THR and 53 for TKR were identified, reviewed and selected by the panel. 301 052 and 416 030 patients newly consulting between 1992 and 2015 were identified in the hip and knee cohorts, respectively (median follow-up 6 years). The resultant model C-statistics is 0.73 (0.72, 0.73) and 0.79 (0.78, 0.79) for THR (with 20 predictors) and TKR model (with 24 predictors), respectively. The IECV C-statistics ranged between 0.70–0.74 (THR model) and 0.76–0.82 (TKR model); the IECV calibration slope ranged between 0.93–1.07 (THR model) and 0.92–1.12 (TKR model). Conclusions Two prediction models with good discrimination and calibration that estimate individuals’ risk of THR and TKR have been developed and validated in large-scale, nationally representative data, and are readily automated in electronic patient records.
Collapse
Affiliation(s)
- Dahai Yu
- Arthritis Research UK Primary Care Centre, Research Institute for Primary Care & Health Sciences, Keele University, Keele, UK
| | - Kelvin P Jordan
- Arthritis Research UK Primary Care Centre, Research Institute for Primary Care & Health Sciences, Keele University, Keele, UK
| | - Kym I E Snell
- Arthritis Research UK Primary Care Centre, Research Institute for Primary Care & Health Sciences, Keele University, Keele, UK.,Centre for Prognostic Research, Arthritis Research UK Primary Care Centre, Research Institute for Primary Care & Health Sciences, Keele University, Keele, UK
| | - Richard D Riley
- Arthritis Research UK Primary Care Centre, Research Institute for Primary Care & Health Sciences, Keele University, Keele, UK.,Centre for Prognostic Research, Arthritis Research UK Primary Care Centre, Research Institute for Primary Care & Health Sciences, Keele University, Keele, UK
| | - John Bedson
- Arthritis Research UK Primary Care Centre, Research Institute for Primary Care & Health Sciences, Keele University, Keele, UK
| | - John James Edwards
- Arthritis Research UK Primary Care Centre, Research Institute for Primary Care & Health Sciences, Keele University, Keele, UK
| | - Christian D Mallen
- Arthritis Research UK Primary Care Centre, Research Institute for Primary Care & Health Sciences, Keele University, Keele, UK
| | - Valerie Tan
- Arthritis Research UK Primary Care Centre, Research Institute for Primary Care & Health Sciences, Keele University, Keele, UK
| | - Vincent Ukachukwu
- Arthritis Research UK Primary Care Centre, Research Institute for Primary Care & Health Sciences, Keele University, Keele, UK
| | - Daniel Prieto-Alhambra
- GREMPAL (Grup de Recerca en Epidemiologia de les Malalties Prevalents de l'Aparell Locomotor), Idiap Jordi Gol Primary Care Research Institute and CIBERFes, Universitat Autònoma de Barcelona and Instituto de Salud Carlos III, Barcelona, Spain.,Musculoskeletal Pharmaco- and Device Epidemiology - Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology, and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Christine Walker
- Arthritis Research UK Primary Care Centre, Research Institute for Primary Care & Health Sciences, Keele University, Keele, UK
| | - George Peat
- Arthritis Research UK Primary Care Centre, Research Institute for Primary Care & Health Sciences, Keele University, Keele, UK
| |
Collapse
|
40
|
Liao J, Muniz-Terrera G, Scholes S, Hao Y, Chen YM. Lifestyle index for mortality prediction using multiple ageing cohorts in the USA, UK and Europe. Sci Rep 2018; 8:6644. [PMID: 29703919 PMCID: PMC5923240 DOI: 10.1038/s41598-018-24778-1] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2017] [Accepted: 04/04/2018] [Indexed: 11/09/2022] Open
Abstract
Current mortality prediction indexes are mainly based on functional morbidity and comorbidity, with limited information for risk prevention. This study aimed to develop and validate a modifiable lifestyle-based mortality predication index for older adults. Data from 51,688 participants (56% women) aged ≥50 years in 2002 Health and Retirement Study, 2002 English Longitudinal Study of Ageing and 2004 Survey of Health Ageing and Retirement in Europe were used to estimate coefficients of the index with cohort-stratified Cox regression. Models were validated across studies and compared to the Lee index (having comorbid and morbidity predictors). Over an average of 11-year follow-up, 10,240 participants died. The lifestyle index includes smoking, drinking, exercising, sleep quality, BMI, sex and age; showing adequate model performance in internal validation (C-statistic 0.79; D-statistic 1.94; calibration slope 1.13) and in all combinations of internal-external cross-validation. It outperformed Lee index (e.g. differences in C-statistic = 0.01, D-statistic = 0.17, P < 0.001) consistently across health status. The lifestyle index stratified participants into varying mortality risk groups, with those in the top quintile having 13.5% excess absolute mortality risk over 10 years than those in the bottom 50th centile. Our lifestyle index with easy-assessable behavioural factors and improved generalizability may maximize its usability for personalized risk management.
Collapse
Affiliation(s)
- Jing Liao
- Department of Medical Statistics and Epidemiology, School of Public Health, Sun Yat-sen University, No.74 Zhongshan 2nd Road, Guangzhou, 510080, P.R. China.,Sun Yat-sen Global Health Institute, Institute of State Governance, Sun Yat-sen University, No. 135 Xingang West Road, Guangzhou, 510275, P.R. China
| | - Graciela Muniz-Terrera
- Biostatistics and Epidemiology, Centre for Dementia Prevention, University of Edinburgh, Edinburgh, Scotland
| | - Shaun Scholes
- UCL Research Department of Epidemiology and Public Health, Faculty of Population Health Sciences, University College London, 1-19 Torrington Place, London, WC1E 6BT, UK
| | - Yuantao Hao
- Department of Medical Statistics and Epidemiology, School of Public Health, Sun Yat-sen University, No.74 Zhongshan 2nd Road, Guangzhou, 510080, P.R. China.,Sun Yat-sen Global Health Institute, Institute of State Governance, Sun Yat-sen University, No. 135 Xingang West Road, Guangzhou, 510275, P.R. China
| | - Yu-Ming Chen
- Department of Medical Statistics and Epidemiology, School of Public Health, Sun Yat-sen University, No.74 Zhongshan 2nd Road, Guangzhou, 510080, P.R. China.
| |
Collapse
|
41
|
Ng R, Kornas K, Sutradhar R, Wodchis WP, Rosella LC. The current application of the Royston-Parmar model for prognostic modeling in health research: a scoping review. Diagn Progn Res 2018; 2:4. [PMID: 31093554 PMCID: PMC6460777 DOI: 10.1186/s41512-018-0026-5] [Citation(s) in RCA: 20] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/15/2017] [Accepted: 01/30/2018] [Indexed: 12/29/2022] Open
Abstract
BACKGROUND Prognostic models incorporating survival analysis predict the risk (i.e., probability) of experiencing a future event over a specific time period. In 2002, Royston and Parmar described a type of flexible parametric survival model called the Royston-Parmar model in Statistics in Medicine, a model which fits a restricted cubic spline to flexibly model the baseline log cumulative hazard on the proportional hazards scale. This feature permits absolute measures of effect (e.g., hazard rates) to be estimated at all time points, an important feature when using the model. The Royston-Parmar model can also incorporate time-dependent effects and be used on different scales (e.g., proportional odds, probit). These features make the Royston-Parmar model attractive for prediction, yet their current uptake for prognostic modeling is unknown. Thus, the objectives were to conduct a scoping review of how the Royston-Parmar model has been applied to prognostic models in health research, to raise awareness of the model, to identify gaps in current reporting, and to offer model building considerations and reporting suggestions for other researchers. METHODS Five electronic databases and gray literature indexed in web sources from 2001 to 2016 were searched to identify articles for inclusion in the scoping review. Two reviewers independently screened 1429 articles, and after applying exclusion criteria through a two-step screening process, data from 12 studies were abstracted. RESULTS Since 2001, only 12 studies were identified that used the Royston-Parmar model in some capacity for prognostic modeling, 10 of which used the model as the basis for their prognostic model. The restricted cubic spline varied across studies in the number of interior knots (range 1 to 6), and only three studies reported knot placement. Three studies provided details about the baseline function, with two studies using a figure and the third providing coefficients. However, no studies provided adequate information on their restricted cubic spline to permit others to validate or completely use the model. CONCLUSIONS Despite the advantages of the Royston-Parmar model for prognostic models, they are not widely used in health research. Better reporting of details about the restricted cubic spline is needed, so the prognostic model can be used and validated by others. REGISTRATION The protocol was registered with Open Science Framework (https://osf.io/r3232/).
Collapse
Affiliation(s)
- Ryan Ng
- 0000 0001 2157 2938grid.17063.33Dalla Lana School of Public Health, University of Toronto, 155 College St, Toronto, ON M5T 3M7 Canada
| | - Kathy Kornas
- 0000 0001 2157 2938grid.17063.33Dalla Lana School of Public Health, University of Toronto, 155 College St, Toronto, ON M5T 3M7 Canada
| | - Rinku Sutradhar
- 0000 0000 8849 1617grid.418647.8Institute for Clinical Evaluative Sciences, 2075 Bayview Ave, Toronto, ON M4N 3M5 Canada
| | - Walter P. Wodchis
- 0000 0000 8849 1617grid.418647.8Institute for Clinical Evaluative Sciences, 2075 Bayview Ave, Toronto, ON M4N 3M5 Canada
- 0000 0001 2157 2938grid.17063.33Institute of Health Policy, Management and Evaluation, University of Toronto, 155 College Street, Toronto, ON M5T 3M6 Canada
| | - Laura C. Rosella
- 0000 0001 2157 2938grid.17063.33Dalla Lana School of Public Health, University of Toronto, 155 College St, Toronto, ON M5T 3M7 Canada
- 0000 0000 8849 1617grid.418647.8Institute for Clinical Evaluative Sciences, 2075 Bayview Ave, Toronto, ON M4N 3M5 Canada
| |
Collapse
|
42
|
Martin GP, Mamas MA, Peek N, Buchan I, Sperrin M. A multiple-model generalisation of updating clinical prediction models. Stat Med 2017; 37:1343-1358. [PMID: 29250812 PMCID: PMC5873448 DOI: 10.1002/sim.7586] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2017] [Revised: 11/15/2017] [Accepted: 11/19/2017] [Indexed: 12/23/2022]
Abstract
There is growing interest in developing clinical prediction models (CPMs) to aid local healthcare decision‐making. Frequently, these CPMs are developed in isolation across different populations, with repetitive de novo derivation a common modelling strategy. However, this fails to utilise all available information and does not respond to changes in health processes through time and space. Alternatively, model updating techniques have previously been proposed that adjust an existing CPM to suit the new population, but these techniques are restricted to a single model. Therefore, we aimed to develop a generalised method for updating and aggregating multiple CPMs. The proposed “hybrid method” re‐calibrates multiple CPMs using stacked regression while concurrently revising specific covariates using individual participant data (IPD) under a penalised likelihood. The performance of the hybrid method was compared with existing methods in a clinical example of mortality risk prediction after transcatheter aortic valve implantation, and in 2 simulation studies. The simulation studies explored the effect of sample size and between‐population‐heterogeneity on the method, with each representing a situation of having multiple distinct CPMs and 1 set of IPD. When the sample size of the IPD was small, stacked regression and the hybrid method had comparable but highest performance across modelling methods. Conversely, in large IPD samples, development of a new model and the hybrid method gave the highest performance. Hence, the proposed strategy can inform the choice between utilising existing CPMs or developing a model de novo, thereby incorporating IPD, existing research, and prior (clinical) knowledge into the modelling strategy.
Collapse
Affiliation(s)
- Glen P Martin
- Farr Institute, Faculty of Biology, Medicine and Health, University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | - Mamas A Mamas
- Farr Institute, Faculty of Biology, Medicine and Health, University of Manchester, Manchester Academic Health Science Centre, Manchester, UK.,Keele Cardiovascular Research Group, Keele University, Stoke-on-Trent, UK
| | - Niels Peek
- Farr Institute, Faculty of Biology, Medicine and Health, University of Manchester, Manchester Academic Health Science Centre, Manchester, UK.,NIHR Greater Manchester Primary Care Patient Safety Translational Research Centre, University of Manchester, Manchester, UK
| | - Iain Buchan
- Farr Institute, Faculty of Biology, Medicine and Health, University of Manchester, Manchester Academic Health Science Centre, Manchester, UK.,Microsoft Research, Cambridge, UK
| | - Matthew Sperrin
- Farr Institute, Faculty of Biology, Medicine and Health, University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| |
Collapse
|
43
|
Murphy GJ, Mumford AD, Rogers CA, Wordsworth S, Stokes EA, Verheyden V, Kumar T, Harris J, Clayton G, Ellis L, Plummer Z, Dott W, Serraino F, Wozniak M, Morris T, Nath M, Sterne JA, Angelini GD, Reeves BC. Diagnostic and therapeutic medical devices for safer blood management in cardiac surgery: systematic reviews, observational studies and randomised controlled trials. PROGRAMME GRANTS FOR APPLIED RESEARCH 2017. [DOI: 10.3310/pgfar05170] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
Abstract
BackgroundAnaemia, coagulopathic bleeding and transfusion are strongly associated with organ failure, sepsis and death following cardiac surgery.ObjectiveTo evaluate the clinical effectiveness and cost-effectiveness of medical devices used as diagnostic and therapeutic tools for the management of anaemia and bleeding in cardiac surgery.Methods and resultsWorkstream 1 – in the COagulation and Platelet laboratory Testing in Cardiac surgery (COPTIC) study we demonstrated that risk assessment using baseline clinical factors predicted bleeding with a high degree of accuracy. The results from point-of-care (POC) platelet aggregometry or viscoelastometry tests or an expanded range of laboratory reference tests for coagulopathy did not improve predictive accuracy beyond that achieved with the clinical risk score alone. The routine use of POC tests was not cost-effective. A systematic review concluded that POC-based algorithms are not clinically effective. We developed two new clinical risk prediction scores for transfusion and bleeding that are available as e-calculators. Workstream 2 – in the PAtient-SPecific Oxygen monitoring to Reduce blood Transfusion during heart surgery (PASPORT) trial and a systematic review we demonstrated that personalised near-infrared spectroscopy-based algorithms for the optimisation of tissue oxygenation, or as indicators for red cell transfusion, were neither clinically effective nor cost-effective. Workstream 3 – in the REDWASH trial we failed to demonstrate a reduction in inflammation or organ injury in recipients of mechanically washed red cells compared with standard (unwashed) red cells.LimitationsExisting studies evaluating the predictive accuracy or effectiveness of POC tests of coagulopathy or near-infrared spectroscopy were at high risk of bias. Interventions that alter red cell transfusion exposure, a common surrogate outcome in most trials, were not found to be clinically effective.ConclusionsA systematic assessment of devices in clinical use as blood management adjuncts in cardiac surgery did not demonstrate clinical effectiveness or cost-effectiveness. The contribution of anaemia and coagulopathy to adverse clinical outcomes following cardiac surgery remains poorly understood. Further research to define the pathogenesis of these conditions may lead to more accurate diagnoses, more effective treatments and potentially improved clinical outcomes.Study registrationCurrent Controlled Trials ISRCTN20778544 (COPTIC study) and PROSPERO CRD42016033831 (systematic review) (workstream 1); Current Controlled Trials ISRCTN23557269 (PASPORT trial) and PROSPERO CRD4201502769 (systematic review) (workstream 2); and Current Controlled Trials ISRCTN27076315 (REDWASH trial) (workstream 3).FundingThis project was funded by the National Institute for Health Research (NIHR) Programme Grants for Applied Research programme and will be published in full inProgramme Grants for Applied Research; Vol. 5, No. 17. See the NIHR Journals Library website for further project information.
Collapse
Affiliation(s)
- Gavin J Murphy
- Department of Cardiovascular Sciences and NIHR Leicester Biomedical Research Unit in Cardiovascular Medicine, University of Leicester, Leicester, UK
| | - Andrew D Mumford
- School of Cellular and Molecular Medicine, University of Bristol, Bristol, UK
| | - Chris A Rogers
- Clinical Trials and Evaluation Unit, School of Clinical Sciences, University of Bristol, Bristol, UK
| | - Sarah Wordsworth
- Health Economics Research Centre, Nuffield Department of Population Health, University of Oxford, Oxford, UK
| | - Elizabeth A Stokes
- Health Economics Research Centre, Nuffield Department of Population Health, University of Oxford, Oxford, UK
| | - Veerle Verheyden
- Department of Cardiovascular Sciences and NIHR Leicester Biomedical Research Unit in Cardiovascular Medicine, University of Leicester, Leicester, UK
| | - Tracy Kumar
- Department of Cardiovascular Sciences and NIHR Leicester Biomedical Research Unit in Cardiovascular Medicine, University of Leicester, Leicester, UK
| | - Jessica Harris
- Clinical Trials and Evaluation Unit, School of Clinical Sciences, University of Bristol, Bristol, UK
| | - Gemma Clayton
- Clinical Trials and Evaluation Unit, School of Clinical Sciences, University of Bristol, Bristol, UK
| | - Lucy Ellis
- Clinical Trials and Evaluation Unit, School of Clinical Sciences, University of Bristol, Bristol, UK
| | - Zoe Plummer
- Clinical Trials and Evaluation Unit, School of Clinical Sciences, University of Bristol, Bristol, UK
| | - William Dott
- Department of Cardiovascular Sciences and NIHR Leicester Biomedical Research Unit in Cardiovascular Medicine, University of Leicester, Leicester, UK
| | - Filiberto Serraino
- Department of Cardiovascular Sciences and NIHR Leicester Biomedical Research Unit in Cardiovascular Medicine, University of Leicester, Leicester, UK
| | - Marcin Wozniak
- Department of Cardiovascular Sciences and NIHR Leicester Biomedical Research Unit in Cardiovascular Medicine, University of Leicester, Leicester, UK
| | - Tom Morris
- Leicester Clinical Trials Unit, University of Leicester, Leicester, UK
| | - Mintu Nath
- Department of Cardiovascular Sciences and NIHR Leicester Biomedical Research Unit in Cardiovascular Medicine, University of Leicester, Leicester, UK
| | - Jonathan A Sterne
- School of Social and Community Medicine, University of Bristol, Bristol, UK
| | - Gianni D Angelini
- Bristol Heart Institute, School of Clinical Sciences, University of Bristol, Bristol, UK
| | - Barnaby C Reeves
- Clinical Trials and Evaluation Unit, School of Clinical Sciences, University of Bristol, Bristol, UK
| |
Collapse
|
44
|
Hilkens NA, Algra A, Diener HC, Reitsma JB, Bath PM, Csiba L, Hacke W, Kappelle LJ, Koudstaal PJ, Leys D, Mas JL, Sacco RL, Amarenco P, Sissani L, Greving JP. Predicting major bleeding in patients with noncardioembolic stroke on antiplatelets: S 2TOP-BLEED. Neurology 2017; 89:936-943. [PMID: 28768848 DOI: 10.1212/wnl.0000000000004289] [Citation(s) in RCA: 27] [Impact Index Per Article: 3.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2016] [Accepted: 05/01/2017] [Indexed: 12/21/2022] Open
Abstract
OBJECTIVE To develop and externally validate a prediction model for major bleeding in patients with a TIA or ischemic stroke on antiplatelet agents. METHODS We combined individual patient data from 6 randomized clinical trials (CAPRIE, ESPS-2, MATCH, CHARISMA, ESPRIT, and PRoFESS) investigating antiplatelet therapy after TIA or ischemic stroke. Cox regression analyses stratified by trial were performed to study the association between predictors and major bleeding. A risk prediction model was derived and validated in the PERFORM trial. Performance was assessed with the c statistic and calibration plots. RESULTS Major bleeding occurred in 1,530 of the 43,112 patients during 94,833 person-years of follow-up. The observed 3-year risk of major bleeding was 4.6% (95% confidence interval [CI] 4.4%-4.9%). Predictors were male sex, smoking, type of antiplatelet agents (aspirin-clopidogrel), outcome on modified Rankin Scale ≥3, prior stroke, high blood pressure, lower body mass index, elderly, Asian ethnicity, and diabetes (S2TOP-BLEED). The S2TOP-BLEED score had a c statistic of 0.63 (95% CI 0.60-0.64) and showed good calibration in the development data. Major bleeding risk ranged from 2% in patients aged 45-54 years without additional risk factors to more than 10% in patients aged 75-84 years with multiple risk factors. In external validation, the model had a c statistic of 0.61 (95% CI 0.59-0.63) and slightly underestimated major bleeding risk. CONCLUSIONS The S2TOP-BLEED score can be used to estimate 3-year major bleeding risk in patients with a TIA or ischemic stroke who use antiplatelet agents, based on readily available characteristics. The discriminatory performance may be improved by identifying stronger predictors of major bleeding.
Collapse
Affiliation(s)
- Nina A Hilkens
- From the Julius Center for Health Sciences and Primary Care (N.A.H., A.A., J.B.R., J.P.G.) and Department of Neurology and Neurosurgery (A.A., L.J.K.), Brain Center Rudolf Magnus, University Medical Center Utrecht, the Netherlands; Department of Neurology (H.-C.D.), University Hospital Essen, Germany; Stroke Trials Unit (P.M.B.), Division of Clinical Neuroscience, University of Nottingham, UK; Department of Neurology (L.C.), University of Debrecen Medical and Health Science Center, Hungary; Department of Neurology (W.H.), University of Heidelberg, Germany; Department of Neurology (P.J.K.), Erasmus Medical Center, Rotterdam, the Netherlands; Department of Neurology (D.L.), Roger Salengro Hospital, Lille, France; Department of Neurology (J.-L.M.), Hôpital Sainte-Anne, Université Paris Descartes, France; Department of Neurology (R.L.S.), Miller School of Medicine, University of Miami, FL; and Department of Neurology and Stroke Center (P.A., L.S.), Bichat University Hospital, Paris, France.
| | - Ale Algra
- From the Julius Center for Health Sciences and Primary Care (N.A.H., A.A., J.B.R., J.P.G.) and Department of Neurology and Neurosurgery (A.A., L.J.K.), Brain Center Rudolf Magnus, University Medical Center Utrecht, the Netherlands; Department of Neurology (H.-C.D.), University Hospital Essen, Germany; Stroke Trials Unit (P.M.B.), Division of Clinical Neuroscience, University of Nottingham, UK; Department of Neurology (L.C.), University of Debrecen Medical and Health Science Center, Hungary; Department of Neurology (W.H.), University of Heidelberg, Germany; Department of Neurology (P.J.K.), Erasmus Medical Center, Rotterdam, the Netherlands; Department of Neurology (D.L.), Roger Salengro Hospital, Lille, France; Department of Neurology (J.-L.M.), Hôpital Sainte-Anne, Université Paris Descartes, France; Department of Neurology (R.L.S.), Miller School of Medicine, University of Miami, FL; and Department of Neurology and Stroke Center (P.A., L.S.), Bichat University Hospital, Paris, France
| | - Hans-Christoph Diener
- From the Julius Center for Health Sciences and Primary Care (N.A.H., A.A., J.B.R., J.P.G.) and Department of Neurology and Neurosurgery (A.A., L.J.K.), Brain Center Rudolf Magnus, University Medical Center Utrecht, the Netherlands; Department of Neurology (H.-C.D.), University Hospital Essen, Germany; Stroke Trials Unit (P.M.B.), Division of Clinical Neuroscience, University of Nottingham, UK; Department of Neurology (L.C.), University of Debrecen Medical and Health Science Center, Hungary; Department of Neurology (W.H.), University of Heidelberg, Germany; Department of Neurology (P.J.K.), Erasmus Medical Center, Rotterdam, the Netherlands; Department of Neurology (D.L.), Roger Salengro Hospital, Lille, France; Department of Neurology (J.-L.M.), Hôpital Sainte-Anne, Université Paris Descartes, France; Department of Neurology (R.L.S.), Miller School of Medicine, University of Miami, FL; and Department of Neurology and Stroke Center (P.A., L.S.), Bichat University Hospital, Paris, France
| | - Johannes B Reitsma
- From the Julius Center for Health Sciences and Primary Care (N.A.H., A.A., J.B.R., J.P.G.) and Department of Neurology and Neurosurgery (A.A., L.J.K.), Brain Center Rudolf Magnus, University Medical Center Utrecht, the Netherlands; Department of Neurology (H.-C.D.), University Hospital Essen, Germany; Stroke Trials Unit (P.M.B.), Division of Clinical Neuroscience, University of Nottingham, UK; Department of Neurology (L.C.), University of Debrecen Medical and Health Science Center, Hungary; Department of Neurology (W.H.), University of Heidelberg, Germany; Department of Neurology (P.J.K.), Erasmus Medical Center, Rotterdam, the Netherlands; Department of Neurology (D.L.), Roger Salengro Hospital, Lille, France; Department of Neurology (J.-L.M.), Hôpital Sainte-Anne, Université Paris Descartes, France; Department of Neurology (R.L.S.), Miller School of Medicine, University of Miami, FL; and Department of Neurology and Stroke Center (P.A., L.S.), Bichat University Hospital, Paris, France
| | - Philip M Bath
- From the Julius Center for Health Sciences and Primary Care (N.A.H., A.A., J.B.R., J.P.G.) and Department of Neurology and Neurosurgery (A.A., L.J.K.), Brain Center Rudolf Magnus, University Medical Center Utrecht, the Netherlands; Department of Neurology (H.-C.D.), University Hospital Essen, Germany; Stroke Trials Unit (P.M.B.), Division of Clinical Neuroscience, University of Nottingham, UK; Department of Neurology (L.C.), University of Debrecen Medical and Health Science Center, Hungary; Department of Neurology (W.H.), University of Heidelberg, Germany; Department of Neurology (P.J.K.), Erasmus Medical Center, Rotterdam, the Netherlands; Department of Neurology (D.L.), Roger Salengro Hospital, Lille, France; Department of Neurology (J.-L.M.), Hôpital Sainte-Anne, Université Paris Descartes, France; Department of Neurology (R.L.S.), Miller School of Medicine, University of Miami, FL; and Department of Neurology and Stroke Center (P.A., L.S.), Bichat University Hospital, Paris, France
| | - Laszlo Csiba
- From the Julius Center for Health Sciences and Primary Care (N.A.H., A.A., J.B.R., J.P.G.) and Department of Neurology and Neurosurgery (A.A., L.J.K.), Brain Center Rudolf Magnus, University Medical Center Utrecht, the Netherlands; Department of Neurology (H.-C.D.), University Hospital Essen, Germany; Stroke Trials Unit (P.M.B.), Division of Clinical Neuroscience, University of Nottingham, UK; Department of Neurology (L.C.), University of Debrecen Medical and Health Science Center, Hungary; Department of Neurology (W.H.), University of Heidelberg, Germany; Department of Neurology (P.J.K.), Erasmus Medical Center, Rotterdam, the Netherlands; Department of Neurology (D.L.), Roger Salengro Hospital, Lille, France; Department of Neurology (J.-L.M.), Hôpital Sainte-Anne, Université Paris Descartes, France; Department of Neurology (R.L.S.), Miller School of Medicine, University of Miami, FL; and Department of Neurology and Stroke Center (P.A., L.S.), Bichat University Hospital, Paris, France
| | - Werner Hacke
- From the Julius Center for Health Sciences and Primary Care (N.A.H., A.A., J.B.R., J.P.G.) and Department of Neurology and Neurosurgery (A.A., L.J.K.), Brain Center Rudolf Magnus, University Medical Center Utrecht, the Netherlands; Department of Neurology (H.-C.D.), University Hospital Essen, Germany; Stroke Trials Unit (P.M.B.), Division of Clinical Neuroscience, University of Nottingham, UK; Department of Neurology (L.C.), University of Debrecen Medical and Health Science Center, Hungary; Department of Neurology (W.H.), University of Heidelberg, Germany; Department of Neurology (P.J.K.), Erasmus Medical Center, Rotterdam, the Netherlands; Department of Neurology (D.L.), Roger Salengro Hospital, Lille, France; Department of Neurology (J.-L.M.), Hôpital Sainte-Anne, Université Paris Descartes, France; Department of Neurology (R.L.S.), Miller School of Medicine, University of Miami, FL; and Department of Neurology and Stroke Center (P.A., L.S.), Bichat University Hospital, Paris, France
| | - L Jaap Kappelle
- From the Julius Center for Health Sciences and Primary Care (N.A.H., A.A., J.B.R., J.P.G.) and Department of Neurology and Neurosurgery (A.A., L.J.K.), Brain Center Rudolf Magnus, University Medical Center Utrecht, the Netherlands; Department of Neurology (H.-C.D.), University Hospital Essen, Germany; Stroke Trials Unit (P.M.B.), Division of Clinical Neuroscience, University of Nottingham, UK; Department of Neurology (L.C.), University of Debrecen Medical and Health Science Center, Hungary; Department of Neurology (W.H.), University of Heidelberg, Germany; Department of Neurology (P.J.K.), Erasmus Medical Center, Rotterdam, the Netherlands; Department of Neurology (D.L.), Roger Salengro Hospital, Lille, France; Department of Neurology (J.-L.M.), Hôpital Sainte-Anne, Université Paris Descartes, France; Department of Neurology (R.L.S.), Miller School of Medicine, University of Miami, FL; and Department of Neurology and Stroke Center (P.A., L.S.), Bichat University Hospital, Paris, France
| | - Peter J Koudstaal
- From the Julius Center for Health Sciences and Primary Care (N.A.H., A.A., J.B.R., J.P.G.) and Department of Neurology and Neurosurgery (A.A., L.J.K.), Brain Center Rudolf Magnus, University Medical Center Utrecht, the Netherlands; Department of Neurology (H.-C.D.), University Hospital Essen, Germany; Stroke Trials Unit (P.M.B.), Division of Clinical Neuroscience, University of Nottingham, UK; Department of Neurology (L.C.), University of Debrecen Medical and Health Science Center, Hungary; Department of Neurology (W.H.), University of Heidelberg, Germany; Department of Neurology (P.J.K.), Erasmus Medical Center, Rotterdam, the Netherlands; Department of Neurology (D.L.), Roger Salengro Hospital, Lille, France; Department of Neurology (J.-L.M.), Hôpital Sainte-Anne, Université Paris Descartes, France; Department of Neurology (R.L.S.), Miller School of Medicine, University of Miami, FL; and Department of Neurology and Stroke Center (P.A., L.S.), Bichat University Hospital, Paris, France
| | - Didier Leys
- From the Julius Center for Health Sciences and Primary Care (N.A.H., A.A., J.B.R., J.P.G.) and Department of Neurology and Neurosurgery (A.A., L.J.K.), Brain Center Rudolf Magnus, University Medical Center Utrecht, the Netherlands; Department of Neurology (H.-C.D.), University Hospital Essen, Germany; Stroke Trials Unit (P.M.B.), Division of Clinical Neuroscience, University of Nottingham, UK; Department of Neurology (L.C.), University of Debrecen Medical and Health Science Center, Hungary; Department of Neurology (W.H.), University of Heidelberg, Germany; Department of Neurology (P.J.K.), Erasmus Medical Center, Rotterdam, the Netherlands; Department of Neurology (D.L.), Roger Salengro Hospital, Lille, France; Department of Neurology (J.-L.M.), Hôpital Sainte-Anne, Université Paris Descartes, France; Department of Neurology (R.L.S.), Miller School of Medicine, University of Miami, FL; and Department of Neurology and Stroke Center (P.A., L.S.), Bichat University Hospital, Paris, France
| | - Jean-Louis Mas
- From the Julius Center for Health Sciences and Primary Care (N.A.H., A.A., J.B.R., J.P.G.) and Department of Neurology and Neurosurgery (A.A., L.J.K.), Brain Center Rudolf Magnus, University Medical Center Utrecht, the Netherlands; Department of Neurology (H.-C.D.), University Hospital Essen, Germany; Stroke Trials Unit (P.M.B.), Division of Clinical Neuroscience, University of Nottingham, UK; Department of Neurology (L.C.), University of Debrecen Medical and Health Science Center, Hungary; Department of Neurology (W.H.), University of Heidelberg, Germany; Department of Neurology (P.J.K.), Erasmus Medical Center, Rotterdam, the Netherlands; Department of Neurology (D.L.), Roger Salengro Hospital, Lille, France; Department of Neurology (J.-L.M.), Hôpital Sainte-Anne, Université Paris Descartes, France; Department of Neurology (R.L.S.), Miller School of Medicine, University of Miami, FL; and Department of Neurology and Stroke Center (P.A., L.S.), Bichat University Hospital, Paris, France
| | - Ralph L Sacco
- From the Julius Center for Health Sciences and Primary Care (N.A.H., A.A., J.B.R., J.P.G.) and Department of Neurology and Neurosurgery (A.A., L.J.K.), Brain Center Rudolf Magnus, University Medical Center Utrecht, the Netherlands; Department of Neurology (H.-C.D.), University Hospital Essen, Germany; Stroke Trials Unit (P.M.B.), Division of Clinical Neuroscience, University of Nottingham, UK; Department of Neurology (L.C.), University of Debrecen Medical and Health Science Center, Hungary; Department of Neurology (W.H.), University of Heidelberg, Germany; Department of Neurology (P.J.K.), Erasmus Medical Center, Rotterdam, the Netherlands; Department of Neurology (D.L.), Roger Salengro Hospital, Lille, France; Department of Neurology (J.-L.M.), Hôpital Sainte-Anne, Université Paris Descartes, France; Department of Neurology (R.L.S.), Miller School of Medicine, University of Miami, FL; and Department of Neurology and Stroke Center (P.A., L.S.), Bichat University Hospital, Paris, France
| | - Pierre Amarenco
- From the Julius Center for Health Sciences and Primary Care (N.A.H., A.A., J.B.R., J.P.G.) and Department of Neurology and Neurosurgery (A.A., L.J.K.), Brain Center Rudolf Magnus, University Medical Center Utrecht, the Netherlands; Department of Neurology (H.-C.D.), University Hospital Essen, Germany; Stroke Trials Unit (P.M.B.), Division of Clinical Neuroscience, University of Nottingham, UK; Department of Neurology (L.C.), University of Debrecen Medical and Health Science Center, Hungary; Department of Neurology (W.H.), University of Heidelberg, Germany; Department of Neurology (P.J.K.), Erasmus Medical Center, Rotterdam, the Netherlands; Department of Neurology (D.L.), Roger Salengro Hospital, Lille, France; Department of Neurology (J.-L.M.), Hôpital Sainte-Anne, Université Paris Descartes, France; Department of Neurology (R.L.S.), Miller School of Medicine, University of Miami, FL; and Department of Neurology and Stroke Center (P.A., L.S.), Bichat University Hospital, Paris, France
| | - Leila Sissani
- From the Julius Center for Health Sciences and Primary Care (N.A.H., A.A., J.B.R., J.P.G.) and Department of Neurology and Neurosurgery (A.A., L.J.K.), Brain Center Rudolf Magnus, University Medical Center Utrecht, the Netherlands; Department of Neurology (H.-C.D.), University Hospital Essen, Germany; Stroke Trials Unit (P.M.B.), Division of Clinical Neuroscience, University of Nottingham, UK; Department of Neurology (L.C.), University of Debrecen Medical and Health Science Center, Hungary; Department of Neurology (W.H.), University of Heidelberg, Germany; Department of Neurology (P.J.K.), Erasmus Medical Center, Rotterdam, the Netherlands; Department of Neurology (D.L.), Roger Salengro Hospital, Lille, France; Department of Neurology (J.-L.M.), Hôpital Sainte-Anne, Université Paris Descartes, France; Department of Neurology (R.L.S.), Miller School of Medicine, University of Miami, FL; and Department of Neurology and Stroke Center (P.A., L.S.), Bichat University Hospital, Paris, France
| | - Jacoba P Greving
- From the Julius Center for Health Sciences and Primary Care (N.A.H., A.A., J.B.R., J.P.G.) and Department of Neurology and Neurosurgery (A.A., L.J.K.), Brain Center Rudolf Magnus, University Medical Center Utrecht, the Netherlands; Department of Neurology (H.-C.D.), University Hospital Essen, Germany; Stroke Trials Unit (P.M.B.), Division of Clinical Neuroscience, University of Nottingham, UK; Department of Neurology (L.C.), University of Debrecen Medical and Health Science Center, Hungary; Department of Neurology (W.H.), University of Heidelberg, Germany; Department of Neurology (P.J.K.), Erasmus Medical Center, Rotterdam, the Netherlands; Department of Neurology (D.L.), Roger Salengro Hospital, Lille, France; Department of Neurology (J.-L.M.), Hôpital Sainte-Anne, Université Paris Descartes, France; Department of Neurology (R.L.S.), Miller School of Medicine, University of Miami, FL; and Department of Neurology and Stroke Center (P.A., L.S.), Bichat University Hospital, Paris, France
| | | |
Collapse
|
45
|
Clark CE, Boddy K, Warren FC, Taylor RS, Aboyans V, Cloutier L, McManus RJ, Shore AC, Campbell JL. Associations between interarm differences in blood pressure and cardiovascular disease outcomes: protocol for an individual patient data meta-analysis and development of a prognostic algorithm. BMJ Open 2017; 7:e016844. [PMID: 28674148 PMCID: PMC5734572 DOI: 10.1136/bmjopen-2017-016844] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/13/2022] Open
Abstract
INTRODUCTION Individual cohort studies in various populations and study-level meta-analyses have shown interarm differences (IAD) in blood pressure to be associated with increased cardiovascular and all-cause mortality. However, key questions remain, such as follows: (1) What is the additional contribution of IAD to prognostic risk estimation for cardiovascular and all-cause mortality? (2) What is the minimum cut-off value for IAD that defines elevated risk? (3) Is there a prognostic value of IAD and do different methods of IAD measurement impact on the prognostic value of IAD? We aim to address these questions by conducting an individual patient data (IPD) meta-analysis. METHODS AND ANALYSIS This study will identify prospective cohort studies that measured blood pressure in both arms during recruitment, and invite authors to contribute IPD datasets to this collaboration. All patient data received will be combined into a single dataset. Using one-stage meta-analysis, we will undertake multivariable time-to-event regression modelling, with the aim of developing a new prognostic model for cardiovascular risk estimation that includes IAD. We will explore variations in risk contribution of IAD across predefined population subgroups (eg, hypertensives, diabetics), establish the lower limit of IAD that is associated with additional cardiovascular risk and assess the impact of different methods of IAD measurement on risk prediction. ETHICS AND DISSEMINATION This study will not include any patient identifiable data. Included datasets will already have ethical approval and consent from their sponsors. Findings will be presented to international conferences and published in peer reviewed journals, and we have a comprehensive dissemination strategy in place with integrated patient and public involvement. PROSPERO REGISTRATION NUMBER CRD42015031227.
Collapse
Affiliation(s)
- Christopher E Clark
- Primary Care Research Group, Institute of Health Services Research, University of Exeter Medical School, Exeter, Devon, UK
| | - Kate Boddy
- Patient and Public Involvement Team, PenCLAHRC, University of Exeter Medical School, Exeter, Devon, UK
| | - Fiona C Warren
- Primary Care Research Group, Institute of Health Services Research, University of Exeter Medical School, Exeter, Devon, UK
| | - Rod S Taylor
- Primary Care Research Group, Institute of Health Services Research, University of Exeter Medical School, Exeter, Devon, UK
| | - Victor Aboyans
- Department of Cardiology, Dupuytren University Hospital, and Inserm 1098, Tropical Neuroepidemiology, Limoges, France
| | - Lyne Cloutier
- Département des sciences infirmières, Université du Québec à Trois-Rivières, Trois-Rivières, Canada
| | - Richard J McManus
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
| | - Angela C Shore
- NIHR Exeter Clinical Research Facility, Royal Devon and Exeter Hospital and University of Exeter Medical School, Exeter, Devon, UK
| | - John L Campbell
- Primary Care Research Group, Institute of Health Services Research, University of Exeter Medical School, Exeter, Devon, UK
| |
Collapse
|
46
|
Willis BH, Riley RD. Measuring the statistical validity of summary meta-analysis and meta-regression results for use in clinical practice. Stat Med 2017. [PMID: 28620945 PMCID: PMC5575530 DOI: 10.1002/sim.7372] [Citation(s) in RCA: 82] [Impact Index Per Article: 11.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/30/2023]
Abstract
An important question for clinicians appraising a meta‐analysis is: are the findings likely to be valid in their own practice—does the reported effect accurately represent the effect that would occur in their own clinical population? To this end we advance the concept of statistical validity—where the parameter being estimated equals the corresponding parameter for a new independent study. Using a simple (‘leave‐one‐out’) cross‐validation technique, we demonstrate how we may test meta‐analysis estimates for statistical validity using a new validation statistic, Vn, and derive its distribution. We compare this with the usual approach of investigating heterogeneity in meta‐analyses and demonstrate the link between statistical validity and homogeneity. Using a simulation study, the properties of Vn and the Q statistic are compared for univariate random effects meta‐analysis and a tailored meta‐regression model, where information from the setting (included as model covariates) is used to calibrate the summary estimate to the setting of application. Their properties are found to be similar when there are 50 studies or more, but for fewer studies Vn has greater power but a higher type 1 error rate than Q. The power and type 1 error rate of Vn are also shown to depend on the within‐study variance, between‐study variance, study sample size, and the number of studies in the meta‐analysis. Finally, we apply Vn to two published meta‐analyses and conclude that it usefully augments standard methods when deciding upon the likely validity of summary meta‐analysis estimates in clinical practice. © 2017 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.
Collapse
Affiliation(s)
- Brian H Willis
- Institute of Applied Health Research, University of Birmingham, U.K
| | - Richard D Riley
- Research Institute for Primary Care and Health Sciences, Keele University, U.K
| |
Collapse
|
47
|
Andreiuolo F, Le Teuff G, Bayar MA, Kilday JP, Pietsch T, von Bueren AO, Witt H, Korshunov A, Modena P, Pfister SM, Pagès M, Castel D, Giangaspero F, Chimelli L, Varlet P, Rutkowski S, Frappaz D, Massimino M, Grundy R, Grill J. Integrating Tenascin-C protein expression and 1q25 copy number status in pediatric intracranial ependymoma prognostication: A new model for risk stratification. PLoS One 2017; 12:e0178351. [PMID: 28617804 PMCID: PMC5472261 DOI: 10.1371/journal.pone.0178351] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2016] [Accepted: 05/11/2017] [Indexed: 12/19/2022] Open
Abstract
PURPOSE Despite multimodal therapy, prognosis of pediatric intracranial ependymomas remains poor with a 5-year survival rate below 70% and frequent late deaths. EXPERIMENTAL DESIGN This multicentric European study evaluated putative prognostic biomarkers. Tenascin-C (TNC) immunohistochemical expression and copy number status of 1q25 were retained for a pooled analysis of 5 independent cohorts. The prognostic value of TNC and 1q25 on the overall survival (OS) was assessed using a Cox model adjusted to age at diagnosis, tumor location, WHO grade, extent of resection, radiotherapy and stratified by cohort. Stratification on a predictor that did not satisfy the proportional hazards assumption was considered. Model performance was evaluated and an internal-external cross validation was performed. RESULTS Among complete cases with 5-year median follow-up (n = 470; 131 deaths), TNC and 1q25 gain were significantly associated with age at diagnosis and posterior fossa tumor location. 1q25 status added independent prognostic value for death beyond the classical variables with a hazard ratio (HR) = 2.19 95%CI = [1.29; 3.76] (p = 0.004), while TNC prognostic relation was tumor location-dependent with HR = 2.19 95%CI = [1.29; 3.76] (p = 0.004) in posterior fossa and HR = 0.64 [0.28; 1.48] (p = 0.295) in supratentorial (interaction p value = 0.015). The derived prognostic score identified 3 different robust risk groups. The omission of upfront RT was not associated with OS for good and intermediate prognostic groups while the absence of upfront RT was negatively associated with OS in the poor risk group. CONCLUSION Integrated TNC expression and 1q25 status are useful to better stratify patients and to eventually adapt treatment regimens in pediatric intracranial ependymoma.
Collapse
Affiliation(s)
- Felipe Andreiuolo
- Université Paris-Sud, Gustave Roussy, CNRS UMR 8203 "Vectorologie et Thérapeutiques Anticancéreuses", Villejuif, France
- Département de Neuropathologie, Hôpital Sainte-Anne, Paris, France
- Departamento de Patologia, Universidade Federal do Rio de Janeiro, Rio de Janeiro, Brazil
| | - Gwénaël Le Teuff
- Departement de Biostatistique et Epidemiologie, Gustave Roussy, Cancer Campus, Grand Paris, Villejuif, France
- CESP Centre for Research in Epidemiology and Population Health, INSERM U1018, Paris-Sud Univ., Villejuif, France
| | - Mohamed Amine Bayar
- Departement de Biostatistique et Epidemiologie, Gustave Roussy, Cancer Campus, Grand Paris, Villejuif, France
- CESP Centre for Research in Epidemiology and Population Health, INSERM U1018, Paris-Sud Univ., Villejuif, France
| | - John-Paul Kilday
- Children’s Brain Tumour Research Network (CBTRN), Royal Manchester Children’s Hospital, Manchester, United Kingdom
- The Centre for Paediatric, Teenage and Young Adult Cancer, Institute of Cancer Sciences, The University of Manchester, Manchester, United Kingdom
| | - Torsten Pietsch
- Institute of Neuropathology, University of Bonn Medical Center, Bonn, Germany
| | - André O. von Bueren
- Department of Paediatric Hematology and Oncology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
- Division of Pediatric Hematology and Oncology, Department of Pediatrics and Adolescent Medicine, University Medical Center Goettingen, Goettingen, Germany
| | - Hendrik Witt
- Division of Paediatric Neurooncology, German Cancer Research Center (DKFZ) and Department of Paediatric Oncology, Heidelberg University Hospital, Heidelberg, Germany
| | - Andrey Korshunov
- Clinical Cooperation Unit Neuropathology, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | | | - Stefan M. Pfister
- Division of Paediatric Neurooncology, German Cancer Research Center (DKFZ) and Department of Paediatric Oncology, Heidelberg University Hospital, Heidelberg, Germany
| | - Mélanie Pagès
- Département de Neuropathologie, Hôpital Sainte-Anne, Paris, France
- Université Sorbonne Paris Cité, Paris, France
| | - David Castel
- Université Paris-Sud, Gustave Roussy, CNRS UMR 8203 "Vectorologie et Thérapeutiques Anticancéreuses", Villejuif, France
- Département de Cancérologie de l'Enfant et de l'Adolescent, Gustave Roussy, Villejuif, France
| | - Felice Giangaspero
- Department of Radiology, Oncology and Anatomo-Pathology, Sapienza University, Roma, Italy
- IRCCS Neuromed, Pozzilli, Isernia, Italy
| | - Leila Chimelli
- Departamento de Patologia, Universidade Federal do Rio de Janeiro, Rio de Janeiro, Brazil
| | - Pascale Varlet
- Département de Neuropathologie, Hôpital Sainte-Anne, Paris, France
- Université Sorbonne Paris Cité, Paris, France
| | - Stefan Rutkowski
- Department of Paediatric Hematology and Oncology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Didier Frappaz
- Institut d'Hématologie-Oncologie Pédiatrique, Lyon, France
| | - Maura Massimino
- Paediatric Unit, Fondazione Istituto Di Ricovero e Cura a Carattere Scientifico, Istituto Nazionale dei Tumori, Milano, Italy
| | - Richard Grundy
- The Children's Brain Tumour Research Centre, University of Nottingham, Nottingham, United Kingdom
| | - Jacques Grill
- Université Paris-Sud, Gustave Roussy, CNRS UMR 8203 "Vectorologie et Thérapeutiques Anticancéreuses", Villejuif, France
- Département de Cancérologie de l'Enfant et de l'Adolescent, Gustave Roussy, Villejuif, France
| | | |
Collapse
|
48
|
Allotey J, Snell KIE, Chan C, Hooper R, Dodds J, Rogozinska E, Khan KS, Poston L, Kenny L, Myers J, Thilaganathan B, Chappell L, Mol BW, Von Dadelszen P, Ahmed A, Green M, Poon L, Khalil A, Moons KGM, Riley RD, Thangaratinam S. External validation, update and development of prediction models for pre-eclampsia using an Individual Participant Data (IPD) meta-analysis: the International Prediction of Pregnancy Complication Network (IPPIC pre-eclampsia) protocol. Diagn Progn Res 2017; 1:16. [PMID: 31093545 PMCID: PMC6460674 DOI: 10.1186/s41512-017-0016-z] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/06/2017] [Accepted: 09/19/2017] [Indexed: 01/13/2023] Open
Abstract
BACKGROUND Pre-eclampsia, a condition with raised blood pressure and proteinuria is associated with an increased risk of maternal and offspring mortality and morbidity. Early identification of mothers at risk is needed to target management. METHODS/DESIGN We aim to systematically review the existing literature to identify prediction models for pre-eclampsia. We have established the International Prediction of Pregnancy Complication Network (IPPIC), made up of 72 researchers from 21 countries who have carried out relevant primary studies or have access to existing registry databases, and collectively possess data from more than two million patients. We will use the individual participant data (IPD) from these studies to externally validate these existing prediction models and summarise model performance across studies using random-effects meta-analysis for any, late (after 34 weeks) and early (before 34 weeks) onset pre-eclampsia. If none of the models perform well, we will recalibrate (update), or develop and validate new prediction models using the IPD. We will assess the differential accuracy of the models in various settings and subgroups according to the risk status. We will also validate or develop prediction models based on clinical characteristics only; clinical and biochemical markers; clinical and ultrasound parameters; and clinical, biochemical and ultrasound tests. DISCUSSION Numerous systematic reviews with aggregate data meta-analysis have evaluated various risk factors separately or in combination for predicting pre-eclampsia, but these are affected by many limitations. Our large-scale collaborative IPD approach encourages consensus towards well developed, and validated prognostic models, rather than a number of competing non-validated ones. The large sample size from our IPD will also allow development and validation of multivariable prediction model for the relatively rare outcome of early onset pre-eclampsia. TRIAL REGISTRATION The project was registered on Prospero on the 27 November 2015 with ID: CRD42015029349.
Collapse
Affiliation(s)
- John Allotey
- 0000 0001 2171 1133grid.4868.2Women’s Health Research Unit, Barts and the London School of Medicine and Dentistry, Queen Mary University of London, London, UK
- 0000 0001 2171 1133grid.4868.2Pragmatic Clinical Trials Unit, Barts and the London School of Medicine and Dentistry, Queen Mary University of London, London, UK
- 0000 0001 2171 1133grid.4868.2Multidisciplinary Evidence Synthesis Hub (MESH), Queen Mary University of London, London, UK
| | - Kym I. E. Snell
- 0000 0004 0415 6205grid.9757.cResearch Institute for Primary Care and Health Sciences, Keele University, Keele, UK
| | - Claire Chan
- 0000 0001 2171 1133grid.4868.2Pragmatic Clinical Trials Unit, Barts and the London School of Medicine and Dentistry, Queen Mary University of London, London, UK
| | - Richard Hooper
- 0000 0001 2171 1133grid.4868.2Pragmatic Clinical Trials Unit, Barts and the London School of Medicine and Dentistry, Queen Mary University of London, London, UK
| | - Julie Dodds
- 0000 0001 2171 1133grid.4868.2Women’s Health Research Unit, Barts and the London School of Medicine and Dentistry, Queen Mary University of London, London, UK
- 0000 0001 2171 1133grid.4868.2Pragmatic Clinical Trials Unit, Barts and the London School of Medicine and Dentistry, Queen Mary University of London, London, UK
- 0000 0001 2171 1133grid.4868.2Multidisciplinary Evidence Synthesis Hub (MESH), Queen Mary University of London, London, UK
| | - Ewelina Rogozinska
- 0000 0001 2171 1133grid.4868.2Women’s Health Research Unit, Barts and the London School of Medicine and Dentistry, Queen Mary University of London, London, UK
- 0000 0001 2171 1133grid.4868.2Pragmatic Clinical Trials Unit, Barts and the London School of Medicine and Dentistry, Queen Mary University of London, London, UK
- 0000 0001 2171 1133grid.4868.2Multidisciplinary Evidence Synthesis Hub (MESH), Queen Mary University of London, London, UK
| | - Khalid S. Khan
- 0000 0001 2171 1133grid.4868.2Women’s Health Research Unit, Barts and the London School of Medicine and Dentistry, Queen Mary University of London, London, UK
- 0000 0001 2171 1133grid.4868.2Pragmatic Clinical Trials Unit, Barts and the London School of Medicine and Dentistry, Queen Mary University of London, London, UK
- 0000 0001 2171 1133grid.4868.2Multidisciplinary Evidence Synthesis Hub (MESH), Queen Mary University of London, London, UK
| | - Lucilla Poston
- 0000 0001 2322 6764grid.13097.3cDivision of Women’s Health, Women’s Health Academic Centre, King’s College London, London, UK
| | - Louise Kenny
- 0000000123318773grid.7872.aIrish Centre for Fetal and Neonatal Translational Research [INFANT], University College Cork, Cork, Ireland
| | - Jenny Myers
- 0000000121662407grid.5379.8Maternal and Fetal Heath Research Centre, Manchester Academic Health Science Centre, University of Manchester, Central Manchester NHS Trust, Manchester, UK
| | - Basky Thilaganathan
- grid.264200.2Fetal Medicine Unit, St George’s University Hospitals NHS Foundation Trust and Molecular and Clinical Sciences Research Institute, St George’s University of London, London, UK
| | - Lucy Chappell
- 0000 0001 2322 6764grid.13097.3cDivision of Women’s Health, Women’s Health Academic Centre, King’s College London, London, UK
| | - Ben W. Mol
- 0000 0004 1936 7304grid.1010.0The Robinson Research Institute, School of Paediatrics and Reproductive Health, University of Adelaide, Adelaide, Australia
| | - Peter Von Dadelszen
- 0000 0001 2161 2573grid.4464.2Institute of Cardiovascular and Cell Sciences, St George’s, University of London, London, UK
| | - Asif Ahmed
- 0000 0004 0376 4727grid.7273.1Aston Medical School, Aston University, Birmingham, UK
| | - Marcus Green
- Action on Pre-eclampsia (APEC) Charity, Worcestershire, UK
| | - Liona Poon
- 0000 0004 0391 9020grid.46699.34Harris Birthright Research Centre for Fetal Medicine, King’s College Hospital, London, UK
- 0000 0004 1937 0482grid.10784.3aDepartment of Obstetrics and Gynaecology, The Chinese University of Hong Kong, Hong Kong, Hong Kong
| | - Asma Khalil
- grid.264200.2Fetal Medicine Unit, St George’s University Hospitals NHS Foundation Trust and Molecular and Clinical Sciences Research Institute, St George’s University of London, London, UK
| | - Karel G. M. Moons
- 0000000090126352grid.7692.aJulius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht, the Netherlands
| | - Richard D. Riley
- 0000 0004 0415 6205grid.9757.cResearch Institute for Primary Care and Health Sciences, Keele University, Keele, UK
| | - Shakila Thangaratinam
- 0000 0001 2171 1133grid.4868.2Women’s Health Research Unit, Barts and the London School of Medicine and Dentistry, Queen Mary University of London, London, UK
- 0000 0001 2171 1133grid.4868.2Pragmatic Clinical Trials Unit, Barts and the London School of Medicine and Dentistry, Queen Mary University of London, London, UK
- 0000 0001 2171 1133grid.4868.2Multidisciplinary Evidence Synthesis Hub (MESH), Queen Mary University of London, London, UK
| | | |
Collapse
|
49
|
Kondofersky I, Laimighofer M, Kurz C, Krautenbacher N, Söllner JF, Dargatz P, Scherb H, Ankerst DP, Fuchs C. Three general concepts to improve risk prediction: good data, wisdom of the crowd, recalibration. F1000Res 2016. [DOI: 10.12688/f1000research.8680.1] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/19/2022] Open
Abstract
In today's information age, the necessary means exist for clinical risk prediction to capitalize on a multitude of data sources, increasing the potential for greater accuracy and improved patient care. Towards this objective, the Prostate Cancer DREAM Challenge posted comprehensive information from three clinical trials recording survival for patients with metastatic castration-resistant prostate cancer treated with first-line docetaxel. A subset of an independent clinical trial was used for interim evaluation of model submissions, providing critical feedback to participating teams for tailoring their models to the desired target. Final submitted models were evaluated and ranked on the independent clinical trial. Our team, called "A Bavarian Dream", utilized many of the common statistical methods for data dimension reduction and summarization during the trial. Three general modeling principles emerged that were deemed helpful for building accurate risk prediction tools and ending up among the winning teams of both sub-challenges. These principles included: first, good data, encompassing the collection of important variables and imputation of missing data; second, wisdom of the crowd, extending beyond the usual model ensemble notion to the inclusion of experts on specific risk ranges; and third, recalibration, entailing transfer learning to the target source. In this study, we illustrate the application and impact of these principles applied to data from the Prostate Cancer DREAM Challenge.
Collapse
|
50
|
Ensor J, Riley RD, Jowett S, Monahan M, Snell KI, Bayliss S, Moore D, Fitzmaurice D. Prediction of risk of recurrence of venous thromboembolism following treatment for a first unprovoked venous thromboembolism: systematic review, prognostic model and clinical decision rule, and economic evaluation. Health Technol Assess 2016; 20:i-xxxiii, 1-190. [PMID: 26879848 DOI: 10.3310/hta20120] [Citation(s) in RCA: 22] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/06/2023] Open
Abstract
BACKGROUND Unprovoked first venous thromboembolism (VTE) is defined as VTE in the absence of a temporary provoking factor such as surgery, immobility and other temporary factors. Recurrent VTE in unprovoked patients is highly prevalent, but easily preventable with oral anticoagulant (OAC) therapy. The unprovoked population is highly heterogeneous in terms of risk of recurrent VTE. OBJECTIVES The first aim of the project is to review existing prognostic models which stratify individuals by their recurrence risk, therefore potentially allowing tailored treatment strategies. The second aim is to enhance the existing research in this field, by developing and externally validating a new prognostic model for individual risk prediction, using a pooled database containing individual patient data (IPD) from several studies. The final aim is to assess the economic cost-effectiveness of the proposed prognostic model if it is used as a decision rule for resuming OAC therapy, compared with current standard treatment strategies. METHODS Standard systematic review methodology was used to identify relevant prognostic model development, validation and cost-effectiveness studies. Bibliographic databases (including MEDLINE, EMBASE and The Cochrane Library) were searched using terms relating to the clinical area and prognosis. Reviewing was undertaken by two reviewers independently using pre-defined criteria. Included full-text articles were data extracted and quality assessed. Critical appraisal of included full texts was undertaken and comparisons made of model performance. A prognostic model was developed using IPD from the pooled database of seven trials. A novel internal-external cross-validation (IECV) approach was used to develop and validate a prognostic model, with external validation undertaken in each of the trials iteratively. Given good performance in the IECV approach, a final model was developed using all trials data. A Markov patient-level simulation was used to consider the economic cost-effectiveness of using a decision rule (based on the prognostic model) to decide on resumption of OAC therapy (or not). RESULTS Three full-text articles were identified by the systematic review. Critical appraisal identified methodological and applicability issues; in particular, all three existing models did not have external validation. To address this, new prognostic models were sought with external validation. Two potential models were considered: one for use at cessation of therapy (pre D-dimer), and one for use after cessation of therapy (post D-dimer). Model performance measured in the external validation trials showed strong calibration performance for both models. The post D-dimer model performed substantially better in terms of discrimination (c = 0.69), better separating high- and low-risk patients. The economic evaluation identified that a decision rule based on the final post D-dimer model may be cost-effective for patients with predicted risk of recurrence of over 8% annually; this suggests continued therapy for patients with predicted risks ≥ 8% and cessation of therapy otherwise. CONCLUSIONS The post D-dimer model performed strongly and could be useful to predict individuals' risk of recurrence at any time up to 2-3 years, thereby aiding patient counselling and treatment decisions. A decision rule using this model may be cost-effective for informing clinical judgement and patient opinion in treatment decisions. Further research may investigate new predictors to enhance model performance and aim to further externally validate to confirm performance in new, non-trial populations. Finally, it is essential that further research is conducted to develop a model predicting bleeding risk on therapy, to manage the balance between the risks of recurrence and bleeding. STUDY REGISTRATION This study is registered as PROSPERO CRD42013003494. FUNDING The National Institute for Health Research Health Technology Assessment programme.
Collapse
Affiliation(s)
- Joie Ensor
- Public Health, Epidemiology and Biostatistics, School of Health and Population Sciences, University of Birmingham, Birmingham, UK.,Research Institute of Primary Care and Health Sciences, Keele University, Staffordshire, UK
| | - Richard D Riley
- Public Health, Epidemiology and Biostatistics, School of Health and Population Sciences, University of Birmingham, Birmingham, UK.,Research Institute of Primary Care and Health Sciences, Keele University, Staffordshire, UK
| | - Sue Jowett
- Health Economics, School of Health and Population Sciences, University of Birmingham, Birmingham, UK
| | - Mark Monahan
- Health Economics, School of Health and Population Sciences, University of Birmingham, Birmingham, UK
| | - Kym Ie Snell
- Public Health, Epidemiology and Biostatistics, School of Health and Population Sciences, University of Birmingham, Birmingham, UK
| | - Susan Bayliss
- Public Health, Epidemiology and Biostatistics, School of Health and Population Sciences, University of Birmingham, Birmingham, UK
| | - David Moore
- Public Health, Epidemiology and Biostatistics, School of Health and Population Sciences, University of Birmingham, Birmingham, UK
| | - David Fitzmaurice
- Primary Care Clinical Sciences, School of Health and Population Sciences, University of Birmingham, Birmingham, UK
| | | |
Collapse
|