1
|
Zahra A, van Smeden M, Abbink EJ, van den Berg JM, Blom MT, van den Dries CJ, Gussekloo J, Wouters F, Joling KJ, Melis R, Mooijaart SP, Peters JB, Polinder-Bos HA, van Raaij BFM, Appelman B, la Roi-Teeuw HM, Moons KGM, Luijken K. External validation of six COVID-19 prognostic models for predicting mortality risk in older populations in a hospital, primary care, and nursing home setting. J Clin Epidemiol 2024; 168:111270. [PMID: 38311188 DOI: 10.1016/j.jclinepi.2024.111270] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2023] [Revised: 01/26/2024] [Accepted: 01/26/2024] [Indexed: 02/10/2024]
Abstract
OBJECTIVES To systematically evaluate the performance of COVID-19 prognostic models and scores for mortality risk in older populations across three health-care settings: hospitals, primary care, and nursing homes. STUDY DESIGN AND SETTING This retrospective external validation study included 14,092 older individuals of ≥70 years of age with a clinical or polymerase chain reaction-confirmed COVID-19 diagnosis from March 2020 to December 2020. The six validation cohorts include three hospital-based (CliniCo, COVID-OLD, COVID-PREDICT), two primary care-based (Julius General Practitioners Network/Academisch network huisartsgeneeskunde/Network of Academic general Practitioners, PHARMO), and one nursing home cohort (YSIS) in the Netherlands. Based on a living systematic review of COVID-19 prediction models using Prediction model Risk Of Bias ASsessment Tool for quality and risk of bias assessment and considering predictor availability in validation cohorts, we selected six prognostic models predicting mortality risk in adults with COVID-19 infection (GAL-COVID-19 mortality, 4C Mortality Score, National Early Warning Score 2-extended model, Xie model, Wang clinical model, and CURB65 score). All six prognostic models were validated in the hospital cohorts and the GAL-COVID-19 mortality model was validated in all three healthcare settings. The primary outcome was in-hospital mortality for hospitals and 28-day mortality for primary care and nursing home settings. Model performance was evaluated in each validation cohort separately in terms of discrimination, calibration, and decision curves. An intercept update was performed in models indicating miscalibration followed by predictive performance re-evaluation. MAIN OUTCOME MEASURE In-hospital mortality for hospitals and 28-day mortality for primary care and nursing home setting. RESULTS All six prognostic models performed poorly and showed miscalibration in the older population cohorts. In the hospital settings, model performance ranged from calibration-in-the-large -1.45 to 7.46, calibration slopes 0.24-0.81, and C-statistic 0.55-0.71 with 4C Mortality Score performing as the most discriminative and well-calibrated model. Performance across health-care settings was similar for the GAL-COVID-19 model, with a calibration-in-the-large in the range of -2.35 to -0.15 indicating overestimation, calibration slopes of 0.24-0.81 indicating signs of overfitting, and C-statistic of 0.55-0.71. CONCLUSION Our results show that most prognostic models for predicting mortality risk performed poorly in the older population with COVID-19, in each health-care setting: hospital, primary care, and nursing home settings. Insights into factors influencing predictive model performance in the older population are needed for pandemic preparedness and reliable prognostication of health-related outcomes in this demographic.
Collapse
Affiliation(s)
- Anum Zahra
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands.
| | - Maarten van Smeden
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Evertine J Abbink
- Department of Internal Medicine, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Jesse M van den Berg
- Department of General Practice, Amsterdam UMC Location Vrije Universiteit Amsterdam, Amsterdam, The Netherlands; Amsterdam Public Health, Health Behaviors & Chronic Diseases, Amsterdam, The Netherlands; PHARMO Institute for Drug Outcomes Research, Utrecht, The Netherlands
| | - Marieke T Blom
- Department of General Practice, Amsterdam UMC Location Vrije Universiteit Amsterdam, Amsterdam, The Netherlands; Amsterdam Public Health, Health Behaviors & Chronic Diseases, Amsterdam, The Netherlands
| | - Carline J van den Dries
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Jacobijn Gussekloo
- Section Gerontology and Geriatrics, LUMC Center for Medicine for Older People & Department of Public Health and Primary Care & Department of Internal Medicine, Leiden University Medical Center, Leiden, The Netherlands
| | - Fenne Wouters
- Department of Medicine for Older People, Amsterdam UMC, Location Vrije Universiteit Amsterdam, De Boelelaan 1117, Amsterdam, The Netherlands; Amsterdam Public Health Research Institute, Aging & Later Life, Amsterdam, The Netherlands
| | - Karlijn J Joling
- Department of Medicine for Older People, Amsterdam UMC, Location Vrije Universiteit Amsterdam, De Boelelaan 1117, Amsterdam, The Netherlands; Amsterdam Public Health Research Institute, Aging & Later Life, Amsterdam, The Netherlands
| | - René Melis
- Department of Geriatric Medicine, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Simon P Mooijaart
- LUMC Center for Medicine for Older People, LUMC, Leiden, The Netherlands
| | - Jeannette B Peters
- Department of Pulmonary Diseases, Radboud University Medical Center, Radboud Institute for Health Sciences, Nijmegen, The Netherlands
| | - Harmke A Polinder-Bos
- Section of Geriatric Medicine, Department of Internal Medicine, Erasmus MC, University Medical Center Rotterdam, The Netherlands
| | - Bas F M van Raaij
- LUMC Center for Medicine for Older People, LUMC, Leiden, The Netherlands
| | - Brent Appelman
- Amsterdam UMC Location University of Amsterdam, Center for Experimental and Molecular Medicine, Amsterdam, The Netherlands
| | - Hannah M la Roi-Teeuw
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Karel G M Moons
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Kim Luijken
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| |
Collapse
|
2
|
Pellekooren S, Ben ÂJ, van Dongen JM, Pool-Goudzwaard AL, van Tulder MW, van den Berg JM, Ostelo RW. Predicting direct healthcare costs of general practitioner-guided care in patients with musculoskeletal complaints. Pain 2024; 165:404-411. [PMID: 37590126 PMCID: PMC10785053 DOI: 10.1097/j.pain.0000000000003028] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2023] [Revised: 06/23/2023] [Accepted: 07/02/2023] [Indexed: 08/19/2023]
Abstract
ABSTRACT Information on healthcare utilization and costs of general practitioner (GP)-guided care in patients with musculoskeletal complaints is important for keeping healthcare affordable and accessible. A registry-based study was performed to describe healthcare utilization and costs of GP-guided care in patients with musculoskeletal complaints and to predict having higher direct healthcare costs. Healthcare costs of GP-guided care included all healthcare resources used by patients due to a musculoskeletal condition in 2018. Data were extracted from the database with a 1-year follow-up and descriptively analyzed. A general linear model was developed to predict having higher direct healthcare costs. In total, 403,719 patients were included, of whom 92% only received a single consultation. The number of referrals varied across the different types of complaints. Total annual direct healthcare costs amounted to €39,180,531, of which a key cost driver was referrals. Primary care consultations accounted for the largest part of referral-related costs. For all musculoskeletal conditions combined, the mean annual direct healthcare cost per patient was €97 (SEM = €0.18). Older age, being a woman, low socioeconomic status, spine complaints, high number of musculoskeletal diagnoses, and a high comorbidity score were predictive of having higher direct healthcare costs and explained 0.7% of the variance. This study showed that mean annual direct healthcare costs of GP-guided care in patients with musculoskeletal conditions were relatively low and did not differ considerably across conditions. The predictive model explained a negligible part of the variance in costs. Thus, it is unclear which factors do predict high direct healthcare costs in this population.
Collapse
Affiliation(s)
- Sylvia Pellekooren
- Department Human Movement Sciences, Faculty of Behavioral and Movement Sciences, Amsterdam Movement Sciences Research Institute, Amsterdam, the Netherlands
- Department of Health Sciences, Faculty of Science, Vrije Universiteit Amsterdam, Amsterdam Movement Sciences Research Institute Amsterdam, the Netherlands
| | - Ângela J. Ben
- Department of Health Sciences, Faculty of Science, Vrije Universiteit Amsterdam, Amsterdam Public Health Research Institute, Amsterdam, the Netherlands
| | - Johanna M. van Dongen
- Department of Health Sciences, Faculty of Science, Vrije Universiteit Amsterdam, Amsterdam Movement Sciences Research Institute Amsterdam, the Netherlands
- Department of Health Sciences, Faculty of Science, Vrije Universiteit Amsterdam, Amsterdam Public Health Research Institute, Amsterdam, the Netherlands
| | - Annelies L. Pool-Goudzwaard
- Department Human Movement Sciences, Faculty of Behavioral and Movement Sciences, Amsterdam Movement Sciences Research Institute, Amsterdam, the Netherlands
- Somt University of Physiotherapy, Amersfoort, the Netherlands
| | - Maurits W. van Tulder
- Department Human Movement Sciences, Faculty of Behavioral and Movement Sciences, Amsterdam Movement Sciences Research Institute, Amsterdam, the Netherlands
| | - Jesse M. van den Berg
- PHARMO Institute for Drug Outcomes Research, Utrecht, the Netherlands
- Department of General Practice, Amsterdam UMC Location Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
- Amsterdam Public Health, Health Behaviors and Chronic Diseases, Amsterdam, the Netherlands
| | - Raymond W.J.G. Ostelo
- Department of Health Sciences, Faculty of Science, Vrije Universiteit Amsterdam, Amsterdam Movement Sciences Research Institute Amsterdam, the Netherlands
- Department of Epidemiology and Biostatistics, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam Movement Sciences Research Institute, Amsterdam, the Netherlands
| |
Collapse
|
3
|
Zahra A, Luijken K, Abbink EJ, van den Berg JM, Blom MT, Elders P, Festen J, Gussekloo J, Joling KJ, Melis R, Mooijaart S, Peters JB, Polinder-Bos HA, van Raaij BFM, Smorenberg A, la Roi-Teeuw HM, Moons KGM, van Smeden M. A study protocol of external validation of eight COVID-19 prognostic models for predicting mortality risk in older populations in a hospital, primary care, and nursing home setting. Diagn Progn Res 2023; 7:8. [PMID: 37013651 PMCID: PMC10069944 DOI: 10.1186/s41512-023-00144-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/10/2022] [Accepted: 01/27/2023] [Indexed: 04/05/2023] Open
Abstract
BACKGROUND The COVID-19 pandemic has a large impact worldwide and is known to particularly affect the older population. This paper outlines the protocol for external validation of prognostic models predicting mortality risk after presentation with COVID-19 in the older population. These prognostic models were originally developed in an adult population and will be validated in an older population (≥ 70 years of age) in three healthcare settings: the hospital setting, the primary care setting, and the nursing home setting. METHODS Based on a living systematic review of COVID-19 prediction models, we identified eight prognostic models predicting the risk of mortality in adults with a COVID-19 infection (five COVID-19 specific models: GAL-COVID-19 mortality, 4C Mortality Score, NEWS2 + model, Xie model, and Wang clinical model and three pre-existing prognostic scores: APACHE-II, CURB65, SOFA). These eight models will be validated in six different cohorts of the Dutch older population (three hospital cohorts, two primary care cohorts, and a nursing home cohort). All prognostic models will be validated in a hospital setting while the GAL-COVID-19 mortality model will be validated in hospital, primary care, and nursing home settings. The study will include individuals ≥ 70 years of age with a highly suspected or PCR-confirmed COVID-19 infection from March 2020 to December 2020 (and up to December 2021 in a sensitivity analysis). The predictive performance will be evaluated in terms of discrimination, calibration, and decision curves for each of the prognostic models in each cohort individually. For prognostic models with indications of miscalibration, an intercept update will be performed after which predictive performance will be re-evaluated. DISCUSSION Insight into the performance of existing prognostic models in one of the most vulnerable populations clarifies the extent to which tailoring of COVID-19 prognostic models is needed when models are applied to the older population. Such insight will be important for possible future waves of the COVID-19 pandemic or future pandemics.
Collapse
Affiliation(s)
- Anum Zahra
- Julius Center for Health Sciences and Primary Care, University Medical Center, Utrecht University, Utrecht, the Netherlands.
| | - Kim Luijken
- Julius Center for Health Sciences and Primary Care, University Medical Center, Utrecht University, Utrecht, the Netherlands
| | - Evertine J Abbink
- Department of Internal Medicine, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Jesse M van den Berg
- Department of General Practice, Amsterdam UMC Location Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
- Amsterdam Public Health, Health Behaviors & Chronic Diseases, Amsterdam, the Netherlands
- PHARMO Institute for Drug Outcomes Research, Utrecht, the Netherlands
| | - Marieke T Blom
- Department of General Practice, Amsterdam UMC Location Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
- Amsterdam Public Health, Health Behaviors & Chronic Diseases, Amsterdam, the Netherlands
| | - Petra Elders
- Department of General Practice, Amsterdam Public Health Research Institute, Amsterdam UMC, Amsterdam, the Netherlands
| | | | - Jacobijn Gussekloo
- Department of Public Health and Primary Care & Department of Internal Medicine, Section of Gerontology and Geriatrics, Leiden University Medical Center, Leiden, the Netherlands
| | - Karlijn J Joling
- Department of Medicine for Older People, Amsterdam UMC, Location Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
- Amsterdam Public Health, Aging & Later Life, Amsterdam, the Netherlands
| | - René Melis
- Department of Geriatric Medicine, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Simon Mooijaart
- Department of Internal Medicine, Leiden University Medical Center, Leiden, the Netherlands
| | - Jeannette B Peters
- Department of Pulmonary Diseases, Radboud Institute for Health Sciences, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Harmke A Polinder-Bos
- Department of Internal Medicine, Section of Geriatric Medicine, Erasmus MC, University Medical Center Rotterdam, Rotterdam, the Netherlands
| | - Bas F M van Raaij
- Department of Internal Medicine, Leiden University Medical Center, Leiden, the Netherlands
| | - Annemieke Smorenberg
- Department of Internal Medicine, Section of Geriatric Medicine, Amsterdam UMC, Amsterdam, the Netherlands
| | - Hannah M la Roi-Teeuw
- Julius Center for Health Sciences and Primary Care, University Medical Center, Utrecht University, Utrecht, the Netherlands
| | - Karel G M Moons
- Julius Center for Health Sciences and Primary Care, University Medical Center, Utrecht University, Utrecht, the Netherlands
| | - Maarten van Smeden
- Julius Center for Health Sciences and Primary Care, University Medical Center, Utrecht University, Utrecht, the Netherlands
| |
Collapse
|