1
|
Estratificación de riesgo cardiovascular: conceptos, análisis crítico, desafíos e historia de su desarrollo en Chile. REVISTA MÉDICA CLÍNICA LAS CONDES 2022. [DOI: 10.1016/j.rmclc.2022.08.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022] Open
|
2
|
Wessler BS, Nelson J, Park JG, McGinnes H, Gulati G, Brazil R, Van Calster B, van Klaveren D, Venema E, Steyerberg E, Paulus JK, Kent DM. External Validations of Cardiovascular Clinical Prediction Models: A Large-Scale Review of the Literature. Circ Cardiovasc Qual Outcomes 2021; 14:e007858. [PMID: 34340529 PMCID: PMC8366535 DOI: 10.1161/circoutcomes.121.007858] [Citation(s) in RCA: 28] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Abstract
BACKGROUND There are many clinical prediction models (CPMs) available to inform treatment decisions for patients with cardiovascular disease. However, the extent to which they have been externally tested, and how well they generally perform has not been broadly evaluated. METHODS A SCOPUS citation search was run on March 22, 2017 to identify external validations of cardiovascular CPMs in the Tufts Predictive Analytics and Comparative Effectiveness CPM Registry. We assessed the extent of external validation, performance heterogeneity across databases, and explored factors associated with model performance, including a global assessment of the clinical relatedness between the derivation and validation data. RESULTS We identified 2030 external validations of 1382 CPMs. Eight hundred seven (58%) of the CPMs in the Registry have never been externally validated. On average, there were 1.5 validations per CPM (range, 0-94). The median external validation area under the receiver operating characteristic curve was 0.73 (25th-75th percentile [interquartile range (IQR)], 0.66-0.79), representing a median percent decrease in discrimination of -11.1% (IQR, -32.4% to +2.7%) compared with performance on derivation data. 81% (n=1333) of validations reporting area under the receiver operating characteristic curve showed discrimination below that reported in the derivation dataset. 53% (n=983) of the validations report some measure of CPM calibration. For CPMs evaluated more than once, there was typically a large range of performance. Of 1702 validations classified by relatedness, the percent change in discrimination was -3.7% (IQR, -13.2 to 3.1) for closely related validations (n=123), -9.0 (IQR, -27.6 to 3.9) for related validations (n=862), and -17.2% (IQR, -42.3 to 0) for distantly related validations (n=717; P<0.001). CONCLUSIONS Many published cardiovascular CPMs have never been externally validated, and for those that have, apparent performance during development is often overly optimistic. A single external validation appears insufficient to broadly understand the performance heterogeneity across different settings.
Collapse
Affiliation(s)
- Benjamin S Wessler
- Predictive Analytics and Comparative Effectiveness (PACE) (B.S.W., J.N., J.G.P., H.G., G.G., R.B., D.v.K., J.K.P., D.M.K.), Tufts Medical Center, Boston, MA.,Division of Cardiology (B.S.W., G.G.), Tufts Medical Center, Boston, MA
| | - Jason Nelson
- Predictive Analytics and Comparative Effectiveness (PACE) (B.S.W., J.N., J.G.P., H.G., G.G., R.B., D.v.K., J.K.P., D.M.K.), Tufts Medical Center, Boston, MA
| | - Jinny G Park
- Predictive Analytics and Comparative Effectiveness (PACE) (B.S.W., J.N., J.G.P., H.G., G.G., R.B., D.v.K., J.K.P., D.M.K.), Tufts Medical Center, Boston, MA
| | - Hannah McGinnes
- Predictive Analytics and Comparative Effectiveness (PACE) (B.S.W., J.N., J.G.P., H.G., G.G., R.B., D.v.K., J.K.P., D.M.K.), Tufts Medical Center, Boston, MA
| | - Gaurav Gulati
- Predictive Analytics and Comparative Effectiveness (PACE) (B.S.W., J.N., J.G.P., H.G., G.G., R.B., D.v.K., J.K.P., D.M.K.), Tufts Medical Center, Boston, MA.,Division of Cardiology (B.S.W., G.G.), Tufts Medical Center, Boston, MA
| | - Riley Brazil
- Predictive Analytics and Comparative Effectiveness (PACE) (B.S.W., J.N., J.G.P., H.G., G.G., R.B., D.v.K., J.K.P., D.M.K.), Tufts Medical Center, Boston, MA
| | - Ben Van Calster
- KU Leuven, Department of Development and Regeneration, Belgium (B.V.C.)
| | - David van Klaveren
- Predictive Analytics and Comparative Effectiveness (PACE) (B.S.W., J.N., J.G.P., H.G., G.G., R.B., D.v.K., J.K.P., D.M.K.), Tufts Medical Center, Boston, MA.,Department of Biomedical Data Sciences (D.v.K.), Leiden University Medical Centre, Netherlands
| | - Esmee Venema
- Department of Public Health (E.V., E.S.), Erasmus MC University Medical Center, Rotterdam, the Netherlands.,Department of Neurology (E.V.), Erasmus MC University Medical Center, Rotterdam, the Netherlands
| | - Ewout Steyerberg
- Department of Biomedical Data Sciences (E.S.), Leiden University Medical Centre, Netherlands.,Department of Public Health (E.V., E.S.), Erasmus MC University Medical Center, Rotterdam, the Netherlands
| | - Jessica K Paulus
- Predictive Analytics and Comparative Effectiveness (PACE) (B.S.W., J.N., J.G.P., H.G., G.G., R.B., D.v.K., J.K.P., D.M.K.), Tufts Medical Center, Boston, MA
| | - David M Kent
- Predictive Analytics and Comparative Effectiveness (PACE) (B.S.W., J.N., J.G.P., H.G., G.G., R.B., D.v.K., J.K.P., D.M.K.), Tufts Medical Center, Boston, MA
| |
Collapse
|
3
|
Ban JW, Chan MS, Muthee TB, Paez A, Stevens R, Perera R. Design, methods, and reporting of impact studies of cardiovascular clinical prediction rules are suboptimal: a systematic review. J Clin Epidemiol 2021; 133:111-120. [PMID: 33515655 DOI: 10.1016/j.jclinepi.2021.01.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2020] [Revised: 01/08/2021] [Accepted: 01/21/2021] [Indexed: 10/22/2022]
Abstract
OBJECTIVES To evaluate design, methods, and reporting of impact studies of cardiovascular clinical prediction rules (CPRs). STUDY DESIGN AND SETTING We conducted a systematic review. Impact studies of cardiovascular CPRs were identified by forward citation and electronic database searches. We categorized the design of impact studies as appropriate for randomized and nonrandomized experiments, excluding uncontrolled before-after study. For impact studies with appropriate study design, we assessed the quality of methods and reporting. We compared the quality of methods and reporting between impact and matched control studies. RESULTS We found 110 impact studies of cardiovascular CPRs. Of these, 65 (59.1%) used inappropriate designs. Of 45 impact studies with appropriate design, 31 (68.9%) had substantial risk of bias. Mean number of reporting domains that impact studies with appropriate study design adhered to was 10.2 of 21 domains (95% confidence interval, 9.3 and 11.1). The quality of methods and reporting was not clearly different between impact and matched control studies. CONCLUSION We found most impact studies either used inappropriate study design, had substantial risk of bias, or poorly complied with reporting guidelines. This appears to be a common feature of complex interventions. Users of CPRs should critically evaluate evidence showing the effectiveness of CPRs.
Collapse
Affiliation(s)
- Jong-Wook Ban
- Centre for Evidence-Based Medicine, Nuffield Department of Primary Care Health Sciences, University of Oxford, Radcliffe Observatory Quarter, Woodstock Road, Oxford, OX2 6GG, United Kingdom; Department for Continuing Education, University of Oxford, Rewley House, 1 Wellington Square, Oxford, OX1 2JA, United Kingdom.
| | - Mei Sum Chan
- Nuffield Department of Population Health, University of Oxford, Richard Doll Building, Old Road Campus, Oxford, OX3 7LF, United Kingdom
| | - Tonny Brian Muthee
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Radcliffe Observatory Quarter, Woodstock Road, Oxford, OX2 6GG, United Kingdom
| | - Arsenio Paez
- Centre for Evidence-Based Medicine, Nuffield Department of Primary Care Health Sciences, University of Oxford, Radcliffe Observatory Quarter, Woodstock Road, Oxford, OX2 6GG, United Kingdom; Department for Continuing Education, University of Oxford, Rewley House, 1 Wellington Square, Oxford, OX1 2JA, United Kingdom
| | - Richard Stevens
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Radcliffe Observatory Quarter, Woodstock Road, Oxford, OX2 6GG, United Kingdom
| | - Rafael Perera
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Radcliffe Observatory Quarter, Woodstock Road, Oxford, OX2 6GG, United Kingdom
| |
Collapse
|
4
|
Ban JW, Perera R, Stevens R. GPs' familiarity with and use of cardiovascular clinical prediction rules: a UK survey study. BJGP Open 2020; 4:bjgpopen20X101081. [PMID: 33023870 PMCID: PMC7880194 DOI: 10.3399/bjgpopen20x101081] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2020] [Accepted: 02/19/2020] [Indexed: 12/20/2022] Open
Abstract
BACKGROUND Clinical prediction rules (CPRs) can help general practitioners (GPs) address challenges in cardiovascular disease. A survey published in 2014 evaluated GPs' awareness and use of CPRs in the UK. However, many new CPRs have been published since and it is unknown which cardiovascular CPRs are currently recognised and used. AIM To identify cardiovascular CPRs recognised and used by GPs, and to assess how GPs' familiarity and use have changed over time. DESIGN & SETTING An online survey of GPs in the UK was undertaken. METHOD Using comparable methods to the 2014 survey, GPs were recruited from a network of doctors in the UK. They were asked how familiar they were with cardiovascular CPRs, how frequently they used them, and why they used them. The results were compared with the 2014 survey. RESULTS Most of 401 GPs were familiar with QRISK scores, ABCD scores, CHADS scores, HAS-BLED score, Wells scores for deep vein thrombosis, and Wells scores for pulmonary embolism. The proportions of GPs using these CPRs were 96.3%, 65.1%, 97.3%, 93.0%, 92.5%, and 82.0%, respectively. GPs' use increased by 31.2% for QRISK scores, by 13.5% for ABCD scores, by 54.6% for CHADS scores, by 33.2% for Wells scores for deep vein thrombosis, and by 43.6% for Wells scores for pulmonary embolism; and decreased by 45.9% for the Joint British Societies (JBS) risk calculator, by 38.7% for Framingham risk scores, and by 8.7% for New Zealand tables. GPs most commonly used cardiovascular CPRs to guide therapy and referral. CONCLUSION The study found GPs' familiarity and use of cardiovascular CPRs changed substantially. Integrating CPRs into guidelines and practice software might increase familiarity and use.
Collapse
Affiliation(s)
- Jong-Wook Ban
- Evidence-Based Health Care Programme, Centre for Evidence-Based Medicine, University of Oxford, Oxford, UK
| | - Rafael Perera
- Nuffield Department of Primary Care Health Sciences, Medical Science Division, University of Oxford, Oxford, UK
| | - Richard Stevens
- Nuffield Department of Primary Care Health Sciences, Medical Science Division, University of Oxford, Oxford, UK
| |
Collapse
|
5
|
Cocomello L, Caputo M, Cornish R, Lawlor D. External validation of the improving partial risk adjustment in surgery (PRAIS-2) model for 30-day mortality after paediatric cardiac surgery. BMJ Open 2020; 10:e039236. [PMID: 33247009 PMCID: PMC7703410 DOI: 10.1136/bmjopen-2020-039236] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/31/2022] Open
Abstract
OBJECTIVE Independent temporal external validation of the improving partial risk adjustment in surgery model (PRAIS-2) to predict 30-day mortality in patients undergoing paediatric cardiac surgery. DESIGN Retrospective analysis of prospectively collected data. SETTING Paediatric cardiac surgery. INTERVENTION PRAIS-2 validation was carried out using a two temporally different single centre (Bristol, UK) cohorts: Cohort 1 surgery undertaken from April 2004 to March 2009 and Cohort 2 from April 2015 to July 2019. For each subject PRAIS-2 score was calculated according to the original formula. PARTICIPANTS A total of 1352 (2004-2009) and 1197 (2015-2019) paediatric cardiac surgical procedures were included in the Cohort 1 and Cohort 2, respectively (median age at the procedure 6.3 and 7.1 months). PRIMARY AND SECONDARY OUTCOME MEASURES PRAIS-2 performance was assessed in terms of discrimination by means of ROC (receiver operating characteristic) curve analysis and calibration by using the calibration belt method. RESULTS PRAIS-2 score showed excellent discrimination for both cohorts (AUC 0.72 (95%CI: 0.65 to 0.80) and 0.88 (95%CI: 0.82 to 0.93), respectively). While PRAIS-2 was only marginally calibrated in Cohort 1, with a tendency to underestimate risk in lowrisk and overestimate risk in high risk procedures (P-value = 0.033), validation in Cohort 2 showed good calibration with the 95% confidence belt containing the bisector for predicted mortality (P-value = 0.143). We also observed good prediction accuracy in the non-elective procedures (N = 483;AUC 0.78 (95%CI 0.68 to 0.87); Calibration belt containing the bisector (P-value=0.589). CONCLUSIONS In a single centre UK-based cohort, PRAIS-2 showed excellent discrimination and calibration in predicting 30-day mortality in paediatric cardiac surgery including in those undergoing non-elective procedures. Our results support a wider adoption of PRAIS-2 score in the clinical practice.
Collapse
Affiliation(s)
- Lucia Cocomello
- MRC Integrative Epidemiology Unit, University of Bristol, Bristol, UK
| | - Massimo Caputo
- Bristol Heart Institute, University of Bristol, Bristol, UK
| | - Rosie Cornish
- Population Health Science, Bristol Medical School University of Bristol, Bristol, United Kingdom
| | - Deborah Lawlor
- MRC Integrative Epidemiology Unit, Department of Social Medicine, University of Bristol, Bristol, UK
| |
Collapse
|
6
|
Wallisch C, Heinze G, Rinner C, Mundigler G, Winkelmayer WC, Dunkler D. Re-estimation improved the performance of two Framingham cardiovascular risk equations and the Pooled Cohort equations: A nationwide registry analysis. Sci Rep 2020; 10:8140. [PMID: 32424214 PMCID: PMC7235230 DOI: 10.1038/s41598-020-64629-6] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2019] [Accepted: 04/17/2020] [Indexed: 12/04/2022] Open
Abstract
Equations predicting the risk of occurrence of cardiovascular disease (CVD) are used in primary care to identify high-risk individuals among the general population. To improve the predictive performance of such equations, we updated the Framingham general CVD 1991 and 2008 equations and the Pooled Cohort equations for atherosclerotic CVD within five years in a contemporary cohort of individuals who participated in the Austrian health-screening program from 2009-2014. The cohort comprised 1.7 M individuals aged 30-79 without documented CVD history. CVD was defined by hospitalization or death from cardiovascular cause. Using baseline and follow-up data, we recalibrated and re-estimated the equations. We evaluated the gain in discrimination and calibration and assessed explained variation. A five-year general CVD risk of 4.61% was observed. As expected, discrimination c-statistics increased only slightly and ranged from 0.73-0.79. The two original Framingham equations overestimated the CVD risk, whereas the original Pooled Cohort equations underestimated it. Re-estimation improved calibration of all equations adequately, especially for high-risk individuals. Half of the individuals were reclassified into another risk category using the re-estimated equations. Predictors in the re-estimated Framingham equations explained 7.37% of the variation, whereas the Pooled Cohort equations explained 5.81%. Age was the most important predictor.
Collapse
Affiliation(s)
- Christine Wallisch
- Medical University of Vienna, CEMSIIS, Section for Clinical Biometrics, Vienna, Austria
| | - Georg Heinze
- Medical University of Vienna, CEMSIIS, Section for Clinical Biometrics, Vienna, Austria
| | - Christoph Rinner
- Medical University of Vienna, CEMSIIS, Section for Medical Information Management, Vienna, Austria
| | - Gerald Mundigler
- Medical University of Vienna, Department of Medicine 2, Division of Cardiology, Vienna, Austria
| | | | - Daniela Dunkler
- Medical University of Vienna, CEMSIIS, Section for Clinical Biometrics, Vienna, Austria.
| |
Collapse
|
7
|
Haller MC, Wallisch C, Mjøen G, Holdaas H, Dunkler D, Heinze G, Oberbauer R. Predicting donor, recipient and graft survival in living donor kidney transplantation to inform pretransplant counselling: the donor and recipient linked iPREDICTLIVING tool - a retrospective study. Transpl Int 2020; 33:729-739. [PMID: 31970822 PMCID: PMC7383676 DOI: 10.1111/tri.13580] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2019] [Revised: 10/23/2019] [Accepted: 01/17/2020] [Indexed: 01/02/2023]
Abstract
Although separate prediction models for donors and recipients were previously published, we identified a need to predict outcomes of donor/recipient simultaneously, as they are clearly not independent of each other. We used characteristics from transplantations performed at the Oslo University Hospital from 1854 live donors and from 837 recipients of a live donor kidney transplant to derive Cox models for predicting donor mortality up to 20 years, and recipient death, and graft loss up to 10 years. The models were developed using the multivariable fractional polynomials algorithm optimizing Akaike’s information criterion, and optimism‐corrected performance was assessed. Age, year of donation, smoking status, cholesterol and creatinine were selected to predict donor mortality (C‐statistic of 0.81). Linear predictors for donor mortality served as summary of donor prognosis in recipient models. Age, sex, year of transplantation, dialysis vintage, primary renal disease, cerebrovascular disease, peripheral vascular disease and HLA mismatch were selected to predict recipient mortality (C‐statistic of 0.77). Age, dialysis vintage, linear predictor of donor mortality, HLA mismatch, peripheral vascular disease and heart disease were selected to predict graft loss (C‐statistic of 0.66). Our prediction models inform decision‐making at the time of transplant counselling and are implemented as online calculators.
Collapse
Affiliation(s)
- Maria C Haller
- Section for Clinical Biometrics, Center for Medical Statistics, Informatics and Intelligent Systems (CeMSIIS), Medical University of Vienna, Vienna, Austria.,Nephrology, Ordensklinikum Linz, Elisabethinen, Linz, Austria
| | - Christine Wallisch
- Section for Clinical Biometrics, Center for Medical Statistics, Informatics and Intelligent Systems (CeMSIIS), Medical University of Vienna, Vienna, Austria
| | - Geir Mjøen
- Department of Transplant Medicine, Oslo University Hospital, Oslo, Norway
| | - Hallvard Holdaas
- Department of Transplant Medicine, Oslo University Hospital, Oslo, Norway
| | - Daniela Dunkler
- Section for Clinical Biometrics, Center for Medical Statistics, Informatics and Intelligent Systems (CeMSIIS), Medical University of Vienna, Vienna, Austria
| | - Georg Heinze
- Section for Clinical Biometrics, Center for Medical Statistics, Informatics and Intelligent Systems (CeMSIIS), Medical University of Vienna, Vienna, Austria
| | - Rainer Oberbauer
- Division of Nephrology and Dialysis, Department of Medicine III, Medical University of Vienna, Vienna, Austria
| |
Collapse
|
8
|
Cowley LE, Farewell DM, Maguire S, Kemp AM. Methodological standards for the development and evaluation of clinical prediction rules: a review of the literature. Diagn Progn Res 2019; 3:16. [PMID: 31463368 PMCID: PMC6704664 DOI: 10.1186/s41512-019-0060-y] [Citation(s) in RCA: 120] [Impact Index Per Article: 24.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/13/2018] [Accepted: 05/12/2019] [Indexed: 12/20/2022] Open
Abstract
Clinical prediction rules (CPRs) that predict the absolute risk of a clinical condition or future outcome for individual patients are abundant in the medical literature; however, systematic reviews have demonstrated shortcomings in the methodological quality and reporting of prediction studies. To maximise the potential and clinical usefulness of CPRs, they must be rigorously developed and validated, and their impact on clinical practice and patient outcomes must be evaluated. This review aims to present a comprehensive overview of the stages involved in the development, validation and evaluation of CPRs, and to describe in detail the methodological standards required at each stage, illustrated with examples where appropriate. Important features of the study design, statistical analysis, modelling strategy, data collection, performance assessment, CPR presentation and reporting are discussed, in addition to other, often overlooked aspects such as the acceptability, cost-effectiveness and longer-term implementation of CPRs, and their comparison with clinical judgement. Although the development and evaluation of a robust, clinically useful CPR is anything but straightforward, adherence to the plethora of methodological standards, recommendations and frameworks at each stage will assist in the development of a rigorous CPR that has the potential to contribute usefully to clinical practice and decision-making and have a positive impact on patient care.
Collapse
Affiliation(s)
- Laura E. Cowley
- Division of Population Medicine, School of Medicine, Neuadd Meirionnydd, Heath Park, Cardiff University, Wales, CF14 4YS UK
| | - Daniel M. Farewell
- Division of Population Medicine, School of Medicine, Neuadd Meirionnydd, Heath Park, Cardiff University, Wales, CF14 4YS UK
| | - Sabine Maguire
- Division of Population Medicine, School of Medicine, Neuadd Meirionnydd, Heath Park, Cardiff University, Wales, CF14 4YS UK
| | - Alison M. Kemp
- Division of Population Medicine, School of Medicine, Neuadd Meirionnydd, Heath Park, Cardiff University, Wales, CF14 4YS UK
| |
Collapse
|