Methodological standards for the development and evaluation of clinical prediction rules: a review of the literature

Development and Validation of a Multivariable Exercise Adherence Prediction Model for Patients with COPD: A Prospective Cohort Study

Purpose

Pulmonary rehabilitation (PR) is considered a cost-effective method of improving health-related quality of life in patients with chronic obstructive pulmonary disease (COPD). However, increasing demand and increasing costs of supply demands for sustainable and affordable care. One of the possible solutions to keep care affordable is self-management. A challenge here is non-adherence. Understanding who are adherent and who are non-adherent could be helpful to differentiate between patients who need more or less support. Therefore, the aim of this study was to develop and validate a model to predict adherence to PR in patients with COPD.

Patients and methods

A multivariable logistic regression model for exercise adherence was developed. Eight candidate predictors, that were prespecified, were obtained in a prospective cohort study from 196 patients with COPD following PR in 53 primary physiotherapy practices in the Netherlands and Belgium, between January 2021 and August 2022. To create a parsimonious model, variable selection using backward selection was performed with a p-value of >0.05 for elimination. Model performance was assessed by discrimination, calibration and clinical utility. Internal validation was assessed by bootstrapping (n = 500).

Results

The final model included four predictors: intention, depression, MRC-score and alliance. The optimism-corrected AUC after bootstrap internal validation was 0.79 (95% CI, 0.72-0.85). Calibration plots suggested good calibration and decision curve analysis showed great net benefit in a wide range of risk thresholds.

Conclusion

The exercise adherence prediction model has potential for clinical utility to predict adherence in patients with COPD. Information from such a model can be used to manage the patient instead of managing the disease, and thereby to determine the treatment frequency for each individual patient. As a result, healthcare capacity might be better distributed, potentially reducing pressure on healthcare without compromising the effectiveness of PR for the individual patient.

Severity of Illness Scores and Biomarkers for Prognosis of Patients with Coronavirus Disease 2019

The spectrum of disease severity and the insidiousness of clinical presentation make it difficult to recognize patients with coronavirus disease 2019 (COVID-19) at higher risk of worse outcomes or death when they are seen in the early phases of the disease. There are now well-established risk factors for worse outcomes in patients with COVID-19. These should be factored in when assessing the prognosis of these patients. However, a more precise prognostic assessment in an individual patient may warrant the use of predictive tools. In this manuscript, we conduct a literature review on the severity of illness scores and biomarkers for the prognosis of patients with COVID-19. Several COVID-19-specific scores have been developed since the onset of the pandemic. Some of them are promising and can be integrated into the assessment of these patients. We also found that the well-known pneumonia severity index (PSI) and CURB-65 (confusion, uremia, respiratory rate, BP, age ≥ 65 years) are good predictors of mortality in hospitalized patients with COVID-19. While neither the PSI nor the CURB-65 should be used for the triage of outpatient versus inpatient treatment, they can be integrated by a clinician into the assessment of disease severity and can be used in epidemiological studies to determine the severity of illness in patient populations. Biomarkers also provide valuable prognostic information and, importantly, may depict the main physiological derangements in severe disease. We, however, do not advocate the isolated use of severity of illness scores or biomarkers for decision-making in an individual patient. Instead, we suggest the use of these tools on a case-by-case basis with the goal of enhancing clinician judgment.

External validation of the ADA score for predicting thrombosis among acutely ill hospitalized medical patients from the APEX Trial

The ADA (Age-D-dimer-Albumin) score was developed to identify hospitalized patients at an increased risk for thrombosis in the coronavirus infectious disease-19 (COVID-19) setting. The study aimed to validate the ADA score for predicting thrombosis in a non-COVID-19 medically ill population from the APEX trial. The APEX trial was a multinational, randomized trial that evaluated the efficacy and safety of betrixaban vs. enoxaparin among acutely ill hospitalized patients at risk for venous thromboembolism. The study endpoints included the composite of arterial or venous thrombosis and its components. Metrics of model calibration and discrimination were computed for assessing the performance of the ADA score as compared to the IMPROVE score, a well-validated VTE risk assessment model. Among 7,119 medical inpatients, 209 (2.9%) had a thrombosis event up to 77 days of follow-up. The ADA score demonstrated good calibration for both arterial and venous thrombosis, whereas the IMPROVE score had adequate calibration for venous thrombosis (p > 0.05 from the Hosmer-Lemeshow test). For discriminating arterial and venous thrombosis, there was no significant difference between the ADA vs. IMPROVE score (c statistic = 0.620 [95% CI: 0.582 to 0.657] vs. 0.590 [95% CI: 0.556 to 0.624]; ∆ c statistic = 0.030 [95% CI: -0.022 to 0.081]; p = 0.255). Similarly, for discriminating arterial thrombosis, there was no significant difference between the ADA vs. IMPROVE score (c statistic = 0.582 [95% CI: 0.534 to 0.629] vs. 0.609 [95% CI: 0.564 to 0.653]; ∆ c statistic = -0.027 [95% CI: -0.091 to 0.036]; p = 0.397). For discriminating venous thrombosis, the ADA score was modestly superior to the IMPROVE score (c statistic = 0.664 [95% CI: 0.607 to 0.722] vs. 0.573 [95% CI: 0.521 to 0.624]; ∆ c statistic = 0.091 [95% CI: 0.011 to 0.172]; p = 0.026). The ADA score had a higher sensitivity (0.579 [95% CI: 0.512 to 0.646]; vs. 0.440 [95% CI: 0.373 to 0.507]) but lower specificity (0.625 [95% CI: 0.614 to 0.637] vs. 0.747 [95% CI: 0.737 to 0.758]) than the IMPROVE score for predicting thrombosis. Among acutely ill hospitalized medical patients enrolled in the APEX trial, the ADA score demonstrated good calibration but suboptimal discrimination for predicting thrombosis. The findings support the use of either the ADA or IMPROVE score for thrombosis risk assessment. The applicability of the ADA score to non-COVID-19 populations warrants further research.Clinical Trial Registration: http://www.clinicaltrials.gov . Unique identifier: NCT01583218.

Healthcare Providers and Parents Highlight Challenges of Pediatric Hand Fracture Care

Pediatric hand fractures are common, and many are referred to hand surgeons despite less than 10% of referrals requiring surgical intervention. We explored healthcare provider and parent perspectives to inform a new care pathway.

Methods

We conducted a qualitative descriptive study using virtual focus groups. Emergency physicians, hand therapists, plastic surgeons, and parents of children treated for hand fractures were asked to discuss their experiences with existing care for pediatric hand fractures, and perceptions surrounding the implementation of a new care pathway. Data were analyzed using directed content analysis with an inductive approach.

Results

Four focus groups included 24 participants: 18 healthcare providers and six parents. Four themes were identified: educating parents throughout the hand fracture journey, streamlining the referral process for simple hand fractures, identifying the most appropriate care provider for simple hand fractures, and maintaining strong multidisciplinary connections to facilitate care. Participants described gaps in the current care, including a need to better inform parents, and elucidated the motivations behind emergency medicine physicians' existing referral practices. Participants also generally agreed on the need for more efficient management of simple hand fractures that do not require surgical care. Healthcare providers believed the strong preexisting relationship between surgeons and hand therapists would facilitate the changes brought forward by the new care pathway.

Conclusion

These findings highlighted shortcomings of existing care for pediatric hand fractures and will inform the co-development and implementation of a new care pathway to enable more efficient management while preserving good patient outcomes.

Delivery of a therapeutic antibody to the lower gastrointestinal tract for the treatment of Clostridium difficile infection (CDI)

The colonic delivery system of toxin neutralizing antibody is a promising method for treating Clostridium difficile infection (CDI) and has some advantages over the parental administration of a neutralizing antibody. However, colonic delivery of biologics presents several challenges, including instability of biologics during encapsulation into the delivery system and harsh conditions in the upper GI tract. In this work, we described a multi-particulate delivery system encapsulating a tetra-valent antibody ABAB-IgG1 with the potential to treat CDI. This work first approved that the cecum injection of ABAB-IgG1 into the lower GI tract of mice could relieve the symptoms, enhance the clinical score, and improve the survival rate of mice during CDI. Then, the antibody was spray layered onto mannitol beads and then enteric coated with pH-sensitive polymers to achieve colon-targeting release. The in vitro release of antibody from the multi-particulate system and the pH-sensitive release of antibody was monitored. The in vivo efficacy of this system was further examined and confirmed in mice and hamsters. In summary, the findings of this study should provide practical information and potential treatment options for CDI through colonic delivery of antibody therapeutics to the lower GI tract using a multi-particulate delivery system.

Use of machine learning in pediatric surgical clinical prediction tools: A systematic review

PURPOSE

Clinical prediction tools (CPTs) are decision-making instruments utilizing patient data to predict specific clinical outcomes, risk-stratify patients, or suggest personalized diagnostic or therapeutic options. Recent advancements in artificial intelligence have resulted in a proliferation of CPTs created using machine learning (ML)-yet the clinical applicability of ML-based CPTs and their validation in clinical settings remain unclear. This systematic review aims to compare the validity and clinical efficacy of ML-based to traditional CPTs in pediatric surgery.

METHODS

Nine databases were searched from 2000 until July 9, 2021 to retrieve articles reporting on CPTs and ML for pediatric surgical conditions. PRISMA standards were followed, and screening was performed by two independent reviewers in Rayyan, with a third reviewer resolving conflicts. Risk of bias was assessed using the PROBAST.

RESULTS

Out of 8300 studies, 48 met the inclusion criteria. The most represented surgical specialties were pediatric general (14), neurosurgery (13) and cardiac surgery (12). Prognostic (26) CPTs were the most represented type of surgical pediatric CPTs followed by diagnostic (10), interventional (9), and risk stratifying (2). One study included a CPT for diagnostic, interventional and prognostic purposes. 81% of studies compared their CPT to ML-based CPTs, statistical CPTs, or the unaided clinician, but lacked external validation and/or evidence of clinical implementation.

CONCLUSIONS

While most studies claim significant potential improvements by incorporating ML-based CPTs in pediatric surgical decision-making, both external validation and clinical application remains limited. Further studies must focus on validating existing instruments or developing validated tools, and incorporating them in the clinical workflow.

TYPE OF STUDY

Systematic Review LEVEL OF EVIDENCE: Level III.

Validity of constructed-response situational judgment tests in training programs for the health professions: A systematic review and meta-analysis protocol

BACKGROUND

Situational judgments tests have been increasingly used to help training programs for the health professions incorporate professionalism attributes into their admissions process. While such tests have strong psychometric properties for testing professional attributes and are feasible to implement in high-volume, high-stakes selection, little is known about constructed-response situational judgment tests and their validity.

METHODS

We will conduct a systematic review of primary published or unpublished studies reporting on the association between scores on constructed-response situational judgment tests and scores on other tests that measure personal, interpersonal, or professional attributes in training programs for the health professions. In addition to searching electronic databases, we will contact academics and researchers and undertake backward and forward searching. Two reviewers will independently screen the papers and decide on their inclusion, first based on the titles and abstracts of all citations, and then according to the full texts. Data extraction will be done independently by two reviewers using a data extraction form to chart study details and key findings. Studies will be assessed for the risk of bias and quality by two reviewers using the "Quality In Prognosis Studies" tool. To synthesize evidence, we will test the statistical heterogeneity and conduct a psychometric meta-analysis using a random-effects model. If adequate data are available, we will explore whether the meta-analytic correlation varies across different subgroups (e.g., race, gender).

DISCUSSION

The findings of this study will inform best practices for admission and selection of applicants for training programs for the health professions and encourage further research on constructed-response situational judgment tests, in particular their validity.

TRIAL REGISTRATION

The protocol for this systematic review has been registered in PROSPERO [CRD42022314561]. https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022314561.

Prehospital ABC (Age, Bystander and Cardiogram) scoring system to predict neurological outcomes of cardiopulmonary arrest on arrival: post hoc analysis of a multicentre prospective observational study

BACKGROUND

There is currently limited evidence to guide prehospital identification of patients with cardiopulmonary arrest on arrival (CPAOA) to hospital who have potentially favourable neurological function. This study aimed to develop a simple scoring system that can be determined at the contact point with emergency medical services to predict neurological outcomes.

METHODS

We analysed data from patients with CPAOA using a regional Japanese database (SOS-KANTO), from January 2012 to March 2013. Patients were randomly assigned into derivation and validation cohorts. Favourable neurological outcomes were defined as cerebral performance category 1 or 2. We developed a new scoring system using logistic regression analysis with the following predictors: age, no-flow time, initial cardiac rhythm and arrest place. The model was internally validated by assessing discrimination and calibration.

RESULTS

Among 4907 patients in the derivation cohort and 4908 patients in the validation cohort, the probabilities of favourable outcome were 0.9% and 0.8%, respectively. In the derivation cohort, age ≤70 years (OR 5.11; 95% CI 2.35 to 11.14), no-flow time ≤5 min (OR 4.06; 95% CI 2.06 to 8.01) and ventricular tachycardia or fibrillation as initial cardiac rhythm (OR 6.66; 95% CI 3.45 to 12.88) were identified as predictors of favourable outcome. The ABC score consisting of Age, information from Bystander and Cardiogram was created. The areas under the receiver operating characteristic curves of this score were 0.863 in the derivation and 0.885 in the validation cohorts. Positive likelihood ratios were 6.15 and 6.39 in patients with scores >2 points and were 11.06 and 17.75 in those with 3 points.

CONCLUSION

The ABC score showed good accuracy for predicting favourable neurological outcomes in patients with CPAOA. This simple scoring system could potentially be used to select patients for extracorporeal cardiopulmonary resuscitation and minimise low-flow time.

Development and Validation of Predictive Model for a Diagnosis of First Episode Psychosis Using the Multinational EU-GEI Case-control Study and Modern Statistical Learning Methods

Background and Hypothesis

It is argued that availability of diagnostic models will facilitate a more rapid identification of individuals who are at a higher risk of first episode psychosis (FEP). Therefore, we developed, evaluated, and validated a diagnostic risk estimation model to classify individual with FEP and controls across six countries.

Study Design

We used data from a large multi-center study encompassing 2627 phenotypically well-defined participants (aged 18-64 years) recruited from six countries spanning 17 research sites, as part of the European Network of National Schizophrenia Networks Studying Gene-Environment Interactions study. To build the diagnostic model and identify which of important factors for estimating an individual risk of FEP, we applied a binary logistic model with regularization by the least absolute shrinkage and selection operator. The model was validated employing the internal-external cross-validation approach. The model performance was assessed with the area under the receiver operating characteristic curve (AUROC), calibration, sensitivity, and specificity.

Study Results

Having included preselected 22 predictor variables, the model was able to discriminate adults with FEP and controls with high accuracy across all six countries (rangesAUROC = 0.84-0.86). Specificity (range = 73.9-78.0%) and sensitivity (range = 75.6-79.3%) were equally good, cumulatively indicating an excellent model accuracy; though, calibration slope for the diagnostic model showed a presence of some overfitting when applied specifically to participants from France, the UK, and The Netherlands.

Conclusions

The new FEP model achieved a good discrimination and good calibration across six countries with different ethnic contributions supporting its robustness and good generalizability.

Retained Food During Esophagogastroduodenoscopy Is a Risk Factor for Gastric-to-Pulmonary Aspiration

BACKGROUND

Residual food (RF) during esophagogastroduodenoscopy (EGD) is thought, but not proven, to be a risk factor for gastric-to-pulmonary aspiration.

AIMS

The aims of this study were to determine the prevalence of RF during EGD, to investigate whether RF was associated with an increased risk of aspiration, especially when monitored anesthesia care (MAC) or general anesthesia (GA) were administered, and to determine whether aspiration associated with RF led to a more severe clinical outcome.

METHODS

Patients undergoing EGD between October 2012 and September 2018 were identified. Patient age, sex, aspiration events, RF, sedation type, structural foregut abnormalities, and diagnoses associated with impaired esophageal or gastric motility were noted. The clinical course after an aspiration event was evaluated.

RESULTS

RF was identified during 4% of 81,367 EGDs. Aspiration events occurred during 41 (5/10,000) procedures. Aspiration was more likely to occur in patients with RF (odds ratio [OR] 15.1) or those receiving MAC or GA (OR 9.6 and 16.8 relative to conscious sedation, respectively). RF and MAC/GA were synergistically associated with increased odds of aspiration. In a multivariate nominal logistic regression model, older age (OR 2.6), MAC (OR 3.8), GA (OR 4.4), vagotomy (OR 5.2), achalasia (OR 3.8), and RF (OR 10.0) were risk factors for aspiration. Aspiration events in the presence or absence of RF led to similar clinical outcomes.

CONCLUSIONS

While aspiration events are rare in patients undergoing EGD, RF and the use of MAC or GA were associated with substantially increased odds of aspiration.

FAIL-T (AFP, AST, tumor sIze, ALT, and Tumor number): a model to predict intermediate-stage HCC patients who are not good candidates for TACE

Background

Patients with un-resectable hepatocellular carcinoma (HCC) treated with transarterial chemoembolization (TACE) are a diverse group with varying overall survival (OS). Despite the availability of several scoring systems for predicting OS, one of the unsolved problems is identifying patients who might not benefit from TACE. We aim to develop and validate a model for identifying HCC patients who would survive <6 months after their first TACE.

Methods

Patients with un-resectable HCC, BCLC stage 0-B, who received TACE as their first and only treatment between 2007 and 2020 were included in this study. Before the first TACE, demographic data, laboratory data, and tumor characteristics were obtained. Eligible patients were randomly allocated in a 2:1 ratio to training and validation sets. The former was used for model development using stepwise multivariate logistic regression, and the model was validated in the latter set.

Results

A total of 317 patients were included in the study (210 for the training set and 107 for the validation set). The baseline characteristics of the two sets were comparable. The final model (FAIL-T) included AFP, AST, tumor sIze, ALT, and Tumor number. The FAIL-T model yielded AUROCs of 0.855 and 0.806 for predicting 6-month mortality after TACE in the training and validation sets, respectively, while the "six-and-twelve" score showed AUROCs of 0.751 (P < 0.001) in the training set and 0.729 (P = 0.099) in the validation sets for the same purpose.

Conclusion

The final model is useful for predicting 6-month mortality in naive HCC patients undergoing TACE. HCC patients with high FAIL-T scores may not benefit from TACE, and other treatment options, if available, should be considered.

Development of a Clinical Prediction Rule for Treatment Success with Transcranial Direct Current Stimulation for Knee Osteoarthritis Pain: A Secondary Analysis of a Double-Blind Randomized Controlled Trial

The study’s objective was to develop a clinical prediction rule that predicts a clinically significant analgesic effect on chronic knee osteoarthritis pain after transcranial direct current stimulation treatment. This is a secondary analysis from a double-blind randomized controlled trial. Data from 51 individuals with chronic knee osteoarthritis pain and an impaired descending pain inhibitory system were used. The intervention comprised a 15-session protocol of anodal primary motor cortex transcranial direct current stimulation. Treatment success was defined by the Western Ontario and McMaster Universities’ Osteoarthritis Index pain subscale. Accuracy statistics were calculated for each potential predictor and for the final model. The final logistic regression model was statistically significant (p < 0.01) and comprised five physical and psychosocial predictor variables that together yielded a positive likelihood ratio of 14.40 (95% CI: 3.66−56.69) and an 85% (95%CI: 60−96%) post-test probability of success. This is the first clinical prediction rule proposed for transcranial direct current stimulation in patients with chronic pain. The model underscores the importance of both physical and psychosocial factors as predictors of the analgesic response to transcranial direct current stimulation treatment. Validation of the proposed clinical prediction rule should be performed in other datasets.

Clinical risk calculators informing the decision to admit: A methodologic evaluation and assessment of applicability

INTRODUCTION

Clinical prediction and decision tools that generate outcome-based risk stratification and/or intervention recommendations are prevalent. Appropriate use and validity of these tools, especially those that inform complex clinical decisions, remains unclear. The objective of this study was to assess the methodologic quality and applicability of clinical risk scoring tools used to guide hospitalization decision-making.

METHODS

In February 2021, a comprehensive search was performed of a clinical calculator online database (mdcalc.com) that is publicly available and well-known to clinicians. The primary reference for any calculator tool informing outpatient versus inpatient disposition was considered for inclusion. Studies were restricted to the adult, acute care population. Those focused on obstetrics/gynecology or critical care admission were excluded. The Wasson-Laupacis framework of methodologic standards for clinical prediction rules was applied to each study.

RESULTS

A total of 22 calculators provided hospital admission recommendations for 9 discrete medical conditions using adverse events (14/22), mortality (6/22), or confirmatory diagnosis (2/22) as outcomes of interest. The most commonly met methodologic standards included mathematical technique description (22/22) and clinical sensibility (22/22) and least commonly met included reproducibility of the rule (1/22) and measurement of effect on clinical use (1/22). Description of the studied population was often lacking, especially patient race/ethnicity (2/22) and mental or behavioral health (0/22). Only one study reported any item related to social determinants of health.

CONCLUSION

Studies commonly do not meet rigorous methodologic standards and often fail to report pertinent details that would guide applicability. These clinical tools focus primarily on specific disease entities and clinical variables, missing the breadth of information necessary to make a disposition determination and raise significant validation and generalizability concerns.

Methodological guidance for the evaluation and updating of clinical prediction models: a systematic review

BACKGROUND

Clinical prediction models are often not evaluated properly in specific settings or updated, for instance, with information from new markers. These key steps are needed such that models are fit for purpose and remain relevant in the long-term. We aimed to present an overview of methodological guidance for the evaluation (i.e., validation and impact assessment) and updating of clinical prediction models.

METHODS

We systematically searched nine databases from January 2000 to January 2022 for articles in English with methodological recommendations for the post-derivation stages of interest. Qualitative analysis was used to summarize the 70 selected guidance papers.

RESULTS

Key aspects for validation are the assessment of statistical performance using measures for discrimination (e.g., C-statistic) and calibration (e.g., calibration-in-the-large and calibration slope). For assessing impact or usefulness in clinical decision-making, recent papers advise using decision-analytic measures (e.g., the Net Benefit) over simplistic classification measures that ignore clinical consequences (e.g., accuracy, overall Net Reclassification Index). Commonly recommended methods for model updating are recalibration (i.e., adjustment of intercept or baseline hazard and/or slope), revision (i.e., re-estimation of individual predictor effects), and extension (i.e., addition of new markers). Additional methodological guidance is needed for newer types of updating (e.g., meta-model and dynamic updating) and machine learning-based models.

CONCLUSION

Substantial guidance was found for model evaluation and more conventional updating of regression-based models. An important development in model evaluation is the introduction of a decision-analytic framework for assessing clinical usefulness. Consensus is emerging on methods for model updating.

Predictors and outcomes in primary depression care (POKAL) - a research training group develops an innovative approach to collaborative care

BACKGROUND

The interdisciplinary research training group (POKAL) aims to improve care for patients with depression and multimorbidity in primary care. POKAL includes nine projects within the framework of the Chronic Care Model (CCM). In addition, POKAL will train young (mental) health professionals in research competences within primary care settings. POKAL will address specific challenges in diagnosis (reliability of diagnosis, ignoring suicidal risks), in treatment (insufficient patient involvement, highly fragmented care and inappropriate long-time anti-depressive medication) and in implementation of innovations (insufficient guideline adherence, use of irrelevant patient outcomes, ignoring relevant context factors) in primary depression care.

METHODS

In 2021 POKAL started with a first group of 16 trainees in general practice (GPs), pharmacy, psychology, public health, informatics, etc. The program is scheduled for at least 6 years, so a second group of trainees starting in 2024 will also have three years of research-time. Experienced principal investigators (PIs) supervise all trainees in their specific projects. All projects refer to the CCM and focus on the diagnostic, therapeutic, and implementation challenges.

RESULTS

The first cohort of the POKAL research training group will develop and test new depression-specific diagnostics (hermeneutical strategies, predicting models, screening for suicidal ideation), treatment (primary-care based psycho-education, modulating factors in depression monitoring, strategies of de-prescribing) and implementation in primary care (guideline implementation, use of patient-assessed data, identification of relevant context factors). Based on those results the second cohort of trainees and their PIs will run two major trials to proof innovations in primary care-based a) diagnostics and b) treatment for depression.

CONCLUSION

The research and training programme POKAL aims to provide appropriate approaches for depression diagnosis and treatment in primary care.

Development and internal validation of a clinical prediction model for the diagnosis of immune thrombocytopenia

BACKGROUND

Immune thrombocytopenia (ITP) is a diagnosis of exclusion that can resemble other thrombocytopenic disorders.

OBJECTIVES

To develop a clinical prediction model (CPM) for the diagnosis of ITP to aid hematogists in investigating patients presenting with undifferentiated thrombocytopenia.

METHODS

We designed a CPM for ITP diagnosis at the time of the initial hematology consultation using penalized logistic regression based on data from patients with thrombocytopenia enrolled in the McMaster ITP registry (n = 523) called the Predict-ITP Tool. The case definition for ITP was a platelet count less than 100 × 109 /L and a platelet count response after high-dose corticosteroids or intravenous immune globulin, defined as the achievement of a platelet count above 50 × 109 /L and at least a doubling of baseline. Internal validation was done using bootstrap resampling. Model discrimination was assessed by the c-statistic, and calibration was assessed by the calibration slope, calibration-in-the-large, and calibration plot.

RESULTS

The final model included the following variables: (1) platelet count variability (based on three or more platelet count values), (2) lowest platelet count value, (3) maximum mean platelet volume, and (4) history of major bleeding (defined by the ITP bleeding scale). The optimism-corrected c-statistic was 0.83, the calibration slope was 0.88, and calibration-in-the-large for all performance measures was <0.001 with standard error <0.001, indicating good discrimination and excellent calibration.

CONCLUSIONS

The Predict-ITP Tool can estimate the likelihood of ITP for a given patient with thrombocytopenia at the time of the initial hematology consultation. The tool had high predictive accuracy for the diagnosis of ITP.

Predictors of pain reduction following a program of manual therapies for patients with temporomandibular disorders: A prospective observational study

BACKGROUND

Clinical guidelines recommend conservative treatment for the management of temporomandibular disorders (TMD), and manual therapy (MT) is commonly applied to reduce pain and improve function.

OBJECTIVES

To identify predictors of pain reduction and functional improvement following a program of manual therapies (MTP) in patients with TMD and develop a first screening tool that could be used in clinical practice to facilitate decision-making.

DESIGN

A cohort of 102 adults with a diagnosis of TMD were treated with four weekly sessions within a MTP applied to craniomandibular structures. Candidate predictors were demographic variables, general health variables, psychosocial features, TMD characteristics and related clinical tests. A reduction of pain intensity by at least 30% after the MTP was considered a good outcome. Logistic regression was adopted to develop the predictive model and its performance was assessed considering the explained variance, calibration, and discrimination. Internal validation of the prediction models was further evaluated in 500 bootstrapped samples.

RESULTS

Patients experiencing pain intensity greater than 2/10 during mouth opening, positive expectations of outcome following a MTP, pain localized in the craniocervical region, and a low Central Sensitization Inventory score obtained a good outcome following the MTP. Predictive performance of the identified physical and psychological variables was characterized by high explained variance (R2 = 58%) and discrimination (AUC = 89%) after internal validation. A preliminary screening clinical tool was developed and presented as a nomogram.

CONCLUSIONS

The high discrimination of the prediction model revealed promising findings, although these need to be externally validated in future research.

TRIAL REGISTRATION NUMBER

NCT03990662.

All-Cause Mortality Risk Prediction in Older Adults with Cancer: Practical Approaches and Limitations

PURPOSE OF REVIEW

The prediction of all-cause mortality is an important component of shared decision-making across the cancer care continuum, particularly in older adults with limited life expectancy, for whom there is an increased risk of over-diagnosis and treatment.

RECENT FINDINGS

Currently, several international societies recommend the use of all-cause mortality risk prediction tools when making decisions regarding screening and treatment in geriatric oncology. Here, we review some practical aspects of the utilization of those tools and dissect the characteristics of those most employed in geriatric oncology, highlighting both their advantages and their limitations.

Canadian infants presenting with Brief Resolved Unexplained Events (BRUEs) and validation of clinical prediction rules for risk stratification: a protocol for a multicentre, retrospective cohort study

INTRODUCTION

Brief Resolved Unexplained Events (BRUEs) are a common presentation among infants. While most of these events are benign and self-limited, guidelines published by the American Academy of Pediatrics inaccurately identify many patients as higher-risk of a serious underlying aetiology (positive predictive value 5%). Recently, new clinical prediction rules have been derived to more accurately stratify patients. This data were however geographically limited to the USA, with no large studies to date assessing the BRUE population in a different healthcare setting. The study's aim is to describe the clinical management and outcomes of infants presenting to Canadian hospitals with BRUEs and to externally validate the BRUE clinical prediction rules in identified cases.

METHODS AND ANALYSIS

This is a multicentre retrospective study, conducted within the Canadian Paediatric Inpatient Research Network (PIRN). Infants (<1 year) presenting with a BRUE at one of 11 Canadian paediatric centres between 1 January 2017 and 31 December 2021 will be included. Eligible patients will be identified using diagnostic codes.The primary outcome will be the presence of a serious underlying illness. Secondary outcomes will include BRUE recurrence and length of hospital stay. We will describe the rates of hospital admissions and whether hospitalisation was associated with an earlier diagnosis or treatment. Variation across Canadian hospitals will be assessed using intraclass correlation coefficient. To validate the newly developed clinical prediction rule, measures of goodness of fit will be evaluated. For this validation, a sample size of 1182 is required to provide a power of 80% to detect patients with a serious underlying illness with a significance level of 5%.

ETHICS AND DISSEMINATION

Ethics approval has been granted by the UBC Children's and Women's Research Board (H21-02357). The results of this study will be disseminated as peer-reviewed manuscripts and presentations at national and international conferences.

Development and internal validation of risk scores to diagnose infectious mononucleosis among college students

BACKGROUND

Individual symptoms and signs of infectious mononucleosis (IM) are of limited value for diagnosis.

OBJECTIVE

To develop and validate risk scores based on signs and symptoms with and without haematologic parameters for the diagnosis of IM.

DESIGN AND SETTING

Data were extracted from electronic health records of a university health centre and were divided into derivation (9/1/2015-10/31/2017) and a prospective temporal internal validation (11/1/2017-1/31/2019) cohort.

METHOD

Independent predictors for the diagnosis of IM were identified in univariate analysis using the derivation cohort. Logistic regression models were used to develop 2 risk scores: 1 with only symptoms and signs (IM-NoLab) and 1 adding haematologic parameters (IM-Lab). Point scores were created based on the regression coefficients, and patients were grouped into risk groups. Primary outcomes were area under the receiver operating characteristic curve (AUROCC) and classification accuracy.

RESULTS

The IM-NoLab model had 4 predictors and identified a low-risk group (7.9% with IM) and a high-risk group (22.2%) in the validation cohort. The AUROCC was 0.75 in the derivation cohort and 0.69 in the validation cohort. The IM-Lab model had 3 predictors and identified a low-risk group (3.6%), a moderate-risk group (12.5%), and a high-risk group (87.6%). The AUROCC was 0.97 in the derivation cohort and 0.93 in the validation cohort.

CONCLUSION

We derived and internally validated the IM-NoLab and IM-Lab risk scores. The IM-Lab score in particular had very good discrimination and have the potential to reduce the need for diagnostic testing for IM.

Clinical Risk Score to Predict Pathogenic Genotypes in Patients With Dilated Cardiomyopathy

BACKGROUND

Although genotyping allows family screening and influences risk-stratification in patients with nonischemic dilated cardiomyopathy (DCM) or isolated left ventricular systolic dysfunction (LVSD), its result is negative in a significant number of patients, limiting its widespread adoption.

OBJECTIVES

This study sought to develop and externally validate a score that predicts the probability for a positive genetic test result (G+) in DCM/LVSD.

METHODS

Clinical, electrocardiogram, and echocardiographic variables were collected in 1,015 genotyped patients from Spain with DCM/LVSD. Multivariable logistic regression analysis was used to identify variables independently predicting G+, which were summed to create the Madrid Genotype Score. The external validation sample comprised 1,097 genotyped patients from the Maastricht and Trieste registries.

RESULTS

A G+ result was found in 377 (37%) and 289 (26%) patients from the derivation and validation cohorts, respectively. Independent predictors of a G+ result in the derivation cohort were: family history of DCM (OR: 2.29; 95% CI: 1.73-3.04; P < 0.001), low electrocardiogram voltage in peripheral leads (OR: 3.61; 95% CI: 2.38-5.49; P < 0.001), skeletal myopathy (OR: 3.42; 95% CI: 1.60-7.31; P = 0.001), absence of hypertension (OR: 2.28; 95% CI: 1.67-3.13; P < 0.001), and absence of left bundle branch block (OR: 3.58; 95% CI: 2.57-5.01; P < 0.001). A score containing these factors predicted a G+ result, ranging from 3% when all predictors were absent to 79% when ≥4 predictors were present. Internal validation provided a C-statistic of 0.74 (95% CI: 0.71-0.77) and a calibration slope of 0.94 (95% CI: 0.80-1.10). The C-statistic in the external validation cohort was 0.74 (95% CI: 0.71-0.78).

CONCLUSIONS

The Madrid Genotype Score is an accurate tool to predict a G+ result in DCM/LVSD.

The clinical effectiveness of the STUMBL score for the management of ED patients with blunt chest trauma compared to clinical evaluation alone

The STUMBL (STUdy of the Management of BLunt chest wall trauma) score is a new prognostic score to assist ED (Emergency Department) decision making in the management of blunt chest trauma. This is a retrospective cohort chart review study conducted in a UK University Hospital ED seeing 120,000 patients a year, comparing its performance characteristics to ED clinician judgement. All blunt chest trauma patients that presented to our ED over a 6-month period were included. Patients were excluded if age < 18, if they had immediate life-threatening injury, required critical care admission for other injuries or in case of missing identification data. Primary endpoint was complication defined as any of lower respiratory tract infection, pulmonary consolidation, empyema, pneumothorax, haemothorax, splenic or hepatic injury and 30-day mortality. Clinician judgement (clinician decision to admit) and STUMBL score were compared using the receiver-operating curve (ROC) and sensitivity analysis. Three hundred and sixty-nine patients were included. ED clinicians admitted 95 of 369 patients. ED clinician decision to admit had a sensitivity of 83.9% and specificity of 86.0% for predicting complications. STUMBL score ≥ 11 had a sensitivity of 79.0% and specificity of 77.9% for the same and would have led to 117 of 369 patients being admitted. Area under the curve (AUC) of STUMBL score and ED clinician decision to admit was 0.84 (95% CI 0.78-0.90) and 0.85 (95% CI 0.79-0.91), respectively. Our findings show that a STUMBL score ≥ 11 performs no better than ED clinician judgement and leads to more patients being admitted to hospital.

Risk Prediction After a Brief Resolved Unexplained Event

OBJECTIVES

Only 4% of brief resolved unexplained events (BRUE) are caused by a serious underlying illness. The American Academy of Pediatrics (AAP) guidelines do not distinguish patients who would benefit from further investigation and hospitalization. We aimed to derive and validate a clinical decision rule for predicting the risk of a serious underlying diagnosis or event recurrence.

METHODS

We retrospectively identified infants presenting with a BRUE to 15 children's hospitals (2015-2020). We used logistic regression in a split-sample to derive and validate a risk prediction model.

RESULTS

Of 3283 eligible patients, 565 (17.2%) had a serious underlying diagnosis (n = 150) or a recurrent event (n = 469). The AAP's higher-risk criteria were met in 91.5% (n = 3005) and predicted a serious diagnosis with 95.3% sensitivity, 8.6% specificity, and an area under the curve of 0.52 (95% confidence interval [CI]: 0.47-0.57). A derived model based on age, previous events, and abnormal medical history demonstrated an area under the curve of 0.64 (95%CI: 0.59-0.70). In contrast to the AAP criteria, patients >60 days were more likely to have a serious underlying diagnosis (odds ratio:1.43, 95%CI: 1.03-1.98, P = .03).

CONCLUSIONS

Most infants presenting with a BRUE do not have a serious underlying pathology requiring prompt diagnosis. We derived 2 models to predict the risk of a serious diagnosis and event recurrence. A decision support tool based on this model may aid clinicians and caregivers in the discussion on the benefit of diagnostic testing and hospitalization (https://www.mdcalc.com/calc/10400/brief-resolved-unexplained-events-2.0-brue-2.0-criteria-infants).

Development and validation of automated computer-aided risk scores to predict in-hospital mortality for emergency medical admissions with COVID-19: a retrospective cohort development and validation study

OBJECTIVES

There are no established mortality risk equations specifically for unplanned emergency medical admissions which include patients with SARS-19 (COVID-19). We aim to develop and validate a computer-aided risk score (CARMc19) for predicting mortality risk by combining COVID-19 status, the first electronically recorded blood test results and the National Early Warning Score (NEWS2).

DESIGN

Logistic regression model development and validation study.

SETTING

Two acute hospitals (York Hospital-model development data; Scarborough Hospital-external validation data).

PARTICIPANTS

Adult (aged ≥16 years) medical admissions discharged over a 24-month period with electronic NEWS and blood test results recorded on admission. We used logistic regression modelling to predict the risk of in-hospital mortality using two models: (1) CARMc19_N: age+sex+NEWS2 including subcomponents+COVID19; (2) CARMc19_NB: CARMc19_N in conjunction with seven blood test results and acute kidney injury score. Model performance was evaluated according to discrimination (c-statistic), calibration (graphically) and clinical usefulness at NEWS2 thresholds of 4+, 5+, 6+.

RESULTS

The risk of in-hospital mortality following emergency medical admission was similar in development and validation datasets (8.4% vs 8.2%). The c-statistics for predicting mortality for CARMc19_NB is better than CARMc19_N in the validation dataset (CARMc19_NB=0.88 (95% CI 0.86 to 0.90) vs CARMc19_N=0.86 (95% CI 0.83 to 0.88)). Both models had good calibration (CARMc19_NB=1.01 (95% CI 0.88 to 1.14) and CARMc19_N:0.95 (95% CI 0.83 to 1.06)). At all NEWS2 thresholds (4+, 5+, 6+) model, CARMc19_NB had better sensitivity and similar specificity.

CONCLUSIONS

We have developed a validated CARMc19 scores with good performance characteristics for predicting the risk of in-hospital mortality. Since the CARMc19 scores place no additional data collection burden on clinicians, it may now be carefully introduced and evaluated in hospitals with sufficient informatics infrastructure.

Liquid biopsies for early diagnosis of brain tumours: in silico mathematical biomarker modelling

Brain tumours are the biggest cancer killer in those under 40 and reduce life expectancy more than any other cancer. Blood-based liquid biopsies may aid early diagnosis, prediction and prognosis for brain tumours. It remains unclear whether known blood-based biomarkers, such as glial fibrillary acidic protein (GFAP), have the required sensitivity and selectivity. We have developed a novel in silico model which can be used to assess and compare blood-based liquid biopsies. We focused on GFAP, a putative biomarker for astrocytic tumours and glioblastoma multi-formes (GBMs). In silico modelling was paired with experimental measurement of cell GFAP concentrations and used to predict the tumour volumes and identify key parameters which limit detection. The average GBM volumes of 449 patients at Leeds Teaching Hospitals NHS Trust were also measured and used as a benchmark. Our model predicts that the currently proposed GFAP threshold of 0.12 ng ml-1 may not be suitable for early detection of GBMs, but that lower thresholds may be used. We found that the levels of GFAP in the blood are related to tumour characteristics, such as vasculature damage and rate of necrosis, which are biological markers of tumour aggressiveness. We also demonstrate how these models could be used to provide clinical insight.

Modifiable prognostic factors of high costs related to healthcare utilization among older people seeking primary care due to back pain: an identification and replication study

BACKGROUND

Back pain is an extensive burden to our healthcare system, yet few studies have explored modifiable prognostic factors associated with high costs related to healthcare utilization, especially among older back pain patients. The aims of this study were to identify modifiable prognostic factors for high costs related to healthcare utilization among older people seeking primary care with a new episode of back pain; and to replicate the identified associations in a similar cohort, in a different country.

METHODS

Data from two cohort studies within the BACE consortium were used, including 452 and 675 people aged ≥55 years seeking primary care with a new episode of back pain. High costs were defined as costs in the top 25th percentile. Healthcare utilization was self-reported, aggregated for one-year of follow-up and included: primary care consultations, medications, examinations, hospitalization, rehabilitation stay and operations. Costs were estimated based on unit costs collected from national pricelists. Nine potential modifiable prognostic factors were selected based on previous literature. Univariable and multivariable binary logistic regression models were used to identify and replicate associations (crude and adjusted for selected covariates) between each modifiable prognostic factor and high costs related to healthcare utilization.

RESULTS

Four modifiable prognostic factors associated with high costs related to healthcare utilization were identified and replicated: a higher degree of pain severity, disability, depression, and a lower degree of physical health-related quality of life. Kinesiophobia and recovery expectations showed no prognostic value. There were inconsistent results across the two cohorts with regards to comorbidity, radiating pain below the knee and mental health-related quality of life.

CONCLUSION

The factors identified in this study may be future targets for intervention with the potential to reduce high costs related to healthcare utilization among older back pain patients.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04261309, 07 February 2020. Retrospectively registered.

Prognostic value of serum high mobility group box 1 protein and histone H3 levels in patients with disseminated intravascular coagulation: a multicenter prospective cohort study

Background

We compared the prognostic value of serum high mobility group box 1 protein (HMGB1) and histone H3 levels with the International Society on Thrombosis and Haemostasis (ISTH) disseminated intravascular coagulation (DIC) scores for 28-day in-hospital mortality in patients with DIC caused by various underlying diseases.

Methods

We conducted a multicenter prospective cohort study including two hematology departments, four emergency departments, and one general medicine department in Japan, between August 2017 and July 2021. We included patients diagnosed with DIC by the ISTH DIC scoring system.

Results

Overall, 104 patients were included: 50 with hematopoietic disorders, 41 with infections, and 13 with the other diseases. The 28-day in-hospital mortality rate was 21%. The receiver operator characteristic (ROC) curve showed that a DIC score of 6 points, serum HMGB1 level of 8 ng/mL, and serum histone H3 level of 2 ng/mL were the optimal cutoff points. The odds ratios of more than these optimal cutoff points of the DIC score, serum HMGB1, and histone H3 levels were 1.58 (95% confidence interval [CI]: 0.60 to 4.17, p = 0.36), 5.47 (95% CI: 1.70 to 17.6, p = 0.004), and 9.07 (95% CI: 2.00 to 41.3, p = 0.004), respectively. The area under the ROC curve of HMGB1 (0.74, 95% CI: 0.63 to 0.85) was better than that of the ISTH DIC scores (0.55, 95% CI: 0.43 to 0.67, p = 0.03), whereas that of histone H3 was not (0.71, 95% CI: 0.60 to 0.82, p = 0.07). Calibration and net reclassification plots of HMGB1 identified some high-risk patients, whereas the ISTH DIC scores and histone H3 did not. The category-free net reclassification improvement of HMGB1 was 0.45 (95% CI: 0.01 to 0.90, p = 0.04) and that of histone H3 was 0.37 (95% CI: − 0.05 to 0.78, p = 0.08).

Conclusions

Serum HMGB1 levels have a prognostic value for mortality in patients with DIC. This finding may help physicians develop treatment strategies.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12959-022-00390-2.

Improving ambulance dispatch triage to trauma: A scoping review using the framework of development and evaluation of clinical prediction rules

OBJECTIVE

Ambulance dispatch algorithms should function as clinical prediction rules, identifying high acuity patients for advanced life support, and low acuity patients for non-urgent transport. Systematic reviews of dispatch algorithms are rare and focus on study types specific to the final phases of rule development, such as impact studies, and may miss the complete value-added evidence chain. We sought to summarise the literature for studies seeking to improve dispatch in trauma by performing a scoping review according to standard frameworks for developing and evaluating clinical prediction rules.

METHODS

We performed a scoping review searching MEDLINE, EMBASE, CINAHL, the CENTRAL trials registry, and grey literature from January 2005 to October 2021. We included all study types investigating dispatch triage to injured patients in the English language. We reported the clinical prediction rule phase (derivation, validation, impact analysis, or user acceptance) and the performance and outcomes measured for high and low acuity trauma patients.

RESULTS

Of 2067 papers screened, we identified 12 low and 30 high acuity studies. Derivation studies were most common (52%) and rule-based computer-aided dispatch was the most frequently investigated (23 studies). Impact studies rarely reported a prior validation phase, and few validation studies had their impact investigated. Common outcome measures in each phase were infrequent (0 to 27%), making a comparison between protocols difficult. A series of papers for low acuity patients and another for pediatric trauma followed clinical prediction rule development. Some low acuity Medical Priority Dispatch System codes are associated with the infrequent requirement for advanced life support and clinician review of computer-aided dispatch may enhance dispatch triage accuracy in studies of helicopter emergency medical services.

CONCLUSIONS

Few derivation and validation studies were followed by an impact study, indicating important gaps in the value-added evidence chain. While impact studies suggest clinician oversight may enhance dispatch, the opportunity exists to standardize outcomes, identify trauma-specific low acuity codes, and develop intelligent dispatch systems.

The Ability of Clinical Decision Rules to Detect Peripheral Arterial Disease: A Narrative Review

Peripheral arterial disease (PAD) is a common cause of lower extremity wound. Consequently, PAD leads to a cause of leg amputation nowadays, especially in diabetic patients. In general practice (GP), confrontation with PAD prevention is a challenge. In general, ankle-brachial index (ABI) measurement can be used as a PAD diagnostic tool, but this takes some time. The tool is not generally available and this need to train healthcare workers to perform. Multiple independent predictors developed the diagnostic prediction model known as clinical decision rules (CDRs) to identify patients with high-risk PAD. This might therefore limit the number of patients (only high-risk patients) to refer for ABI evaluation. This narrative review summarized existing CDRs for PAD.

Quality and transparency of reporting derivation and validation prognostic studies of recurrent stroke in patients with TIA and minor stroke: a systematic review

BACKGROUND

Clinical prediction models/scores help clinicians make optimal evidence-based decisions when caring for their patients. To critically appraise such prediction models for use in a clinical setting, essential information on the derivation and validation of the models needs to be transparently reported. In this systematic review, we assessed the quality of reporting of derivation and validation studies of prediction models for the prognosis of recurrent stroke in patients with transient ischemic attack or minor stroke.

METHODS

MEDLINE and EMBASE databases were searched up to February 04, 2020. Studies reporting development or validation of multivariable prognostic models predicting recurrent stroke within 90 days in patients with TIA or minor stroke were included. Included studies were appraised for reporting quality and conduct using a select list of items from the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) Statement.

RESULTS

After screening 7026 articles, 60 eligible articles were retained, consisting of 100 derivation and validation studies of 27 unique prediction models. Four models were newly derived while 23 were developed by validating and updating existing models. Of the 60 articles, 15 (25%) reported an informative title. Among the 100 derivation and validation studies, few reported whether assessment of the outcome (24%) and predictors (12%) was blinded. Similarly, sample size justifications (49%), description of methods for handling missing data (16.1%), and model calibration (5%) were seldom reported. Among the 96 validation studies, 17 (17.7%) clearly reported on similarity (in terms of setting, eligibility criteria, predictors, and outcomes) between the validation and the derivation datasets. Items with the highest prevalence of adherence were the source of data (99%), eligibility criteria (93%), measures of discrimination (81%) and study setting (65%).

CONCLUSIONS

The majority of derivation and validation studies for the prognosis of recurrent stroke in TIA and minor stroke patients suffer from poor reporting quality. We recommend that all prediction model derivation and validation studies follow the TRIPOD statement to improve transparency and promote uptake of more reliable prediction models in practice.

TRIAL REGISTRATION

The protocol for this review was registered with PROSPERO (Registration number CRD42020201130 ).

An opportunity for primary prevention research in psychotic disorders

An opportunity has opened for research into primary prevention of psychotic disorders, based on progress in endophenotypes, genetics, and genomics. Primary prevention requires reliable prediction of susceptibility before any symptoms are present. We studied a battery of measures where published data supports abnormalities of these measurements prior to appearance of initial psychosis symptoms. These neurobiological and behavioral measurements included cognition, eye movement tracking, Event Related Potentials, and polygenic risk scores. They generated an acceptably precise separation of healthy controls from outpatients with a psychotic disorder. METHODS: The Bipolar and Schizophrenia Network on Intermediate Phenotypes (B-SNIP) measured this battery in an ancestry-diverse series of consecutively recruited adult outpatients with a psychotic disorder and healthy controls. Participants include all genders, 16 to 50 years of age, 261 with psychotic disorders (Schizophrenia (SZ) 109, Bipolar with psychosis (BPP) 92, Schizoaffective disorder (SAD) 60), 110 healthy controls. Logistic Regression, and an extension of the Linear Mixed Model to include analysis of pairwise interactions between measures (Environmental kernel Relationship Matrices (ERM)) with multiple iterations, were performed to predict case-control status. Each regression analysis was validated with four-fold cross-validation. RESULTS AND CONCLUSIONS: Sensitivity, specificity, and Area Under the Curve of Receiver Operating Characteristic of 85%, 62%, and 86%, respectively, were obtained for both analytic methods. These prediction metrics demonstrate a promising diagnostic distinction based on premorbid risk variables. There were also statistically significant pairwise interactions between measures in the ERM model. The strong prediction metrics of both types of analytic model provide proof-of-principle for biologically-based laboratory tests as a first step toward primary prevention studies. Prospective studies of adolescents at elevated risk, vs. healthy adolescent controls, would be a next step toward development of primary prevention strategies.

Predicting 2-year survival in stage I-III non-small cell lung cancer: the development and validation of a scoring system from an Australian cohort

Background

There are limited data on survival prediction models in contemporary inoperable non-small cell lung cancer (NSCLC) patients. The objective of this study was to develop and validate a survival prediction model in a cohort of inoperable stage I-III NSCLC patients treated with radiotherapy.

Methods

Data from inoperable stage I-III NSCLC patients diagnosed from 1/1/2016 to 31/12/2017 were collected from three radiation oncology clinics. Patient, tumour and treatment-related variables were selected for model inclusion using univariate and multivariate analysis. Cox proportional hazards regression was used to develop a 2-year overall survival prediction model, the South West Sydney Model (SWSM) in one clinic (n = 117) and validated in the other clinics (n = 144). Model performance, assessed internally and on one independent dataset, was expressed as Harrell’s concordance index (c-index).

Results

The SWSM contained five variables: Eastern Cooperative Oncology Group performance status, diffusing capacity of the lung for carbon monoxide, histological diagnosis, tumour lobe and equivalent dose in 2 Gy fractions. The SWSM yielded a c-index of 0.70 on internal validation and 0.72 on external validation. Survival probability could be stratified into three groups using a risk score derived from the model.

Conclusions

A 2-year survival model with good discrimination was developed. The model included tumour lobe as a novel variable and has the potential to guide treatment decisions. Further validation is needed in a larger patient cohort.

Quantitative prediction error analysis to investigate predictive performance under predictor measurement heterogeneity at model implementation

BACKGROUND

When a predictor variable is measured in similar ways at the derivation and validation setting of a prognostic prediction model, yet both differ from the intended use of the model in practice (i.e., "predictor measurement heterogeneity"), performance of the model at implementation needs to be inferred. This study proposed an analysis to quantify the impact of anticipated predictor measurement heterogeneity.

METHODS

A simulation study was conducted to assess the impact of predictor measurement heterogeneity across validation and implementation setting in time-to-event outcome data. The use of the quantitative prediction error analysis was illustrated using an example of predicting the 6-year risk of developing type 2 diabetes with heterogeneity in measurement of the predictor body mass index.

RESULTS

In the simulation study, calibration-in-the-large of prediction models was poor and overall accuracy was reduced in all scenarios of predictor measurement heterogeneity. Model discrimination decreased with increasing random predictor measurement heterogeneity.

CONCLUSIONS

Heterogeneity of predictor measurements across settings of validation and implementation reduced predictive performance at implementation of prognostic models with a time-to-event outcome. When validating a prognostic model, the targeted clinical setting needs to be considered and analyses can be conducted to quantify the impact of anticipated predictor measurement heterogeneity on model performance at implementation.

REcognizing DElirium in geriatric Emergency Medicine: The REDEEM risk stratification score

OBJECTIVE

The objective was to derive a risk score that uses variables available early during the emergency department (ED) encounter to identify high-risk geriatric patients who may benefit from delirium screening.

METHODS

This was an observational study of older adults age ≥ 75 years who presented to an academic ED and who were screened for delirium during their ED visit. Variable selection from candidate predictors was performed through a LASSO-penalized logistic regression. A risk score was derived from the final prediction model, and predictive accuracy characteristics were calculated with 95% confidence intervals (CIs).

RESULTS

From the 967 eligible ED visits, delirium was detected in 107 (11.1%). The area under the curve for the REcognizing DElirium in Emergency Medicine (REDEEM) score was 0.901 (95% CI = 0.864-0.938). The REEDEM risk score included 10 different variables (seven based on triage information and three obtained during early history taking) with a score ranging from -3 to 66. Using an optimal cutoff of ≥11, we found a sensitivity of 84.1% (90 of 107 ED delirium patients, 95% CI = 75.5%-90.2%) and a specificity of 86.6% (745 of 860 non-ED delirium patients, 95% CI = 84.1%-88.8%). A lower cutoff of ≥5 was found to minimize false negatives with an improved sensitivity at 91.6% (98 of 107 ED delirium patients, 95% CI = 84.2%-95.8%).

CONCLUSION

A risk stratification score was derived with the potential to augment delirium recognition in geriatric ED patients. This has the potential to assist on delirium-targeted screening of high-risk patients in the ED. Validation of REDEEM, however, is needed prior to implementation.

Composite Outcomes in Clinical Prediction Modeling: Are We Trying to Predict Apples and Oranges?

Composite outcomes are widely used in clinical research. Existing literature has considered the pros and cons of composite outcomes in clinical trials, but their extensive use in clinical prediction has received much less attention. Clinical prediction assists decision-making by directing patients with higher risks of adverse outcomes toward interventions that provide the greatest benefits to those at the greatest risk. In this article, we summarize our existing understanding of the advantages and disadvantages of composite outcomes, consider how these relate to clinical prediction, and highlight the problem of key predictors having markedly different associations with individual components of the composite outcome. We suggest that a "composite outcome fallacy" may occur when a clinical prediction model is based on strong associations between key predictors and one component of a composite outcome (such as mortality) and used to direct patients toward intervention when these predictors actually have an inverse association with a more relevant component of the composite outcome (such as the use of a lifesaving intervention). We propose that clinical prediction scores using composite outcomes should report their accuracy for key components of the composite outcome and examine for inconsistencies among predictor variables.

Opioid2MME: Standardizing opioid prescriptions to morphine milligram equivalents from electronic health records

BACKGROUND

The national increase in opioid use and misuse has become a public health crisis in the U.S. To tackle this crisis, the systematic evaluation and monitoring of opioid prescribing patterns is necessary. Thus, opioid prescriptions from electronic health records (EHRs) must be standardized to morphine milligram equivalent (MME) to facilitate monitoring and surveillance. While most studies report MMEs to describe opioid prescribing patterns, there is a lack of transparency regarding their data pre-processing and conversion processes for replication or comparison purposes.

METHODS

In this work, we developed Opioid2MME, a SQL-based open-source framework, to convert opioid prescriptions to MMEs using EHR prescription data. The MME conversions were validated internally using F-measures through manual chart review; were compared with two existing tools, as MedEx and MedXN; and the framework was tested in an external academic EHR system.

RESULTS

We identified 232,913 prescriptions for 49,060 unique patients in the EHRs, 2008-2019. We manually annotated a sample of prescriptions to assess the performance of the framework. The internal evaluation for medication information extraction achieved F-measures from 0.98 to 1.00 for each piece of the extracted information, outperforming MedEx and MedXN (F-Scores 0.98 and 0.94, respectively). MME values in the internal EHR system obtained a F-measure of 0.97 and identified 3% of the data as outliers and 7% missing values. The MME conversion in the external EHR system obtained 78.3% agreement between the MME values obtained with the development site.

CONCLUSIONS

The results demonstrated that the framework is replicable and capable of converting opioid prescriptions to MMEs across different medical institutions. In summary, this work sets the groundwork for the systematic evaluation and monitoring of opioid prescribing patterns across healthcare systems.

Prehospital activation of a coordinated multidisciplinary hospital response in preparation for patients with severe hemorrhage. A state-wide data linkage study of the New South Wales "Code Crimson" pathway

BACKGROUND

Hemorrhage is a leading cause of preventable death in trauma. Prehospital medical teams can streamline access to massive transfusion and definitive hemorrhage control by alerting in-hospital trauma teams of suspected life-threatening bleeding in unstable patients. This study reports the initial experience of an Australian "Code Crimson" pathway facilitating early multidisciplinary care for these patients.

METHODS

This data-linkage study combined prehospital databases with a trauma registry of patients with an ISS > 12 between 2017 and 2019. Four groups were created; prehospital Code Crimson (CC) activation with and without in-hospital links and patients with inpatient treatment consistent with CC, without one being activated. Diagnostic accuracy was estimated using capture-recapture methodology to replace the missing cell (no prehospital CC and ISS < 12).

RESULTS

Of 72 prehospital CC patients, 50 were linked with hospital data. Of 154 potentially missed patients, 42 had a prehospital link. Most CC patients were young males who sustained blunt trauma and required more prehospital interventions than non-CC patients. CC patients had more multisystem trauma, especially complex thoracic injuries (80%), while missed-CC patients more frequently had single organ injuries (59%). CC patients required fewer hemorrhage control procedures (60% vs 86%). Lower mortality was observed in CC patients despite greater hospital and ICU length of stay. Despite a low sensitivity (0.49, 95%CI 0.38-0.61) and good specificity (0.92, 95%CI 0.86-0.96), the positive likelihood ratio was acceptable (6.42, 95%CI 3.30-12.48).

CONCLUSIONS

The initiation of a state-wide Code Crimson process was highly specific for the need for hemorrhage control intervention in hospital, but further work is required to improve the sensitivity of prehospital activation. Patients who had a Code Crimson activation sustained more multisystem trauma but had lower mortality than those who did not. These results guide measures to improve this pathway.

LEVEL OF EVIDENCE

Level III, Therapeutic/Care management.

Neck Pain Clinical Prediction Rule to Prescribe Combined Aerobic and Neck-Specific Exercises: Secondary Analysis of a Randomized Controlled Trial

OBJECTIVE

A previous randomized controlled trial revealed that combined aerobic and neck-specific exercises yielded greater improvement than neck-specific exercises alone after a 6-month intervention in outpatients with nonspecific neck pain (NP). The aim of this secondary analysis was to identify subgroups of patients in the combined exercises group most likely to benefit from the intervention.

METHODS

Sixty-nine patients were included. The original trial was conducted in multiple physical therapy outpatient clinics twice a week for 6 weeks; follow-up was 6 months after assignment. The primary outcome was the therapeutic success rate (Global Rating of Change Score ≥ +5, "quite a bit better") after 6 weeks of training and at the 6-month follow-up. Candidate predictors from patients' medical history and physical examination were selected for univariable regression analysis to determine their association with treatment response status. Multivariable logistic regression analysis was used to derive preliminary clinical prediction rules.

RESULTS

The clinical prediction rule contained 3 predictor variables: (1) symptom duration ≤6 months, (2) neck flexor endurance ≥18 seconds, and (3) absence of referred pain (Nagelkerke R2 = .40 and -2 log likelihood = 60.30). The pre-test probability of success was 61.0% in the short term and 77.0% in the long term. The post-test probability of success for patients with at least 2 of the 3 predictor variables was 84.0% in the short term and 87.0% in the long term; such patients will likely benefit from this program.

CONCLUSION

A simple 3-item assessment, derived from easily obtainable baseline data, can identify patients with NP who may respond best to combined aerobic and neck-specific exercises. Validation is required before clinical recommendation.

IMPACT

Patients experiencing NP symptoms ≤6 months who have no referred pain and exhibit neck flexor endurance ≥18 seconds may benefit from a simple self-training program of combined aerobic and neck-specific exercises.

Evaluation of Prognostic and Predictive Models in the Oncology Clinic

Predictive and prognostic models hold great potential to support clinical decision making in oncology and could ultimately facilitate a paradigm shift to a more personalised form of treatment. While a large number of models relevant to the field of oncology have been developed, few have been translated into clinical use and assessment of clinical utility is not currently considered a routine part of model development. In this narrative review of the clinical evaluation of prediction models in oncology, we propose a high-level process diagram for the life cycle of a clinical model, encompassing model commissioning, clinical implementation and ongoing quality assurance, which aims to bridge the gap between model development and clinical implementation.

A simple mortality risk prediction score for viper envenoming in India (VENOMS): A model development and validation study

BACKGROUND

Snakebite is a neglected problem with a high mortality in India. There are no simple clinical prognostic tools which can predict mortality in viper envenomings. We aimed to develop and validate a mortality-risk prediction score for patients of viper envenoming from Southern India.

METHODS

We used clinical predictors from a prospective cohort of 248 patients with syndromic diagnosis of viper envenoming and had a positive 20-minute whole blood clotting test (WBCT 20) from a tertiary-care hospital in Puducherry, India. We applied multivariable logistic regression with backward elimination approach. External validation of this score was done among 140 patients from the same centre and its performance was assessed with concordance statistic and calibration plots.

FINDINGS

The final model termed VENOMS from the term "Viper ENvenOming Mortality Score included 7 admission clinical parameters (recorded in the first 48 hours after bite): presence of overt bleeding manifestations, presence of capillary leak syndrome, haemoglobin <10 g/dL, bite to antivenom administration time > 6.5 h, systolic blood pressure < 100 mm Hg, urine output <20 mL/h in 24 h and female gender. The lowest possible VENOMS score of 0 predicted an in-hospital mortality risk of 0.06% while highest score of 12 predicted a mortality of 99.1%. The model had a concordance statistic of 0·86 (95% CI 0·79-0·94) in the validation cohort. Calibration plots indicated good agreement of predicted and observed outcomes.

CONCLUSIONS

The VENOMS score is a good predictor of the mortality in viper envenoming in southern India where Russell's viper envenoming burden is high. The score may have potential applications in triaging patients and guiding management after further validation.

Developing clinical prediction models when adhering to minimum sample size recommendations: The importance of quantifying bootstrap variability in tuning parameters and predictive performance

Recent minimum sample size formula (Riley et al.) for developing clinical prediction models help ensure that development datasets are of sufficient size to minimise overfitting. While these criteria are known to avoid excessive overfitting on average, the extent of variability in overfitting at recommended sample sizes is unknown. We investigated this through a simulation study and empirical example to develop logistic regression clinical prediction models using unpenalised maximum likelihood estimation, and various post-estimation shrinkage or penalisation methods. While the mean calibration slope was close to the ideal value of one for all methods, penalisation further reduced the level of overfitting, on average, compared to unpenalised methods. This came at the cost of higher variability in predictive performance for penalisation methods in external data. We recommend that penalisation methods are used in data that meet, or surpass, minimum sample size requirements to further mitigate overfitting, and that the variability in predictive performance and any tuning parameters should always be examined as part of the model development process, since this provides additional information over average (optimism-adjusted) performance alone. Lower variability would give reassurance that the developed clinical prediction model will perform well in new individuals from the same population as was used for model development.

Simple-to-use nomogram for predicting the risk of syphilis among MSM in Guangdong Province: results from a serial cross-sectional study

Background

The purpose of this study was to develop and validate a simple-to-use nomogram for the prediction of syphilis infection among men who have sex with men (MSM) in Guangdong Province.

Methods

A serial cross-sectional data of 2184 MSM from 2017 to 2019 was used to develop and validate the nomogram risk assessment model. The eligible MSM were randomly assigned to the training and validation dataset. Factors included in the nomogram were determined by multivariate logistic regression analysis based on the training dataset. The receiver operating characteristic (ROC) curves was used to assess its predictive accuracy and discriminative ability.

Results

A total of 2184 MSM were recruited in this study. The prevalence of syphilis was 18.1% (396/2184). Multivariate logistic analysis found that age, the main venue used to find sexual partners, condom use in the past 6 months, commercial sex in the past 6 months, infection with sexually transmitted diseases (STD) in the past year were associated with syphilis infection using the training dataset. All these factors were included in the nomogram model that was well calibrated. The C-index was 0.80 (95% CI 0.76–0.84) in the training dataset, and 0.79 (95% CI 0.75–0.84) in the validation dataset.

Conclusions

A simple-to-use nomogram for predicting the risk of syphilis has been developed and validated among MSM in Guangdong Province. The proposed nomogram shows good assessment performance.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12879-021-06912-z.

Stroke risk associated with NSAIDs uses in women with dysmenorrhea: A population-based cohort study

OBJECTIVE

Dysmenorrhea is among the most common type of gynecological problem, affecting young women across the globe. This study assessed the comparative risk of stroke associated with the use of nonsteroidal anti-inflammatory drugs (NSAIDs) and non-NSAIDs in women with dysmenorrhea while taking into account the following factors such as age, history of pregnancy, NSAIDs uses and its duration of use, and selected comorbidities.

METHODOLOGY

We used a quantitative research approach based on a comparative case-control study design. The study data was selected from the Longitudinal Health Insurance Database (LHID) 2000, of the Taiwan National Health Research Institutes. Among the estimated 23.4 million insured Taiwanese, who were covered by the Taiwan health insurance system, in the 2000 registry of beneficiaries, one million individuals were randomly selected from the database. A total of 24,955 females suffering from dysmenorrhea were selected for the study. Out of those 3238 (13%) participated in the study group and 21,717 (87%) were randomly distributed into the controls group. Women in the age range, 15-49 years, who did not have any history of stroke, hysterectomy, and/or ovariectomy, were included in the study. A comparative proportional distribution analysis was used for data analysis.

RESULTS

Age and use of NSAIDs and its duration of usage were factors associated with an increased incidence of stroke. The stroke incidence rate was 12.77 per 10,000 person-years, and 1.83-fold higher in NSAIDs use cohort than in comparisons with adjusted hazard ratio (aHR) of 1.47 (95% CI = 0.93-2.32). Among women with dysmenorrhea use of NSAIDs, the stroke incidence increased to 2.29-fold (aHR 95% CI = 1.36-3.84) in those use for ≧24 days per month and to 0.51-fold (aHR 95% CI = 0.13-2.10) in those use for 6-12 days per month.

CONCLUSIONS

Women with dysmenorrhea who use NSAIDs have a higher risk of stroke. Especially young women, the risk of stroke is further increased, and the longer the medication, the higher the risk of stroke. Every woman with symptoms of dysmenorrhea deserves specialized outpatient treatment and care.

The development and validation of prognostic models for overall survival in the presence of missing data in the training dataset: a strategy with a detailed example

BACKGROUND

The United Kingdom Myeloma Research Alliance (UK-MRA) Myeloma Risk Profile is a prognostic model for overall survival. It was trained and tested on clinical trial data, aiming to improve the stratification of transplant ineligible (TNE) patients with newly diagnosed multiple myeloma. Missing data is a common problem which affects the development and validation of prognostic models, where decisions on how to address missingness have implications on the choice of methodology.

METHODS

Model building The training and test datasets were the TNE pathways from two large randomised multicentre, phase III clinical trials. Potential prognostic factors were identified by expert opinion. Missing data in the training dataset was imputed using multiple imputation by chained equations. Univariate analysis fitted Cox proportional hazards models in each imputed dataset with the estimates combined by Rubin's rules. Multivariable analysis applied penalised Cox regression models, with a fixed penalty term across the imputed datasets. The estimates from each imputed dataset and bootstrap standard errors were combined by Rubin's rules to define the prognostic model. Model assessment Calibration was assessed by visualising the observed and predicted probabilities across the imputed datasets. Discrimination was assessed by combining the prognostic separation D-statistic from each imputed dataset by Rubin's rules. Model validation The D-statistic was applied in a bootstrap internal validation process in the training dataset and an external validation process in the test dataset, where acceptable performance was pre-specified. Development of risk groups Risk groups were defined using the tertiles of the combined prognostic index, obtained by combining the prognostic index from each imputed dataset by Rubin's rules.

RESULTS

The training dataset included 1852 patients, 1268 (68.47%) with complete case data. Ten imputed datasets were generated. Five hundred twenty patients were included in the test dataset. The D-statistic for the prognostic model was 0.840 (95% CI 0.716-0.964) in the training dataset and 0.654 (95% CI 0.497-0.811) in the test dataset and the corrected D-Statistic was 0.801.

CONCLUSION

The decision to impute missing covariate data in the training dataset influenced the methods implemented to train and test the model. To extend current literature and aid future researchers, we have presented a detailed example of one approach. Whilst our example is not without limitations, a benefit is that all of the patient information available in the training dataset was utilised to develop the model.

TRIAL REGISTRATION

Both trials were registered; Myeloma IX- ISRCTN68454111 , registered 21 September 2000. Myeloma XI- ISRCTN49407852 , registered 24 June 2009.

Development and Assessment of an Interpretable Machine Learning Triage Tool for Estimating Mortality After Emergency Admissions

IMPORTANCE

Triage in the emergency department (ED) is a complex clinical judgment based on the tacit understanding of the patient's likelihood of survival, availability of medical resources, and local practices. Although a scoring tool could be valuable in risk stratification, currently available scores have demonstrated limitations.

OBJECTIVES

To develop an interpretable machine learning tool based on a parsimonious list of variables available at ED triage; provide a simple, early, and accurate estimate of patients' risk of death; and evaluate the tool's predictive accuracy compared with several established clinical scores.

DESIGN, SETTING, AND PARTICIPANTS

This single-site, retrospective cohort study assessed all ED patients between January 1, 2009, and December 31, 2016, who were subsequently admitted to a tertiary hospital in Singapore. The Score for Emergency Risk Prediction (SERP) tool was derived using a machine learning framework. To estimate mortality outcomes after emergency admissions, SERP was compared with several triage systems, including Patient Acuity Category Scale, Modified Early Warning Score, National Early Warning Score, Cardiac Arrest Risk Triage, Rapid Acute Physiology Score, and Rapid Emergency Medicine Score. The initial analyses were completed in October 2020, and additional analyses were conducted in May 2021.

MAIN OUTCOMES AND MEASURES

Three SERP scores, namely SERP-2d, SERP-7d, and SERP-30d, were developed using the primary outcomes of interest of 2-, 7-, and 30-day mortality, respectively. Secondary outcomes included 3-day mortality and inpatient mortality. The SERP's predictive power was measured using the area under the curve in the receiver operating characteristic analysis.

RESULTS

The study included 224 666 ED episodes in the model training cohort (mean [SD] patient age, 63.60 [16.90] years; 113 426 [50.5%] female), 56 167 episodes in the validation cohort (mean [SD] patient age, 63.58 [16.87] years; 28 427 [50.6%] female), and 42 676 episodes in the testing cohort (mean [SD] patient age, 64.85 [16.80] years; 21 556 [50.5%] female). The mortality rates in the training cohort were 0.8% at 2 days, 2.2% at 7 days, and 5.9% at 30 days. In the testing cohort, the areas under the curve of SERP-30d were 0.821 (95% CI, 0.796-0.847) for 2-day mortality, 0.826 (95% CI, 0.811-0.841) for 7-day mortality, and 0.823 (95% CI, 0.814-0.832) for 30-day mortality and outperformed several benchmark scores.

CONCLUSIONS AND RELEVANCE

In this retrospective cohort study, SERP had better prediction performance than existing triage scores while maintaining easy implementation and ease of ascertainment in the ED. It has the potential to be widely applied and validated in different circumstances and health care settings.

Clinical Scoring for Prediction of Acute Kidney Injury in Patients with Acute ST-Segment Elevation Myocardial Infarction after Emergency Primary Percutaneous Coronary Intervention

Acute kidney injury (AKI) after a coronary intervention is common in patients with ST-segment elevation myocardial infarction (STEMI) and is associated with significant morbidity and mortality. Several scores have been developed to predict post-procedural AKI over the years. However, the AKI definitions have also evolved, which causes the definitions used in the past to be obsolete. We aimed to develop a prediction score for AKI in patients with STEMI requiring emergency primary percutaneous coronary intervention (pPCI). This study was based on a retrospective cohort of Thai patients with STEMI who underwent pPCI at the Central Chest Institute of Thailand from December 2014 to September 2019. AKI was defined as an increase in serum creatinine of at least 0.3 mg/dL from baseline within 48 h after pPCI. Logistic regression was used for modeling. A total of 1617 patients were included. Of these, 195 patients had AKI (12.1%). Eight significant predictors were identified: age, baseline creatinine, left ventricular ejection fraction (LVEF) < 40%, multi-vessel pPCI, treated with thrombus aspiration, inserted intra-aortic balloon pump (IABP), pre- and intra-procedural cardiogenic shock, and congestive heart failure. The score showed an area under the receiver operating characteristic curve of 0.78 (95% CI 0.75, 0.82) and was well-calibrated. The pPCI-AKI score showed an acceptable predictive performance and was potentially useful to help interventionists stratify the patients and provide optimal preventive management.

Prediction Models for Type 2 Diabetes Risk in the General Population: A Systematic Review of Observational Studies

OBJECTIVES

This study aimed to provide an overview of prediction models of undiagnosed type 2 diabetes mellitus (U-T2DM) or the incident T2DM (I-T2DM) using the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) checklist and the prediction model risk of the bias assessment tool (PROBAST).

DATA SOURCES

Both PUBMED and EMBASE databases were searched to guarantee adequate and efficient coverage.

STUDY SELECTION

Articles published between December 2011 and October 2019 were considered.

DATA EXTRACTION

For each article, information on model development requirements, discrimination measures, calibration, overall performance, clinical usefulness, overfitting, and risk of bias (ROB) was reported.

RESULTS

The median (interquartile range; IQR) number of the 46 study populations for model development was 5711 (1971 - 27426) and 2457 (2060 - 6995) individuals for I-T2DM and U-T2DM, respectively. The most common reported predictors were age and body mass index, and only the Qrisk-2017 study included social factors (e.g., Townsend score). Univariable analysis was reported in 46% of the studies, and the variable selection procedure was not clear in 17.4% of them. Moreover, internal and external validation was reported in 43% the studies, while over 63% of them reported calibration. The median (IQR) of AUC for I-T2DM models was 0.78 (0.74 - 0.82); the corresponding value for studies derived before October 2011 was 0.80 (0.77 - 0.83). The highest discrimination index was reported for Qrisk-2017 with C-statistics of 0.89 for women and 0.87 for men. Low ROB for I-T2DM and U-T2DM was assessed at 18% and 41%, respectively.

CONCLUSIONS

Among prediction models, an intermediate to poor quality was reassessed in several aspects of model development and validation. Generally, despite its new risk factors or new methodological aspects, the newly developed model did not increase our capability in screening/predicting T2DM, mainly in the analysis part. It was due to the lack of external validation of the prediction models.

Risk Factors and Prediction Models for Venous Thromboembolism in Ambulatory Patients with Lung Cancer

Venous thromboembolism (VTE) is a significant cause of mortality in patients with lung cancer. Despite the availability of a wide range of anticoagulants to help prevent thrombosis, thromboprophylaxis in ambulatory patients is a challenge due to its associated risk of haemorrhage. As a result, anticoagulation is only recommended in patients with a relatively high risk of VTE. Efforts have been made to develop predictive models for VTE risk assessment in cancer patients, but the availability of a reliable predictive model for ambulate patients with lung cancer is unclear. We have analysed the latest information on this topic, with a focus on the lung cancer-related risk factors for VTE, and risk prediction models developed and validated in this group of patients. The existing risk models, such as the Khorana score, the PROTECHT score and the CONKO score, have shown poor performance in external validations, failing to identify many high-risk individuals. Some of the newly developed and updated models may be promising, but their further validation is needed.