Development and External Validation of Models to Predict Persistent Hypoxemic Respiratory Failure for Clinical Trial Enrichment

OBJECTIVES

Improving the efficiency of clinical trials in acute hypoxemic respiratory failure (HRF) depends on enrichment strategies that minimize enrollment of patients who quickly resolve with existing care and focus on patients at high risk for persistent HRF. We aimed to develop parsimonious models predicting risk of persistent HRF using routine data from ICU admission and select research immune biomarkers.

DESIGN

Prospective cohorts for derivation ( n = 630) and external validation ( n = 511).

SETTING

Medical and surgical ICUs at two U.S. medical centers.

PATIENTS

Adults with acute HRF defined as new invasive mechanical ventilation (IMV) and hypoxemia on the first calendar day after ICU admission.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

We evaluated discrimination, calibration, and practical utility of models predicting persistent HRF risk (defined as ongoing IMV and hypoxemia on the third calendar day after admission): 1) a clinical model with least absolute shrinkage and selection operator (LASSO) selecting Pa o2 /F io2 , vasopressors, mean arterial pressure, bicarbonate, and acute respiratory distress syndrome as predictors; 2) a model adding interleukin-6 (IL-6) to clinical predictors; and 3) a comparator model with Pa o2 /F io2 alone, representing an existing strategy for enrichment. Forty-nine percent and 69% of patients had persistent HRF in derivation and validation sets, respectively. In validation, both LASSO (area under the receiver operating characteristic curve, 0.68; 95% CI, 0.64-0.73) and LASSO + IL-6 (0.71; 95% CI, 0.66-0.76) models had better discrimination than Pa o2 /F io2 (0.64; 95% CI, 0.59-0.69). Both models underestimated risk in lower risk deciles, but exhibited better calibration at relevant risk thresholds. Evaluating practical utility, both LASSO and LASSO + IL-6 models exhibited greater net benefit in decision curve analysis, and greater sample size savings in enrichment analysis, compared with Pa o2 /F io2 . The added utility of LASSO + IL-6 model over LASSO was modest.

CONCLUSIONS

Parsimonious, interpretable models that predict persistent HRF may improve enrichment of trials testing HRF-targeted therapies and warrant future validation.

Prospective external validation of the FluScore risk score for influenza in outpatients

BACKGROUND

Testing for influenza in patients with acute lower respiratory tract infection (LRTI) is common and in some cases is performed for all patients with LRTI. A more selective approach to testing could be more efficient.

METHODS

We used data from two prospective studies in the US primary and urgent care settings that enrolled patients with acute LRTI or influenza-like illness. Data were collected in the 2016, 2019, 2021, and 2022 flu seasons. All patients underwent polymerase chain reaction (PCR) testing for influenza and the FluScore was calculated based on patient-reported symptoms at their initial visit. The probability of influenza in each risk group was reported, as well as stratum-specific likelihood ratios (SSLRs) for each risk level.

RESULTS

The prevalence of influenza within risk groups varied based on overall differences in flu seasons and populations. However, the FluScore exhibited consistent performance across various seasons and populations based on the SSLRs. The FluScore had a consistent SSLR range of 0.20 to 0.23 for the low-risk group, 0.63 to 0.99 for the moderate-risk group, and 1.46 to 1.67 for the high-risk group. The diagnostic odds ratio based on the midpoints of these ranges was 7.25.

CONCLUSIONS

The FluScore could streamline patient categorization, identifying patients who could be exempted from testing, while identifying candidates for rapid influenza tests. This has the potential to be more efficient than a "one size fits all" test strategy, as it strategically targets the use of tests on patients most likely to benefit. It is potentially usable in a telehealth setting.

Overuse of Computed Tomography Pulmonary Angiography and Low Utilization of Clinical Prediction Rules in Suspected Pulmonary Embolism Patients at a Regional Australian Hospital

A pulmonary embolism (PE) is an obstruction in the pulmonary arterial system and may include non-specific signs and symptoms. Clinical prediction rules (CPRs) assess the pretest probability (PTP) of a PE to prevent the overuse of computed tomography pulmonary angiography (CTPA). CTPA overuse results in patient harm and health system waste. This study aimed to evaluate CTPA usage in an Australian regional hospital through analyzing CTPA encounters. A retrospective chart analysis was undertaken of 100 CTPAs conducted at an Australian regional hospital from April to May 2023. Analysis was undertaken for parameters including risk factors, signs and symptoms, investigations, and the use of CPRs. Overall, 86% of patients had signs and/or symptoms of a PE within a week of examination, and 6% of the population had signs of deep vein thrombosis. More than half of the population had no risk factors, while the most prevalent risk factors were a recent history of immobilization/trauma and/or having surgery that required general anesthesia in the last 4 weeks. The most common co-morbidity was chronic lung disease (11%). For the pre-test diagnostic workup, the ECG was the most ordered investigation. The Wells' score was used at 10%, while most patients did not have any CPRs applied. The prevalence of PEs discovered on CTPAs was 9%. CPRs were under-utilized in this Australian regional hospital. The D-dimers for ruling out subjects with low PTP derived from CPRs were also underused. This led to the inappropriate overordering of CTPAs, resulting in negative implications for patients and unnecessary costs to the health system.

Predicting in-hospital mortality in patients admitted from the emergency department for pulmonary embolism: Incidence and prognostic value of deep vein thrombosis. A retrospective study

BACKGROUND

Pulmonary embolism (PE) is one of the most common causes of death from cardiovascular disease. Although deep vein thrombosis (DVT) is the leading cause of PE, its prognostic role is unclear. This study investigated the incidence and prognostic value of DVT in predicting in-hospital mortality (IHM) in patients admitted from the emergency department (ED) for PE.

METHODS

This retrospective cohort study was conducted in the ED of a third-level university hospital. Patients over 18 years admitted for PE between 1 January 2018 and 31 December 2022 were included.

RESULTS

Five hundred and thirty patients (mean age 73.13 years, 6% IHM) were included. 69.1% of cases had DVT (36.4% unilateral femoral vein, 3.6% bilateral, 39.1% unilateral popliteal vein, 2.8% bilateral, 45.7% distal vein thrombosis and 7.4% iliocaval involvement). Patients who died in hospital had a higher Pulmonary Embolism Severity Index (PESI) (138.6 vs. 99.65, p < 0.001), European Society of Cardiology risk class (15.6% vs. 1%, intermediate-high in 50% vs. 6.4%, p < 0.001) and more DVT involving the iliac-caval vein axis (18.8% vs. 6.6%, p = 0.011). PESI class >II, right ventricular dysfunction, increased blood markers of myocardial damage and involvement of the iliocaval venous axis were independent predictors of IHM on multivariate analysis.

CONCLUSIONS

Although further studies are needed to confirm the prognostic role of DVT at PE, involvement of the iliocaval venous axis should considered to be a sign of a higher risk of IHM and may be a key factor in prognostic stratification.

Enabling knowledge translation: implementation of a web-based tool for independent walking prediction after traumatic spinal cord injury

Introduction

Several clinical prediction rules (CPRs) have been published, but few are easily accessible or convenient for clinicians to use in practice. We aimed to develop, implement, and describe the process of building a web-based CPR for predicting independent walking 1-year after a traumatic spinal cord injury (TSCI).

Methods

Using the published and validated CPR, a front-end web application called "Ambulation" was built using HyperText Markup Language (HTML), Cascading Style Sheets (CSS), and JavaScript. A survey was created using QualtricsXM Software to gather insights on the application's usability and user experience. Website activity was monitored using Google Analytics. Ambulation was developed with a core team of seven clinicians and researchers. To refine the app's content, website design, and utility, 20 professionals from different disciplines, including persons with lived experience, were consulted.

Results

After 11 revisions, Ambulation was uploaded onto a unique web domain and launched (www.ambulation.ca) as a pilot with 30 clinicians (surgeons, physiatrists, and physiotherapists). The website consists of five web pages: Home, Calculation, Team, Contact, and Privacy Policy. Responses from the user survey (n = 6) were positive and provided insight into the usability of the tool and its clinical utility (e.g., helpful in discharge planning and rehabilitation), and the overall face validity of the CPR. Since its public release on February 7, 2022, to February 28, 2023, Ambulation had 594 total users, 565 (95.1%) new users, 26 (4.4%) returning users, 363 (61.1%) engaged sessions (i.e., the number of sessions that lasted 10 seconds/longer, had one/more conversion events e.g., performing the calculation, or two/more page or screen views), and the majority of the users originating from the United States (39.9%) and Canada (38.2%).

Discussion

Ambulation is a CPR for predicting independent walking 1-year after TSCI and it can assist frontline clinicians with clinical decision-making (e.g., time to surgery or rehabilitation plan), patient education and goal setting soon after injury. This tool is an example of adapting a validated CPR for independent walking into an easily accessible and usable web-based tool for use in clinical practice. This study may help inform how other CPRs can be adopted into clinical practice.

Development and validation of a clinical prediction rule for acute appendicitis in children in primary care

BACKGROUND

Recognising acute appendicitis in children presenting with acute abdominal pain in primary care is challenging. General practitioners (GPs) may benefit from a clinical prediction rule.

OBJECTIVES

To develop and validate a clinical prediction rule for acute appendicitis in children presenting with acute abdominal pain in primary care.

METHODS

In a historical cohort study data was retrieved from GP electronic health records included in the Integrated Primary Care Information database. We assigned children aged 4-18 years presenting with acute abdominal pain (≤ 7 days) to development (2010-2012) and validation (2013-2016) cohorts, using acute appendicitis within six weeks as the outcome. Multiple logistic regression was used to develop a prediction model based on predictors with > 50% data availability derived from existing rules for secondary care. We performed internal and external temporal validation and derived a point score to stratify risk of appendicitis into three groups, i.e. low-risk, medium-risk and high-risk.

RESULTS

The development and validation cohorts included 2,041 and 3,650 children, of whom 95 (4.6%) and 195 (5.3%) had acute appendicitis. The model included male sex, pain duration (<24, 24-48, > 48 h), nausea/vomiting, elevated temperature (≥ 37.3 °C), abnormal bowel sounds, right lower quadrant tenderness, and peritoneal irritation. Internal and temporal validation showed good discrimination (C-statistics: 0.93 and 0.90, respectively) and excellent calibration. In the three groups, the risks of acute appendicitis were 0.5%, 7.5%, and 41%.

CONCLUSION

Combined with further testing in the medium-risk group, the prediction rule could improve clinical decision making and outcomes.

Clinical predictors of pulmonary embolism for inpatients: are computed tomography pulmonary angiograms being requested appropriately?

BACKGROUND

The heterogeneity of inpatient pulmonary embolism (PE) presentations may lead to computed tomography pulmonary angiograms (CTPA) being over-requested. Current clinical predictors for PE, including Wells criteria and Pulmonary Embolism Rule-out Criteria (PERC), have predominantly focussed on outpatient and emergency department populations.

AIM

To determine the clinical indicators for ordering inpatient CTPA and the predictors of positive scans for PE.

METHODS

Consecutive inpatient CTPA (performed >24 h after admission) from January 2017 to December 2017 were retrospectively reviewed. Variables including baseline characteristics, vital signs and risk factors for PE were extracted.

RESULTS

A total of 312 CTPA was reviewed (average patient age 67 years; 46% male) and 36 CTPA were positive for PE (11.5%). The average time to inpatient CTPA request was 7 days. Clinical indicators associated with positive scans were hypoxia (odds ratio (OR) 2.4; 95% confidence interval (CI) 1.1-5.6), tachypnoea (OR 2.5; 95% CI 1.2-6.0), recent surgery or immobilisation (OR 2.7; 95% CI 1.2-6.4), S1Q3T3 pattern on electrocardiogram (ECG; OR 7.2; 95% CI 1.4-35.7) and right bundle branch block pattern on ECG (OR 4.7; 95% CI 1.6-13.1). Hypotension, fever and malignancy were not significant. Both PERC and Wells criteria had poor positive predictive value (12% and 27% respectively), but the negative predictive value for PERC and Wells was 100% and 95.8% respectively.

CONCLUSION

Inpatient CTPA appear to be over-requested and can potentially be rationalised based on a combination of clinical predictors and Wells criteria and/or PERC rule. Further prospective studies are needed to develop accurate clinical decision tools targeted towards inpatients.

Prognostic models of quality of life after total knee replacement: A systematic review

OBJECTIVE

To systematically review and critically appraise prognostic models for quality of life (QOL) in patients with total knee replacement (TKA).

METHODS

Subjects were TKA recipients recruited from inpatient postoperative settings. Searches were made on June 2022 and updated on April 2023. Databases included PubMed.gov, CINAHL, The Cochrane Library, Web of Science. Two authors performed all review stages independently. Risk of bias assessments on participants, predictors, outcomes and analysis methods followed the Prediction study Risk Of Bias ASsessment Tool (PROBAST).

RESULTS

After screening 2204 studies, 9 were eligible for inclusion. Twelve prognostic models were reported, of which 10 models were developed from data without validation and 2 were both developed and validated. The most frequently applied predictor was the pre-TKA QOL score. Discriminatory measures were reported for 9 (75.0%) models with areas under the curve values of 0.66-0.95. All models showed a high risk of bias, mostly due to limitations in statistical methods and outcome assessments.

CONCLUSION

Several prognostic models have been developed for QOL in patients with TKA, but all models show a high risk of bias and are unreliable in clinical practice. Future, prognostic models overcoming the risk of bias identified in this study are needed.

A New Clinical Prediction Rule for Infective Endocarditis in Emergency Department Patients With Fever: Definition and First Validation of the CREED Score

Background Infective endocarditis (IE) could be suspected in any febrile patients admitted to the emergency department (ED). This study was aimed at assessing clinical criteria predictive of IE and identifying and prospectively validating a sensible and easy-to-use clinical prediction score for the diagnosis of IE in the ED. Methods and Results We conducted a retrospective observational study, enrolling consecutive patients with fever admitted to the ED between January 2015 and December 2019 and subsequently hospitalized. Several clinical and anamnestic standardized variables were collected and evaluated for the association with IE diagnosis. We derived a multivariate prediction model by logistic regression analysis. The identified predictors were assigned a score point value to obtain the Clinical Rule for Infective Endocarditis in the Emergency Department (CREED) score. To validate the CREED score we conducted a prospective observational study between January 2020 and December 2021, enrolling consecutive febrile patients hospitalized after the ED visit, and evaluating the association between the CREED score values and the IE diagnosis. A total of 15 689 patients (median age, 71 [56-81] years; 54.1% men) were enrolled in the retrospective cohort, and IE was diagnosed in 267 (1.7%). The CREED score included 12 variables: male sex, anemia, dialysis, pacemaker, recent hospitalization, recent stroke, chest pain, specific infective diagnosis, valvular heart disease, valvular prosthesis, previous endocarditis, and clinical signs of suspect endocarditis. The CREED score identified 4 risk groups for IE diagnosis, with an area under the receiver operating characteristic curve of 0.874 (0.849-0.899). The prospective cohort included 13 163 patients, with 130 (1.0%) IE diagnoses. The CREED score had an area under the receiver operating characteristic curve of 0.881 (0.848-0.913) in the validation cohort, not significantly different from the one calculated in the retrospective cohort (P=0.578). Conclusions In this study, we propose and prospectively validate the CREED score, a clinical prediction rule for the diagnosis of IE in patients with fever admitted to the ED. Our data reflect the difficulty of creating a meaningful tool able to identify patients with IE among this general and heterogeneous population because of the complexity of the disease and its low prevalence in the ED setting.

The prediction of 24-h mortality by the respiratory rate and oxygenation index compared with National Early Warning Score in emergency department patients: an observational study

BACKGROUND

The ROX index combines respiratory rate and oxygenation to predict the response to oxygen therapy in pneumonia. It is calculated by dividing the patient's oxygen saturation, by the inspired oxygen concentration, and then by the respiratory rate (e.g. 95%/0.21/16 = 28). Since this index includes the most essential physiological variables to detect deterioration, it may be a helpful risk tool in the emergency department (ED). Although small studies suggest it can predict early mortality, no large study has compared it with the National Early Warning Score (NEWS), the most widely validated risk score for death within 24 h.

AIM

The aim of this study was to compare the ability of the ROX index with the NEWS to predict mortality within 24 h of arrival at the hospital.

METHODS

This was a retrospective observational multicentre analysis of data in the Netherlands Emergency Department Evaluation Database (NEED) on 270 665 patients attending four participating Dutch EDs. The ROX index and NEWS were determined on ED arrival and prior to ED treatment.

RESULTS

The risk of death within 24 h increased with falling ROX and rising NEWS values. The area under the receiving operating characteristic curves for 24-h mortality of NEWS was significantly higher than for the ROX index [0.92; 95% confidence interval (CI), 0.91-0.92 versus 0.87; 95% CI, 0.86-0.88; P  < 0.01]. However, the observed and predicted mortality by the ROX index was identical to mortality of 5%, after which mortality was underestimated. In contrast, up to a predicted 24-h mortality of 3% NEWS slightly underestimates mortality, and above this level over-estimates it. The standardized net benefit of ROX is slightly higher than NEWS up to a predicted 24-h mortality of 3%.

CONCLUSION

The prediction of 24-h mortality by the ROX index is more accurate than NEWS for most patients likely to be encountered in the ED. ROX may be used as a first screening tool in the ED.

Development and Validation of a Novel Score for Predicting Long-Term Mortality after an Acute Ischemic Stroke

BACKGROUND

Long-term mortality prediction can guide feasible discharge care plans and coordinate appropriate rehabilitation services. We aimed to develop and validate a prediction model to identify patients at risk of mortality after acute ischemic stroke (AIS).

METHODS

The primary outcome was all-cause mortality, and the secondary outcome was cardiovascular death. This study included 21,463 patients with AIS. Three risk prediction models were developed and evaluated: a penalized Cox model, a random survival forest model, and a DeepSurv model. A simplified risk scoring system, called the C-HAND (history of Cancer before admission, Heart rate, Age, eNIHSS, and Dyslipidemia) score, was created based on regression coefficients in the multivariate Cox model for both study outcomes.

RESULTS

All experimental models achieved a concordance index of 0.8, with no significant difference in predicting poststroke long-term mortality. The C-HAND score exhibited reasonable discriminative ability for both study outcomes, with concordance indices of 0.775 and 0.798.

CONCLUSIONS

Reliable prediction models for long-term poststroke mortality were developed using information routinely available to clinicians during hospitalization.

A Clinical Prediction Rule for Thrombosis in Critically Ill COVID-19 Patients: Step 1 Results of the Thromcco Study

The incidence of thrombosis in COVID-19 patients is exceptionally high among intensive care unit (ICU)-admitted individuals. We aimed to develop a clinical prediction rule for thrombosis in hospitalized COVID-19 patients. Data were taken from the Thromcco study (TS) database, which contains information on consecutive adults (aged ≥ 18) admitted to eight Spanish ICUs between March 2020 and October 2021. Diverse logistic regression model analysis, including demographic data, pre-existing conditions, and blood tests collected during the first 24 h of hospitalization, was performed to build a model that predicted thrombosis. Once obtained, the numeric and categorical variables considered were converted to factor variables giving them a score. Out of 2055 patients included in the TS database, 299 subjects with a median age of 62.4 years (IQR 51.5-70) (79% men) were considered in the final model (SE = 83%, SP = 62%, accuracy = 77%). Seven variables with assigned scores were delineated as age 25-40 and ≥70 = 12, age 41-70 = 13, male = 1, D-dimer ≥ 500 ng/mL = 13, leukocytes ≥ 10 × 10³/µL = 1, interleukin-6 ≥ 10 pg/mL = 1, and C-reactive protein (CRP) ≥ 50 mg/L = 1. Score values ≥28 had a sensitivity of 88% and specificity of 29% for thrombosis. This score could be helpful in recognizing patients at higher risk for thrombosis, but further research is needed.

Development of Clinical Prediction Score for Chemotherapy Response in Advanced Non-Small Cell Lung Cancer Patients

The outcomes of advanced non-small cell lung cancer (NSCLC) patients have been significantly improved with novel therapies, such as tyrosine kinase inhibitors and immune checkpoint inhibitors. However, in resource-limited countries, platinum-doublet chemotherapy is mainly used as a first-line treatment. We investigate clinical parameters to predict the response after chemotherapy, which may be useful for patient selection. A clinical prediction score (CPS) was developed, based on data from a retrospective cohort study of unresectable stage IIIB or IV NSCLC patients who were treated with platinum-doublet chemotherapy in the first-line setting with at least two cycles and an evaluated response by RECIST 1.1 at Surin Hospital Cancer Center, Thailand, between July 2014 and December 2018. The clinical parameters in the prediction model were derived by risk regression analysis. There were 117 responders (CR or PR) and 90 non-responders (SD or PD). The clinical prediction score was developed by six clinical parameters including gender, age, smoking status, ECOG, pre-treatment albumin, and histologic subtype. The AuROC of the model was 0.71 (95% CI 0.63-0.78). The internal validation was performed using a bootstrap technique and showed a consistent AuROC of 0.66 (95% CI 0.59-0.72). The prediction score ranged from 0-13, with a score of 0-8 meaning a low probability (PPV = 50%) and a score of 8.5-13 meaning a high probability (PPV = 83.7%) for chemotherapy response. Advanced NSCLC patients who cannot access novel therapies and have a CPS of 8.5-13 have a high probability for chemotherapy response in the first-line setting. This CPS could be used for risk communication and making decisions with patients, especially in regard to chemotherapy.

Development of a Clinical Prediction Rule for Treatment Success with Transcranial Direct Current Stimulation for Knee Osteoarthritis Pain: A Secondary Analysis of a Double-Blind Randomized Controlled Trial

The study’s objective was to develop a clinical prediction rule that predicts a clinically significant analgesic effect on chronic knee osteoarthritis pain after transcranial direct current stimulation treatment. This is a secondary analysis from a double-blind randomized controlled trial. Data from 51 individuals with chronic knee osteoarthritis pain and an impaired descending pain inhibitory system were used. The intervention comprised a 15-session protocol of anodal primary motor cortex transcranial direct current stimulation. Treatment success was defined by the Western Ontario and McMaster Universities’ Osteoarthritis Index pain subscale. Accuracy statistics were calculated for each potential predictor and for the final model. The final logistic regression model was statistically significant (p < 0.01) and comprised five physical and psychosocial predictor variables that together yielded a positive likelihood ratio of 14.40 (95% CI: 3.66−56.69) and an 85% (95%CI: 60−96%) post-test probability of success. This is the first clinical prediction rule proposed for transcranial direct current stimulation in patients with chronic pain. The model underscores the importance of both physical and psychosocial factors as predictors of the analgesic response to transcranial direct current stimulation treatment. Validation of the proposed clinical prediction rule should be performed in other datasets.

Development and internal validation of a clinical prediction model for the diagnosis of immune thrombocytopenia

BACKGROUND

Immune thrombocytopenia (ITP) is a diagnosis of exclusion that can resemble other thrombocytopenic disorders.

OBJECTIVES

To develop a clinical prediction model (CPM) for the diagnosis of ITP to aid hematogists in investigating patients presenting with undifferentiated thrombocytopenia.

METHODS

We designed a CPM for ITP diagnosis at the time of the initial hematology consultation using penalized logistic regression based on data from patients with thrombocytopenia enrolled in the McMaster ITP registry (n = 523) called the Predict-ITP Tool. The case definition for ITP was a platelet count less than 100 × 109 /L and a platelet count response after high-dose corticosteroids or intravenous immune globulin, defined as the achievement of a platelet count above 50 × 109 /L and at least a doubling of baseline. Internal validation was done using bootstrap resampling. Model discrimination was assessed by the c-statistic, and calibration was assessed by the calibration slope, calibration-in-the-large, and calibration plot.

RESULTS

The final model included the following variables: (1) platelet count variability (based on three or more platelet count values), (2) lowest platelet count value, (3) maximum mean platelet volume, and (4) history of major bleeding (defined by the ITP bleeding scale). The optimism-corrected c-statistic was 0.83, the calibration slope was 0.88, and calibration-in-the-large for all performance measures was <0.001 with standard error <0.001, indicating good discrimination and excellent calibration.

CONCLUSIONS

The Predict-ITP Tool can estimate the likelihood of ITP for a given patient with thrombocytopenia at the time of the initial hematology consultation. The tool had high predictive accuracy for the diagnosis of ITP.

External Validation of the COVID-NoLab and COVID-SimpleLab Prognostic Tools

Our objective was to externally validate 2 simple risk scores for mortality among a mostly inpatient population with COVID-19 in Canada (588 patients for COVID-NoLab and 479 patients for COVID-SimpleLab). The mortality rates in the low-, moderate-, and high-risk groups for COVID-NoLab were 1.1%, 9.6%, and 21.2%, respectively. The mortality rates for COVID-SimpleLab were 0.0%, 9.8%, and 20.0%, respectively. These values were similar to those in the original derivation cohort. The 2 simple risk scores, now successfully externally validated, offer clinicians a reliable way to quickly identify low-risk inpatients who could potentially be managed as outpatients in the event of a bed shortage. Both are available online (https://ebell-projects.shinyapps.io/covid_nolab/ and https://ebell-projects.shinyapps.io/COVID-SimpleLab/).

Prediction of 2-year major adverse cardiac events from myocardial perfusion scintigraphy and clinical risk factors

BACKGROUND

We developed CRAX2MACE, a new tool derived from clinical and SPECT myocardial perfusion imaging (MPI) variables, to predict 2-year probability of major adverse cardiac event (MACE) comprising death, hospitalized acute myocardial infarction or coronary revascularization.

METHODS

Consecutive individuals with SPECT MPI 2001-2008 had two-year MACE determined from population-based health services data. CRAX2MACE included age, sex, diabetes, recent cardiac hospitalization, pharmacologic stress, stress total perfusion deficit (TPD), ischemic (stress-rest) TPD, left ventricular ejection fraction and transient ischemic dilation ratio. Two-year event rates were classified as low (< 5%), moderate (5.0-9.9%), high (10-19.9%) and very high (20% or greater).

RESULTS

The study population comprised 3896 individuals for the development and 1946 for the validation subgroups with subsequent MACE in 589 (15.1%) and 272 (14.0%), respectively. CRAX2MACE, derived from the development subgroups, accurately stratified MACE risk in the validation subgroup (area under the receiver operating characteristics curve 0.79) with stepwise increase in the observed event rate with increasing predicted risk category (low, 2.3%; moderate, 5.5%; high, 18.8%; very high 33.2%; P-trend < 0.001).

CONCLUSIONS

A simple tool based upon clinical risk factors and MPI variables predicts 2-year cardiac events. Risk stratification between the low and very high groups was greater than tenfold.

Performance of the model for end-stage liver disease score for mortality prediction and the potential role of etiology

BACKGROUND & AIMS

Although the discriminative ability of the model for end-stage liver disease (MELD) score is generally considered acceptable, its calibration is still unclear. In a validation study, we assessed the discriminative performance and calibration of 3 versions of the model: original MELD-TIPS, used to predict survival after transjugular intrahepatic portosystemic shunt (TIPS); classic MELD-Mayo; and MELD-UNOS, used by the United Network for Organ Sharing (UNOS). We also explored recalibrating and updating the model.

METHODS

In total, 776 patients who underwent elective TIPS (TIPS cohort) and 445 unselected patients (non-TIPS cohort) were included. Three, 6 and 12-month mortality predictions were calculated by the 3 MELD versions: discrimination was assessed by c-statistics and calibration by comparing deciles of predicted and observed risks. Cox and Fine and Grey models were used for recalibration and prognostic analyses.

RESULTS

In the TIPS/non-TIPS cohorts, the etiology of liver disease was viral in 402/188, alcoholic in 185/130, and non-alcoholic steatohepatitis in 65/33; mean follow-up±SD was 25±9/19±21 months; and the number of deaths at 3-6-12 months was 57-102-142/31-47-99, respectively. C-statistics ranged from 0.66 to 0.72 in TIPS and 0.66 to 0.76 in non-TIPS cohorts across prediction times and scores. A post hoc analysis revealed worse c-statistics in non-viral cirrhosis with more pronounced and significant worsening in the non-TIPS cohort. Calibration was acceptable with MELD-TIPS but largely unsatisfactory with MELD-Mayo and -UNOS whose performance improved much after recalibration. A prognostic analysis showed that age, albumin, and TIPS indication might be used to update the MELD.

CONCLUSIONS

In this validation study, the performance of the MELD score was largely unsatisfactory, particularly in non-viral cirrhosis. MELD recalibration and candidate variables for an update to the MELD score are proposed.

LAY SUMMARY

While the discriminative performance of the model for end-stage liver disease (MELD) score is credited to be fair to good, its calibration, the correspondence of observed to predicted mortality, is still unsettled. We found that application of 3 different versions of the MELD in 2 independent cirrhosis cohorts yielded largely imprecise mortality predictions particularly in non-viral cirrhosis. Thus, we propose a recalibration and suggest candidate variables for an update to the model.

External validation of a magnetic resonance imaging-based algorithm for prediction of side-specific extracapsular extension in prostate cancer

Introduction

Recently developed algorithm for prediction of side-specific extracapsular extension (ECE) of prostate cancer required validation before being recommended to use. The algorithm assumed that ECE on a particular side was not likely with same side maximum tumor diameter (MTD) <15 mm AND cancerous tissue in ipsilateral biopsy <15% AND PSA <20 ng/mL (both sides condition). The aim of the study was to validate this predictive tool in patients from another department.

Material and methods

Data of 154 consecutive patients (308 prostatic lateral lobes) were used for validation. Predictive factors chosen in the development set of patients were assessed together with other preoperative parameters using logistic regression to check for their significance. Sensitivity, specificity, negative and positive predictive values were calculated for bootstrapped risk-stratified validation dataset.

Results

Validation cohort did not differ significantly from development cohort regarding PSA, PSA density, Gleason score (GS), MTD, age, ECE and seminal vesicle invasion rate. In bootstrapped data set (n = 200 random sampling) algorithm revealed 70.2% sensitivity (95% confidence interval (CI) 58.8–83.0%), 49.9% specificity (95%CI: 42.0–57.7%), 83.9% negative predictive value (NPV; 95%CI: 76.1–91.4%) and 31.1% positive predictive value (PPV; 95%CI: 19.6–39.7%). When limiting analysis to high-risk patients (Gleason score >7) the algorithm improved its performance: sensitivity 91%, specificity 47%, PPV 53%, NPV 89%.

Conclusions

Analyzed algorithm is useful for identifying prostate lobes without ECE and deciding on ipsilateral nerve-sparing technique during radical prostatectomy, especially in patients with GS >7. Due to significant number of false positives in case of: MTD ≥15 mm OR cancer in biopsy ≥15% OR PSA ≥20 ng/mL additional evaluation is necessary to aid decision-making.

The prognostic value of respiratory symptoms and performance status in ambulatory cancer patients and unsuspected pulmonary embolism; analysis of an international, prospective, observational cohort study

BACKGROUND

Optimal risk stratification of unsuspected pulmonary embolism (UPE) in ambulatory cancer patients (ACPs) remains unclear. Existing clinical predictive rules (CPRs) are derived from retrospective databases and have limitations. The UPE registry is a prospective international registry with pre-specified characteristics of ACPs with a recent UPE. The aim of this study was to assess the utility of risk factors captured in the UPE registry in predicting proximate (30-, 90- and 180-day) mortality and how they performed when applied to an existing CPR.

OBJECTIVES

To evaluate risk factors for proximate mortality, overall survival, recurrent venous thromboembolism and major bleeding, in the patients enrolled in the UPE registry cohort.

METHODS

Data from the 695 ACPs in this registry were subjected to multivariate logistic regression analyses to identify predictors independently associated with proximate mortality and overall survival. The most consistent predictors were applied to the Hull CPR, an existing 5-point prediction rule.

RESULTS

The most consistent predictors of mortality were patient-reported respiratory symptoms within 14 days before, and ECOG performance status at the time of UPE. These predictors applied to the Hull-CPR produced a consistent correlation with proximate mortality and overall survival (area under the curve [AUC] = 0.70 [95% CI 0.63, 077], AUC = 0.65 [95% CI 0.60, 070], AUC = 0.64 [95% CI 0.59, 068], and AUC = 0.61, 95% CI 0.57, 0.65, respectively).

CONCLUSION

In ACPs with UPE, ECOG performance status logged contemporaneously to the UPE diagnosis and respiratory symptoms prior to UPE diagnosis can stratify mortality risk. When applied to the HULL-CPR these risk predictors confirmed the risk stratification clusters of low-intermediate and high-risk for proximate mortality as seen in the original derivation cohort.

Prehospital troponin as a predictor of early clinical deterioration

BACKGROUND AND OBJECTIVES

Elevated troponin T (cTnT) values are associated with comorbidities and early mortality, in both cardiovascular and noncardiovascular diseases. The objective of this study is to evaluate the prognostic accuracy of the sole utilization of prehospital point-of-care cardiac troponin T to identify the risk of early in-hospital deterioration, including mortality within 28 days.

METHODS

We conducted a prospective, multicentric, controlled, ambulance-based, observational study in adults with acute diseases transferred with high priority by ambulance to emergency departments, between 1 January and 30 September 2020. Patients with hospital diagnosis of acute coronary syndrome were excluded. The discriminative power of the predictive cTnT was assessed through a discrimination model trained using a derivation cohort and evaluated by the area under the curve of the receiver operating characteristic on a validation cohort.

RESULTS

A total of 848 patients were included in our study. The median age was 68 years (25th-75th percentiles: 50-81 years), and 385 (45.4%) were women. The mortality rate within 28 days was 12.4% (156 cases). The predictive ability of cTnT to predict mortality presented an area under the curve of 0.903 (95% CI: 0.85-0.954; P < .001). Risk stratification was performed, resulting in three categories with the following optimal cTnT cut-off points: high risk greater than or equal to 100, intermediate risk 40-100 and low risk less than 40 ng/L. In the high-risk group, the mortality rate was 61.7%, and on the contrary, the low-risk group presented a mortality of 2.3%.

CONCLUSIONS

The implementation of a routine determination of cTnT on the ambulance in patients transferred with high priority to the emergency department can help to stratify the risk of these patients and to detect unknown early clinical deterioration.

Age-Adjusted Endothelial Activation and Stress Index for Coronavirus Disease 2019 at Admission Is a Reliable Predictor for 28-Day Mortality in Hospitalized Patients With Coronavirus Disease 2019

Background: Endothelial Activation and Stress Index (EASIX) predict death in patients undergoing allogeneic hematopoietic stem cell transplantation who develop endothelial complications. Because coronavirus disease 2019 (COVID-19) patients also have coagulopathy and endotheliitis, we aimed to assess whether EASIX predicts death within 28 days in hospitalized COVID-19 patients. Methods: We performed a retrospective study on COVID-19 patients from two different cohorts [derivation (n = 1,200 patients) and validation (n = 1,830 patients)]. The endpoint was death within 28 days. The main factors were EASIX [(lactate dehydrogenase * creatinine)/thrombocytes] and aEASIX-COVID (EASIX * age), which were log2-transformed for analysis. Results: Log2-EASIX and log2-aEASIX-COVID were independently associated with an increased risk of death in both cohorts (p < 0.001). Log2-aEASIX-COVID showed a good predictive performance for 28-day mortality both in the derivation cohort (area under the receiver-operating characteristic = 0.827) and in the validation cohort (area under the receiver-operating characteristic = 0.820), with better predictive performance than log2-EASIX (p < 0.001). For log2 aEASIX-COVID, patients with low/moderate risk (<6) had a 28-day mortality probability of 5.3% [95% confidence interval (95% CI) = 4-6.5%], high (6-7) of 17.2% (95% CI = 14.7-19.6%), and very high (>7) of 47.6% (95% CI = 44.2-50.9%). The cutoff of log2 aEASIX-COVID = 6 showed a positive predictive value of 31.7% and negative predictive value of 94.7%, and log2 aEASIX-COVID = 7 showed a positive predictive value of 47.6% and negative predictive value of 89.8%. Conclusion: Both EASIX and aEASIX-COVID were associated with death within 28 days in hospitalized COVID-19 patients. However, aEASIX-COVID had significantly better predictive performance than EASIX, particularly for discarding death. Thus, aEASIX-COVID could be a reliable predictor of death that could help to manage COVID-19 patients.

Appendicitis Inflammatory Response Score in Comparison to Alvarado Score in Acute Appendicitis

Appendicitis is a common differential diagnosis of right lower quadrant pain. Clinical evaluation alone results in high negative appendicectomy rates. Alvarado scoring is the most commonly used clinical prediction rule. The study aimed to compare the recently developed appendicitis inflammatory response (AIR) score with the Alvarado score. This cross-sectional observational study included patients who underwent appendicectomy for clinical suspicion of appendicitis. The clinical and laboratory parameters required for obtaining Alvarado score and AIRS were gathered. Area under ROC curve was calculated for both Alvarado score and AIRS. The study included 130 patients (77 males and 53 females). The negative appendicectomy rate was 10.7%. The perforation rate was 10.3%. The area under ROC for Alvarado score was 0.821 and for AIR score was 0.901. The Alvarado score had a sensitivity of 72% and a specificity of 79% at score ≥6. The appendicitis inflammatory response score had a sensitivity of 98% for scores ≥5 and a specificity of 97% for score ≥6. The C-reactive protein (CRP) value was the best performing individual parameter with an area under ROC of 0.789, followed by WBC count with an area under ROC of 0.762. Appendicitis inflammatory response score is a recently developed score that outperforms the Alvarado score. AIR score has a higher specificity. The sound construction, gradation of parameters, the inclusion of CRP, and avoidance of subjective parameters make the AIR score an attractive clinical prediction rule which can decrease the rate of negative appendicectomy.

The Effect of Amoxicillin in Adult Patients Presenting to Primary Care with Acute Cough Predicted to Have Pneumonia or a Combined Viral-Bacterial Infection

While most cases of acute cough are self-limiting, antibiotics are prescribed to over 50%. This proportion is inappropriately high given that benefit from treatment with amoxicillin could only be demonstrated in adults with pneumonia (based on chest radiograph) or combined viral-bacterial infection (based on modern microbiological methodology). As routine use of chest radiographs and microbiological testing is costly, clinical prediction rules could be used to identify these patient subsets. In this secondary analysis of data from a multicentre randomised controlled trial in adults presenting to primary care with acute cough, we used prediction rules for pneumonia or combined infection and assessed the effect of amoxicillin in patients predicted to have pneumonia or combined infection on symptom duration, symptom severity and illness deterioration. In total, 2056 patients that fulfilled all inclusion criteria were randomised, 1035 to amoxicillin, 1021 to placebo. Neither patients with a predicted pneumonia nor patients with a predicted combined infection were significantly more likely to benefit from amoxicillin. While the studied clinical prediction rules may help primary care clinicians to reduce antibiotic prescribing for low-risk patients, they did not identify adult acute cough patients that would benefit from amoxicillin treatment.

A Diagnostic Assistant Tool for Work-Related Low Back Pain in Hospital Workers

Aims:

The study objective was to develop a clinical risk score to assist occupational medicine physicians in diagnosing hospital workers’ occupational lower back pain (LBP).

Settings and Design:

A cross-sectional data collection design was conducted at Saraburi Hospital, Thailand.

Methods and Materials:

The sample consisted of 220 hospital workers who cared for patients and had LBP. They were assessed for the frequency of targeted activities (CPR, lifting, transferring patients) and other activities from work as well as ergonomic assessments, and diagnosed with LBP by three occupational medicine physicians.

Statistical Analysis Used:

Predicted factors of multivariable logistic regression were analysed to find clinical risk scores to help the diagnosis.

Results:

The physicians agreed on the diagnosis, based on ergonomic risk factors and their experiences that 86 persons have occupational LBP. A diagnostic assistant tool consists of six predictors: the duration of LBP, having LBP within the last 7 days, bending, twisting, lateral bending, and reaching. The scores predicted occupational LBP correctly with an AuROC of 90.0% (95% CI; 84.8–93.5%). The positive likelihood ratio for occupational LBP was 0 in the low risk category (<6 points) and 16.8 (95% CI; 8.0–35.6) in the high risk (>8 points).

Conclusions:

A diagnostic assistant tool is used to assist occupational medicine physicians in diagnosing hospital workers' occupational LBP.

A predictive model for postthrombotic syndrome in proximal deep vein thrombosis patients

Background

Postthrombotic syndrome (PTS) is the most common long-term complication of deep vein thrombosis (DVT). Predictive models for PTS after hospitalized DVT patients, especially those with proximal DVT for whom preventative intervention decisions need to be made, are rare. We aimed to develop and externally validate a clinical predictive model for PTS in patients with proximal DVT.

Methods

This study was a retrospective, single-center, case-control study. The data used in our model were retrospectively collected from a prospective registry database in which 210 (derivation) and 90 (validation) consecutive patients were first diagnosed with proximal DVT. We developed a nomogram using the multivariate logistic regression model. External validation of our predictive model and previous predictive models in our validation set was assessed by discrimination, calibration, and clinical utility.

Results

Of the 30 candidate predictors, 5 were significantly associated with PTS in our final multivariable model, including the number of signs and symptoms (OR 1.33, 95% CI: 1.17 to 1.53, P<0.001), male sex (OR 1.79, 95% CI: 1.07 to 3.06, P=0.028), varicose vein history (OR 3.02, 95% CI: 1.04 to 7.60, P<0.001), BMI (OR 1.06, 95% CI: 1.00 to 1.12, P=0.052), and chronic DVT (OR 2.66, 95% CI: 1.49 to 4.79, P<0.001). The area under the curve was 0.724 in our predictive model, indicating suitable external performance.

Conclusions

A simple-to-use nomogram effectively predicts the risk of PTS in patients with proximal DVT. This predictive model may be considered for use in clinical care.

Development of a prediction model of treatment response in patients with cutaneous arteritis: Insights from a cohort of 33 patients

Cutaneous arteritis (CA) is necrotizing vasculitis invading the small- to medium-sized arteries of the skin. The majority of patients can be favorably managed by low- to medium-dose systemic corticosteroids (prednisolone, <0.5 mg/kg/day) or other oral medications such as non-steroidal anti-inflammatory drugs, dapsone, and azathioprine. Meanwhile, some patients require more intensive therapy including high-dose systemic corticosteroids (prednisolone, ≥0.5 mg/kg/day), i.v. immunoglobulin, and i.v. cyclophosphamide therapy. Although predicting such treatment response among CA patients is critical in clinical decision-making, prediction rules have not yet been established. Herein, we retrospectively reviewed 33 patients regularly visiting our clinic to reveal predictive factors of their treatment response. Clinical data were collected from electronic medical records. Association between each factor and treatment response was examined by logistic regression analysis. Progression-free time was calculated by Kaplan-Meier's method and analyzed by log-rank test and Cox progression hazard model. Potential predictive factors were selected, given 1 point for each, and integrated into a classification model. Discrimination of the model was examined by the receiver operating characteristic (ROC) curve analysis. In total, 33 CA patients were enrolled in our study. Of these, 11 patients required intensive therapy, classified as treatment non-responders. Logistic analyses revealed that treatment response was significantly associated with male sex, presence of skin ulcers, and elevated serum levels of C-reactive protein at the initial work-up. Kaplan-Meier analyses also demonstrated that those factors are predictive of progression-free time. The area under the ROC curve of our classification model was 0.92 (95% confidence interval, 0.83-1.00), which classified non-responders from the others with a sensitivity of 90.9% and specificity of 81.8% at the cut-off point of 2 or more. Collectively, treatment response of CA could be predictable by a combination of sex, presence of skin ulcers, and serum levels of C-reactive protein.

A Diagnostic Algorithm Based on a Simple Clinical Prediction Rule for the Diagnosis of Cranial Giant Cell Arteritis

Background: Risk stratification based on pre-test probability may improve the diagnostic accuracy of temporal artery high-resolution compression sonography (hrTCS) in the diagnostic workup of cranial giant cell arteritis (cGCA). Methods: A logistic regression model with candidate items was derived from a cohort of patients with suspected cGCA (n = 87). The diagnostic accuracy of the model was tested in the derivation cohort and in an independent validation cohort (n = 114) by receiver operator characteristics (ROC) analysis. The clinical items were composed of a clinical prediction rule, integrated into a stepwise diagnostic algorithm together with C-reactive protein (CRP) values and hrTCS values. Results: The model consisted of four clinical variables (age > 70, headache, jaw claudication, and anterior ischemic optic neuropathy). The diagnostic accuracy of the model for discrimination of patients with and without a final clinical diagnosis of cGCA was excellent in both cohorts (area under the curve (AUC) 0.96 and AUC 0.92, respectively). The diagnostic algorithm improved the positive predictive value of hrCTS substantially. Within the algorithm, 32.8% of patients (derivation cohort) and 49.1% (validation cohort) would not have been tested by hrTCS. None of these patients had a final diagnosis of cGCA. Conclusion: A diagnostic algorithm based on a clinical prediction rule improves the diagnostic accuracy of hrTCS.

Using a clinical prediction rule to prioritize diagnostic testing leads to reduced transmission and hospital burden: A modeling example of early SARS-CoV-2

BACKGROUND

Prompt identification of infections is critical for slowing the spread of infectious diseases. However, diagnostic testing shortages are common in emerging diseases, low resource settings, and during outbreaks. This forces difficult decisions regarding who receives a test, often without knowing the implications of those decisions on population-level transmission dynamics. Clinical prediction rules (CPRs) are commonly used tools to guide clinical decisions.

METHODS

Using early SARS-CoV-2 as an example, we used data from electronic health records to develop a parsimonious 5-variable CPR to identify those who are most likely to test positive. To consider the implications of gains in daily case detection at the population level, we incorporated testing using the CPR into a compartmentalized model of SARS-CoV-2.

RESULTS

We found that applying this CPR (AUC: 0.69 (95% CI: 0.68 - 0.70)) to prioritize testing increased the proportion of those testing positive in settings of limited testing capacity. We found that prioritized testing led to a delayed and lowered infection peak (i.e., "flattens the curve"), with the greatest impact at lower values of the effective reproductive number (such as with concurrent community mitigation efforts), and when higher proportions of infectious persons seek testing. Additionally, prioritized testing resulted in reductions in overall infections as well as hospital and intensive care unit (ICU) burden.

CONCLUSION

We highlight the population-level benefits of evidence-based allocation of limited diagnostic capacity.

A modular approach to integrating multiple data sources into real-time clinical prediction for pediatric diarrhea

Traditional clinical prediction models focus on parameters of the individual patient. For infectious diseases, sources external to the patient, including characteristics of prior patients and seasonal factors, may improve predictive performance. We describe the development of a predictive model that integrates multiple sources of data in a principled statistical framework using a post-test odds formulation. Our method enables electronic real-time updating and flexibility, such that components can be included or excluded according to data availability. We apply this method to the prediction of etiology of pediatric diarrhea, where 'pre-test’ epidemiologic data may be highly informative. Diarrhea has a high burden in low-resource settings, and antibiotics are often over-prescribed. We demonstrate that our integrative method outperforms traditional prediction in accurately identifying cases with a viral etiology, and show that its clinical application, especially when used with an additional diagnostic test, could result in a 61% reduction in inappropriately prescribed antibiotics.

Matching depression management to severity prognosis in primary care: results of the Target-D randomised controlled trial

BACKGROUND

Mental health treatment rates are increasing, but the burden of disease has not reduced. Tools to support efficient resource distribution are required.

AIM

To investigate whether a person-centred e-health (Target-D) platform matching depression care to symptom severity prognosis can improve depressive symptoms relative to usual care.

DESIGN AND SETTING

Stratified individually randomised controlled trial in 14 general practices in Melbourne, Australia, from April 2016 to February 2019. In total, 1868 participants aged 18-65 years who had current depressive symptoms; internet access; no recent change to antidepressant; no current antipsychotic medication; and no current psychological therapy were randomised (1:1) via computer-generated allocation to intervention or usual care.

METHOD

The intervention was an e-health platform accessed in the GP waiting room, comprising symptom feedback, priority-setting, and prognosis-matched management options (online self-help, online guided psychological therapy, or nurse-led collaborative care). Management options were flexible, neither participants nor staff were blinded, and there were no substantive protocol deviations. The primary outcome was depressive symptom severity (9-item Patient Health Questionnaire [PHQ-9]) at 3 months.

RESULTS

In intention to treat analysis, estimated between- arm difference in mean PHQ-9 scores at 3 months was -0.88 (95% confidence interval [CI] = -1.45 to -0.31) favouring the intervention, and -0.59 at 12 months (95% CI = -1.18 to 0.01); standardised effect sizes of -0.16 (95% CI = -0.26 to -0.05) and -0.10 (95% CI = -0.21 to 0.002), respectively. No serious adverse events were reported.

CONCLUSION

Matching management to prognosis using a person-centred e-health platform improves depressive symptoms at 3 months compared to usual care and could feasibly be implemented at scale. Scope exists to enhance the uptake of management options.

Machine Learning to Identify Persons at High-Risk of Human Immunodeficiency Virus Acquisition in Rural Kenya and Uganda

BACKGROUND

In generalized epidemic settings, strategies are needed to prioritize individuals at higher risk of human immunodeficiency virus (HIV) acquisition for prevention services. We used population-level HIV testing data from rural Kenya and Uganda to construct HIV risk scores and assessed their ability to identify seroconversions.

METHODS

During 2013-2017, >75% of residents in 16 communities in the SEARCH study were tested annually for HIV. In this population, we evaluated 3 strategies for using demographic factors to predict the 1-year risk of HIV seroconversion: membership in ≥1 known "risk group" (eg, having a spouse living with HIV), a "model-based" risk score constructed with logistic regression, and a "machine learning" risk score constructed with the Super Learner algorithm. We hypothesized machine learning would identify high-risk individuals more efficiently (fewer persons targeted for a fixed sensitivity) and with higher sensitivity (for a fixed number targeted) than either other approach.

RESULTS

A total of 75 558 persons contributed 166 723 person-years of follow-up; 519 seroconverted. Machine learning improved efficiency. To achieve a fixed sensitivity of 50%, the risk-group strategy targeted 42% of the population, the model-based strategy targeted 27%, and machine learning targeted 18%. Machine learning also improved sensitivity. With an upper limit of 45% targeted, the risk-group strategy correctly classified 58% of seroconversions, the model-based strategy 68%, and machine learning 78%.

CONCLUSIONS

Machine learning improved classification of individuals at risk of HIV acquisition compared with a model-based approach or reliance on known risk groups and could inform targeting of prevention strategies in generalized epidemic settings.

CLINICAL TRIALS REGISTRATION

NCT01864603.

Comparison of Multivariable Logistic Regression and Other Machine Learning Algorithms for Prognostic Prediction Studies in Pregnancy Care: Systematic Review and Meta-Analysis

Background

Predictions in pregnancy care are complex because of interactions among multiple factors. Hence, pregnancy outcomes are not easily predicted by a single predictor using only one algorithm or modeling method.

Objective

This study aims to review and compare the predictive performances between logistic regression (LR) and other machine learning algorithms for developing or validating a multivariable prognostic prediction model for pregnancy care to inform clinicians’ decision making.

Methods

Research articles from MEDLINE, Scopus, Web of Science, and Google Scholar were reviewed following several guidelines for a prognostic prediction study, including a risk of bias (ROB) assessment. We report the results based on the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Studies were primarily framed as PICOTS (population, index, comparator, outcomes, timing, and setting): Population: men or women in procreative management, pregnant women, and fetuses or newborns; Index: multivariable prognostic prediction models using non-LR algorithms for risk classification to inform clinicians’ decision making; Comparator: the models applying an LR; Outcomes: pregnancy-related outcomes of procreation or pregnancy outcomes for pregnant women and fetuses or newborns; Timing: pre-, inter-, and peripregnancy periods (predictors), at the pregnancy, delivery, and either puerperal or neonatal period (outcome), and either short- or long-term prognoses (time interval); and Setting: primary care or hospital. The results were synthesized by reporting study characteristics and ROBs and by random effects modeling of the difference of the logit area under the receiver operating characteristic curve of each non-LR model compared with the LR model for the same pregnancy outcomes. We also reported between-study heterogeneity by using τ² and I².

Results

Of the 2093 records, we included 142 studies for the systematic review and 62 studies for a meta-analysis. Most prediction models used LR (92/142, 64.8%) and artificial neural networks (20/142, 14.1%) among non-LR algorithms. Only 16.9% (24/142) of studies had a low ROB. A total of 2 non-LR algorithms from low ROB studies significantly outperformed LR. The first algorithm was a random forest for preterm delivery (logit AUROC 2.51, 95% CI 1.49-3.53; I²=86%; τ²=0.77) and pre-eclampsia (logit AUROC 1.2, 95% CI 0.72-1.67; I²=75%; τ²=0.09). The second algorithm was gradient boosting for cesarean section (logit AUROC 2.26, 95% CI 1.39-3.13; I²=75%; τ²=0.43) and gestational diabetes (logit AUROC 1.03, 95% CI 0.69-1.37; I²=83%; τ²=0.07).

Conclusions

Prediction models with the best performances across studies were not necessarily those that used LR but also used random forest and gradient boosting that also performed well. We recommend a reanalysis of existing LR models for several pregnancy outcomes by comparing them with those algorithms that apply standard guidelines.

Trial Registration

PROSPERO (International Prospective Register of Systematic Reviews) CRD42019136106; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=136106

Two Clinical Prediction Tools to Improve Tuberculosis Contact Investigation

BACKGROUND

Efficient contact investigation strategies are needed for the early diagnosis of tuberculosis (TB) disease and treatment of latent TB infections.

METHODS

Between September 2009 and August 2012, we conducted a prospective cohort study in Lima, Peru, in which we enrolled and followed 14 044 household contacts of adults with pulmonary TB. We used information from a subset of this cohort to derive 2 clinical prediction tools that identify contacts of TB patients at elevated risk of progressing to active disease by training multivariable models that predict (1) coprevalent TB among all household contacts and (2) 1-year incident TB among adult contacts. We validated the models in a geographically distinct subcohort and compared the relative utilities of clinical decisions based on these tools to existing strategies.

RESULTS

In our cohort, 296 (2.1%) household contacts had coprevalent TB and 145 (1.9%) adult contacts developed incident TB within 1 year of index patient diagnosis. We predicted coprevalent disease using information that could be readily obtained at the time an index patient was diagnosed and predicted 1-year incident TB by including additional contact-specific characteristics. The area under the receiver operating characteristic curves for coprevalent TB and incident TB were 0.86 (95% confidence interval [CI], .83-.89]) and 0.72 (95% CI, .67-.77), respectively. These clinical tools give 5%-10% higher relative utilities than existing methods.

CONCLUSIONS

We present 2 tools that identify household contacts at high risk for TB disease based on reportable information from patient and contacts alone. The performance of these tools is comparable to biomarkers that are both more costly and less feasible than this approach.

Predicting risk of multiple levels of recurrence and progression after initial diagnosis of nonmuscle-invasive bladder cancer in a multisite, community-based cohort

BACKGROUND

Nonmuscle-invasive bladder cancer (NMIBC) has heterogeneous recurrence and progression outcomes. Available risk calculators estimate recurrence and progression but do not predict the recurrence stage or grade, which may influence downstream treatment. The objective of this study was to predict risk-stratified NMIBC recurrence and progression based on recurrence tumor classification and grade.

METHODS

In total, 2956 patients with NMIBC (<T2) who were diagnosed at Kaiser Permanente Northwest and Geisinger from 1994 to 2015 were identified. Recurrences were annotated for tumor classification and grade. Four risk-stratified outcomes were created based on the tumor classification and grade of the recurrence: 1) any recurrence, 2) intermediate-risk recurrence (Ta high grade, carcinoma in situ, T1 low grade) or higher, 3) high-risk recurrence (T1 high grade) or progression (clinical T2), and 4) progression. Multivariable Cox proportional hazards regression was used to compute 1-year and 5-year risk estimates for each outcome based on initial tumor classification and grade.

RESULTS

Over a median follow-up of 29.4 months, there were 1062 recurrences (35.9%), including 111 progressions (3.8%). The adjusted hazard of high-risk recurrence or progression increased, depending on initial tumor classification and grade: The adjusted hazard ratio was 2.60 (95% CI, 1.62-4.15) for Ta high-grade tumors, 4.74 (95% CI, 3.01-7.47) for tumor in situ or Ta with carcinoma in situ, and 7.14 (95% CI, 4.97-10.26) T1 high-grade tumors. Using Ta high-grade tumors as an example, the 1-year and 5-year predicted rates of adjusted risk of a high-risk recurrence or progression were 4.4% and 7.9%, respectively.

CONCLUSIONS

The 1-year and 5-year predicted risk of high-risk recurrences and progression increased with higher tumor classification and grade at diagnosis. These granular risk estimates may further inform risk-stratified treatment and surveillance for patients with NMIBC.

AutoScore: A Machine Learning-Based Automatic Clinical Score Generator and Its Application to Mortality Prediction Using Electronic Health Records

BACKGROUND

Risk scores can be useful in clinical risk stratification and accurate allocations of medical resources, helping health providers improve patient care. Point-based scores are more understandable and explainable than other complex models and are now widely used in clinical decision making. However, the development of the risk scoring model is nontrivial and has not yet been systematically presented, with few studies investigating methods of clinical score generation using electronic health records.

OBJECTIVE

This study aims to propose AutoScore, a machine learning-based automatic clinical score generator consisting of 6 modules for developing interpretable point-based scores. Future users can employ the AutoScore framework to create clinical scores effortlessly in various clinical applications.

METHODS

We proposed the AutoScore framework comprising 6 modules that included variable ranking, variable transformation, score derivation, model selection, score fine-tuning, and model evaluation. To demonstrate the performance of AutoScore, we used data from the Beth Israel Deaconess Medical Center to build a scoring model for mortality prediction and then compared the data with other baseline models using the receiver operating characteristic analysis. A software package in R 3.5.3 (R Foundation) was also developed to demonstrate the implementation of AutoScore.

RESULTS

Implemented on the data set with 44,918 individual admission episodes of intensive care, the AutoScore-created scoring models performed comparably well as other standard methods (ie, logistic regression, stepwise regression, least absolute shrinkage and selection operator, and random forest) in terms of predictive accuracy and model calibration but required fewer predictors and presented high interpretability and accessibility. The nine-variable, AutoScore-created, point-based scoring model achieved an area under the curve (AUC) of 0.780 (95% CI 0.764-0.798), whereas the model of logistic regression with 24 variables had an AUC of 0.778 (95% CI 0.760-0.795). Moreover, the AutoScore framework also drives the clinical research continuum and automation with its integration of all necessary modules.

CONCLUSIONS

We developed an easy-to-use, machine learning-based automatic clinical score generator, AutoScore; systematically presented its structure; and demonstrated its superiority (predictive performance and interpretability) over other conventional methods using a benchmark database. AutoScore will emerge as a potential scoring tool in various medical applications.

The STAPH Score: A Predictor of Staphylococcus aureus as the Causative Microorganism of Native Vertebral Osteomyelitis

Background

Staphylococcus aureus (SA) is the most common causative microorganism in native vertebral osteomyelitis (NVO). Few studies have compared the clinical features of NVO due to SA (SA-NVO) and NVO due to other organisms (NSA-NVO). This study was conducted to validate a predictive score for SA-NVO to facilitate NVO treatment without broad-spectrum antimicrobial agents.

Methods

This retrospective study compared the clinical features of patients with SA-NVO and NSA-NVO who were diagnosed from 2004 to 2019. Univariate associations were assessed using χ ², Fisher's exact, or Mann-Whitney U test. Multivariable analysis was conducted using logistic regression. The optimal age cutoff point was determined by classification and regression tree analysis.

Results

Among 155 NVO patients, 98 (63.2%) had a microbiologically confirmed diagnosis: 40 (25.8%) with SA-NVO and 58 (37.4%) with NSA-NVO. Six predictors, either independently associated with SA-NVO or clinically relevant, were used to develop the STAPH prediction score: atopic dermatitis (Skin) (3 points); recent Trauma (2 points); Age < 67 years (1 point); Abscess (1 point); central venous Port catheter (2 points); and History of puncture (2 points). In a receiver operating characteristic analysis, the area under the curve was 0.84 (95% confidence interval, 0.76-0.91). The best cutoff point was 3. A score ≥3 had a sensitivity, specificity, positive predictive value, and negative predictive value of 58%, 84%, 84%, and 73%, respectively.

Conclusions

The STAPH score has relatively high specificity for use by clinicians to predict SA as the causative microorganism in patients with NVO until results of a confirmatory culture are available.

Recent advances in understanding, diagnosing and treating venous thrombosis

Focusing on the current state of the art, this article (a) describes recent advances in the understanding of the pathogenesis of venous thromboembolism (VTE), (b) discusses current approaches for the prevention, diagnosis and treatment of VTE, (c) outlines the role of aspirin for VTE prevention and treatment, and (d) highlights the unmet needs in VTE management and describes novel approaches to address them.

Emergency admission risk stratification tools in UK primary care: a cross-sectional survey of availability and use

Background

Stratifying patient populations by risk of adverse events was believed to support preventive care for those identified, but recent evidence does not support this. Emergency admission risk stratification (EARS) tools have been widely promoted in UK policy and GP contracts.

Aim

To describe availability and use of EARS tools across the UK, and identify factors perceived to influence implementation.

Design and setting

Cross-sectional survey in UK.

Method

Online survey of 235 organisations responsible for UK primary care: 209 clinical commissioning groups (CCGs) in England; 14 health boards in Scotland; seven health boards in Wales; and five local commissioning groups (LCGs) in Northern Ireland. Analysis results are presented using descriptive statistics for closed questions and by theme for open questions.

Results

Responses were analysed from 171 (72.8%) organisations, of which 148 (86.5%) reported that risk tools were available in their areas. Organisations identified 39 different EARS tools in use. Promotion by NHS commissioners, involvement of clinical leaders, and engagement of practice managers were identified as the most important factors in encouraging use of tools by general practices. High staff workloads and information governance were identified as important barriers. Tools were most frequently used to identify individual patients, but also for service planning. Nearly 40% of areas using EARS tools reported introducing or realigning services as a result, but relatively few reported use for service evaluation.

Conclusion

EARS tools are widely available across the UK, although there is variation by region. There remains a need to align policy and practice with research evidence.

Prognostic models versus single risk factor approach in first-trimester selective screening for gestational diabetes mellitus: a prospective population-based multicentre cohort study

Objectives

To evaluate whether (1) first‐trimester prognostic models for gestational diabetes mellitus (GDM) outperform the currently used single risk factor approach, and (2) a first‐trimester random venous glucose measurement improves model performance.

Design

Prospective population‐based multicentre cohort.

Setting

Thirty‐one independent midwifery practices and six hospitals in the Netherlands.

Population

Women recruited before 14 weeks of gestation without pre‐existing diabetes.

Methods

The single risk factor approach (presence of at least one risk factor: BMI ≥30 kg/m², previous macrosomia, history of GDM, positive first‐degree family history of diabetes, non‐western ethnicity) was compared with the four best performing models in our previously published external validation study (Gabbay‐Benziv 2014, Nanda 2011, Teede 2011, van Leeuwen 2010) with and without the addition of glucose.

Main outcome measures

Discrimination was assessed by c‐statistics, calibration by calibration plots, added value of glucose by the likelihood ratio chi‐square test, net benefit by decision curve analysis and reclassification by reclassification plots.

Results

Of the 3723 women included, a total of 181 (4.9%) developed GDM. The c‐statistics of the prognostic models were higher, ranging from 0.74 to 0.78 without glucose and from 0.78 to 0.80 with glucose, compared with the single risk factor approach (0.72). Models showed adequate calibration, and yielded a higher net benefit than the single risk factor approach for most threshold probabilities. Teede 2011 performed best in the reclassification analysis.

Conclusions

First‐trimester prognostic models seem to outperform the currently used single risk factor approach in screening for GDM, particularly when glucose was added as a predictor.

Tweetable abstract

Prognostic models seem to outperform the currently used single risk factor approach in screening for gestational diabetes.

Prognostic models seem to outperform the currently used single risk factor approach in screening for gestational diabetes.

Evaluating an Algorithm and Clinical Prediction Rule for Diagnosis of Bone Stress Injuries

BACKGROUND

A novel algorithm and clinical prediction rule (CPR), with 18 variables, was created in 2014. The CPR generated a bone stress injury (BSI) score, which was used to determine the necessity of imaging in suspected BSI. To date, there are no validated algorithms for imaging selection in patients with suspected BSI.

HYPOTHESIS

A simplified CPR will assist clinicians with diagnosis and decision making in patients with suspected BSI.

STUDY DESIGN

Prospective cohort study.

LEVEL OF EVIDENCE

Level 3.

METHODS

A total of 778 military trainees with lower extremity pain were enrolled. All trainees were evaluated for 18 clinical variables suggesting BSI. Participants were monitored via electronic medical record review. Then, a prediction model was developed using logistic regression to identify clinical variables with the greatest predictive value and assigned appropriate weight. Test characteristics for various BSI score thresholds were calculated.

RESULTS

Of the enrolled trainees, 204 had imaging-confirmed BSI in or distal to the femoral condyles. The optimized CPR selected 4 clinical variables (weighted score): bony tenderness (3), prior history of BSI (2), pes cavus (2), and increased walking/running volume (1). The optimized CPR with a score ≥3 yielded 97.5% sensitivity, 54.2% specificity, and 98.2% negative predictive value. An isolated measure, bony tenderness, demonstrated similar statistical performance.

CONCLUSION

The optimized CPR, which uses bony tenderness, prior history of BSI, pes cavus, and increased walking/running volume, is valid for detecting BSI in or distal to the femoral condyles. However, bony tenderness alone provides a simpler criterion with an equally strong negative predictive value for BSI decision making.

CLINICAL RELEVANCE

For suspected BSI in or distal to the femoral condyles, imaging can be deferred when there is no bony tenderness. When bony tenderness is present in the setting of 1 or more proven risk factors and no clinical evidence of high-risk bone involvement, presumptive treatment for BSI and serial radiographs may be appropriate.

Simplified Prehospital Prediction Rule to Estimate the Likelihood of 4 Types of Stroke: The 7-Item Japan Urgent Stroke Triage (JUST-7) Score

OBJECTIVE

Prehospital prediction models to estimate the likelihood of several types of stroke (large vessel occlusion [LVO], intracranial hemorrhage [ICH], and subarachnoid hemorrhage [SAH], and other types of stroke) should be useful to transfer those with suspected stroke to appropriate facilities. We recently reported Japan Urgent Stroke Triage (JUST) score with 21 items had excellent predictive abilities, and we further tried to simplify the score with parsimonious items and comparable predictive abilities.

METHODS

We conducted historical and prospective multicenter cohort studies at 8 centers from June 2015 to March 2018. We developed the prediction rules with select variables from JUST score for LVO, ICH, SAH and other types of stroke in 2236 patients with suspected stroke in historical derivation cohort. We validated the developed prediction rules in 964 patients in prospective validation cohort.

RESULTS

There were 1150 stroke, including 235 LVO, 352 ICH, 107 SAH and 456 other types of stroke in the derivation cohort. We developed the scores with 7 items (high blood pressure, arrhythmia, conjugate deviation, headache, dysarthria, disturbance of consciousness, paralysis of upper limbs) and the developed scores had area under the receiver-operating curve (AUC) of 0.84 for any type of stroke, 0.89 for LVO, 0.79 for ICH, and 0.90 for SAH in the derivation cohort. There were 490 stroke, including 102 LVO, 138 ICH, 28 SAH and 222 other types of stroke in the validation cohort. The scores well discriminated these strokes in the validation cohort (AUC of 0.76 for any type of stroke; 0.81 for LVO, 0.73 for ICH, and 0.85 for SAH).

CONCLUSIONS

The simplified 7-item JUST (JUST-7) score had good predictive ability and can help healthcare providers to estimate the likelihood of different types of stroke and decide the referral hospital.

Evaluating the Impact of the Grading and Assessment of Predictive Tools Framework on Clinicians and Health Care Professionals' Decisions in Selecting Clinical Predictive Tools: Randomized Controlled Trial

BACKGROUND

While selecting predictive tools for implementation in clinical practice or for recommendation in clinical guidelines, clinicians and health care professionals are challenged with an overwhelming number of tools. Many of these tools have never been implemented or evaluated for comparative effectiveness. To overcome this challenge, the authors developed and validated an evidence-based framework for grading and assessment of predictive tools (the GRASP framework). This framework was based on the critical appraisal of the published evidence on such tools.

OBJECTIVE

The aim of the study was to examine the impact of using the GRASP framework on clinicians' and health care professionals' decisions in selecting clinical predictive tools.

METHODS

A controlled experiment was conducted through a web-based survey. Participants were randomized to either review the derivation publications, such as studies describing the development of the predictive tools, on common traumatic brain injury predictive tools (control group) or to review an evidence-based summary, where each tool had been graded and assessed using the GRASP framework (intervention group). Participants in both groups were asked to select the best tool based on the greatest validation or implementation. A wide group of international clinicians and health care professionals were invited to participate in the survey. Task completion time, rate of correct decisions, rate of objective versus subjective decisions, and level of decisional conflict were measured.

RESULTS

We received a total of 194 valid responses. In comparison with not using GRASP, using the framework significantly increased correct decisions by 64%, from 53.7% to 88.1% (88.1/53.7=1.64; t₁₉₃=8.53; P<.001); increased objective decision making by 32%, from 62% (3.11/5) to 82% (4.10/5; t₁₈₉=9.24; P<.001); decreased subjective decision making based on guessing by 20%, from 49% (2.48/5) to 39% (1.98/5; t₁₈₈=-5.47; P<.001); and decreased prior knowledge or experience by 8%, from 71% (3.55/5) to 65% (3.27/5; t₁₈₇=-2.99; P=.003). Using GRASP significantly decreased decisional conflict and increased the confidence and satisfaction of participants with their decisions by 11%, from 71% (3.55/5) to 79% (3.96/5; t₁₈₈=4.27; P<.001), and by 13%, from 70% (3.54/5) to 79% (3.99/5; t₁₈₈=4.89; P<.001), respectively. Using GRASP decreased the task completion time, on the 90th percentile, by 52%, from 12.4 to 6.4 min (t₁₉₃=-0.87; P=.38). The average System Usability Scale of the GRASP framework was very good: 72.5% and 88% (108/122) of the participants found the GRASP useful.

CONCLUSIONS

Using GRASP has positively supported and significantly improved evidence-based decision making. It has increased the accuracy and efficiency of selecting predictive tools. GRASP is not meant to be prescriptive; it represents a high-level approach and an effective, evidence-based, and comprehensive yet simple and feasible method to evaluate, compare, and select clinical predictive tools.

Prediction Model for Timing of Death in Potential Donors After Circulatory Death (DCD III): Protocol for a Multicenter Prospective Observational Cohort Study

Background

Controlled donation after circulatory death (cDCD) is a major source of organs for transplantation. A potential cDCD donor poses considerable challenges in terms of identification of those dying within the predefined time frame of warm ischemia after withdrawal of life-sustaining treatment (WLST) to circulatory arrest. Several attempts have been made to develop models predicting the time between treatment withdrawal and circulatory arrest. This time window determines whether organ donation can occur and influences the quality of the donated organs. However, the selected patients used for these models were not always restricted to potential cDCD donors (eg, patients with cancer or severe infections were also included). This severely limits the generalizability of those data.

Objective

The objectives of this study are the following: (1) to develop a model predicting time to death within 60 minutes in potential cDCD patients; (2) to validate and update previous prediction models on time to death after WLST; (3) to determine timing and patient characteristics that are associated with prognostication and the decision-making process that leads to initiating end-of-life care; (4) to evaluate the impact of timing of family approach on organ donation approval; and (5) to assess the influence of variation in WLST processes on postmortem organ donor potential and actual postmortem organ donors.

Methods

In this multicenter observational prospective cohort study, all patients admitted to the intensive care unit of 3 university hospitals and 3 teaching hospitals who met the criteria of the cDCD protocol as defined by the Dutch Transplant Foundation were included. The target of enrolment was set to 400 patients. Previously developed models will be refitted in our data set. To further update previous prediction models, we will apply least absolute shrinkage and selection operator (LASSO) as a tool for efficient variable selection to develop the multivariable logistic regression model.

Results

This protocol was funded in August 2014 by the Dutch Transplant Foundation. We expect to have the results of this study in July 2020. Patient enrolment was completed in July 2018 and data collection was completed in April 2020.

Conclusions

This study will provide a robust multimodal prediction model, based on clinical and physiological parameters, that can predict time to circulatory arrest in cDCD donors. In addition, it will add valuable insight in the process of WLST in cDCD donors and will fill an important knowledge gap in this essential field of health care.

Trial Registration

ClinicalTrials.gov NCT04123275; https://clinicaltrials.gov/ct2/show/NCT04123275

International Registered Report Identifier (IRRID)

DERR1-10.2196/16733

External validation of the SOX-PTS score in a prospective multicenter trial of patients with proximal deep vein thrombosis

BACKGROUND

Using data from the SOX Trial, we recently developed a clinical prediction model for occurrence of the postthrombotic syndrome (PTS) after proximal deep vein thrombosis (DVT), termed the SOX-PTS score. The score includes anatomical extent of DVT; body mass index; and baseline Villalta score.

OBJECTIVE

To externally validate the SOX-PTS score.

METHODS

Logistic regression analysis of data from the ATTRACT Trial that evaluated pharmacomechanical catheter directed thrombolysis in patients with proximal DVT. The primary outcome was the occurrence of PTS (defined as Villalta score ≥ 5) from 6 to 24 months after DVT. Secondary outcomes included moderate-severe PTS (Villalta scale ≥ 10) and severe PTS (Villalta scale ≥ 14). Predictive performance was assessed by discrimination and calibration. An updated score was evaluated in an exploratory analysis.

RESULTS

Six hundred and ninety-one ATTRACT patients were included, of whom 328 (47%) developed PTS. The c-statistic was 0.63; 95% confidence interval (CI) 0.59-0.67 for PTS. The model's performance appeared to be better for the outcomes moderate to severe PTS and severe PTS (c-statistic 0.67; 95% CI 0.62-0.72 for moderate-severe PTS and 0.70; 0.64-0.77 for severe PTS). An updated model with age as an additional variable performed similarly to the original model.

CONCLUSION

We externally validated the SOX-PTS score for estimating the risk of developing PTS, moderate to severe PTS, and severe PTS, in patients with proximal DVT. The score may be useful to predict PTS at the time of DVT diagnosis. Further external validation in different patient cohorts is required.

Mobile Clinical Decision Tools Among Emergency Department Clinicians: Web-Based Survey and Analytic Data for Evaluation of The Ottawa Rules App

BACKGROUND

The Canadian CT Head Rule (CCHR), the Canadian Transient Ischemic Attack (TIA) Score, and the Subarachnoid Hemorrhage (SAH) Rule have all previously demonstrated the potential to significantly standardize care and improve the management of patients in emergency departments (EDs). On the basis of user feedback, we believe that the addition of these rules to the Ottawa Rules App has the potential to increase the app's usability and user acceptability.

OBJECTIVE

This study aimed to evaluate the perceived usefulness, acceptability, and uptake of the enhanced Ottawa Rules App (which now includes CCHR, TIA, and SAH Rules) among ED clinicians (medical students, residents, nurses, and physicians).

METHODS

The enhanced Ottawa Rules App was publicly released for free on iOS and Android operating systems in November 2018. This study was conducted across 2 tertiary EDs in Ottawa, Canada. Posters, direct enrollment, snowball sampling, and emails were used for study recruitment. A 24-question Web-based survey was administered to participants via email, and this was used to determine user acceptability of the app and Technology Readiness Index (TRI) scores. In-app user analytics were collected to track user behavior, such as the number of app sessions, length of app sessions, frequency of rule use, and the date app was first opened.

RESULTS

A total of 77 ED clinicians completed the study, including 34 nurses, 12 residents, 14 physicians, and 17 medical students completing ED rotations. The median TRI score for this group was 3.38, indicating a higher than average propensity to embrace and adopt new technologies to accomplish goals in their work or daily lives. The majority of respondents agreed or strongly agreed that the app helped participants accurately carry out the clinical rules (56/77, 73%) and that they would recommend this app to their colleagues (64/77, 83%). Feedback from study participants suggested further expansion of the app-more clinical decision rules (CDRs) and different versions of the app tailored to the clinician role. Analysis and comparison of Google Analytics data and in-app data revealed similar usage behavior among study-enrolled users and all app users globally.

CONCLUSIONS

This study provides evidence that using the Ottawa Rules App (version 3.0.2) to improve and guide patient care would be feasible and widely accepted. The ability to verify self-reported user data (via a Web-based survey) against server analytics data is a notable strength of this study. Participants' continued app use and request for the addition of more CDRs warrant the further development of this app and call for additional studies to evaluate its feasibility and usability in different settings as well as assessment of clinical impact.

Usefulness of the pediatric appendicitis score for assessing the severity of acute appendicitis in children

BACKGROUND

We investigated relationships between the Pediatric Appendicitis Score (PAS) and pathological progression and disease severity in pediatric acute appendicitis.

METHODS

We retrospectively evaluated 72 children who underwent surgery for acute appendicitis. We divided them into groups: simple appendicitis (n = 28) or complicated appendicitis (n = 44). We compared the influence of age, body temperature, blood test findings, hospitalization period, number of complications, and PAS between the groups. We calculated the sensitivity, specificity, positive predictive value, and negative predictive value of the PAS for diagnosing complicated appendicitis. A receiver operating characteristic curve was constructed to evaluate the cut-off value for diagnosing complicated appendicitis. To assess the severity of acute appendicitis, we divided the patients into groups according to that cut-off value.

RESULTS

There were statistically significant differences in the PAS between simple appendicitis and complicated appendicitis (5.8 versus 7.9). The receiver operating characteristic curve indicated a PAS cut-off value of 8. A PAS ≥ 8 had a sensitivity of 73%, a specificity of 89%, a positive predictive value of 91%, and a negative predictive value of 68%. A PAS ≥ 8 was associated with significantly longer hospitalization and more complications than a PAS < 8.

CONCLUSIONS

The PAS may be associated with pathological progression and disease severity in appendicitis.

A clinical decision rule to predict intracranial hypertension in severe traumatic brain injury

OBJECTIVE

While existing guidelines support the treatment of intracranial hypertension in severe traumatic brain injury (TBI), it is unclear when to suspect and initiate treatment for high intracranial pressure (ICP). The objective of this study was to derive a clinical decision rule that accurately predicts intracranial hypertension.

METHODS

Using Delphi methods, the authors identified a set of potential predictors of intracranial hypertension and a clinical decision rule a priori by consensus among a group of 43 neurosurgeons and intensivists who have extensive experience managing severe TBI without ICP monitoring. To validate these predictors, the authors used data from a Latin American trial (n = 150; BEST TRIP). To report on the performance of the rule, they calculated sensitivity, specificity, and positive and negative predictive values with 95% confidence intervals. In a secondary analysis, the rule was validated using data from a North American trial (n = 131; COBRIT).

RESULTS

The final predictors and the clinical decision rule were approved by 97% of participants in the consensus working group. The predictors are divided into major and minor criteria. High ICP would be considered suspected in the presence of 1 major or ≥ 2 minor criteria. Major criteria are: compressed cisterns (CT classification of Marshall diffuse injury [DI] III), midline shift > 5 mm (Marshall DI IV), or nonevacuated mass lesion. Minor criteria are: Glasgow Coma Scale (GCS) motor score ≤ 4, pupillary asymmetry, abnormal pupillary reactivity, or Marshall DI II. The area under the curve for the logistic regression model that contains all the predictors was 0.86. When high ICP was defined as > 22 mm Hg, the decision rule performed with a sensitivity of 93.9% (95% CI 85.0%-98.3%), a specificity of 42.3% (95% CI 31.7%-53.6%), a positive predictive value of 55.5% (95% CI 50.7%-60.2%), and a negative predictive value of 90% (95% CI 77.1%-96.0%). The sensitivity of the clinical decision rule improved with higher ICP cutoffs up to a sensitivity of 100% when intracranial hypertension was defined as ICP > 30 mm Hg. Similar results were found in the North American cohort.

CONCLUSIONS

A simple clinical decision rule based on a combination of clinical and imaging findings was found to be highly sensitive in distinguishing patients with severe TBI who would suffer intracranial hypertension. It could be used to identify patients who require ICP monitoring in high-resource settings or start ICP-lowering treatment in environments where resource limitations preclude invasive monitoring.Clinical trial registration no.: NCT02059941 (clinicaltrials.gov).

Predicting venous thromboembolism in multiple myeloma: development and validation of the IMPEDE VTE score

Venous thromboembolism (VTE) is a common cause of morbidity and mortality among patients with multiple myeloma (MM). The International Myeloma Working Group (IMWG) developed guidelines recommending primary thromboprophylaxis, in those identified at high-risk of VTE by the presence of risk factors. The National Comprehensive Cancer Network (NCCN) has adopted these guidelines; however, they lack validation. We sought to develop and validate a risk prediction score for VTE in MM and to evaluate the performance of the current IMWG/NCCN guidelines. Using 4446 patients within the Veterans Administration Central Cancer Registry, we used time-to-event analyses to develop a risk score for VTE in patients with newly diagnosed MM starting chemotherapy. We externally validated the score using the Surveillance, Epidemiology, End Results (SEER)-Medicare database (N = 4256). After identifying independent predictors of VTE, we combined the variables to develop the IMPEDE VTE score (Immunomodulatory agent; Body Mass Index ≥25 kg/m2 ; Pelvic, hip or femur fracture; Erythropoietin stimulating agent; Dexamethasone/Doxorubicin; Asian Ethnicity/Race; VTE history; Tunneled line/central venous catheter; Existing thromboprophylaxis). The score showed satisfactory discrimination in the derivation cohort, c-statistic = 0.66. Risk of VTE significantly increased as score increased (hazard ratio 1.20, P = <.0001). Within the external validation cohort, IMPEDE VTE had a c-statistic of 0.64. For comparison, when evaluating the performance of the IMWG/NCCN guidelines, the c-statistic was 0.55. In summary, the IMPEDE VTE score outperformed the current IMWG/NCCN guidelines and could be considered as the new standard risk stratification for VTE in MM.