Assessing the discriminative ability of risk models for more than two outcome categories

Development of machine learning models for patients in the high intrahepatic cholangiocarcinoma incidence age group

BACKGROUND

Intrahepatic cholangiocarcinoma (ICC) has a poor prognosis and is understudied. Based on the clinical features of patients with ICC, we constructed machine learning models to understand their importance on survival and to accurately determine patient prognosis, aiming to develop reference values to guide physicians in developing more effective treatment plans.

METHODS

This study used machine learning (ML) algorithms to build prediction models using ICC data on 1,751 patients from the SEER (Surveillance, Epidemiology, and End Results) database and 58 hospital cases. The models' performances were compared using receiver operating characteristic curve analysis, C-index, and Brier scores.

RESULTS

A total of eight variables were used to construct the ML models. Our analysis identified the random survival forest model as the best for prognostic prediction. In the training cohort, its C-index, Brier score, and Area Under the Curve values were 0.76, 0.124, and 0.882, respectively, and it also performed well in the test cohort. Kaplan-Meier survival analysis revealed that the model could effectively determine patient prognosis.

CONCLUSIONS

To our knowledge, this is the first study to develop ML prognostic models for ICC in the high-incidence age group. Of the ML models, the random survival forest model was best at prognosis prediction.

Diagnostic accuracy validation study of the M6 model without initial serum progesterone (M6NP) in triage of pregnancy of unknown location

OBJECTIVES

The M6 prediction model stratifies the risk of development of ectopic pregnancy (EP) for women with pregnancy of unknown location (PUL) into low risk or high risk, using human chorionic gonadotrophin (hCG) and progesterone levels at the initial visit to a gynaecological emergency room and hCG level at 48 h. This study evaluated a second model, the M6NP model, which does not include the progesterone level at the initial visit. The main aim of this study was to validate the diagnostic accuracy of the M6NP model in a population of French women.

STUDY DESIGN

Between January and December 2021, all women with an hCG measurement from the gynaecological emergency department of a teaching hospital were screened for inclusion in this study. Women with a pregnancy location determined before or at the second visit were excluded. The diagnostic test was based on logistic regression of the M6NP model, with classification into two groups: high risk of EP (≥5%) and low risk of EP (<5%). The reference test was the final outcome based on clinical, biological and sonographic results: failed PUL (FPUL), intrauterine pregnancy (IUP) or EP. Diagnostic performance for risk prediction of EP, and also IUP and FPUL, was calculated.

RESULTS

In total, 759 women with possible PUL were identified. After screening, 341 women with PUL were included in the main analysis. Of these, 186 (54.5%) were classified as low risk, including three (1.6%) with a final outcome of EP. The remaining 155 women with PUL were classified as high risk, of whom 60 (38.7%), 66 (42.8%) and 29 (18.7%) had a final outcome of FPUL, IUP and EP, respectively. Of the 32 women with PUL with a final outcome of EP, 29 (90.6%) were classified as high risk and three (9.4%) were classified as low risk. Therefore, the performance of the M6NP model to predict EP had a negative predictive value of 98.4%, a positive predictive value of 18.7%, sensitivity of 90.6% and specificity of 59.2%. If the prediction model had been used, it is estimated that 4.5 visits per patient could have been prevented.

CONCLUSION

The M6NP model could be used safely in the French population for risk stratification of PUL. Its use in clinical practice should result in a substantial reduction in the number of visits to a gynaecological emergency room.

Comparison of the ADNEX and ROMA risk prediction models for the diagnosis of ovarian cancer: a multicentre external validation in patients who underwent surgery

BACKGROUND

Several diagnostic prediction models to help clinicians discriminate between benign and malignant adnexal masses are available. This study is a head-to-head comparison of the performance of the Assessment of Different NEoplasias in the adneXa (ADNEX) model with that of the Risk of Ovarian Malignancy Algorithm (ROMA).

METHODS

This is a retrospective study based on prospectively included consecutive women with an adnexal tumour scheduled for surgery at five oncology centres and one non-oncology centre in four countries between 2015 and 2019. The reference standard was histology. Model performance for ADNEX and ROMA was evaluated regarding discrimination, calibration, and clinical utility.

RESULTS

The primary analysis included 894 patients, of whom 434 (49%) had a malignant tumour. The area under the receiver operating characteristic curve (AUC) was 0.92 (95% CI 0.88-0.95) for ADNEX with CA125, 0.90 (0.84-0.94) for ADNEX without CA125, and 0.85 (0.80-0.89) for ROMA. ROMA, and to a lesser extent ADNEX, underestimated the risk of malignancy. Clinical utility was highest for ADNEX. ROMA had no clinical utility at decision thresholds <27%.

CONCLUSIONS

ADNEX had better ability to discriminate between benign and malignant adnexal tumours and higher clinical utility than ROMA.

CLINICAL TRIAL REGISTRATION

clinicaltrials.gov NCT01698632 and NCT02847832.

Updating M6 pregnancy of unknown location risk-prediction model including evaluation of clinical factors

OBJECTIVES

Ectopic pregnancy (EP) is a major high-risk outcome following a pregnancy of unknown location (PUL) classification. Biochemical markers are used to triage PUL as high vs low risk to guide appropriate follow-up. The M6 model is currently the best risk-prediction model. We aimed to update the M6 model and evaluate whether performance can be improved by including clinical factors.

METHODS

This prospective cohort study recruited consecutive PUL between January 2015 and January 2017 at eight units (Phase 1), with two centers continuing recruitment between January 2017 and March 2021 (Phase 2). Serum samples were collected routinely and sent for β-human chorionic gonadotropin (β-hCG) and progesterone measurement. Clinical factors recorded were maternal age, pain score, bleeding score and history of EP. Based on transvaginal ultrasonography and/or biochemical confirmation during follow-up, PUL were classified subsequently as failed PUL (FPUL), intrauterine pregnancy (IUP) or EP (including persistent PUL (PPUL)). The M6 models with (M6P ) and without (M6NP ) progesterone were refitted and extended with clinical factors. Model validation was performed using internal-external cross-validation (IECV) (Phase 1) and temporal external validation (EV) (Phase 2). Missing values were handled using multiple imputation.

RESULTS

Overall, 5473 PUL were recruited over both phases. A total of 709 PUL were excluded because maternal age was < 16 years or initial β-hCG was ≤ 25 IU/L, leaving 4764 (87%) PUL for analysis (2894 in Phase 1 and 1870 in Phase 2). For the refitted M6P model, the area under the receiver-operating-characteristics curve (AUC) for EP/PPUL vs IUP/FPUL was 0.89 for IECV and 0.84-0.88 for EV, with respective sensitivities of 94% and 92-93%. For the refitted M6NP model, the AUCs were 0.85 for IECV and 0.82-0.86 for EV, with respective sensitivities of 92% and 93-94%. Calibration performance was good overall, but with heterogeneity between centers. Net Benefit confirmed clinical utility. The change in AUC when M6P was extended to include maternal age, bleeding score and history of EP was between -0.02 and 0.01, depending on center and phase. The corresponding change in AUC when M6NP was extended was between -0.01 and 0.03. At the 5% threshold to define high risk of EP/PPUL, extending M6P altered sensitivity by -0.02 to -0.01, specificity by 0.03 to 0.04 and Net Benefit by -0.005 to 0.006. Extending M6NP altered sensitivity by -0.03 to -0.01, specificity by 0.05 to 0.07 and Net Benefit by -0.005 to 0.006.

CONCLUSIONS

The updated M6 model offers accurate diagnostic performance, with excellent sensitivity for EP. Adding clinical factors to the model improved performance in some centers, especially when progesterone levels were not suitable or unavailable. © 2023 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of International Society of Ultrasound in Obstetrics and Gynecology.

ADNEX risk prediction model for diagnosis of ovarian cancer: systematic review and meta-analysis of external validation studies

Objectives

To conduct a systematic review of studies externally validating the ADNEX (Assessment of Different Neoplasias in the adnexa) model for diagnosis of ovarian cancer and to present a meta-analysis of its performance.

Design

Systematic review and meta-analysis of external validation studies.

Data sources

Medline, Embase, Web of Science, Scopus, and Europe PMC, from 15 October 2014 to 15 May 2023.

Eligibility criteria for selecting studies

All external validation studies of the performance of ADNEX, with any study design and any study population of patients with an adnexal mass. Two independent reviewers extracted the data. Disagreements were resolved by discussion. Reporting quality of the studies was scored with the TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) reporting guideline, and methodological conduct and risk of bias with PROBAST (Prediction model Risk Of Bias Assessment Tool). Random effects meta-analysis of the area under the receiver operating characteristic curve (AUC), sensitivity and specificity at the 10% risk of malignancy threshold, and net benefit and relative utility at the 10% risk of malignancy threshold were performed.

Results

47 studies (17 007 tumours) were included, with a median study sample size of 261 (range 24-4905). On average, 61% of TRIPOD items were reported. Handling of missing data, justification of sample size, and model calibration were rarely described. 91% of validations were at high risk of bias, mainly because of the unexplained exclusion of incomplete cases, small sample size, or no assessment of calibration. The summary AUC to distinguish benign from malignant tumours in patients who underwent surgery was 0.93 (95% confidence interval 0.92 to 0.94, 95% prediction interval 0.85 to 0.98) for ADNEX with the serum biomarker, cancer antigen 125 (CA125), as a predictor (9202 tumours, 43 centres, 18 countries, and 21 studies) and 0.93 (95% confidence interval 0.91 to 0.94, 95% prediction interval 0.85 to 0.98) for ADNEX without CA125 (6309 tumours, 31 centres, 13 countries, and 12 studies). The estimated probability that the model has use clinically in a new centre was 95% (with CA125) and 91% (without CA125). When restricting analysis to studies with a low risk of bias, summary AUC values were 0.93 (with CA125) and 0.91 (without CA125), and estimated probabilities that the model has use clinically were 89% (with CA125) and 87% (without CA125).

Conclusions

The results of the meta-analysis indicated that ADNEX performed well in distinguishing between benign and malignant tumours in populations from different countries and settings, regardless of whether the serum biomarker, CA125, was used as a predictor. A key limitation was that calibration was rarely assessed.

Systematic review registration

PROSPERO CRD42022373182.

Automatic triage of twelve-lead electrocardiograms using deep convolutional neural networks: a first implementation study

Aims

Expert knowledge to correctly interpret electrocardiograms (ECGs) is not always readily available. An artificial intelligence (AI)-based triage algorithm (DELTAnet), able to support physicians in ECG prioritization, could help reduce current logistic burden of overreading ECGs and improve time to treatment for acute and life-threatening disorders. However, the effect of clinical implementation of such AI algorithms is rarely investigated.

Methods and results

Adult patients at non-cardiology departments who underwent ECG testing as a part of routine clinical care were included in this prospective cohort study. DELTAnet was used to classify 12-lead ECGs into one of the following triage classes: normal, abnormal not acute, subacute, and acute. Performance was compared with triage classes based on the final clinical diagnosis. Moreover, the associations between predicted classes and clinical outcomes were investigated. A total of 1061 patients and ECGs were included. Performance was good with a mean concordance statistic of 0.96 (95% confidence interval 0.95-0.97) when comparing DELTAnet with the clinical triage classes. Moreover, zero ECGs that required a change in policy or referral to the cardiologist were missed and there was a limited number of cases predicted as acute that did not require follow-up (2.6%).

Conclusion

This study is the first to prospectively investigate the impact of clinical implementation of an ECG-based AI triage algorithm. It shows that DELTAnet is efficacious and safe to be used in clinical practice for triage of 12-lead ECGs in non-cardiology hospital departments.

External validation of a multivariable prediction model for identification of pneumonia and other serious bacterial infections in febrile immunocompromised children

OBJECTIVE

To externally validate and update the Feverkids tool clinical prediction model for differentiating bacterial pneumonia and other serious bacterial infections (SBIs) from non-SBI causes of fever in immunocompromised children.

DESIGN

International, multicentre, prospective observational study embedded in PErsonalised Risk assessment in Febrile illness to Optimise Real-life Management across the European Union (PERFORM).

SETTING

Fifteen teaching hospitals in nine European countries.

PARTICIPANTS

Febrile immunocompromised children aged 0-18 years.

METHODS

The Feverkids clinical prediction model predicted the probability of bacterial pneumonia, other SBI or no SBI. Model discrimination, calibration and diagnostic performance at different risk thresholds were assessed. The model was then re-fitted and updated.

RESULTS

Of 558 episodes, 21 had bacterial pneumonia, 104 other SBI and 433 no SBI. Discrimination was 0.83 (95% CI 0.71 to 0.90) for bacterial pneumonia, with moderate calibration and 0.67 (0.61 to 0.72) for other SBIs, with poor calibration. After model re-fitting, discrimination improved to 0.88 (0.79 to 0.96) and 0.71 (0.65 to 0.76) and calibration improved. Predicted risk <1% ruled out bacterial pneumonia with sensitivity 0.95 (0.86 to 1.00) and negative likelihood ratio (LR) 0.09 (0.00 to 0.32). Predicted risk >10% ruled in bacterial pneumonia with specificity 0.91 (0.88 to 0.94) and positive LR 6.51 (3.71 to 10.3). Predicted risk <10% ruled out other SBIs with sensitivity 0.92 (0.87 to 0.97) and negative LR 0.32 (0.13 to 0.57). Predicted risk >30% ruled in other SBIs with specificity 0.89 (0.86 to 0.92) and positive LR 2.86 (1.91 to 4.25).

CONCLUSION

Discrimination and calibration were good for bacterial pneumonia but poorer for other SBIs. The rule-out thresholds have the potential to reduce unnecessary investigations and antibiotics in this high-risk group.

Multiclass risk models for ovarian malignancy: an illustration of prediction uncertainty due to the choice of algorithm

BACKGROUND

Assessing malignancy risk is important to choose appropriate management of ovarian tumors. We compared six algorithms to estimate the probabilities that an ovarian tumor is benign, borderline malignant, stage I primary invasive, stage II-IV primary invasive, or secondary metastatic.

METHODS

This retrospective cohort study used 5909 patients recruited from 1999 to 2012 for model development, and 3199 patients recruited from 2012 to 2015 for model validation. Patients were recruited at oncology referral or general centers and underwent an ultrasound examination and surgery ≤ 120 days later. We developed models using standard multinomial logistic regression (MLR), Ridge MLR, random forest (RF), XGBoost, neural networks (NN), and support vector machines (SVM). We used nine clinical and ultrasound predictors but developed models with or without CA125.

RESULTS

Most tumors were benign (3980 in development and 1688 in validation data), secondary metastatic tumors were least common (246 and 172). The c-statistic (AUROC) to discriminate benign from any type of malignant tumor ranged from 0.89 to 0.92 for models with CA125, from 0.89 to 0.91 for models without. The multiclass c-statistic ranged from 0.41 (SVM) to 0.55 (XGBoost) for models with CA125, and from 0.42 (SVM) to 0.51 (standard MLR) for models without. Multiclass calibration was best for RF and XGBoost. Estimated probabilities for a benign tumor in the same patient often differed by more than 0.2 (20% points) depending on the model. Net Benefit for diagnosing malignancy was similar for algorithms at the commonly used 10% risk threshold, but was slightly higher for RF at higher thresholds. Comparing models, between 3% (XGBoost vs. NN, with CA125) and 30% (NN vs. SVM, without CA125) of patients fell on opposite sides of the 10% threshold.

CONCLUSION

Although several models had similarly good performance, individual probability estimates varied substantially.

A clinical model to predict postoperative improvement in sub-domains of the modified Japanese Orthopedic Association score for degenerative cervical myelopathy

PURPOSE

The modified Japanese Orthopedic Association (mJOA) score consists of six sub-domains and is used to quantify the severity of cervical myelopathy. The current study aimed to assess for predictors of postoperative mJOA sub-domains scores following elective surgical management for patients with cervical myelopathy and develop the first clinical prediction model for 12-month mJOA sub-domain scores.Please confirm if the author names are presented accurately and in the correct sequence (given name, middle name/initial, family name). Author 1 Given name: [Byron F.] Last name [Stephens], Author 2 Given name: [Lydia J.] Last name [McKeithan], Author 3 Given name: [W. Hunter] Last name [Waddell], Author 4 Given name: [Anthony M.] Last name [Steinle], Author 5 Given name: [Wilson E.] Last name [Vaughan], Author 6 Given name: [Jacquelyn S.] Last name [Pennings], Author 7 Given name: [Jacquelyn S.] Last name [Pennings], Author 8 Given name: [Scott L.] Last name [Zuckerman], Author 9 Given name: [Kristin R.] Last name [Archer], Author 10 Given name: [Amir M.] Last name [Abtahi] Also, kindly confirm the details in the metadata are correct.Last Author listed should be Kristin R. Archer METHODS: A multivariable proportional odds ordinal regression model was developed for patients with cervical myelopathy. The model included patient demographic, clinical, and surgery covariates along with baseline sub-domain scores. The model was internally validated using bootstrap resampling to estimate the likely performance on a new sample of patients.

RESULTS

The model identified mJOA baseline sub-domains to be the strongest predictors of 12-month scores, with numbness in legs and ability to walk predicting five of the six mJOA items. Additional covariates predicting three or more items included age, preoperative anxiety/depression, gender, race, employment status, duration of symptoms, smoking status, and radiographic presence of listhesis. Surgical approach, presence of motor deficits, number of surgical levels involved, history of diabetes mellitus, workers' compensation claim, and patient insurance had no impact on 12-month mJOA scores.

CONCLUSION

Our study developed and validated a clinical prediction model for improvement in mJOA scores at 12 months following surgery. The results highlight the importance of assessing preoperative numbness, walking ability, modifiable variables of anxiety/depression, and smoking status. This model has the potential to assist surgeons, patients, and families when considering surgery for cervical myelopathy.

LEVEL OF EVIDENCE

Level III.

Minimum sample size for developing a multivariable prediction model using multinomial logistic regression

AIMS

Multinomial logistic regression models allow one to predict the risk of a categorical outcome with > 2 categories. When developing such a model, researchers should ensure the number of participants (n ) is appropriate relative to the number of events (E k ) and the number of predictor parameters (p k ) for each category k. We propose three criteria to determine the minimum n required in light of existing criteria developed for binary outcomes.

PROPOSED CRITERIA

The first criterion aims to minimise the model overfitting. The second aims to minimise the difference between the observed and adjusted R 2 Nagelkerke. The third criterion aims to ensure the overall risk is estimated precisely. For criterion (i), we show the sample size must be based on the anticipated Cox-snell R 2 of distinct 'one-to-one' logistic regression models corresponding to the sub-models of the multinomial logistic regression, rather than on the overall Cox-snell R 2 of the multinomial logistic regression.

EVALUATION OF CRITERIA

We tested the performance of the proposed criteria (i) through a simulation study and found that it resulted in the desired level of overfitting. Criterion (ii) and (iii) were natural extensions from previously proposed criteria for binary outcomes and did not require evaluation through simulation.

SUMMARY

We illustrated how to implement the sample size criteria through a worked example considering the development of a multinomial risk prediction model for tumour type when presented with an ovarian mass. Code is provided for the simulation and worked example. We will embed our proposed criteria within the pmsampsize R library and Stata modules.

Effective Dose Range of Intrathecal Isobaric Bupivacaine to Achieve T5–T10 Sensory Block Heights for Elderly and Overweight Patients: An Observational Study

Background and Objectives: The dose selection for isobaric bupivacaine determines the success of spinal anesthesia (SA). A dose higher than the optimal dose causes high SA, whereas an underdose leads to inadequate spread of cephalad. As it involves anatomical and physiological alterations, the dosing should be reduced with advancing age and body mass index values. Therefore, this study aimed to demonstrate the association between the isobaric bupivacaine dose and block height, and to determine the dose intervals of bupivacaine to achieve the T5–T10 sensory block with a low probability of high SA in elderly and overweight patients. Material and Methods: This retrospective observational study recruited 1079 adult patients who underwent SA with 0.5% isobaric bupivacaine from 2018 to 2021. The patients were divided into four categories: category 1 (age < 60, BMI < 25), category 2 (age < 60, BMI ≥ 25), category 3 (age ≥ 60, BMI < 25), and category 4 (age ≥ 60, BMI ≥ 25). The bupivacaine dose and sensory block height (classified into three levels: high (T1–T4), favorable (T5–T10), and low (T11–L2)) were recorded. Results: The sensory block level increased significantly with increasing doses of bupivacaine for patients in categories 1 and 2. The suggested dose ranges for the favorable block heights were 15–17 and 10.5–16 mg in patient categories 1–2 and 3–4, respectively. In these dose ranges, the probability range of high SA was 10–15%. Conclusions: The sensory block height following SA was associated with the bupivacaine dose in patients aged <60 years. Regardless of the BMI, the suggested dose ranges of 0.5% isobaric bupivacaine are 15–17 mg (3.0–3.4 mL) and 10.5–16 mg (2.1–3.2 mL) for patients aged <60 and ≥60 years, respectively.

Benign descriptors and ADNEX in two-step strategy to estimate risk of malignancy in ovarian tumors: retrospective validation in IOTA5 multicenter cohort

OBJECTIVE

Previous work has suggested that the ultrasound-based benign simple descriptors (BDs) can reliably exclude malignancy in a large proportion of women presenting with an adnexal mass. This study aimed to validate a modified version of the BDs and to validate a two-step strategy to estimate the risk of malignancy, in which the modified BDs are followed by the Assessment of Different NEoplasias in the adneXa (ADNEX) model if modified BDs do not apply.

METHODS

This was a retrospective analysis using data from the 2-year interim analysis of the International Ovarian Tumor Analysis (IOTA) Phase-5 study, in which consecutive patients with at least one adnexal mass were recruited irrespective of subsequent management (conservative or surgery). The main outcome was classification of tumors as benign or malignant, based on histology or on clinical and ultrasound information during 1 year of follow-up. Multiple imputation was used when outcome based on follow-up was uncertain according to predefined criteria.

RESULTS

A total of 8519 patients were recruited at 36 centers between 2012 and 2015. We excluded patients who were already in follow-up at recruitment and all patients from 19 centers that did not fulfil our criteria for good-quality surgical and follow-up data, leaving 4905 patients across 17 centers for statistical analysis. Overall, 3441 (70%) tumors were benign, 978 (20%) malignant and 486 (10%) uncertain. The modified BDs were applicable in 1798/4905 (37%) tumors, of which 1786 (99.3%) were benign. The two-step strategy based on ADNEX without CA125 had an area under the receiver-operating-characteristics curve (AUC) of 0.94 (95% CI, 0.92-0.96). The risk of malignancy was slightly underestimated, but calibration varied between centers. A sensitivity analysis in which we expanded the definition of uncertain outcome resulted in 1419 (29%) tumors with uncertain outcome and an AUC of the two-step strategy without CA125 of 0.93 (95% CI, 0.91-0.95).

CONCLUSION

A large proportion of adnexal masses can be classified as benign by the modified BDs. For the remaining masses, the ADNEX model can be used to estimate the risk of malignancy. This two-step strategy is convenient for clinical use. © 2022 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of International Society of Ultrasound in Obstetrics and Gynecology.

A network approach to compute hypervolume under receiver operating characteristic manifold for multi-class biomarkers

Computation of hypervolume under ROC manifold (HUM) is necessary to evaluate biomarkers for their capability to discriminate among multiple disease types or diagnostic groups. However the original definition of HUM involves multiple integration and thus a medical investigation for multi-class receiver operating characteristic (ROC) analysis could suffer from huge computational cost when the formula is implemented naively. We introduce a novel graph-based approach to compute HUM efficiently in this article. The computational method avoids the time-consuming multiple summation when sample size or the number of categories is large. We conduct extensive simulation studies to demonstrate the improvement of our method over existing R packages. We apply our method to two real biomedical data sets to illustrate its application.

AMPREDICT PROsthetics-Predicting Prosthesis Mobility to Aid in Prosthetic Prescription and Rehabilitation Planning

OBJECTIVE

To develop and validate a patient-specific multivariable prediction model that uses variables readily available in the electronic medical record to predict 12-month mobility at the time of initial post-amputation prosthetic prescription. The prediction model is designed for patients who have undergone their initial transtibial (TT) or transfemoral (TF) amputation because of complications of diabetes and/or peripheral artery disease.

DESIGN

Multi-methodology cohort study that identified patients retrospectively through a large Veteran's Affairs (VA) dataset then prospectively collected their patient-reported mobility.

SETTING

The VA Corporate Data Warehouse, the National Prosthetics Patient Database, participant mailings, and phone calls.

PARTICIPANTS

Three-hundred fifty-seven veterans who underwent an incident dysvascular TT or TF amputation and received a qualifying lower limb prosthesis between March 1, 2018, and November 30, 2020 (N=357).

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURE

The Amputee Single Item Mobility Measure (AMPSIMM) was divided into a 4-category outcome to predict wheelchair mobility (0-2), and household (3), basic community (4), or advanced community ambulation (5-6).

RESULTS

Multinomial logistic lasso regression, a machine learning methodology designed to select variables that most contribute to prediction while controlling for overfitting, led to a final model including 23 predictors of the 4-category AMPSIMM outcome that effectively discriminates household ambulation from basic community ambulation and from advanced community ambulation-levels of key clinical importance when estimating future prosthetic demands. The overall model performance was modest as it did not discriminate wheelchair from household mobility as effectively.

CONCLUSIONS

The AMPREDICT PROsthetics model can assist providers in estimating individual patients' future mobility at the time of prosthetic prescription, thereby aiding in the formulation of appropriate mobility goals, as well as facilitating the prescription of a prosthetic device that is most appropriate for anticipated functional goals.

Examination of Tobacco-Related Messaging and Tobacco Use over Time among U.S. Military Young Adults

Background: People from minoritized populations have historically been targeted by tobacco companies. Little is known about exposure to tobacco-related messages among military personnel from disadvantaged backgrounds. Objectives: The current study aimed to examine exposure to tobacco-related messaging across many nicotine products and through a variety of mediums (i.e., family, friends, advertisements, event promotions, social media) among diverse military populations and use one year later in a sample of young adults who recently enlisted in the U.S. Air Force. Methods: In this study, 8,901 U.S. Air Force trainees reported on demographics, tobacco use, and exposure to positive tobacco messages from social sources (i.e., friends, family, social media) and environmental sources (i.e., advertisements and promotions). Tobacco use was reported one-year later. Results: Compared to others of the same reported racial/ethnic background, Latino/a/x (Relative Risk Ratio [RRR] = 1.354, 95% CI: [1.145, 1.563]) and multiracial (RRR = 1.594, 95% CI: [1.173, 2.016]) participants who were exposed to positive tobacco messages from social sources were significantly more likely to report tobacco product use at one-year follow-up than those who were not exposed to social messages. Exposure to positive tobacco messages from environmental sources were not significantly associated with tobacco use one year later. Conclusions: Social messages may play an important role in increasing risk of tobacco use among some minoritized populations. Cultural as well as systemic factors could be addressed in future tobacco prevention programs to decrease the potency of positive tobacco-related social messages among Latino/a/x and multiracial communities.

External Validation of the Assessment of Different NEoplasias in the adneXa Model Performance in Evaluating the Risk of Ovarian Carcinoma Before Surgery in China: A Tertiary Center Study

OBJECTIVES

The Assessment of Different NEoplasias in the adneXa (ADNEX) model was developed by the International Ovarian Tumor Analysis group to assess the risk of an ovarian mass being malignant. This study aimed to externally validate the ADNEX model performance in a tertiary center in China.

METHODS

This retrospective, single-center university hospital study assessed the model diagnostic accuracy. All patients were examined by transvaginal ultrasonography, and serum CA125 levels were measured. Moreover, clinicopathological information was collected. The diagnostic performance of the ADNEX model was calculated with and without CA125 as a predictor.

RESULTS

We retrieved data of 335 patients, of which 53 were excluded based on the exclusion criteria. Of the included 282 patients, 178 (63.1%) had benign tumors, and 104 (36.9%) had malignant tumors. When CA125 was factored in, the area under the receiver operating characteristic curve (AUC) for the distinction between benign and malignant tumors was 0.93 (95% confidence interval [CI], 0.90-0.96), whereas it was 0.91 (95% CI, 0.88-0.95) without CA125. The concordance between the predicted risk of malignancy and the proportion of observed malignancies was well demonstrated by the calibration plots.

CONCLUSIONS

The proper performance of the ADNEX model was verified externally in a tertiary center in China, showing a good distinction between tumour subtypes. Our findings suggest the ADNEX model is a valuable tool in clinical practice and may help in managing patients with adnexal masses.

Gout and Hospital Admission for Ambulatory Care-Sensitive Conditions: Risks and Trajectories

OBJECTIVE

To investigate the risks and trajectories of hospital admission for ambulatory care-sensitive conditions (ACSCs) in gout.

METHODS

Among individuals aged 35 years to 85 years residing in Skåne, Sweden, in 2005, those with no doctor-diagnosed gout during 1998 to 2005 (n = 576,659) were followed from January 1, 2006, until a hospital admission for an ACSC, death, relocation outside Skåne, or December 31, 2016. Treating a new gout diagnosis (International Classification of Diseases, 10th revision, code M10) as a time-varying exposure, we used Cox proportional and additive hazard models to estimate the effects of gout on hospital admissions for ACSCs. We investigated the trajectory of hospital admissions for ACSCs from 3 years before to 3 years after gout diagnosis using generalized estimating equations and group-based trajectory modeling in an age-and sex-matched cohort study.

RESULTS

Gout was associated with a 41% increased rate of hospital admission for ACSCs (hazard ratio 1.41, 95% CI 1.35-1.47), corresponding to 121 (95% CI 104-138) more hospital admissions for ACSCs per 10,000 person-years compared with those without gout. Our trajectory analysis showed that higher rates of hospital admission for ACSCs among persons with gout were observed from 3 years before to 3 years after diagnosis, with the highest prevalence rate ratio (2.22, 95% CI 1.92-2.53) at the 3-month period after diagnosis. We identified 3 classes with distinct trajectories of hospital admissions for ACSCs among patients with gout: almost none (88.5%), low-rising (9.7%), and moderate-sharply rising (1.8%). The Charlson Comorbidity Index was the most important predictor of trajectory class membership.

CONCLUSION

Increased risk of hospital admissions for ACSCs in gout highlights the need for better management of the disease through outpatient care, especially among foreign-born, older patients with comorbidities.

The Risk of Endometrial Malignancy and Other Endometrial Pathology in Women with Abnormal Uterine Bleeding: An Ultrasound-Based Model Development Study by the IETA Group

OBJECTIVES

The aim of this study was to develop a model that can discriminate between different etiologies of abnormal uterine bleeding.

DESIGN

The International Endometrial Tumor Analysis 1 study is a multicenter observational diagnostic study in 18 bleeding clinics in 9 countries. Consecutive women with abnormal vaginal bleeding presenting for ultrasound examination (n = 2,417) were recruited. The histology was obtained from endometrial sampling, D&C, hysteroscopic resection, hysterectomy, or ultrasound follow-up for >1 year.

METHODS

A model was developed using multinomial regression based on age, body mass index, and ultrasound predictors to distinguish between: (1) endometrial atrophy, (2) endometrial polyp or intracavitary myoma, (3) endometrial malignancy or atypical hyperplasia, (4) proliferative/secretory changes, endometritis, or hyperplasia without atypia and validated using leave-center-out cross-validation and bootstrapping. The main outcomes are the model's ability to discriminate between the four outcomes and the calibration of risk estimates.

RESULTS

The median age in 2,417 women was 50 (interquartile range 43-57). 414 (17%) women had endometrial atrophy; 996 (41%) had a polyp or myoma; 155 (6%) had an endometrial malignancy or atypical hyperplasia; and 852 (35%) had proliferative/secretory changes, endometritis, or hyperplasia without atypia. The model distinguished well between malignant and benign histology (c-statistic 0.88 95% CI: 0.85-0.91) and between all benign histologies. The probabilities for each of the four outcomes were over- or underestimated depending on the centers.

LIMITATIONS

Not all patients had a diagnosis based on histology. The model over- or underestimated the risk for certain outcomes in some centers, indicating local recalibration is advisable.

CONCLUSIONS

The proposed model reliably distinguishes between four histological outcomes. This is the first model to discriminate between several outcomes and is the only model applicable when menopausal status is uncertain. The model could be useful for patient management and counseling, and aid in the interpretation of ultrasound findings. Future research is needed to externally validate and locally recalibrate the model.

Risk prediction models for discrete ordinal outcomes: Calibration and the impact of the proportional odds assumption

Calibration is a vital aspect of the performance of risk prediction models, but research in the context of ordinal outcomes is scarce. This study compared calibration measures for risk models predicting a discrete ordinal outcome, and investigated the impact of the proportional odds assumption on calibration and overfitting. We studied the multinomial, cumulative, adjacent category, continuation ratio, and stereotype logit/logistic models. To assess calibration, we investigated calibration intercepts and slopes, calibration plots, and the estimated calibration index. Using large sample simulations, we studied the performance of models for risk estimation under various conditions, assuming that the true model has either a multinomial logistic form or a cumulative logit proportional odds form. Small sample simulations were used to compare the tendency for overfitting between models. As a case study, we developed models to diagnose the degree of coronary artery disease (five categories) in symptomatic patients. When the true model was multinomial logistic, proportional odds models often yielded poor risk estimates, with calibration slopes deviating considerably from unity even on large model development datasets. The stereotype logistic model improved the calibration slope, but still provided biased risk estimates for individual patients. When the true model had a cumulative logit proportional odds form, multinomial logistic regression provided biased risk estimates, although these biases were modest. Nonproportional odds models require more parameters to be estimated from the data, and hence suffered more from overfitting. Despite larger sample size requirements, we generally recommend multinomial logistic regression for risk prediction modeling of discrete ordinal outcomes.

Differentiating COVID-19 and dengue from other febrile illnesses in co-epidemics: Development and internal validation of COVIDENGUE scores

OBJECTIVES

The purpose of this cohort study was to develop two scores able to differentiate coronavirus 2019 (COVID-19) from dengue and other febrile illnesses (OFIs).

METHODS

All subjects suspected of COVID-19 who attended the SARS-CoV-2 testing center of Saint-Pierre hospital, Reunion, between March 23 and May 10, 2020, were assessed for identifying predictors of both infectious diseases from a multinomial logistic regression model. Two scores were developed after weighting the odd ratios then validated by bootstrapping.

RESULTS

Over 49 days, 80 COVID-19, 60 non-severe dengue and 872 OFIs were diagnosed. The translation of the best fit model yielded two scores composed of 11 criteria: contact with a COVID-19 positive case (+3 points for COVID-19; 0 point for dengue), return from travel abroad within 15 days (+3/-1), previous individual episode of dengue (+1/+3), active smoking (-3/0), body ache (0/+5), cough (0/-2), upper respiratory tract infection symptoms (-1/-1), anosmia (+7/-1), headache (0/+5), retro-orbital pain (-1/+5), and delayed presentation (>3 days) to hospital (+1/0). The area under the receiver operating characteristic curve was 0.79 (95%CI 0.76-0.82) for COVID-19 score and 0.88 (95%CI 0.85-0.90) for dengue score. Calibration was satisfactory for COVID-19 score and excellent for dengue score. For predicting COVID-19, sensitivity was 97% at the 0-point cut-off and specificity 99% at the 10-point cut-off. For predicting dengue, sensitivity was 97% at the 3-point cut-off and specificity 98% at the 11-point cut-off.

CONCLUSIONS

COVIDENGUE scores proved discriminant to differentiate COVID-19 and dengue from OFIs in the context of SARS-CoV-2 testing center during a co-epidemic.

Efficacy of a group-based brief tobacco intervention among young adults aged 18-20 years in the US Air Force

INTRODUCTION

Most smokers begin using tobacco before the age of 25 years, making it important to reduce tobacco use during adolescence and early adulthood. Rates of use are historically higher among military personnel. While 'Tobacco 21' made it illegal for US retailers to sell tobacco to those aged <21 years, the policy did not address cessation for current youth and young adult tobacco users. Additionally, there is limited research on cessation interventions among young adults under 21 years. The current study evaluated the efficacy of a group-based Brief Tobacco Intervention (BTI) among US Air Force trainees, who are predominantly aged 18-20 years and directly impacted by Tobacco 21 legislation.

METHODS

Participants were 2969 US Air Force Trainees from April 2017 through January 2018 cluster randomized to three conditions: 1) BTI + Airman's Guide to Remaining Tobacco Free (AG), 2) AG alone, and 3) the National Cancer Institute's Clearing the Air (CTA) pamphlet. To assess the efficacy of the interventions among people aged 18-20 years, a domain analysis (<21 years, n=2117; and ≥21 years, n=852) of a multinomial logistic regression model was run.

RESULTS

Mono tobacco users aged <21 years at baseline who received the BTI+AG had higher odds of quitting tobacco at 3 months (OR=2.13; 95% CI: 1.02-4.46). Dual and poly users aged <21 years at baseline who received the BTI+AG intervention had higher odds of reducing the number of tobacco products used at 3 months (OR=2.94; 95% CI: 1.03-8.37).

CONCLUSIONS

The BTI was effective for people aged 18-20 years. The current study offers insight into components of interventions that might be successful in helping this age group decrease tobacco use.

Variability in end-of-life healthcare use in patients with osteoarthritis: a population-based matched cohort study

PURPOSE

To investigate the patterns of healthcare use (HCU) at the last year of life in persons with osteoarthritis (OA).

METHODS

Using linked registers, we identified persons aged≥ 65 years who died during 2003-2014 and were resided in the Skåne region during 5-year prior to death. Among these, we randomly matched decedents with a principal OA diagnosis prior to the last year of life (OA cohort, n = 17,993) with up to 4 comparators without OA by sex, age at death, and year of death (n = 59,945). We measured monthly HCU for each decedent during last year of life and applied two-part regression models to estimate HCU attributable to OA. Group-based trajectory modelling (GBTM) was used to detect distinct trajectories of HCU within the OA cohort.

RESULTS

During last 12-month of life, each person with OA had, on average, 2.5 (95% CI 2.2, 2.7) excess healthcare consultations and 1.8 (95% CI 1.3, 2.2) more inpatient days than those without OA. While both cohorts observed increasing trends in HCU towards death, excess healthcare consultations attributable to OA declined and inpatient days increased as death approached. For both healthcare consultations and inpatient days, GBTM identified four distinct trajectory classes. While underlying cause of death and age were the most important predictors of class membership, the overall predictive accuracy was poor.

CONCLUSION

OA was associated with excess HCU especially hospital-based care during the last year of life. However, there seem to be distinct trajectory classes within the OA patient population.

Development of Classification Criteria for the Uveitides

PURPOSE

To develop classification criteria for 25 of the most common uveitides.

DESIGN

Machine learning using 5,766 cases of 25 uveitides.

METHODS

Cases were collected in an informatics-designed preliminary database. Using formal consensus techniques, a final database was constructed of 4,046 cases achieving supermajority agreement on the diagnosis. Cases were analyzed within uveitic class and were split into a training set and a validation set. Machine learning used multinomial logistic regression with lasso regularization on the training set to determine a parsimonious set of criteria for each disease and to minimize misclassification rates. The resulting criteria were evaluated in the validation set. Accuracy of the rules developed to express the machine learning criteria was evaluated by a masked observer in a 10% random sample of cases.

RESULTS

Overall accuracy estimates by uveitic class in the validation set were as follows: anterior uveitides 96.7% (95% confidence interval [CI] 92.4, 98.6); intermediate uveitides 99.3% (95% CI 96.1, 99.9); posterior uveitides 98.0% (95% CI 94.3, 99.3); panuveitides 94.0% (95% CI 89.0, 96.8); and infectious posterior uveitides / panuveitides 93.3% (95% CI 89.1, 96.3). Accuracies of the masked evaluation of the "rules" were anterior uveitides 96.5% (95% CI 91.4, 98.6) intermediate uveitides 98.4% (91.5, 99.7), posterior uveitides 99.2% (95% CI 95.4, 99.9), panuveitides 98.9% (95% CI 94.3, 99.8), and infectious posterior uveitides / panuveitides 98.8% (95% CI 93.4, 99.9).

CONCLUSIONS

The classification criteria for these 25 uveitides had high overall accuracy (ie, low misclassification rates) and seemed to perform well enough for use in clinical and translational research.

A prognostic model for the preoperative identification of patients at risk for receiving transfusion of packed red blood cells in cardiac surgery

BACKGROUND

Patients undergoing cardiothoracic surgery are at substantial risk for blood transfusion. Increased awareness and patient blood management have resulted in a significant reduction over the past years. The next step is preoperative treatment of patients at high risk for packed red blood cells (RBC) transfusion, with the ultimate goal to eventually prevent RBC transfusion. A prediction model was developed to select patients at high risk for RBC transfusion.

MATERIALS AND METHODS

Data of all patients that underwent cardiac surgery in our center between 2008 and 2013 (n = 2951) were used for model development, and between 2014 and 2016 for validation (n = 1136). Only preoperative characteristics were included in a multinomial regression model with three outcome categories (no, RBC, other transfusion). The accuracy of the estimated risks and discriminative ability of the model were assessed. Clinical usefulness was explored.

RESULTS

Risk factors included are sex, type of surgery, redo surgery, age, height, body mass index, preoperative hemoglobin level, and preoperative platelet count. The model has excellent discriminative ability for predicting RBC transfusion versus no transfusion (area under the curve [AUC] = 94%) and good discriminative ability for RBC transfusion versus other transfusion (AUC = 84%). With a cut-off value of RBC risk of 16.8% and higher, the model is well able to identify a high proportion of patients at risk for RBC transfusion (sensitivity = 87.1%, specificity = 82.3%).

CONCLUSION

In the current study, a prediction tool was developed to be used for risk stratification of patients undergoing elective cardiac surgery at risk for blood transfusions.

CT radiomics-based prediction of anaplastic lymphoma kinase and epidermal growth factor receptor mutations in lung adenocarcinoma

PURPOSE

To develop and validate a CT-based radiomic model to simultaneously diagnose anaplastic lymphoma kinase (ALK) rearrangements and epidermal growth factor receptor (EGFR) mutation status of lung adenocarcinoma and to assess whether peritumoural radiomic features add value in the prediction of mutation status.

METHODS

503 patients with pathologically proven lung adenocarcinoma containing information on the mutation status were retrospectively included. Intratumoural and peritumoural radiomic features of the primary lesion were extracted from CT. We proposed two-level stepwise binary radiomics-based classification models to diagnose ALK (step1) and EGFR mutation status (step2). The performance of proposed models and added value of peritumoural radiomic features were evaluated by using the areas under receiver operating characteristic curves (AUC) and Obuchowski index in the development and validation sets.

RESULTS

Regarding the prediction of ALK rearrangement, the diagnostic performance of the intratumoural radiomic model showed the AUC of 0.77 and 0.68 for the development and validation sets, respectively. As for EGFR mutation, the diagnostic performance of the intratumoural radiomic model showed the AUCs of 0.64 and 0.62 for the development and validation sets, respectively. The radiomics added value to the model based on clinical features (development set [radiomics + clinical model vs. clinical model]: Obuchowski index, 0.76 vs. 0.66, p < 0.001; validation set: 0.69 vs. 0.61, p = 0.075). Adding peritumoural features resulted in no improvement in terms of model performance.

CONCLUSION

The CT radiomics-based model allowed the simultaneous prediction of the presence of ALK and EGFR mutations while adding value to the clinical features.

Preoperative Evaluation of the ADNEX Model for the Prediction of the Ovarian Cancer Risk of Adnexal Masses at Siriraj Hospital

INTRODUCTION

Distinguishing benign adnexal masses from malignant tumors plays an important role in preoperative planning and improving patients' survival rates. The International Ovarian Tumor Analysis (IOTA) group developed a model termed the Assessment of Different NEoplasias in the adneXa (ADNEX).

OBJECTIVE

Our objective was to evaluate the performance of the ADNEX model in distinguishing between benign and malignant tumors at a cutoff value of 10%.

METHODS

This was a prospective diagnostic study. 357 patients with an adnexal mass who were scheduled for surgery at Siriraj Hospital were included from May 1, 2018, to May 30, 2019. All patients were undergoing ultrasonography, and serum CA125 was measured. Data were calculated by the ADNEX model via an IOTA ADNEX calculator.

RESULTS

Of the 357 patients, 296 had benign tumors and 61 had malignant tumors. The area under the receiver operating characteristic curve for using the ADNEX model was 0.975 (95% confidence interval, 0.953-0.988). At a 10% cutoff, the sensitivity was 98.4% and specificity was 87.2%. The best cutoff value was at 16.6% in our population.

CONCLUSIONS

The performance of the ADNEX model in differentiating benign and malignant tumors was excellent.

Clinical prediction models to predict the risk of multiple binary outcomes: a comparison of approaches

Clinical prediction models (CPMs) can predict clinically relevant outcomes or events. Typically, prognostic CPMs are derived to predict the risk of a single future outcome. However, there are many medical applications where two or more outcomes are of interest, meaning this should be more widely reflected in CPMs so they can accurately estimate the joint risk of multiple outcomes simultaneously. A potentially naïve approach to multi-outcome risk prediction is to derive a CPM for each outcome separately, then multiply the predicted risks. This approach is only valid if the outcomes are conditionally independent given the covariates, and it fails to exploit the potential relationships between the outcomes. This paper outlines several approaches that could be used to develop CPMs for multiple binary outcomes. We consider four methods, ranging in complexity and conditional independence assumptions: namely, probabilistic classifier chain, multinomial logistic regression, multivariate logistic regression, and a Bayesian probit model. These are compared with methods that rely on conditional independence: separate univariate CPMs and stacked regression. Employing a simulation study and real-world example, we illustrate that CPMs for joint risk prediction of multiple outcomes should only be derived using methods that model the residual correlation between outcomes. In such a situation, our results suggest that probabilistic classification chains, multinomial logistic regression or the Bayesian probit model are all appropriate choices. We call into question the development of CPMs for each outcome in isolation when multiple correlated or structurally related outcomes are of interest and recommend more multivariate approaches to risk prediction.

External validation of models to predict the outcome of pregnancies of unknown location: a multicentre cohort study

Objective

To validate externally five approaches to predict ectopic pregnancy (EP) in pregnancies of unknown location (PUL): the M6P and M6NP risk models, the two‐step triage strategy (2ST, which incorporates M6P), the M4 risk model, and beta human chorionic gonadotropin ratio cut‐offs (BhCG‐RC).

Design

Secondary analysis of a prospective cohort study.

Setting

Eight UK early pregnancy assessment units.

Population

Women presenting with a PUL and BhCG >25 IU/l.

Methods

Women were managed using the 2ST protocol: PUL were classified as low risk of EP if presenting progesterone ≤2 nmol/l; the remaining cases returned 2 days later for triage based on M6P. EP risk ≥5% was used to classify PUL as high risk. Missing values were imputed, and predictions for the five approaches were calculated post hoc. We meta‐analysed centre‐specific results.

Main outcome measures

Discrimination, calibration and clinical utility (decision curve analysis) for predicting EP.

Results

Of 2899 eligible women, the primary analysis excluded 297 (10%) women who were lost to follow up. The area under the ROC curve for EP was 0.89 (95% CI 0.86–0.91) for M6P, 0.88 (0.86–0.90) for 2ST, 0.86 (0.83–0.88) for M6NP and 0.82 (0.78–0.85) for M4. Sensitivities for EP were 96% (M6P), 94% (2ST), 92% (N6NP), 80% (M4) and 58% (BhCG‐RC); false‐positive rates were 35%, 33%, 39%, 24% and 13%. M6P and 2ST had the best clinical utility and good overall calibration, with modest variability between centres.

Conclusions

2ST and M6P performed best for prediction and triage in PUL.

Tweetable abstract

The M6 model, as part of a two‐step triage strategy, is the best approach to characterise and triage PULs.

The M6 model, as part of a two‐step triage strategy, is the best approach to characterise and triage PULs.

Influence of gender on initiation of tobacco and nicotine containing product use among U.S. Air Force trainees

Military personnel are a subgroup of young adults at risk for tobacco and nicotine containing product (TNCP) use. This study of US Air Force (USAF) trainees who were never users of TNCPs examined gender, peer tobacco use, and tobacco use intentions as predictors of TNCP initiation after Basic Military Training (BMT). We used a longitudinal cohort assessment study design with baseline and 1-year surveys completed (2011-2016) among 2393 USAF trainees: 73% men, 95% aged 18-25 years, 36% racial minorities. Overall, initiation of any TNCP use at 1-year was 23% (20% women, 24% men). From a multivariable multinomial logistic regression model predicting TNCP use at 1-year follow-up, significant 2-way interactions were detected between gender and number of close friends using tobacco before BMT (p = 0.015), and between gender and tobacco use intentions (p < 0.0001). Women reporting almost all or many close friends used tobacco were more likely to report TNCP use compared to women with none (Odds ratio [OR] = 5.8, 95% CI 2.5-13.5, Bonferroni corrected p < 0.0001). Having close friends using tobacco had little influence on TNCP use among men. Men with tobacco use intentions were more likely to report TNCP use compared to men having no intentions (OR = 8.0, 95% CI: 4.7-13.6, Bonferroni corrected p < 0.001), but tobacco use intentions had little influence among women. In this sample of USAF trainees, the study provides novel prospective findings on TNCP initiation, and how men and women are influenced differently by peer tobacco use and tobacco use intentions. Gender-specific prevention efforts focused on uptake of TNCPs appear warranted.

Use of the Activity Measure for Post-Acute Care "6 Clicks" Basic Mobility Inpatient Short Form and National Institutes of Health Stroke Scale to Predict Hospital Discharge Disposition After Stroke

OBJECTIVE

Therapists in the hospital are charged with making timely discharge recommendations to improve access to rehabilitation after stroke. The objective of this study was to identify the predictive ability of the Activity Measure for Post-Acute Care "6 Clicks" Basic Mobility Inpatient Short Form (6 Clicks mobility) score and the National Institutes of Health Stroke Scale (NIHSS) score for actual hospital discharge disposition after stroke.

METHODS

In this retrospective cohort study, data were collected from an academic hospital in the United States for 1543 patients with acute stroke and a 6 Clicks mobility score. Discharge to home, a skilled nursing facility (SNF), or an inpatient rehabilitation facility (IRF) was the primary outcome. Associations among these outcomes and 6 Clicks mobility and NIHSS scores, alone or together, were tested using multinomial logistic regression, and the predictive ability of these scores was calculated using concordance statistics.

RESULTS

A higher 6 Clicks mobility score alone was associated with a decreased odds of actual discharge to an IRF or an SNF. The 6 Clicks mobility score alone was a strong predictor of discharge to home versus an IRF or an SNF. However, predicting discharge to an IRF versus an SNF was stronger when the 6 Clicks mobility score was considered in combination with the NIHSS score, age, sex, and race.

CONCLUSION

The 6 Clicks mobility score alone can guide discharge decision making after stroke, particularly for discharge to home versus an SNF or an IRF. Determining discharge to an SNF versus an IRF could be improved by also considering the NIHSS score, age, sex, and race. Future studies should seek to identify which additional characteristics improve predictability for these separate discharge destinations.

IMPACT

The use of outcome measures can improve therapist confidence in making discharge recommendations for people with stroke, can enhance hospital throughput, and can expedite access to rehabilitation, ultimately affecting functional outcomes.

Risk Factors for Developing Severe Acute Kidney Injury in Adult Patients With Refractory Postcardiotomy Cardiogenic Shock Receiving Venoarterial Extracorporeal Membrane Oxygenation

OBJECTIVES

Postcardiotomy cardiogenic shock occurs in 2-6% of patients undergoing cardiac surgery, and 1% of cardiac surgery patients will require mechanical circulatory support using venoarterial extracorporeal membrane oxygenation. Acute kidney injury is a frequent complication in this population and negatively impacts the survival. We aimed to determine whether the timing of extracorporeal membrane oxygenation implantation influences the renal prognosis of these patients.

DESIGN

Retrospective observational cohort study between January 2013 and December 2016.

SETTING

An 18-bed surgical ICU in a university hospital.

PATIENTS

A total of 4,796 consecutive adult patients who underwent cardiac surgery were included in the study, and 347 (7.2%) were assisted with venoarterial extracorporeal membrane oxygenation for refractory postcardiotomy cardiogenic shock. The patients who died during the first 48 hours after venoarterial extracorporeal membrane oxygenation implantation were excluded. The complete-case analysis included 257 patients.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

The primary outcome was the occurrence, within 10 days following the venoarterial extracorporeal membrane oxygenation implantation, of a stage 3 acute kidney injury defined by the Kidney Disease: Improving Global Outcomes group. One hundred sixty-nine patients (65.7%) presented with a Kidney Disease: Improving Global Outcomes stage 3 acute kidney injury; 14 patients (5.4%) died before the end of the follow-up period, without developing the primary outcome. Ninety-two percent of patients with Kidney Disease: Improving Global Outcomes 3 acute kidney injury received renal replacement therapy, for a median duration of 7 days (3-16 d). Late implantation of venoarterial extracorporeal membrane oxygenation was independently associated with an increased risk of Kidney Disease: Improving Global Outcomes stage 3 acute kidney injury (odds ratio, 2.81 [95% CI, 1.31-6.07]; p = 0.008). The other factors associated with Kidney Disease: Improving Global Outcomes stage 3 acute kidney injury were preoperative left ventricular ejection fraction (odds ratio, 1.03 [95% CI, 1.01-1.05]; p = 0.007), intraoperative plasma transfusion (odds ratio, 1.13 [95% CI, 1.02-1.26]; p = 0.022), increased bilirubinemia level (odds ratio, 1.013 [95% CI, 1.001-1.026]; p = 0.032), and increased creatinine levels (odds ratio, 1.012 [95% CI, 1.006-1.018]; p < 0.001) on the day of implantation.

CONCLUSIONS

Significant kidney dysfunction is particularly frequent in patients with refractory postcardiotomy cardiogenic shock assisted with venoarterial extracorporeal membrane oxygenation. Early implantation of extracorporeal membrane oxygenation may help prevent acute kidney injury.

Validation of models to diagnose ovarian cancer in patients managed surgically or conservatively: multicentre cohort study

OBJECTIVE

To evaluate the performance of diagnostic prediction models for ovarian malignancy in all patients with an ovarian mass managed surgically or conservatively.

DESIGN

Multicentre cohort study.

SETTING

36 oncology referral centres (tertiary centres with a specific gynaecological oncology unit) or other types of centre.

PARTICIPANTS

Consecutive adult patients presenting with an adnexal mass between January 2012 and March 2015 and managed by surgery or follow-up.

MAIN OUTCOME MEASURES

Overall and centre specific discrimination, calibration, and clinical utility of six prediction models for ovarian malignancy (risk of malignancy index (RMI), logistic regression model 2 (LR2), simple rules, simple rules risk model (SRRisk), assessment of different neoplasias in the adnexa (ADNEX) with or without CA125). ADNEX allows the risk of malignancy to be subdivided into risks of a borderline, stage I primary, stage II-IV primary, or secondary metastatic malignancy. The outcome was based on histology if patients underwent surgery, or on results of clinical and ultrasound follow-up at 12 (±2) months. Multiple imputation was used when outcome based on follow-up was uncertain.

RESULTS

The primary analysis included 17 centres that met strict quality criteria for surgical and follow-up data (5717 of all 8519 patients). 812 patients (14%) had a mass that was already in follow-up at study recruitment, therefore 4905 patients were included in the statistical analysis. The outcome was benign in 3441 (70%) patients and malignant in 978 (20%). Uncertain outcomes (486, 10%) were most often explained by limited follow-up information. The overall area under the receiver operating characteristic curve was highest for ADNEX with CA125 (0.94, 95% confidence interval 0.92 to 0.96), ADNEX without CA125 (0.94, 0.91 to 0.95) and SRRisk (0.94, 0.91 to 0.95), and lowest for RMI (0.89, 0.85 to 0.92). Calibration varied among centres for all models, however the ADNEX models and SRRisk were the best calibrated. Calibration of the estimated risks for the tumour subtypes was good for ADNEX irrespective of whether or not CA125 was included as a predictor. Overall clinical utility (net benefit) was highest for the ADNEX models and SRRisk, and lowest for RMI. For patients who received at least one follow-up scan (n=1958), overall area under the receiver operating characteristic curve ranged from 0.76 (95% confidence interval 0.66 to 0.84) for RMI to 0.89 (0.81 to 0.94) for ADNEX with CA125.

CONCLUSIONS

Our study found the ADNEX models and SRRisk are the best models to distinguish between benign and malignant masses in all patients presenting with an adnexal mass, including those managed conservatively.

TRIAL REGISTRATION

ClinicalTrials.gov NCT01698632.

A registry-based algorithm to predict ejection fraction in patients with heart failure

Aims

Left ventricular ejection fraction (EF) is required to categorize heart failure (HF) [i.e. HF with preserved (HFpEF), mid‐range (HFmrEF), and reduced (HFrEF) EF] but is often not captured in population‐based cohorts or non‐HF registries. The aim was to create an algorithm that identifies EF subphenotypes for research purposes.

Methods and results

We included 42 061 HF patients from the Swedish Heart Failure Registry. As primary analysis, we performed two logistic regression models including 22 variables to predict (i) EF≥ vs. <50% and (ii) EF≥ vs. <40%. In the secondary analysis, we performed a multivariable multinomial analysis with 22 variables to create a model for all three separate EF subphenotypes: HFrEF vs. HFmrEF vs. HFpEF. The models were validated in the database from the CHECK‐HF study, a cross‐sectional survey of 10 627 patients from the Netherlands. The C‐statistic (discrimination) was 0.78 [95% confidence interval (CI) 0.77–0.78] for EF ≥50% and 0.76 (95% CI 0.75–0.76) for EF ≥40%. Similar results were achieved for HFrEF and HFpEF in the multinomial model, but the C‐statistic for HFmrEF was lower: 0.63 (95% CI 0.63–0.64). The external validation showed similar discriminative ability to the development cohort.

Conclusions

Routine clinical characteristics could potentially be used to identify different EF subphenotypes in databases where EF is not readily available. Accuracy was good for the prediction of HFpEF and HFrEF but lower for HFmrEF. The proposed algorithm enables more effective research on HF in the big data setting.

Automatic Triage of 12-Lead ECGs Using Deep Convolutional Neural Networks

BACKGROUND

The correct interpretation of the ECG is pivotal for the accurate diagnosis of many cardiac abnormalities, and conventional computerized interpretation has not been able to reach physician‐level accuracy in detecting (acute) cardiac abnormalities. This study aims to develop and validate a deep neural network for comprehensive automated ECG triage in daily practice.

METHODS AND RESULTS

We developed a 37‐layer convolutional residual deep neural network on a data set of free‐text physician‐annotated 12‐lead ECGs. The deep neural network was trained on a data set with 336.835 recordings from 142.040 patients and validated on an independent validation data set (n=984), annotated by a panel of 5 cardiologists electrophysiologists. The 12‐lead ECGs were acquired in all noncardiology departments of the University Medical Center Utrecht. The algorithm learned to classify these ECGs into the following 4 triage categories: normal, abnormal not acute, subacute, and acute. Discriminative performance is presented with overall and category‐specific concordance statistics, polytomous discrimination indexes, sensitivities, specificities, and positive and negative predictive values. The patients in the validation data set had a mean age of 60.4 years and 54.3% were men. The deep neural network showed excellent overall discrimination with an overall concordance statistic of 0.93 (95% CI, 0.92–0.95) and a polytomous discriminatory index of 0.83 (95% CI, 0.79–0.87).

CONCLUSIONS

This study demonstrates that an end‐to‐end deep neural network can be accurately trained on unstructured free‐text physician annotations and used to consistently triage 12‐lead ECGs. When further fine‐tuned with other clinical outcomes and externally validated in clinical practice, the demonstrated deep learning–based ECG interpretation can potentially improve time to treatment and decrease healthcare burden.

Differentiation of predominant subtypes of lung adenocarcinoma using a quantitative radiomics approach on CT

OBJECTIVES

To develop a model for differentiating the predominant subtype-based prognostic groups of lung adenocarcinoma using CT radiomic features, and to validate its performance in comparison with radiologists' assessments.

METHODS

A total of 993 patients presenting with invasive lung adenocarcinoma between March 2010 and June 2016 were identified. Predominant histologic subtypes were categorized into three groups according to their prognosis (group 0: lepidic; group 1: acinar/papillary; group 2: solid/micropapillary). Seven hundred eighteen radiomic features were extracted from segmented lung cancers on contrast-enhanced CT. A model-development set was formed from the images of 893 patients, while 100 image sets were reserved for testing. A least absolute shrinkage and selection operator method was used for feature selection. Performance of the radiomic model was evaluated using receiver operating characteristic curve analysis, and accuracy on the test set was compared with that of three radiologists with varying experiences (6, 7, and 19 years in chest CT).

RESULTS

Our model differentiated the three groups with areas under the curve (AUCs) of 0.892 and 0.895 on the development and test sets, respectively. In pairwise discrimination, the AUC was highest for group 0 vs. 2 (0.984). The accuracy of the model on the test set was higher than the averaged accuracy of the three radiologists without statistical significance (73.0% vs. 61.7%, p = 0.059). For group 2, the model achieved higher PPV than the observers (85.7% vs. 35.0-48.4%).

CONCLUSIONS

Predominant subtype-based prognostic groups of lung adenocarcinoma were classified by a CT-based radiomic model with comparable performance to radiologists.

KEY POINTS

• A CT-based radiomic model differentiated three prognosis-based subtype groups of lung adenocarcinoma with areas under the curve (AUCs) of 0.892 and 0.895 on development and test sets, respectively. • The CT-based radiomic model showed near perfect discrimination between group 0 and group 2 (AUCs, 0.984-1.000). • The accuracy of the CT-based radiomic model was comparable to the averaged accuracy of the three radiologists with 6, 7, and 19 years of clinical experience in chest CT (73.0% vs. 61.7%, p = 0.059), achieving a higher positive predictive value for group 2 than the observers (85.7% vs. 35.0-48.4%).

The ADNEX model to triage adnexal masses: An external validation study and comparison with the IOTA two-step strategy and subjective assessment by an experienced ultrasound operator

OBJECTIVES

The ADNEX (Assessment of Different NEoplasias in the adneXa) model was developed using parameters collected by experienced (level III) ultrasound examiners. Our primary aim was to externally validate the ADNEX model. Then, the discriminatory performance of ADNEX was compared with the two-step strategy and subjective assessment by an experienced ultrasound operator.

METHODS

Between February 2013 and January 2017, all patients who were scheduled for surgery for an adnexal mass at the Sant'Anna Hospital in Turin were enrolled in this study. Preoperative transvaginal sonography was performed, and the two-step strategy was applied for triage of the adnexal mass. Two ultrasound examiners, IOTA certified, applied the ADNEX model to all the collected masses based on the ultrasound reports. Finally, an experienced operator assigned the subjective assessment based on recorded ultrasound images. The discrimination and calibration performance of ADNEX were evaluated. The AUC was calculated for the basic discrimination between benign and malignant tumours. In addition, AUCs were computed for each pair of tumour types using the conditional risk method.

RESULTS

A total of 577 patients were included in the analysis: the overall prevalence of malignancy was 25 %. With ADNEX, the AUC to differentiate between benign and malignant masses was 0.9111 (95 % CI 0. 8788-0.9389). At risk cut-offs of 1%, 10 % and 30 %, sensitivities were 100 %, 89.6 % and 79.2 %, respectively, and specificities were 2.8 %, 76.2 % and 89.6 %, respectively. Discrimination between benign and stage II-IV tumours was good (AUC 0.935). The model had the most difficulties discriminating between borderline and stage I tumours (AUC 0.666), and between stages II-IV invasive and secondary metastatic tumours (AUC 0.736). The polytomous discrimination index (PDI) was 0.61 for ADNEX, whereas PDI for random performance would be 0.25. ADNEX proved to be equally or more accurate than the subjective assessment or the two-step strategy in the discrimination between benign and malignant adnexal masses.

CONCLUSIONS

the ADNEX model could probably be successfully applied when an expert examiner is not available and, therefore both a subjective assessment and the two-step strategy cannot be performed.

A Bayesian classification model for discriminating common infectious diseases in Zhejiang province, China

To develop a classification model for accurately discriminating common infectious diseases in Zhejiang province, China.Symptoms and signs, abnormal lab test results, epidemiological features, as well as the incidence rates were treated as predictors, and were collected from the published literature and a national surveillance system of infectious disease. A classification model was established using naïve Bayesian classifier. Dataset from historical outbreaks was applied for model validation, while sensitivity, specificity, accuracy, area under the receiver operating characteristic curve (AUC) and M-index were presented.A total of 146 predictors were included in the classification model, for discriminating 25 common infectious diseases. The sensitivity ranged from 44.44% for hepatitis E to 96.67% for measles. The specificity varied from 96.36% for dengue fever to 100% for 5 diseases. The median of total accuracy was 97.41% (range: 93.85%-99.04%). The AUCs exceeded 0.98 in 11 of 12 diseases, except in dengue fever (0.613). The M-index was 0.960 (95%CI 0.941-0.978).A novel classification model was constructed based on Bayesian approach to discriminate common infectious diseases in Zhejiang province, China. After entering symptoms and signs, abnormal lab test results, epidemiological features and city of disease origin, an output list of possible diseases ranked according to the calculated probabilities can be provided. The discrimination performance was reasonably good, making it useful in epidemiological applications.

Ultrasound and Clinical Preoperative Characteristics for Discrimination Between Ovarian Metastatic Colorectal Cancer and Primary Ovarian Cancer: A Case-Control Study

The aim of this study was to describe the clinical and sonographic features of ovarian metastases originating from colorectal cancer (mCRC), and to discriminate mCRC from primary ovarian cancer (OC). We conducted a multi-institutional, retrospective study of consecutive patients with ovarian mCRC who had undergone ultrasound examination using the International Ovarian Tumor Analysis (IOTA) terminology, with the addition of evaluating signs of necrosis and abdominal staging. A control group included patients with primary OC. Clinical and ultrasound data, subjective assessment (SA), and an assessment of different neoplasias in the adnexa (ADNEX) model were evaluated. Fisher's exact and Student's t-tests, the area under the receiver-operating characteristic curve (AUC), and classification and regression trees (CART) were used to conduct statistical analyses. In total, 162 patients (81 with OC and 81 with ovarian mCRC) were included. None of the patients with OC had undergone chemotherapy for CRC in the past, compared with 40% of patients with ovarian mCRC (p < 0.001). The ovarian mCRC tumors were significantly larger, a necrosis sign was more frequently present, and tumors had an irregular wall or were fixed less frequently; ascites, omental cake, and carcinomatosis were less common in mCRC than in primary OC. In a subgroup of patients with ovarian mCRC who had not undergone treatment for CRC in anamnesis, tumors were larger, and had fewer papillations and more locules compared with primary OC. The highest AUC for the discrimination of ovarian mCRC from primary OC was for CART (0.768), followed by SA (0.735) and ADNEX calculated with CA-125 (0.680). Ovarian mCRC and primary OC can be distinguished based on patient anamnesis, ultrasound pattern recognition, a proposed decision tree model, and an ADNEX model with CA-125 levels.

Fundamentals of Clinical Prediction Modeling for the Neurosurgeon

Clinical prediction models in neurosurgery are increasingly reported. These models aim to provide an evidence-based approach to the estimation of the probability of a neurosurgical outcome by combining 2 or more prognostic variables. Model development and model reporting are often suboptimal. A basic understanding of the methodology of clinical prediction modeling is needed when interpreting these models. We address basic statistical background, 7 modeling steps, and requirements of these models such that they may fulfill their potential for major impact for our daily clinical practice and for future scientific work.

Validation of the Feverkidstool and procalcitonin for detecting serious bacterial infections in febrile children

BackgroundTo validate the Feverkidstool, a prediction model consisting of clinical signs and symptoms and C-reactive protein (CRP) to identify serious bacterial infections (SBIs) in febrile children, and to determine the incremental diagnostic value of procalcitonin.MethodsThis prospective observational study that was carried out at two Dutch emergency departments included children with fever, aged 1 month to 16 years. The prediction models were developed with polytomous logistic regression differentiating "pneumonia" and "other SBIs" from "non-SBIs" using standardized, routinely collected data on clinical signs and symptoms, CRP, and procalcitonin.ResultsA total of 1,085 children were included with a median age of 1.6 years (interquartile range 0.8-3.4); 73 children (7%) had pneumonia and 98 children (9%) had other SBIs. The Feverkidstool showed good discriminative ability in this new population. After adding procalcitonin to the Feverkidstool, c-statistic for "pneumonia" increased from 0.85 (95% confidence interval (CI) 0.76-0.94) to 0.86 (0.77-0.94) and for "other SBI" from 0.81 (0.73-0.90) to 0.83 (0.75- 0.91). A model with clinical features and procalcitonin performed similar to the Feverkidstool.ConclusionThis study confirms the external validity of the Feverkidstool, with CRP and procalcitonin being equally valuable for predicting SBI in our population of febrile children. Our findings do not support routine dual use of CRP and procalcitonin.

Perseverance time of informal caregivers for people with dementia: construct validity, responsiveness and predictive validity

Background

Informal care is essential for many people with dementia (PwD), but it often results in a considerable burden for the caregiver. The perseverance time instrument integrates the aspect of perceived burden with the caregiver’s capacity to cope with the burden, in contrast to most available instruments, which measure solely the burden of caregiving. The aim of this study was to extend insight into psychometric properties of the perseverance time instrument, specifically the construct validity, responsiveness, and predictive validity, within the population of informal caregivers for PwD.

Methods

Data from two studies among informal caregivers of community-dwelling PwD in the Netherlands were used. The first study included 198 caregivers from a single region in the Netherlands and lasted 1 year. The second was a cross-sectional nationwide study with 166 caregivers for PwD. Questionnaires of both studies included questions regarding demographics and informal care, perseverance time, and other informal caregiver outcomes (Caregiver Strain Index, Self-rated Burden scale, Care-related Quality of Life instrument, and visual analogue scale health scores). Construct validity and responsiveness were assessed using a hypothesis-testing approach. The predictive validity of demographic characteristics and perseverance time for living situation after 1 year (living at home, institutionalized, or deceased) was assessed with multivariable multinomial regression.

Results

All but one of the hypotheses regarding construct validity were met. Three of five hypotheses regarding responsiveness were met. Perseverance time scores at baseline were associated with living situation after 1 year (p < 0.01), unlike age, sex, and relationship with PwD. Perseverance time strongly increased predictive power for living situation after 1 year (c-index between 0.671 and 0.775) in addition to demographic characteristics.

Conclusions

This study supports previous findings regarding the construct validity of the perseverance time instrument and adds new evidence of good construct validity, responsiveness, and predictive validity. The predictive power of perseverance time scores for living situation exceeds the predictive power of other burden measures and indicates informal care as an important factor for maintaining the patient at home.

Nonparametric estimation and inference for polytomous discrimination index

Polytomous discrimination index is a novel and important diagnostic accuracy measure for multi-category classification. After reconstructing its probabilistic definition, we propose a nonparametric approach to the estimation of polytomous discrimination index based on an empirical sample of biomarker values. In this paper, we provide the finite-sample and asymptotic properties of the proposed estimators and such analytic results may facilitate the statistical inference. Simulation studies are performed to examine the performance of the nonparametric estimators. Two real data examples are analysed to illustrate our methodology.

Validation and updating of risk models based on multinomial logistic regression

BACKGROUND

Risk models often perform poorly at external validation in terms of discrimination or calibration. Updating methods are needed to improve performance of multinomial logistic regression models for risk prediction.

METHODS

We consider simple and more refined updating approaches to extend previously proposed methods for dichotomous outcomes. These include model recalibration (adjustment of intercept and/or slope), revision (re-estimation of individual model coefficients), and extension (revision with additional markers). We suggest a closed testing procedure to assist in deciding on the updating complexity. These methods are demonstrated on a case study of women with pregnancies of unknown location (PUL). A previously developed risk model predicts the probability that a PUL is a failed, intra-uterine, or ectopic pregnancy. We validated and updated this model on more recent patients from the development setting (temporal updating; n = 1422) and on patients from a different hospital (geographical updating; n = 873). Internal validation of updated models was performed through bootstrap resampling.

RESULTS

Contrary to dichotomous models, we noted that recalibration can also affect discrimination for multinomial risk models. If the number of outcome categories is higher than the number of variables, logistic recalibration is obsolete because straightforward model refitting does not require the estimation of more parameters. Although recalibration strongly improved performance in the case study, the closed testing procedure selected model revision. Further, revision of functional form of continuous predictors had a positive effect on discrimination, whereas penalized estimation of changes in model coefficients was beneficial for calibration.

CONCLUSIONS

Methods for updating of multinomial risk models are now available to improve predictions in new settings. A closed testing procedure is helpful to decide whether revision is preferred over recalibration. Because multicategory outcomes increase the number of parameters to be estimated, we recommend full model revision only when the sample size for each outcome category is large.

Managing pregnancy of unknown location based on initial serum progesterone and serial serum hCG levels: development and validation of a two-step triage protocol

OBJECTIVES

A uniform rationalized management protocol for pregnancies of unknown location (PUL) is lacking. We developed a two-step triage protocol to select PUL at high risk of ectopic pregnancy (EP), based on serum progesterone level at presentation (step 1) and the serum human chorionic gonadotropin (hCG) ratio, defined as the ratio of hCG at 48 h to hCG at presentation (step 2).

METHODS

This was a cohort study of 2753 PUL (301 EP), involving a secondary analysis of prospectively and consecutively collected PUL data from two London-based university teaching hospitals. Using a chronological split we used 1449 PUL for development and 1304 for validation. We aimed to assign PUL as low risk with high confidence (high negative predictive value (NPV)) while classifying most EP as high risk (high sensitivity). The first triage step assigned PUL as low risk using a threshold of serum progesterone at presentation. The remaining PUL were triaged using a novel logistic regression risk model based on hCG ratio and initial serum progesterone (second step), defining low risk as an estimated EP risk of < 5%.

RESULTS

On validation, initial serum progesterone ≤ 2 nmol/L (step 1) classified 16.1% PUL as low risk. Second-step classification with the risk model selected an additional 46.0% of all PUL as low risk. Overall, the two-step protocol classified 62.1% of PUL as low risk, with an NPV of 98.6% and a sensitivity of 92.0%. When the risk model was used in isolation (i.e. without the first step), 60.5% of PUL were classified as low risk with 99.1% NPV and 94.9% sensitivity.

CONCLUSION

PUL can be classified efficiently into being either high or low risk for complications using a two-step protocol involving initial progesterone and hCG levels and the hCG ratio. Copyright © 2016 ISUOG. Published by John Wiley & Sons Ltd.