Validation techniques for logistic regression models

Penalized Regression Methods With Modified Cross-Validation and Bootstrap Tuning Produce Better Prediction Models

Risk prediction models fitted using maximum likelihood estimation (MLE) are often overfitted resulting in predictions that are too extreme and a calibration slope (CS) less than 1. Penalized methods, such as Ridge and Lasso, have been suggested as a solution to this problem as they tend to shrink regression coefficients toward zero, resulting in predictions closer to the average. The amount of shrinkage is regulated by a tuning parameter,λ , $\lambda ,$ commonly selected via cross-validation ("standard tuning"). Though penalized methods have been found to improve calibration on average, they often over-shrink and exhibit large variability in the selected λ $\lambda $ and hence the CS. This is a problem, particularly for small sample sizes, but also when using sample sizes recommended to control overfitting. We consider whether these problems are partly due to selecting λ $\lambda $ using cross-validation with "training" datasets of reduced size compared to the original development sample, resulting in an over-estimation of λ $\lambda $ and, hence, excessive shrinkage. We propose a modified cross-validation tuning method ("modified tuning"), which estimates λ $\lambda $ from a pseudo-development dataset obtained via bootstrapping from the original dataset, albeit of larger size, such that the resulting cross-validation training datasets are of the same size as the original dataset. Modified tuning can be easily implemented in standard software and is closely related to bootstrap selection of the tuning parameter ("bootstrap tuning"). We evaluated modified and bootstrap tuning for Ridge and Lasso in simulated and real data using recommended sample sizes, and sizes slightly lower and higher. They substantially improved the selection of λ $\lambda $ , resulting in improved CS compared to the standard tuning method. They also improved predictions compared to MLE.

Risks posed by invasive species to the provision of ecosystem services in Europe

Invasive species significantly impact biodiversity and ecosystem services, yet understanding these effects at large spatial scales remains a challenge. Our study addresses this gap by assessing the current and potential future risks posed by 94 invasive species to seven key ecosystem services in Europe. We demonstrate widespread potential impacts, particularly on outdoor recreation, habitat maintenance, crop provisioning, and soil and nitrogen retention. Exposure to invasive species was higher in areas with lower provision of ecosystem services, particularly for regulating and cultural services. Exposure was also high in areas where ecosystem contributions to crop provision and nitrogen retention were at their highest. Notably, regions vital for ecosystem services currently have low invasion suitability, but face an average 77% increase in potential invasion area. Here we show that, while high-value ecosystem service areas at the highest risk represent a small fraction of Europe (0-13%), they are disproportionally important for service conservation. Our study underscores the importance of monitoring and protecting these hotspots to align management strategies with international biodiversity targets, considering both invasion vulnerability and ecosystem service sustainability.

Lost and found: Patterns of marine litter accumulation on the remote Island of Santa Luzia, Cabo Verde

Santa Luzia, an uninhabited island in the archipelago of Cabo Verde, serves as a natural laboratory and important nesting site for loggerhead turtles Carettacaretta. The island constitutes an Integral Natural Reserve and a Marine Protected Area. We assessed marine litter accumulation on sandy beaches of the island and analysed their spatial patterns using two sampling methods: at a fine scale, sand samples from 1 × 1 m squares were collected, identifying debris larger than 1 mm; at a coarse scale, drone surveys were conducted to identify visible marine debris (>25 mm) in aerial images. We sampled six points on three beaches of the island: Achados (three points), Francisca (two points) and Palmo Tostão (one point). Then, we modelled the abundance of marine debris using topographical variables as explanatory factors, derived from digital surface models (DSM). Our findings reveal that the island is a significant repository for marine litter (>84% composed of plastics), with up to 917 plastic items per m2 in the sand samples and a maximum of 38 macro-debris items per m2 in the drone surveys. Plastic fragments dominate, followed by plastic pellets (at the fine-scale approach) and fishing materials (at the coarse-scale approach). We observed that north-facing, higher-elevation beaches accumulate more large marine litter, while slope and elevation affect their spatial distribution within the beach. Achados Beach faces severe marine debris pollution challenges, and the upcoming climate changes could exacerbate this problem.

Epidemiological Analyses of the First Incursion of the Epizootic Hemorrhagic Disease Virus Serotype 8 in Tunisia, 2021-2022

Epizootic hemorrhagic disease (EHD) is a non-contagious arthropod-transmitted viral disease and a World Organization for Animal Health (WOAH)-listed disease of domestic and wild ruminants since 2008. EHDV is transmitted among susceptible animals by a few species of midges of genus Culicoides. During the fall of 2021, a large outbreak caused by the epizootic hemorrhagic disease virus (EHDV), identified as serotype 8, was reported in Tunisian dairy and beef farms with Bluetongue virus (BTV)-like clinical signs. The disease was detected later in the south of Italy, in Spain, in Portugal and, more recently, in France, where it caused severe infections in cattle. This was the first evidence of EHDV-8 circulation outside Australia since 1982. In this study, we analyzed the epidemiological situation of the 2021-2022 EHDV outbreaks reported in Tunisia, providing a detailed description of the spatiotemporal evolution of the disease. We attempted to identify the eco-climatic factors associated with infected areas using generalized linear models (GLMs). Our results demonstrated that environmental factors mostly associated with the presence of C. imicola, such as digital elevation model (DEM), slope, normalized difference vegetation index (NDVI), and night-time land surface temperature (NLST)) were by far the most explanatory variables for EHD repartition cases in Tunisia that may have consequences in neighboring countries, both in Africa and Europe through the spread of infected vectors. The risk maps elaborated could be useful for disease control and prevention strategies.

Development and validation of an interpretable Markov-embedded multilabel model for predicting risks of multiple postoperative complications among surgical inpatients: a multicenter prospective cohort study

BACKGROUND

When they encounter various highly related postoperative complications, existing risk evaluation tools that focus on single or any complications are inadequate in clinical practice. This seriously hinders complication management because of the lack of a quantitative basis. An interpretable multilabel model framework that predicts multiple complications simultaneously is urgently needed.

MATERIALS AND METHODS

The authors included 50 325 inpatients from a large multicenter cohort (2014-2017). The authors separated patients from one hospital for external validation and randomly split the remaining patients into training and internal validation sets. A MARKov-EmbeDded (MARKED) multilabel model was proposed, and three models were trained for comparison: binary relevance, a fully connected network (FULLNET), and a deep neural network. Performance was mainly evaluated using the area under the receiver operating characteristic curve (AUC). The authors interpreted the model using Shapley Additive Explanations. Complication-specific risk and risk source inference were provided at the individual level.

RESULTS

There were 26 292, 6574, and 17 459 inpatients in the training, internal validation, and external validation sets, respectively. For the external validation set, MARKED achieved the highest average AUC (0.818, 95% CI: 0.771-0.864) across eight outcomes [compared with binary relevance, 0.799 (0.748-0.849), FULLNET, 0.806 (0.756-0.856), and deep neural network, 0.815 (0.765-0.866)]. Specifically, the AUCs of MARKED were above 0.9 for cardiac complications [0.927 (0.894-0.960)], neurological complications [0.905 (0.870-0.941)], and mortality [0.902 (0.867-0.937)]. Serum albumin, surgical specialties, emergency case, American Society of Anesthesiologists score, age, and sex were the six most important preoperative variables. The interaction between complications contributed more than the preoperative variables, and formed a hierarchical chain of risk factors, mild complications, and severe complications.

CONCLUSION

The authors demonstrated the advantage of MARKED in terms of performance and interpretability. The authors expect that the identification of high-risk patients and the inference of the risk source for specific complications will be valuable for clinical decision-making.

Calibrating machine learning approaches for probability estimation: A comprehensive comparison

Statistical prediction models have gained popularity in applied research. One challenge is the transfer of the prediction model to a different population which may be structurally different from the model for which it has been developed. An adaptation to the new population can be achieved by calibrating the model to the characteristics of the target population, for which numerous calibration techniques exist. In view of this diversity, we performed a systematic evaluation of various popular calibration approaches used by the statistical and the machine learning communities for estimating two-class probabilities. In this work, we first provide a review of the literature and, second, present the results of a comprehensive simulation study. The calibration approaches are compared with respect to their empirical properties and relationships, their ability to generalize precise probability estimates to external populations and their availability in terms of easy-to-use software implementations. Third, we provide code from real data analysis allowing its application by researchers. Logistic calibration and beta calibration, which estimate an intercept plus one and two slope parameters, respectively, consistently showed the best results in the simulation studies. Calibration on logit transformed probability estimates generally outperformed calibration methods on nontransformed estimates. In case of structural differences between training and validation data, re-estimation of the entire prediction model should be outweighted against sample size of the validation data. We recommend regression-based calibration approaches using transformed probability estimates, where at least one slope is estimated in addition to an intercept for updating probability estimates in validation studies.

Climate change affects the distribution of diversity across marine food webs

Many studies predict shifts in species distributions and community size composition in response to climate change, yet few have demonstrated how these changes will be distributed across marine food webs. We use Bayesian Additive Regression Trees to model how climate change will affect the habitat suitability of marine fish species across a range of body sizes and belonging to different feeding guilds, each with different habitat and feeding requirements in the northeast Atlantic shelf seas. Contrasting effects of climate change are predicted for feeding guilds, with spatially extensive decreases in the species richness of consumers lower in the food web (planktivores) but increases for those higher up (piscivores). Changing spatial patterns in predator-prey mass ratios and fish species size composition are also predicted for feeding guilds and across the fish assemblage. In combination, these changes could influence nutrient uptake and transformation, transfer efficiency and food web stability, and thus profoundly alter ecosystem structure and functioning.

Predicting the distribution of Ixodes ricinus and Dermacentor reticulatus in Europe: a comparison of climate niche modelling approaches

BACKGROUND

The ticks Ixodes ricinus and Dermacentor reticulatus are two of the most important vectors in Europe. Climate niche modelling has been used in many studies to attempt to explain their distribution and to predict changes under a range of climate change scenarios. The aim of this study was to assess the ability of different climate niche modelling approaches to explain the known distribution of I. ricinus and D. reticulatus in Europe.

METHODS

A series of climate niche models, using different combinations of input data, were constructed and assessed. Species occurrence records obtained from systematic literature searches and Global Biodiversity Information Facility data were thinned to different degrees to remove sampling spatial bias. Four sources of climate data were used: bioclimatic variables, WorldClim, TerraClimate and MODIS satellite-derived data. Eight different model training extents were examined and three modelling frameworks were used: maximum entropy, generalised additive models and random forest models. The results were validated through internal cross-validation, comparison with an external independent dataset and expert opinion.

RESULTS

The performance metrics and predictive ability of the different modelling approaches varied significantly within and between each species. Different combinations were better able to define the distribution of each of the two species. However, no single approach was considered fully able to capture the known distribution of the species. When considering the mean of the performance metrics of internal and external validation, 24 models for I. ricinus and 11 models for D. reticulatus of the 96 constructed were considered adequate according to the following criteria: area under the receiver-operating characteristic curve > 0.7; true skill statistic > 0.4; Miller's calibration slope 0.25 above or below 1; Boyce index > 0.9; omission rate < 0.15.

CONCLUSIONS

This comprehensive analysis suggests that there is no single 'best practice' climate modelling approach to account for the distribution of these tick species. This has important implications for attempts to predict climate-mediated impacts on future tick distribution. It is suggested here that climate variables alone are not sufficient; habitat type, host availability and anthropogenic impacts, not included in current modelling approaches, could contribute to determining tick presence or absence at the local or regional scale.

Recalibrating prognostic models to improve predictions of in-hospital child mortality in resource-limited settings

BACKGROUND

In an external validation study, model recalibration is suggested once there is evidence of poor model calibration but with acceptable discriminatory abilities. We identified four models, namely RISC-Malawi (Respiratory Index of Severity in Children) developed in Malawi, and three other predictive models developed in Uganda by Lowlaavar et al. (2016). These prognostic models exhibited poor calibration performance in the recent external validation study, hence the need for recalibration.

OBJECTIVE

In this study, we aim to recalibrate these models using regression coefficients updating strategy and determine how much their performances improve.

METHODS

We used data collected by the Clinical Information Network from paediatric wards of 20 public county referral hospitals. Missing data were multiply imputed using chained equations. Model updating entailed adjustment of the model's calibration performance while the discriminatory ability remained unaltered. We used two strategies to adjust the model: intercept-only and the logistic recalibration method.

RESULTS

Eligibility criteria for the RISC-Malawi model were met in 50,669 patients, split into two sets: a model-recalibrating set (n = 30,343) and a test set (n = 20,326). For the Lowlaavar models, 10,782 patients met the eligibility criteria, of whom 6175 were used to recalibrate the models and 4607 were used to test the performance of the adjusted model. The intercept of the recalibrated RISC-Malawi model was 0.12 (95% CI 0.07, 0.17), while the slope of the same model was 1.08 (95% CI 1.03, 1.13). The performance of the recalibrated models on the test set suggested that no model met the threshold of a perfectly calibrated model, which includes a calibration slope of 1 and a calibration-in-the-large/intercept of 0.

CONCLUSIONS

Even after model adjustment, the calibration performances of the 4 models did not meet the recommended threshold for perfect calibration. This finding is suggestive of models over/underestimating the predicted risk of in-hospital mortality, potentially harmful clinically. Therefore, researchers may consider other alternatives, such as ensemble techniques to combine these models into a meta-model to improve out-of-sample predictive performance.

Projected northward shifts in eastern red-backed salamanders due to changing climate

Many species' distributions are being impacted by the acceleration of climate change. Amphibians in particular serve numerous ecosystem functions and are useful indicators of environmental change. Understanding how their distributions have been impacted by climate change and will continue to be impacted is thus important to overall ecosystem health. Plethodon cinereus (Eastern Red-Backed Salamander) is a widespread species of lungless salamander (Plethodontidae) that ranges across northeastern North America. To better understand future potential lungless salamander range shifts, we quantify environmental favorability, the likelihood of membership in a set of sites where environmental conditions are favorable for a species, for P. cinereus in multiple time periods, and examine shifts in the species' distribution. First, utilizing a large data set of georeferenced records, we assessed which bioclimatic variables were associated with environmental favorability in P. cinereus. We then used species distribution modeling for two time periods (1961-1980 and 2001-2020) to determine whether there was a regional shift in environmental favorability in the past 60 years. Models were then used to project future distributions under eight climate change scenarios to quantify potential range shifts. Shifts were assessed using fuzzy logic, avoiding thresholds that oversimplify model predictions into artificial binary outputs. We found that P. cinereus presence is strongly associated with environmental stability. There has been a substantial northward shift in environmental favorability for P. cinereus between 1961-1980 and 2001-2020. This shift is predicted to continue by 2070, with larger shifts under higher greenhouse gas emission scenarios. As climate change accelerates, it is differentially impacting species but has especially strong impacts on dispersal-limited species. Our results show substantial northward shifts in climatic favorability in the last 60 years for P. cinereus, which are likely to be exacerbated by ongoing climate change. Since P. cinereus is dispersal-limited, these models may imply local extirpations along the southern modern range with limited northward dispersal. Continued monitoring of amphibians in the field will reveal microclimatic effects associated with climate change and the accuracy of the model predictions presented here.

Extended-spectrum β-lactamase- and AmpC β-lactamase-producing Enterobacterales associated with urinary tract infections in the New Zealand community: a case-control study

OBJECTIVES

To assess whether having a pet in the home is a risk factor for community-acquired urinary tract infections associated with extended-spectrum β-lactamase (ESBL)- or AmpC β-lactamase (ACBL)- producing Enterobacterales.

METHODS

An unmatched case-control study was conducted between August 2015 and September 2017. Cases (n = 141) were people with community-acquired urinary tract infection (UTI) caused by ESBL- or ACBL-producing Enterobacterales. Controls (n = 525) were recruited from the community. A telephone questionnaire on pet ownership and other factors was administered, and associations were assessed using logistic regression.

RESULTS

Pet ownership was not associated with ESBL- or ACBL-producing Enterobacterales-related human UTIs. A positive association was observed for recent antimicrobial treatment, travel to Asia in the previous year, and a doctor's visit in the last 6 months. Among isolates with an ESBL-/ACBL-producing phenotype, 126/134 (94%) were Escherichia coli, with sequence type 131 being the most common (47/126).

CONCLUSIONS

Companion animals in the home were not found to be associated with ESBL- or ACBL-producing Enterobacterales-related community-acquired UTIs in New Zealand. Risk factors included overseas travel, recent antibiotic use, and doctor visits.

Predictive models for the incidence of Parkinson's disease: systematic review and critical appraisal

Numerous predictive models for Parkinson's disease (PD) incidence have been published recently. However, the model performance and methodological quality of those available models are yet needed to be summarized and assessed systematically. In this systematic review, we systematically reviewed the published predictive models for PD incidence and assessed their risk of bias and applicability. Three international databases were searched. Cohort or nested case-control studies that aimed to develop or validate a predictive model for PD incidence were considered eligible. The Prediction model Risk Of Bias ASsessment Tool (PROBAST) was used for risk of bias and applicability assessment. Ten studies covering 10 predictive models were included. Among them, four studies focused on model development, covering eight models, while the remaining six studies focused on model external validation, covering two models. The discrimination of the eight new development models was generally poor, with only one model reported C index > 0.70. Four out of the six external validation studies showed excellent or outstanding discrimination. All included studies had high risk of bias. Three predictive models (the International Parkinson and Movement Disorder Society [MDS] prodromal PD criteria, the model developed by Karabayir et al. and models validated by Faust et al.) are recommended for clinical application by considering model performance and resource-demanding. In conclusion, the performance and methodological quality of most of the identified predictive models for PD incidence were unsatisfactory. The MDS prodromal PD criteria, model developed by Karabayir et al. and model validated by Faust et al. may be considered for clinical use.

External validation of the Rotterdam prostate cancer risk calculator within a high-risk Dutch clinical cohort

PURPOSE

This study aims to externally validate the Rotterdam Prostate Cancer Risk Calculator (RPCRC)-3/4 and RPCRC-MRI within a Dutch clinical cohort.

METHODS

Men subjected to prostate biopsies, between 2018 and 2021, due to a clinical suspicion of prostate cancer (PCa) were retrospectively included. The performance of the RPCRC-3/4 and RPCRC-MRI was analyzed in terms of discrimination, calibration and net benefit. In addition, the need for recalibration and adjustment of risk thresholds for referral was investigated. Clinically significant (cs) PCa was defined as Gleason score ≥ 3 + 4.

RESULTS

A total of 1575 men were included in the analysis. PCa was diagnosed in 63.2% (996/1575) of men and csPCa in 41.7% (656/1575) of men. Use of the RPCRC-3/4 could have prevented 37.3% (587/1575) of all MRIs within this cohort, thereby missing 18.3% (120/656) of csPCa diagnoses. After recalibration and adjustment of risk thresholds to 20% for PCa and 10% for csPCa, use of the recalibrated RPCRC-3/4 could have prevented 15.1% (238/1575) of all MRIs, resulting in 5.3% (35/656) of csPCa diagnoses being missed. The performance of the RPCRC-MRI was good; use of this risk calculator could have prevented 10.7% (169/1575) of all biopsies, resulting in 1.2% (8/656) of csPCa diagnoses being missed.

CONCLUSION

The RPCRC-3/4 underestimates the probability of having csPCa within this Dutch clinical cohort, resulting in significant numbers of csPCa diagnoses being missed. For optimal performance of a risk calculator in a specific cohort, evaluation of its performance within the population under study is essential.

Prediction Models for Osteoporotic Fractures Risk: A Systematic Review and Critical Appraisal

Osteoporotic fractures (OF) are a global public health problem currently. Many risk prediction models for OF have been developed, but their performance and methodological quality are unclear. We conducted this systematic review to summarize and critically appraise the OF risk prediction models. Three databases were searched until April 2021. Studies developing or validating multivariable models for OF risk prediction were considered eligible. Used the prediction model risk of bias assessment tool to appraise the risk of bias and applicability of included models. All results were narratively summarized and described. A total of 68 studies describing 70 newly developed prediction models and 138 external validations were included. Most models were explicitly developed (n=31, 44%) and validated (n=76, 55%) only for female. Only 22 developed models (31%) were externally validated. The most validated tool was Fracture Risk Assessment Tool. Overall, only a few models showed outstanding (n=3, 1%) or excellent (n=32, 15%) prediction discrimination. Calibration of developed models (n=25, 36%) or external validation models (n=33, 24%) were rarely assessed. No model was rated as low risk of bias, mostly because of an insufficient number of cases and inappropriate assessment of calibration. There are a certain number of OF risk prediction models. However, few models have been thoroughly internally validated or externally validated (with calibration being unassessed for most of the models), and all models showed methodological shortcomings. Instead of developing completely new models, future research is suggested to validate, improve, and analyze the impact of existing models.

A prognostic model for 1-month mortality in the postoperative intensive care unit

PURPOSES

Recognizing which patients admitted postsurgically to the intensive care unit (ICU) are at greater risk of mortality assists medical staff to identify who will benefit most from the care. We developed a prediction model for the 1-month mortality of postsurgical ICU patients.

METHODS

From May, 2019 to May, 2020, we conducted a prospective cohort study in the postsurgical ICU of a teaching hospital affiliated with our University of Medical Sciences. The outcome was death within 1 month of admission and the predictors were a variety of anthropometric and clinical features. The subjects of this analysis were 805 consecutive adult postsurgical patients with a mean (SD) age of 54.8 (18.9) years.

RESULTS

Overall, the resulted logistic model was well-fitted [χ² (26) = 772.097, p < 0.001, Nagelkerke R² = 0.814] accurate (88%), and specific (92%). The adjusted odds ratio for body temperature was 0.51, p < 0.001. Patients with comorbidities and those undergoing multiple operations were at a greater risk of mortality, odds = 10.00 and 10.65 (both p < 0.001).

CONCLUSIONS

Higher body temperature at the time of postoperative ICU admission is a protective factor against 1-month mortality. Our study found that patients with several comorbidities and those who have undergone multiple operations are at a greater risk of a poor outcome.

Prediction of 90-day mortality after total hip arthroplasty

AIMS

To develop and externally validate a parsimonious statistical prediction model of 90-day mortality after elective total hip arthroplasty (THA), and to provide a web calculator for clinical usage.

METHODS

We included 53,099 patients with cemented THA due to osteoarthritis from the Swedish Hip Arthroplasty Registry for model derivation and internal validation, as well as 125,428 patients from England and Wales recorded in the National Joint Register for England, Wales, Northern Ireland, the Isle of Man, and the States of Guernsey (NJR) for external model validation. A model was developed using a bootstrap ranking procedure with a least absolute shrinkage and selection operator (LASSO) logistic regression model combined with piecewise linear regression. Discriminative ability was evaluated by the area under the receiver operating characteristic curve (AUC). Calibration belt plots were used to assess model calibration.

RESULTS

A main effects model combining age, sex, American Society for Anesthesiologists (ASA) class, the presence of cancer, diseases of the central nervous system, kidney disease, and diagnosed obesity had good discrimination, both internally (AUC = 0.78, 95% confidence interval (CI) 0.75 to 0.81) and externally (AUC = 0.75, 95% CI 0.73 to 0.76). This model was superior to traditional models based on the Charlson (AUC = 0.66, 95% CI 0.62 to 0.70) and Elixhauser (AUC = 0.64, 95% CI 0.59 to 0.68) comorbidity indices. The model was well calibrated for predicted probabilities up to 5%.

CONCLUSION

We developed a parsimonious model that may facilitate individualized risk assessment prior to one of the most common surgical interventions. We have published a web calculator to aid clinical decision-making. Cite this article: Bone Joint J 2021;103-B(3):469-478.

Derivation of the first clinical diagnostic models for dehydration severity in patients over five years with acute diarrhea

Diarrheal diseases lead to an estimated 1.3 million deaths each year, with the majority of those deaths occurring in patients over five years of age. As the severity of diarrheal disease can vary widely, accurately assessing dehydration status remains the most critical step in acute diarrhea management. The objective of this study is to empirically derive clinical diagnostic models for assessing dehydration severity in patients over five years with acute diarrhea in low resource settings. We enrolled a random sample of patients over five years with acute diarrhea presenting to the icddr,b Dhaka Hospital. Two blinded nurses independently assessed patients for symptoms/signs of dehydration on arrival. Afterward, consecutive weights were obtained to determine the percent weight change with rehydration, our criterion standard for dehydration severity. Full and simplified ordinal logistic regression models were derived to predict the outcome of none (<3%), some (3-9%), or severe (>9%) dehydration. The reliability and accuracy of each model were assessed. Bootstrapping was used to correct for over-optimism and compare each model's performance to the current World Health Organization (WHO) algorithm. 2,172 patients were enrolled, of which 2,139 (98.5%) had complete data for analysis. The Inter-Class Correlation Coefficient (reliability) was 0.90 (95% CI = 0.87, 0.91) for the full model and 0.82 (95% CI = 0.77, 0.86) for the simplified model. The area under the Receiver-Operator Characteristic curve (accuracy) for severe dehydration was 0.79 (95% CI: 0.76-0.82) for the full model and 0.73 (95% CI: 0.70, 0.76) for the simplified model. The accuracy for both the full and simplified models were significantly better than the WHO algorithm (p<0.001). This is the first study to empirically derive clinical diagnostic models for dehydration severity in patients over five years. Once prospectively validated, the models may improve management of patients with acute diarrhea in low resource settings.

Pain stickiness in pediatric complex regional pain syndrome: A role for the nucleus accumbens

•

Pain nonresponders have decreased nucleus accumbens (NAc) grey matter density.

•

Pain nonresponders have reduced functional connectivity between NAc and dlPFC.

•

Connectivity strength between NAc and dlPFC correlates with changes in pain.

•

Prediction estimate for pain improvement with grey matter and connectivity was 87%.

Some individuals with chronic pain experience improvement in their pain with treatment, whereas others do not. The neurobiological reason is unclear, but an understanding of brain structure and functional patterns may provide insights into pain’s responsivity to treatment. In this investigation, we used magnetic resonance imaging (MRI) techniques to determine grey matter density alterations on resting functional connectivity (RFC) strengths between pain responders and nonresponders in patients with complex regional pain syndrome. Brain metrics of pediatric patients at admission to an intensive pain rehabilitative treatment program were evaluated. Pain responders reported significant pain improvement at discharge and/or follow-up whereas nonresponders reported no improvements in pain, increases in pain, or emergence of new pain symptoms. The pain (responder/nonresponder) groups were compared with pain-free healthy controls to examine predictors of pain responder status via brain metrics. Our results show: (1) on admission, pain nonresponders had decreased grey matter density (GMD) within the nucleus accumbens (NAc) and reduced RFC strength between the NAc and the dorsolateral prefrontal cortex vs. responders; (2) Connectivity strength was positively correlated with change in pain intensity from admission to discharge; (3) Compared with pain-free controls, grey matter and RFC differences emerged only among pain nonresponders; and (4) Using a discriminative model, combining GMD and RFC strengths assessed at admission showed the highest prediction estimate (87%) on potential for pain improvement, warranting testing in a de novo sample. Taken together, these results support the idea that treatment responsiveness on pain is underpinned by concurrent brain structure and resting brain activity.

A multi-modal diagnostic model improves detection of cardiac amyloidosis among patients with diagnostic confirmation by cardiac biopsy

BACKGROUND

Timely recognition of cardiac amyloidosis is clinically important, but the diagnosis is frequently delayed.

OBJECTIVES

We sought to identify a multi-modality approach with the highest diagnostic accuracy in patients evaluated by cardiac biopsy, the diagnostic gold standard.

METHODS

Consecutive patients (N = 242) who underwent cardiac biopsy for suspected amyloidosis within an 18-year period were retrospectively identified. Cardiac biomarker, ECG, and echocardiography results were examined for correlation with biopsy-proven disease. A prediction model for cardiac amyloidosis was derived using multivariable logistic regression.

RESULTS

The overall cohort was characterized by elevated BNP (median 727 ng/mL), increased left ventricular wall thickness (IWT; median 1.7 cm), and reduced voltage-to-mass ratio (median 0.06 mm/[g/m²]). One hundred and thirteen patients (46%) had either light chain (n = 53) or transthyretin (n = 60) amyloidosis by cardiac biopsy. A prediction model including age, relative wall thickness, left atrial pressure by E/e', and low limb lead voltage (<0.5 mV) showed good discrimination for cardiac amyloidosis with an optimism-corrected c-index of 0.87 (95% CI 0.83-0.92). The diagnostic accuracy of this model (79% sensitivity, 84% specificity) surpassed that of traditional screening parameters, such as IWT in the absence of left ventricular hypertrophy on ECG (98% sensitivity, 20% specificity) and IWT with low limb lead voltage (49% sensitivity, 91% specificity).

CONCLUSION

Among patients with an advanced infiltrative cardiomyopathy phenotype, traditional biomarker, ECG, and echocardiography-based screening tests have limited individual diagnostic utility for cardiac amyloidosis. A prediction algorithm including age, relative wall thickness, E/e', and low limb lead voltage improves the detection of cardiac biopsy-proven disease.

Predicting the spatio-temporal spread of West Nile virus in Europe

West Nile virus is a widely spread arthropod-born virus, which has mosquitoes as vectors and birds as reservoirs. Humans, as dead-end hosts of the virus, may suffer West Nile Fever (WNF), which sometimes leads to death. In Europe, the first large-scale epidemic of WNF occurred in 1996 in Romania. Since then, human cases have increased in the continent, where the highest number of cases occurred in 2018. Using the location of WNF cases in 2017 and favorability models, we developed two risk models, one environmental and the other spatio-environmental, and tested their capacity to predict in 2018: 1) the location of WNF; 2) the intensity of the outbreaks (i.e. the number of confirmed human cases); and 3) the imminence of the cases (i.e. the Julian week in which the first case occurred). We found that climatic variables (the maximum temperature of the warmest month and the annual temperature range), human-related variables (rain-fed agriculture, the density of poultry and horses), and topo-hydrographic variables (the presence of rivers and altitude) were the best environmental predictors of WNF outbreaks in Europe. The spatio-environmental model was the most useful in predicting the location of WNF outbreaks, which suggests that a spatial structure, probably related to bird migration routes, has a role in the geographical pattern of WNF in Europe. Both the intensity of cases and their imminence were best predicted using the environmental model, suggesting that these features of the disease are linked to the environmental characteristics of the areas. We highlight the relevance of river basins in the propagation dynamics of the disease, as outbreaks started in the lower parts of the river basins, from where WNF spread towards the upper parts. Therefore, river basins should be considered as operational geographic units for the public health management of the disease.

Predictive approach in managing voiding dysfunction after surgery for deep endometriosis: a personalized nomogram

INTRODUCTION AND HYPOTHESIS

The aim was to develop a nomogram based on clinical and surgical factors to predict the likelihood of voiding dysfunction after surgery for deep endometriosis.

METHODS

This was a retrospective study of 789 patients (training set) who underwent surgery for deep endometriosis with colorectal involvement from January 2005 through December 2017 at Tenon University Hospital. A multivariate logistic regression analysis of selected risk factors was performed to construct a nomogram to predict postoperative voiding dysfunction. The nomogram was externally validated in 333 patients (validation set) from Rouen University Hospital.

RESULTS

Postoperative voiding dysfunction occurred in 23% of the patients (180/789) in the training set. Age, colorectal involvement/management, colpectomy and parametrectomy were the main factors associated with an increased risk of voiding dysfunction and were included in the nomogram. The predictive model had an internal concordance index of 0.79 (95% CI: 0.77-0.81) after the 200 repetitions of bootstrap sample corrections and showed good calibration. The ROC area related to the nomogram for external validation was 0.74 (95% CI: 0.72-0.76).

CONCLUSIONS

The nomogram we present here, based on four clinical and imaging characteristics, could be useful in predicting postoperative voiding dysfunction for women undergoing surgery for deep endometriosis. Patients could thus be better informed about this postoperative risk and the surgical strategy adapted according to individual risk. The accuracy of the tool was validated externally but additional validation is required.

Bedside prediction of intradialytic hemodynamic instability in critically ill patients: the SOCRATE study

BACKGROUND

Despite improvements in intermittent hemodialysis management, intradialytic hemodynamic instability (IHI) remains a common issue that could account for increased mortality and delayed renal recovery. However, predictive factors of IHI remain poorly explored. The objective of this study was to evaluate the relationship between baseline macrohemodynamic, tissue hypoperfusion parameters and IHI occurrence.

METHODS

Prospective observational study conducted in a 18-bed medical ICU of a tertiary teaching hospital. Cardiovascular SOFA score, index capillary refill time (CRT) and lactate level were measured just before (T0) consecutive intermittent hemodialysis sessions performed for AKI. The occurrence of IHI requiring a therapeutic intervention was recorded.

RESULTS

Two hundred eleven sessions, corresponding to 72 (34%) first sessions and 139 (66%) later sessions, were included. As IHI mostly occurred during first sessions (43% vs 12%, P < 0.0001), following analyses were performed on the 72 first sessions. At T0, cardiovascular SOFA score ≥1 (87% vs 51%, P = 0.0021) was more frequent before IHI sessions, as well as index CRT ≥ 3 s (55% vs 15%, P = 0.0004), and hyperlactatemia > 2 mmol/L (68% vs 29%, P = 0.0018). Moreover, the occurrence of IHI increased with the number of macrohemodynamic and tissue perfusion impaired parameters, named SOCRATE score (cardiovascular SOFA, index CRT and lactATE): 10% (95% CI [3%, 30%]), 33% (95% CI [15%, 58%]), 55% (95% CI [35%, 73%]) and 80% (95% CI [55%, 93%]) for 0, 1, 2 and 3 parameters, respectively (AUC = 0.79 [0.69-0.89], P < 0.0001). These results were confirmed by analyzing the 139 later sessions included in the study.

CONCLUSIONS

The SOCRATE score based on 3 easy-to-use bedside parameters correlates with the risk of IHI. By improving risk stratification of IHI, this score could help clinicians to manage intermittent hemodialysis initiation in critically ill AKI patients.

Predicting the likelihood of recurrence of pregnancy-associated breast cancer: Nomogram based on analysis of the French cancer network: Cancer Associé à La Grossesse

OBJECTIVE

Pregnancy associated breast cancer (PABC) are defined as breast cancer diagnosed during pregnancy and during the year following delivery. The prediction of poor prognosis events (PPE) such as recurrence is a major medical challenge of management for women with PABC. The aim of this study was to build a nomogram based on selected clinical and histological variables to predict recurrence.

STUDY DESIGN

This retrospective study included 96 patients with PABC from January 2002 to January 2018. A multivariate Cox analysis of selected risk factors was performed and a nomogram to predict recurrence was built. The nomogram was internally validated.

RESULTS

The overall recurrence rate was 22% (21/95) and the 3-years recurrence rate was 13% (12/95). Age at diagnosis, histological type, immuno-histological class, tumor stage (TNM), node stage (TNM) were associated with PPE in univariate analysis, and were included in the final Cox model to develop the nomogram. The predictive model had a concordance index of 0.83 (95% Confidence Interval (CI), 0.81-0.85) and 0.78 (95% CI, 0.76-0.80) before and after the 200 repetitions of bootstrap sample corrections, respectively, and showed a good calibration.

CONCLUSION

Our results support the use of the present nomogram based on 5 clinical and pathological characteristics to predict PPE in PABC with a high concordance. External validation is required to recommend this nomogram in routine practice.

Evaluation of a five-year predicted survival model for cystic fibrosis in later time periods

We evaluated a multivariable logistic regression model predicting 5-year survival derived from a 1993-1997 cohort from the United States Cystic Fibrosis (CF) Foundation Patient Registry to assess whether therapies introduced since 1993 have altered applicability in cohorts, non-overlapping in time, from 1993-1998, 1999-2004, 2005-2010 and 2011-2016. We applied Kaplan-Meier statistics to assess unadjusted survival. We tested logistic regression model discrimination using the C-index and calibration using Hosmer-Lemeshow tests to examine original model performance and guide updating as needed. Kaplan-Meier age-adjusted 5-year probability of death in the CF population decreased substantially during 1993-2016. Patients in successive cohorts were generally healthier at entry, with higher average age, weight and lung function and fewer pulmonary exacerbations annually. CF-related diabetes prevalence, however, steadily increased. Newly derived multivariable logistic regression models for 5-year survival in new cohorts had similar estimated coefficients to the originals. The original model exhibited excellent calibration and discrimination when applied to later cohorts despite improved survival and remains useful for predicting 5-year survival. All models may be used to stratify patients for new studies, and the original coefficients may be useful as a baseline to search for additional but rare events that affect survival in CF.

Predicting deer–vehicle collision risk across Victoria, Australia

The risk of deer–vehicle collisions (DVCs) is increasing in south-east Australia as populations of introduced deer expand rapidly. There are no investigations of the spatial and temporal patterns of DVC or predictions of where such collisions are most likely to occur. Here, we use an analytical framework to model deer distribution and vehicle movements in order to predict DVC risk across the State of Victoria. We modelled the occurrence of deer using existing occurrence records and geographic climatic variables. We estimated patterns of vehicular movements from records of average annual daily traffic and speeds. Given the low number of DVCs reported in Victoria, we used a generalised linear regression model fitted to DVCs in California, USA. The fitted model coefficients suggested high collision risk on road segments with high predicted deer occurrence, moderate traffic volume and high traffic speed. We used the California deer model to predict collision risk on Victorian roads and validated the predictions with two independent datasets of DVC records from Victoria. The California deer model performed well when comparing predictions of collision risk to the independent DVC datasets and generated plausible DVC risk predictions across the State of Victoria.

The Role of the Gut Microbiome in Predicting Response to Diet and the Development of Precision Nutrition Models-Part I: Overview of Current Methods

Health care is increasingly focused on health at the individual level. In the rapidly evolving field of precision nutrition, researchers aim to identify how genetics, epigenetics, and the microbiome interact to shape an individual's response to diet. With this understanding, personalized responses can be predicted and dietary advice can be tailored to the individual. With the integration of these complex sources of data, an important aspect of precision nutrition research is the methodology used for studying interindividual variability in response to diet. This article stands as the first in a 2-part review of current research investigating the contribution of the gut microbiota to interindividual variability in response to diet. Part I reviews the methods used by researchers to design and carry out such studies as well as the statistical and bioinformatic methods used to analyze results. Part II reviews the findings of these studies, discusses gaps in our current knowledge, and summarizes directions for future research. Taken together, these reviews summarize the current state of knowledge and provide a foundation for future research on the role of the gut microbiome in precision nutrition.

Validation of clinical prediction models: what does the "calibration slope" really measure?

BACKGROUND AND OBJECTIVES

Definitions of calibration, an aspect of model validation, have evolved over time. We examine use and interpretation of the statistic currently referred to as the calibration slope.

METHODS

The history of the term "calibration slope", and usage in papers published in 2016 and 2017, were reviewed. The behaviour of the slope in illustrative hypothetical examples and in two examples in the clinical literature was demonstrated.

RESULTS

The paper in which the statistic was proposed described it as a measure of "spread" and did not use the term "calibration". In illustrative examples, slope of 1 can be associated with good or bad calibration, and this holds true across different definitions of calibration. In data extracted from a previous study, the slope was correlated with discrimination, not overall calibration. Many authors of recent papers interpret the slope as a measure of calibration; a minority interpret it as a measure of discrimination or do not explicitly categorise it as either. Seventeen of thirty-three papers used the slope as the sole measure of calibration.

CONCLUSION

Misunderstanding about this statistic has led to many papers in which it is the sole measure of calibration, which should be discouraged.

Validation of the REVEAL Prognostic Equation and Risk Score Calculator in Incident Systemic Sclerosis-Associated Pulmonary Arterial Hypertension

OBJECTIVE

A prognostic equation and risk score calculator derived from the Registry to Evaluate Early and Long-term Pulmonary Arterial Hypertension Disease Management (REVEAL) are being used to predict 1-year survival in patients with pulmonary arterial hypertension (PAH), but little is known about the performance of these REVEAL survival prediction tools in systemic sclerosis (SSc)-associated PAH (SSc-PAH).

METHODS

Prospectively gathered data from the Johns Hopkins Pulmonary Hypertension Program and Pulmonary Hypertension Assessment and Recognition of Outcome in Scleroderma Registries were used to evaluate the predictive accuracy of the REVEAL models for predicting the probability of 1-year survival in patients with SSc-PAH. Model discrimination was assessed by comparison of the Harrell's C-index, model fit was assessed using multivariable regression techniques, and model calibration was assessed by comparing predicted to observed survival estimates.

RESULTS

The validation cohort consisted of 292 SSc-PAH patients with a 1-year survival rate of 87.4%. The C-index for predictive accuracy of the REVEAL prognostic equation (0.734, 95% confidence interval [95% CI] 0.652-0.816) and for the risk score (0.743, 95% CI 0.663-0.823) demonstrated good discrimination, comparable to that in the model development cohort. The calibration slope was 0.707 (95% CI 0.400-1.014), indicating that the overall model fit was marginal (P = 0.06). The magnitude of risk assigned to low distance on the 6-minute walk test (6MWD) and elevated serum levels of brain natriuretic peptide (BNP) was lower in the validation cohort than was originally seen in the REVEAL derivation cohort. Model calibration was poor, particularly for the highest risk groups.

CONCLUSION

In predicting 1-year survival in patients newly diagnosed as having SSc-PAH, the REVEAL prognostic equation and risk score provide very good discrimination but poor calibration. REVEAL prediction scores should be interpreted with caution in newly diagnosed SSc-PAH patients, particularly those with higher predicted risk of poor 1-year survival resulting from a low 6MWD or a high BNP serum level.

Assessment of heterogeneity in an individual participant data meta-analysis of prediction models: An overview and illustration

Clinical prediction models aim to provide estimates of absolute risk for a diagnostic or prognostic endpoint. Such models may be derived from data from various studies in the context of a meta‐analysis. We describe and propose approaches for assessing heterogeneity in predictor effects and predictions arising from models based on data from different sources. These methods are illustrated in a case study with patients suffering from traumatic brain injury, where we aim to predict 6‐month mortality based on individual patient data using meta‐analytic techniques (15 studies, n = 11 022 patients). The insights into various aspects of heterogeneity are important to develop better models and understand problems with the transportability of absolute risk predictions.

Association between Allogeneic Blood Transfusion and Wound Infection after Total Hip or Knee Arthroplasty: A Retrospective Case-Control Study

Background: To assess using a retrospective case control study, whether patients undergoing primary, elective total hip or knee arthroplasty who receive blood transfusion have a higher rate of post-operative infection compared to those who do not.

Materials and Methods: Data on elective primary total hip or knee arthroplasty patients, including patient characteristics, co-morbidities, type and duration of surgery, blood transfusion, deep and superficial infection was extracted from the Alberta Bone and Joint Health Institute (ABJHI). Logistic regression analysis was used to compare deep infection and superficial infection in blood-transfused and non-transfused cohorts.

Results: Of the 27892 patients identified, 3098 (11.1%) received blood transfusion (TKA 9.7%; THA 13.1%). Overall, the rate of superficial infection (SI) was 0.5% and deep infection (DI) was 1.1%. The infection rates in the transfused cohort were SI 1.0% and DI 1.6%, and in the non-transfused cohort were SI 0.5% and DI 1.0%. The transfused cohort had an increased risk of superficial infection (adjusted odds ratio (OR) 1.9 [95% CI 1.2-2.9, p-value 0.005]) as well as deep infection (adjusted OR 1.6 [95% CI 1.1-2.2, p-value 0.008]).

Conclusion: The odds of superficial and deep wound infection are significantly increased in primary, elective total hip and knee arthroplasty patients who receive blood transfusion compared to those who did not. This study can potentially help in reducing periprosthetic hip or knee infections.

Reliability of reported peri-ictal behavior to identify psychogenic nonepileptic seizures

PURPOSE

Differentiating psychogenic non-epileptic seizures (PNES) from epileptic seizures (ES) can be difficult, even when expert clinicians have video recordings of seizures. Moreover, witnesses who are not trained observers may provide descriptions that differ from the expert clinicians', which often raises concern about whether the patient has both ES and PNES. As such, quantitative, evidence-based tools to help differentiate ES from PNES based on patients' and witnesses' descriptions of seizures may assist in the early, accurate diagnosis of patients.

METHODS

Based on patient- and observer-reported data from 1372 patients with diagnoses documented by video-elect roencephalography (vEEG), we used logistic regression (LR) to compare specific peri-ictal behaviors and seizure triggers in five mutually exclusive groups: ES, PNES, physiologic non-epileptic seizure-like events, mixed PNES plus ES, and inconclusive monitoring. To differentiate PNES-only from ES-only, we retrospectively trained multivariate LR and a forest of decision trees (DF) to predict the documented diagnoses of 246 prospective patients.

RESULTS

The areas under the receiver operating characteristic curve (AUCs) of the DF and LR were 75% and 74%, respectively (empiric 95% CI of chance 37-62%). The overall accuracy was not significantly higher than the naïve assumption that all patients have ES (accuracy DF 71%, LR 70%, naïve 68%, p > 0.05).

CONCLUSIONS

Quantitative analysis of patient- and observer-reported peri-ictal behaviors objectively changed the likelihood that a patient's seizures were psychogenic, but these reports were not reliable enough to be diagnostic in isolation. Instead, our scores may identify patients with "probable" PNES that, in the right clinical context, may warrant further diagnostic assessment.

Sample size considerations and predictive performance of multinomial logistic prediction models

Multinomial Logistic Regression (MLR) has been advocated for developing clinical prediction models that distinguish between three or more unordered outcomes. We present a full‐factorial simulation study to examine the predictive performance of MLR models in relation to the relative size of outcome categories, number of predictors and the number of events per variable. It is shown that MLR estimated by Maximum Likelihood yields overfitted prediction models in small to medium sized data. In most cases, the calibration and overall predictive performance of the multinomial prediction model is improved by using penalized MLR. Our simulation study also highlights the importance of events per variable in the multinomial context as well as the total sample size. As expected, our study demonstrates the need for optimism correction of the predictive performance measures when developing the multinomial logistic prediction model. We recommend the use of penalized MLR when prediction models are developed in small data sets or in medium sized data sets with a small total sample size (ie, when the sizes of the outcome categories are balanced). Finally, we present a case study in which we illustrate the development and validation of penalized and unpenalized multinomial prediction models for predicting malignancy of ovarian cancer.

Importance of gene expression signatures in pancreatic cancer prognosis and the establishment of a prediction model

Background and aim

Pancreatic cancer (PC) is one of the most common tumors with a poor prognosis. The current American Joint Committee on Cancer (AJCC) staging system, based on the anatomical features of tumors, is insufficient to predict PC outcomes. The current study is endeavored to identify important prognosis-related genes and build an effective predictive model.

Methods

Multiple public datasets were used to identify differentially expressed genes (DEGs) and survival-related genes (SRGs). Bioinformatics analysis of DEGs was used to identify the main biological processes and pathways involved in PC. A risk score based on SRGs was computed through a univariate Cox regression analysis. The performance of the risk score in predicting PC prognosis was evaluated with survival analysis, Harrell's concordance index (C-index), area under the curve (AUC), and calibration plots. A predictive nomogram was built through integrating the risk score with clinicopathological information.

Results

A total of 945 DEGs were identified in five Gene Expression Omnibus datasets, and four SRGs (LYRM1, KNTC1, IGF2BP2, and CDC6) were significantly associated with PC progression and prognosis in four datasets. The risk score showed relatively good performance in predicting prognosis in multiple datasets. The predictive nomogram had greater C-index and AUC values, compared with those of the AJCC stage and risk score.

Conclusion

This study identified four new biomarkers that are significantly associated with the carcinogenesis, progression, and prognosis of PC, which may be helpful in studying the underlying mechanism of PC carcinogenesis. The predictive nomogram showed robust performance in predicting PC prognosis. Therefore, the current model may provide an effective and reliable guide for prognosis assessment and treatment decision-making in the clinic.

Development of a Breast Cancer Risk Prediction Model for Women in Nigeria

Background: Risk prediction models have been widely used to identify women at higher risk of breast cancer. We aimed to develop a model for absolute breast cancer risk prediction for Nigerian women.Methods: A total of 1,811 breast cancer cases and 2,225 controls from the Nigerian Breast Cancer Study (NBCS, 1998-2015) were included. Subjects were randomly divided into the training and validation sets. Incorporating local incidence rates, multivariable logistic regressions were used to develop the model.Results: The NBCS model included age, age at menarche, parity, duration of breastfeeding, family history of breast cancer, height, body mass index, benign breast diseases, and alcohol consumption. The model developed in the training set performed well in the validation set. The discriminating accuracy of the NBCS model [area under ROC curve (AUC) = 0.703, 95% confidence interval (CI), 0.687-0.719] was better than the Black Women's Health Study (BWHS) model (AUC = 0.605; 95% CI, 0.586-0.624), Gail model for white population (AUC = 0.551; 95% CI, 0.531-0.571), and Gail model for black population (AUC = 0.545; 95% CI, 0.525-0.565). Compared with the BWHS and two Gail models, the net reclassification improvement of the NBCS model were 8.26%, 13.45%, and 14.19%, respectively.Conclusions: We have developed a breast cancer risk prediction model specific to women in Nigeria, which provides a promising and indispensable tool to identify women in need of breast cancer early detection in Sub-Saharan Africa populations.Impact: Our model is the first breast cancer risk prediction model in Africa. It can be used to identify women at high risk for breast cancer screening. Cancer Epidemiol Biomarkers Prev; 27(6); 636-43. ©2018 AACR.

Detection of early pancreatic ductal adenocarcinoma with thrombospondin-2 and CA19-9 blood markers

Markers are needed to facilitate early detection of pancreatic ductal adenocarcinoma (PDAC), which is often diagnosed too late for effective therapy. Starting with a PDAC cell reprogramming model that recapitulated the progression of human PDAC, we identified secreted proteins and tested a subset as potential markers of PDAC. We optimized an enzyme-linked immunosorbent assay (ELISA) using plasma samples from patients with various stages of PDAC, from individuals with benign pancreatic disease, and from healthy controls. A phase 1 discovery study (n = 20), a phase 2a validation study (n = 189), and a second phase 2b validation study (n = 537) revealed that concentrations of plasma thrombospondin-2 (THBS2) discriminated among all stages of PDAC consistently. The receiver operating characteristic (ROC) c-statistic was 0.76 in the phase 1 study, 0.84 in the phase 2a study, and 0.87 in the phase 2b study. The plasma concentration of THBS2 was able to discriminate resectable stage I cancer as readily as stage III/IV PDAC tumors. THBS2 plasma concentrations combined with those for CA19-9, a previously identified PDAC marker, yielded a c-statistic of 0.96 in the phase 2a study and 0.97 in the phase 2b study. THBS2 data improved the ability of CA19-9 to distinguish PDAC from pancreatitis. With a specificity of 98%, the combination of THBS2 and CA19-9 yielded a sensitivity of 87% for PDAC in the phase 2b study. A THBS2 and CA19-9 blood marker panel measured with a conventional ELISA may improve the detection of patients at high risk for PDAC.

Predictive Model for Neoplastic Potential of Gallbladder Polyp

GOAL

To provide the statistical predictive model for neoplastic potential of gallbladder polyp (GBP).

BACKGROUND

Many studies have attempted to define the risk factors for neoplastic potential of GBP. It remains difficult to precisely adapt the reported risk factors for the decision of surgery. Estimating the probability for neoplastic potential of GBP using a combination of several risk factors before surgical resection would be useful in patient consultation.

STUDY

We collected data of patients confirmed as GBP through cholecystectomy at Samsung Medical Center between January 1997 and March 2015. Those with a definite evidence for malignancy, such as adjacent organ invasion, metastasis on preoperative imaging studies, polyp >15 mm, and absence of proper preoperative ultrasonographic imaging were excluded. A total of 1976 patients were enrolled. To make and validate the predictive model, we divided the cohort into the modeling group (n=979) and validation group (n=997). Clinical information, ultrasonographic findings, and blood tests were retrospectively analyzed.

RESULTS

Clinical factors of older age, single lesion, sessile shape, and polyp size showed statistical significance for neoplastic potential of GBP in the modeling group. A predictive model for neoplastic potential of GBP was constructed utilizing the statistical outcome of the modeling group. Statistical validation was performed with the validation group to determine the optimal clinical sensitivity and specificity of the predictive model. Optimal cut-off value for neoplastic probability was 7.4%.

CONCLUSIONS

The predictive model for neoplastic potential of GBP may support clinical decisions before cholecystectomy.

An objective score to identify psychogenic seizures based on age of onset and history

OBJECTIVE

Psychogenic nonepileptic seizure (PNES) is a common diagnosis after evaluation of medication resistant or atypical seizures with video-electroencephalographic monitoring (VEM), but usually follows a long delay after the development of seizures, during which patients are treated for epilepsy. Therefore, more readily available diagnostic tools are needed for earlier identification of patients at risk for PNES. A tool based on patient-reported psychosocial history would be especially beneficial because it could be implemented in the outpatient clinic.

METHODS

Based on the data from 1375 patients with VEM-confirmed diagnoses, we used logistic regression to compare the frequency of specific patient-reported historical events, demographic information, age of onset, and delay from first seizure until VEM in five mutually exclusive groups of patients: epileptic seizures (ES), PNES, physiologic nonepileptic seizure-like events (PSLE), mixed PNES plus ES, and inconclusive monitoring. To determine the diagnostic utility of this information to differentiate PNES only from ES only, we used multivariate piecewise-linear logistic regression trained using retrospective data from chart review and validated based on data from 246 prospective standardized interviews.

RESULTS

The prospective area under the curve of our weighted multivariate piecewise-linear by-sex score was 73%, with the threshold that maximized overall retrospective accuracy resulting in a prospective sensitivity of 74% (95% CI: 70-79%) and prospective specificity of 71% (95% CI: 64-82%). The linear model and piecewise linear without an interaction term for sex had very similar performance statistics. In the multivariate piecewise-linear sex-split predictive model, the significant factors positively associated with ES were history of febrile seizures, current employment or active student status, history of traumatic brain injury (TBI), and longer delay from first seizure until VEM. The significant factors associated with PNES were female sex, older age of onset, mild TBI, and significant stressful events with sexual abuse, in particular, increasing the likelihood of PNES. Delays longer than 20years, age of onset after 31years for men, and age of onset after 40years for women had no additional effect on the likelihood of PNES.

DISCUSSION

Our promising results suggest that an objective score has the potential to serve as an early outpatient screening tool to identify patients with greater likelihood of PNES when considered in combination with other factors. In addition, our analysis suggests that sexual abuse, more than other psychological stressors including physical abuse, is more associated with PNES. There was a trend of increasing frequency of PNES for women during childbearing years and plateauing outside those years that was not observed in men.

Incidence and predictors of 30-day postoperative readmission in children

BACKGROUND

Hospital readmissions are being used as a quality metric for hospital reimbursement without a clear understanding of the factors that contribute to readmission.

OBJECTIVE

The objective of this study was to report the incidence of 30-day postsurgical readmission in children, identify the predictors for readmission, and create an algorithm to identify high-risk children.

METHODS

Data from the 2012-2014 Pediatric database of the American College of Surgeons National Surgical Quality Improvement Program were analyzed using univariable and multivariable logistical regression analysis.

RESULTS

Among 182 589 children included in the 2012-2014 American College of Surgeons National Surgical Quality Improvement Program Pediatric database, 4.8% (8815/182 589) experienced a readmission within 30 days. Four significant predictors were retained in the multivariable logistic regression model: American Society of Anesthesiologists physical status ≥ 3 (OR: 1.9, 95% CI: 1.8-2.0), presence of congenital heart disease (OR: 1.66, 95% CI: 1.31-2.11), inpatient status at time of surgery (OR: 3.5, 95% CI: 3.3-3.7), and at least 1 postoperative complication (neurologic, renal, wound, cardiac, bleeding, or pulmonary) (OR: 3.14, 95% CI: 2.92-3.34). The multivariable logistic regression model showed reasonably good discrimination in predicting 30-day readmissions with receiver operating characteristic area under the curve of 0.747 (95% CI: 0.73-0.75) and good calibration (Brier score: 0.044). We created a predictive algorithm of 30-day readmission based on the 4 significant predictors.

CONCLUSION

Children with congenital heart disease, high American Society of Anesthesiologist physical class, inpatient status, and at least 1 postoperative complication of any kind are at high risk for postsurgical readmissions. We provide an algorithm for quantifying this risk with the goal of reducing the number of readmissions, improving the care of patients with complex chronic illnesses, and reducing hospital costs.

Incidence and predictive score for delayed hemolytic transfusion reaction in adult patients with sickle cell disease

Delayed hemolytic transfusion reaction (DHTR) is a life-threatening complication of transfusion in sickle cell disease (SCD). The frequency of DHTR is underestimated because its symptoms mimic those of vaso-occlusive crisis and antibodies (Abs) are often not detectable. No predictive factors for identifying patients likely to develop DHTR have yet been defined. We conducted a prospective single-center observational study over 30 months in adult sickle cell patients. We included 694 transfusion episodes (TEs) in 311 patients, divided into occasional TEs (OTEs: 360) and chronic transfusion program (CTEs: 334). During follow-up, 15 cases of DHTR were recorded, exclusively after OTEs. DHTR incidence was 4.2% per OTE (95% CI [2.6; 6.9]) and 6.8% per patient during the 30 months of the study (95% CI [4.2; 11.3]). We studied 11 additional DHTR cases, to construct a predictive score for DHTR. The DHTR mortality is high, 3 (11.5%) of the 26 DHTR patients died. The variables retained in the multivariate model were history of DHTR, number of units previously transfused and immunization status before transfusion. The resulting DHTR-predictive score had an area under the ROC curve of 0.850 [95% CI: 0.780-0.930], a negative-predictive value of 98.4% and a positive-predictive value of 50%. We report in our study population, for the first time, the incidence of DHTR, and, its occurrence exclusively in occasionally transfused patients. We also describe a simple score for predicting DHTR in patients undergoing occasional transfusion, to facilitate the management of blood transfusion in SCD patients.

Regulatory activity based risk model identifies survival of stage II and III colorectal carcinoma

Clinical and pathological indicators are inadequate for prognosis of stage II and III colorectal carcinoma (CRC). In this study, we utilized the activity of regulatory factors, univariate Cox regression and random forest for variable selection and developed a multivariate Cox model to predict the overall survival of Stage II/III colorectal carcinoma in GSE39582 datasets (469 samples). Patients in low-risk group showed a significant longer overall survival and recurrence-free survival time than those in high-risk group. This finding was further validated in five other independent datasets (GSE14333, GSE17536, GSE17537, GSE33113, and GSE37892). Besides, associations between clinicopathological information and risk score were analyzed. A nomogram including risk score was plotted to facilitate the utilization of risk score. The risk score model is also demonstrated to be effective on predicting both overall and recurrence-free survival of chemotherapy received patients. After performing Gene Set Enrichment Analysis (GSEA) between high and low risk groups, we found that several cell-cell interaction KEGG pathways were identified. Funnel plot results showed that there was no publication bias in these datasets. In summary, by utilizing the regulatory activity in stage II and III colorectal carcinoma, the risk score successfully predicts the survival of 1021 stage II/III CRC patients in six independent datasets.

Identifying psychogenic seizures through comorbidities and medication history

OBJECTIVE

Low-cost evidence-based tools are needed to facilitate the early identification of patients with possible psychogenic nonepileptic seizures (PNES). Prior to accurate diagnosis, patients with PNES do not receive interventions that address the cause of their seizures and therefore incur high medical costs and disability due to an uncontrolled seizure disorder. Both seizures and comorbidities may contribute to this high cost.

METHODS

Based on data from 1,365 adult patients with video-electroencephalography-confirmed diagnoses from a single center, we used logistic and Poisson regression to compare the total number of comorbidities, number of medications, and presence of specific comorbidities in five mutually exclusive groups of diagnoses: epileptic seizures (ES) only, PNES only, mixed PNES and ES, physiologic nonepileptic seizurelike events, and inconclusive monitoring. To determine the diagnostic utility of comorbid diagnoses and medication history to differentiate PNES only from ES only, we used multivariate logistic regression, controlling for sex and age, trained using a retrospective database and validated using a prospective database.

RESULTS

Our model differentiated PNES only from ES only with a prospective accuracy of 78% (95% confidence interval =72-84%) and area under the curve of 79%. With a few exceptions, the number of comorbidities and medications was more predictive than a specific comorbidity. Comorbidities associated with PNES were asthma, chronic pain, and migraines (p < 0.01). Comorbidities associated with ES were diabetes mellitus and nonmetastatic neoplasm (p < 0.01). The population-level analysis suggested that patients with mixed PNES and ES may be a population distinct from patients with either condition alone.

SIGNIFICANCE

An accurate patient-reported medical history and medication history can be useful when screening for possible PNES. Our prospectively validated and objective score may assist in the interpretation of the medication and medical history in the context of the seizure description and history.

Sediment quality benchmarks for assessing oil-related impacts to the deep-sea benthos

Paired sediment contaminant and benthic infaunal data from prior studies following the 2010 Deepwater Horizon (DWH) oil spill in the Gulf of Mexico were analyzed using logistic regression models (LRMs) to derive sediment quality benchmarks for assessing risks of oil-related impacts to the deep-sea benthos. Sediment total polycyclic aromatic hydrocarbon (PAH) and total petroleum hydrocarbon (TPH) concentrations were used as measures of oil exposure. Taxonomic richness (average number of taxa/sample) was selected as the primary benthic response variable. Data are from 37 stations (1300-1700 m water depth) in fine-grained sediments (92%-99% silt-clay) sampled within 200 km of the DWH wellhead (most within 40 km) in 2010 and 32 stations sampled in 2011 (29 of which were common to both years). Results suggest the likelihood of impacts to benthic macrofauna and meiofauna communities is low (<20%) at TPH concentrations of less than 606 mg kg^-1 (ppm dry weight) and 700 mg kg^-1 respectively, high (>80%) at concentrations greater than 2144 mg kg^-1 and 2359 mg kg^-1 respectively, and intermediate at concentrations in between. For total PAHs, the probability of impacts is low (<20%) at concentrations of less than 4.0 mg kg^-1 (ppm) for both macrofauna and meiofauna, high (>80%) at concentrations greater than 24 mg kg^-1 and 25 mg kg^-1 for macrofauna and meiofauna, respectively, and intermediate at concentrations in between. Although numerical sediment quality guidelines (SQGs) are available for total PAHs and other chemical contaminants based on bioeffect data for shallower estuarine, marine, and freshwater biota, to our knowledge, none have been developed for measures of total oil (e.g., TPH) or specifically for deep-sea benthic applications. The benchmarks presented herein provide valuable screening tools for evaluating the biological significance of observed oil concentrations in similar deep-sea sediments following future spills and as potential restoration targets to aid in managing recovery. Integr Environ Assess Manag 2017;13:840-851. Published 2017. This article is a US Government work and is in the public domain in the USA.

External validation of the Donor Risk Index and the Eurotransplant Donor Risk Index on the French liver transplantation registry

BACKGROUND & AIMS

A major limitation to liver transplantation is organ shortage leading to the use of non-optimal liver grafts. The Donor Risk Index has been validated and recommended to select donors/organs. The Eurotransplant Donor Risk Index was derived from the Donor Risk Index. The objective of our study was to perform an external validation of both Donor Risk Index and Eurotransplant-Donor Risk Index against the French liver transplantation Cristal registry according to recommendations of the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis.

METHODS

Liver transplantations performed in France between 2009 and 2013 were used to perform the validation study for the Donor Risk Index and the Eurotransplant-Donor Risk Index respectively. We applied on the French data the models used to construct the Donor Risk Index and the Eurotransplant-Donor Risk Index respectively.

RESULTS

Neither the Donor Risk Index nor the Eurotransplant-Donor Risk Index were validated against this dataset. Discrimination and calibration of these scores were not preserved according to our data. Important donor and candidates differences between our dataset and the Organ Procurement and Transplantation Network or the Eurotransplant datasets may explain why the Donor Risk Index and the Eurotransplant-Donor Risk Index appeared unadapted to the French transplant registry.

CONCLUSION

Neither of these risk indexes were suitable to optimize the French liver allocation system. Thus, our next step will be to propose a general adaptive model for a Donor Risk Index.

Using Transfer Learning for Improved Mortality Prediction in a Data-Scarce Hospital Setting

Algorithm–based clinical decision support (CDS) systems associate patient-derived health data with outcomes of interest, such as in-hospital mortality. However, the quality of such associations often depends on the availability of site-specific training data. Without sufficient quantities of data, the underlying statistical apparatus cannot differentiate useful patterns from noise and, as a result, may underperform. This initial training data burden limits the widespread, out-of-the-box, use of machine learning–based risk scoring systems. In this study, we implement a statistical transfer learning technique, which uses a large “source” data set to drastically reduce the amount of data needed to perform well on a “target” site for which training data are scarce. We test this transfer technique with AutoTriage, a mortality prediction algorithm, on patient charts from the Beth Israel Deaconess Medical Center (the source) and a population of 48 249 adult inpatients from University of California San Francisco Medical Center (the target institution). We find that the amount of training data required to surpass 0.80 area under the receiver operating characteristic (AUROC) on the target set decreases from more than 4000 patients to fewer than 220. This performance is superior to the Modified Early Warning Score (AUROC: 0.76) and corresponds to a decrease in clinical data collection time from approximately 6 months to less than 10 days. Our results highlight the usefulness of transfer learning in the specialization of CDS systems to new hospital sites, without requiring expensive and time-consuming data collection efforts.