Improving Clinical Translation of Machine Learning Approaches Through Clinician-Tailored Visual Displays of Black Box Algorithms: Development and Validation

BACKGROUND

Despite the promise of machine learning (ML) to inform individualized medical care, the clinical utility of ML in medicine has been limited by the minimal interpretability and black box nature of these algorithms.

OBJECTIVE

The study aimed to demonstrate a general and simple framework for generating clinically relevant and interpretable visualizations of black box predictions to aid in the clinical translation of ML.

METHODS

To obtain improved transparency of ML, simplified models and visual displays can be generated using common methods from clinical practice such as decision trees and effect plots. We illustrated the approach based on postprocessing of ML predictions, in this case random forest predictions, and applied the method to data from the Left Ventricular (LV) Structural Predictors of Sudden Cardiac Death (SCD) Registry for individualized risk prediction of SCD, a leading cause of death.

RESULTS

With the LV Structural Predictors of SCD Registry data, SCD risk predictions are obtained from a random forest algorithm that identifies the most important predictors, nonlinearities, and interactions among a large number of variables while naturally accounting for missing data. The black box predictions are postprocessed using classification and regression trees into a clinically relevant and interpretable visualization. The method also quantifies the relative importance of an individual or a combination of predictors. Several risk factors (heart failure hospitalization, cardiac magnetic resonance imaging indices, and serum concentration of systemic inflammation) can be clearly visualized as branch points of a decision tree to discriminate between low-, intermediate-, and high-risk patients.

CONCLUSIONS

Through a clinically important example, we illustrate a general and simple approach to increase the clinical translation of ML through clinician-tailored visual displays of results from black box algorithms. We illustrate this general model-agnostic framework by applying it to SCD risk prediction. Although we illustrate the methods using SCD prediction with random forest, the methods presented are applicable more broadly to improving the clinical translation of ML, regardless of the specific ML algorithm or clinical application. As any trained predictive model can be summarized in this manner to a prespecified level of precision, we encourage the use of simplified visual displays as an adjunct to the complex predictive model. Overall, this framework can allow clinicians to peek inside the black box and develop a deeper understanding of the most important features from a model to gain trust in the predictions and confidence in applying them to clinical care.

The predictors of and reasons for non-adherence in an observational cohort of patients with rheumatoid arthritis commencing methotrexate

OBJECTIVE

In order to develop interventions to optimize MTX use for the treatment of RA we evaluated the rate of, reasons for and predictors of MTX non-adherence during the first 6 months of therapy.

METHODS

The Rheumatoid Arthritis Medication Study (RAMS) is a prospective multicentre cohort study of incident MTX users in the UK. Prior to MTX commencement demographic, clinical and psychological data were collected. A weekly patient-completed diary recorded MTX dose, possible side effects and adherence over 26 weeks. The number of non-adherent weeks was calculated. Potential baseline predictors of ever non-adherence (⩾1 week non-adherent) during the first 6 months of MTX therapy were identified using logistic regression analyses.

RESULTS

606 patients with RA were included; 69% female, mean (s.d.) age 60 (13) years and DAS28 score 4.2 (1.2). Over the first 6 months following MTX initiation, 158 (26%) patients were ever non-adherent (71% intentional, 19% non-intentional, 10% unexplained) and mean (s.d.) number of non-adherent weeks was 2.5 (2.1). Multivariable predictors of ever non-adherence included DAS28 [odds ratios (OR) 1.1, 95% CI 1.0, 1.4], fatigue (OR 1.1, 95% CI 1.0, 1.2 per cm), ⩾2 comorbidities vs no comorbidities (OR 1.9, 95% CI 1.1, 3.5) and high medication concerns despite perceived need (OR 1.1, 95% CI 1.0, 1.1 per unit decrease in need/concern differential).

CONCLUSION

This is the largest study evaluating early intentional and non-intentional non-adherence to MTX, which has identified that patient beliefs and multi-morbidity strongly link with non-adherence. These findings can direct the design of and provide potential targets for interventions to improve patient adherence.

European cancer mortality predictions for the year 2020 with a focus on prostate cancer

BACKGROUND

Current cancer mortality figures are important for disease management and resource allocation. We estimated mortality counts and rates for 2020 in the European Union (EU) and for its six most populous countries.

MATERIALS AND METHODS

We obtained cancer death certification and population data from the World Health Organization and Eurostat databases for 1970-2015. We estimated projections to 2020 for 10 major cancer sites plus all neoplasms and calculated the number of avoided deaths over 1989-2020.

RESULTS

Total cancer mortality rates in the EU are predicted to decline reaching 130.1/100 000 men (-5.4% since 2015) and 82.2 in women (-4.1%) in 2020. The predicted number of deaths will increase by 4.7% reaching 1 428 800 in 2020. In women, the upward lung cancer trend is predicted to continue with a rate in 2020 of 15.1/100 000 (higher than that for breast cancer, 13.5) while in men we predicted further falls. Pancreatic cancer rates are also increasing in women (+1.2%) but decreasing in men (-1.9%). In the EU, the prostate cancer predicted rate is 10.0/100 000, declining by 7.1% since 2015; decreases for this neoplasm are ∼8% at age 45-64, 14% at 65-74 and 75-84, and 6% at 85 and over. Poland is the only country with an increasing prostate cancer trend (+18%). Mortality rates for other cancers are predicted to decline further. Over 1989-2020, we estimated over 5 million avoided total cancer deaths and over 400 000 for prostate cancer.

CONCLUSION

Cancer mortality predictions for 2020 in the EU are favourable with a greater decline in men. The number of deaths continue to rise due to population ageing. Due to the persistent amount of predicted lung (and other tobacco-related) cancer deaths, tobacco control remains a public health priority, especially for women. Favourable trends for prostate cancer are largely attributable to continuing therapeutic improvements along with early diagnosis.

Applications of Machine Learning Predictive Models in the Chronic Disease Diagnosis

This paper reviews applications of machine learning (ML) predictive models in the diagnosis of chronic diseases. Chronic diseases (CDs) are responsible for a major portion of global health costs. Patients who suffer from these diseases need lifelong treatment. Nowadays, predictive models are frequently applied in the diagnosis and forecasting of these diseases. In this study, we reviewed the state-of-the-art approaches that encompass ML models in the primary diagnosis of CD. This analysis covers 453 papers published between 2015 and 2019, and our document search was conducted from PubMed (Medline), and Cumulative Index to Nursing and Allied Health Literature (CINAHL) libraries. Ultimately, 22 studies were selected to present all modeling methods in a precise way that explains CD diagnosis and usage models of individual pathologies with associated strengths and limitations. Our outcomes suggest that there are no standard methods to determine the best approach in real-time clinical practice since each method has its advantages and disadvantages. Among the methods considered, support vector machines (SVM), logistic regression (LR), clustering were the most commonly used. These models are highly applicable in classification, and diagnosis of CD and are expected to become more important in medical practice in the near future.

Risk factors and a 3-month risk score for predicting pacemaker implantation in patients with atrial fibrillations

Objectives

To identify risk factors and to develop a predictive risk score for pacemaker implantation in patients with atrial fibrillation (AF).

Methods

Using Danish nationwide registries, patients with newly diagnosed AF from 2000 to 2014 were identified. Cox proportional-hazards regression computed HRs for risk factors of pacemaker implantation. A logistic regression was used to fit a prediction model for 3-month risk of pacemaker implantation and derived a risk score using 80% of the data and its predictive accuracy estimated using the remaining 20%.

Results

Among 155 934 AF patients included, the median age (IQR) was 75 (65–83) and 51.3% were men. During a median follow-up time of 3.4 (1.2–5.0) years, 8348 (5.4%) patients received a pacemaker implantation. Risk factors of pacemaker implantation were (in order of highest risk first) age above 60 years, congenital heart disease, heart failure at age under 60 years, prior syncope, valvular AF, hypertension, ischaemic heart disease, male sex and diabetes mellitus. The derived risk score assigns points ranging from 1 to 14 to each of these risk factors. The 3-month risk of pacemaker implantation increased from 0.4% (95% CI: 0.2 to 0.8) at 1 point to 2.6% (95% CI: 1.9 to 3.6) at 18 points. Area under the receiver operator characteristics curve was 62.9 (95% CI: 60.3 to 65.5).

Conclusion

We highlighted risk factors of pacemaker implantation in newly diagnosed AF patients and created a risk score. The clinical utility of the risk score needs further investigation.

Prediction modeling-part 1: regression modeling

Risk prediction models are statistical models that estimate the probability of individuals having a certain disease or clinical outcome based on a range of characteristics, and they can be used in clinical practice to stratify disease severity and characterize the risk of disease or disease prognosis. With technological advancements and the proliferation of clinical and biological data, prediction models are increasingly being developed in many areas of nephrology practice. This article guides the reader through the process of creating a prediction model, including (i) defining the clinical question and type of model, (ii) data collection and data cleaning, (iii) model building and variable selection, (iv) model performance, (v) model validation, (vi) model presentation and reporting, and (vii) impact evaluation. An example of developing a prediction model to predict mortality after intensive care unit admission for patients with end-stage kidney disease is also provided to illustrate the model development process.

An Introductory Review of Deep Learning for Prediction Models With Big Data

Deep learning models stand for a new learning paradigm in artificial intelligence (AI) and machine learning. Recent breakthrough results in image analysis and speech recognition have generated a massive interest in this field because also applications in many other domains providing big data seem possible. On a downside, the mathematical and computational methodology underlying deep learning models is very challenging, especially for interdisciplinary scientists. For this reason, we present in this paper an introductory review of deep learning approaches including Deep Feedforward Neural Networks (D-FFNN), Convolutional Neural Networks (CNNs), Deep Belief Networks (DBNs), Autoencoders (AEs), and Long Short-Term Memory (LSTM) networks. These models form the major core architectures of deep learning models currently used and should belong in any data scientist's toolbox. Importantly, those core architectural building blocks can be composed flexibly-in an almost Lego-like manner-to build new application-specific network architectures. Hence, a basic understanding of these network architectures is important to be prepared for future developments in AI.

SpineCloud: image analytics for predictive modeling of spine surgery outcomes

Purpose: Data-intensive modeling could provide insight on the broad variability in outcomes in spine surgery. Previous studies were limited to analysis of demographic and clinical characteristics. We report an analytic framework called "SpineCloud" that incorporates quantitative features extracted from perioperative images to predict spine surgery outcome. Approach: A retrospective study was conducted in which patient demographics, imaging, and outcome data were collected. Image features were automatically computed from perioperative CT. Postoperative 3- and 12-month functional and pain outcomes were analyzed in terms of improvement relative to the preoperative state. A boosted decision tree classifier was trained to predict outcome using demographic and image features as predictor variables. Predictions were computed based on SpineCloud and conventional demographic models, and features associated with poor outcome were identified from weighting terms evident in the boosted tree. Results: Neither approach was predictive of 3- or 12-month outcomes based on preoperative data alone in the current, preliminary study. However, SpineCloud predictions incorporating image features obtained during and immediately following surgery (i.e., intraoperative and immediate postoperative images) exhibited significant improvement in area under the receiver operating characteristic (AUC): AUC = 0.72 ( CI 95 = 0.59 to 0.83) at 3 months and AUC = 0.69 ( CI 95 = 0.55 to 0.82) at 12 months. Conclusions: Predictive modeling of lumbar spine surgery outcomes was improved by incorporation of image-based features compared to analysis based on conventional demographic data. The SpineCloud framework could improve understanding of factors underlying outcome variability and warrants further investigation and validation in a larger patient cohort.

Early Prediction of One-Year Mortality in Ischemic and Haemorrhagic Stroke

BACKGROUND

In Denmark 15%-20% of stroke victims die within the first year. Simple and valid tools are needed to assess patients' risk of dying. The aim of this study was to identify potential predictors of 1-year mortality in stroke victims and construct a simple and valid prediction model.

METHODS

Data were collected retrospectively from a cohort of 1031 stroke victims admitted over a period of 18 months at Nordsjællands Hospital, Denmark. Follow-up was 1 year after symptom onset. Multiple logistic regression analysis with backwards selection was used to identify predictors and construction of a prediction model. The model was validated using cross validation with 10,000 repeated random splits of the dataset. Area under the receiver operating characteristic curve (AUC) and Brier score were used as measures of validity.

RESULTS

Within the first year 186 patients died (18.0%) and 4 (0.4%) were lost to follow-up. Age (OR 1.08), gender (OR 2.19), stroke severity (OR 1.03), Early Warning Score (OR 1.17), Performance Status (ECOG) (OR 1.94), Body Mass Index (OR 0.91), the Charlton's Comorbidity Index (OR 1.17), and urinary problems (OR 2.55) were found to be independent predictors of 1-year mortality. A model including age, stroke severity, Early Warning Score, and Performance Status was found to be valid (AUC 86.5 %, Brier Score 9.03).

CONCLUSIONS

A model including only 4 clinical variables available shortly after admission was able to predict the 1-year mortality risk of patients with acute ischemic and haemorrhagic stroke.

Predicting SSRI-Resistance: Clinical Features and tagSNPs Prediction Models Based on Support Vector Machine

BACKGROUND

A large proportion of major depressive patients will experience recurring episodes. Many patients still do not response to available antidepressants. In order to meaningfully predict who will not respond to which antidepressant, it may be necessary to combine multiple biomarkers and clinical variables.

METHODS

Eight hundred fifty-seven patients with recurrent major depressive disorder who were followed up 3-10 years involved 32 variables including socio-demographic, clinical features, and SSRIs treatment features when they received the first treatment. Also, 34 tagSNPs related to 5-HT signaling pathway, were detected by using mass spectrometry analysis. The training samples which had 12 clinical variables and four tagSNPs with statistical differences were learned repeatedly to establish prediction models based on support vector machine (SVM).

RESULTS

Twelve clinical features (psychomotor retardation, psychotic symptoms, suicidality, weight loss, SSRIs average dose, first-course treatment response, sleep disturbance, residual symptoms, personality, onset age, frequency of episode, and duration) were found significantly difference (P< 0.05) between 302 SSRI-resistance and 304 SSRI non-resistance group. Ten SSRI-resistance predicting models were finally selected by using support vector machine, and our study found that mutations in tagSNPs increased the accuracy of these models to a certain degree.

CONCLUSION

Using a data-driven machine learning method, we found 10 predictive models by mining existing clinical data, which might enable prospective identification of patients who are likely to resistance to SSRIs antidepressant.

Methodological challenges in constructing DNA methylation risk scores

Polygenic approaches often access more variance of complex traits than is possible by single variant approaches. For genotype data, genetic risk scores (GRS) are widely used for risk prediction as well as in association and interaction studies. Recently, interest has been growing in transferring GRS approaches to DNA methylation data (methylation risk scores, MRS), which can be used 1) as biomarkers for environmental exposures, 2) in association analyses in which single CpG sites do not achieve significance, 3) as dimension reduction approach in interaction and mediation analyses, and 4) to predict individual risks of disease or treatment success. Most GRS approaches can directly be transferred to methylation data. However, since methylation data is more sensitive to confounding, e.g. by age and tissue, it is more complex to find appropriate external weights. In this review, we will outline the adaption of current GRS approaches to methylation data and highlight occurring challenges.

An Introductory Review of Deep Learning for Prediction Models With Big Data

Deep learning models stand for a new learning paradigm in artificial intelligence (AI) and machine learning. Recent breakthrough results in image analysis and speech recognition have generated a massive interest in this field because also applications in many other domains providing big data seem possible. On a downside, the mathematical and computational methodology underlying deep learning models is very challenging, especially for interdisciplinary scientists. For this reason, we present in this paper an introductory review of deep learning approaches including Deep Feedforward Neural Networks (D-FFNN), Convolutional Neural Networks (CNNs), Deep Belief Networks (DBNs), Autoencoders (AEs), and Long Short-Term Memory (LSTM) networks. These models form the major core architectures of deep learning models currently used and should belong in any data scientist's toolbox. Importantly, those core architectural building blocks can be composed flexibly-in an almost Lego-like manner-to build new application-specific network architectures. Hence, a basic understanding of these network architectures is important to be prepared for future developments in AI.

Kallikrein markers performance in pretreatment blood to predict early prostate cancer recurrence and metastasis after radical prostatectomy among very high-risk men

BACKGROUND

To assess whether a prespecified statistical model based on the four kallikrein markers measured in blood-total, free, and intact prostate-specific antigen (PSA), together with human kallikrein-related peptidase 2 (hK2)-or any individual marker measured in pretreatment serum were associated with biochemical recurrence-free (BCR) or metastasis-free survival after radical prostatectomy (RP) in a subgroup of men with very high-risk disease.

METHODS

We identified 106 men treated at Mayo Clinic from 2004 to 2008 with pathological Gleason grade group 4 to 5 or seminal vesicle invasion at RP. Univariable and multivariable Cox models were used to test the association between standard predictors (Kattan nomogram and GPSM [Gleason, PSA, seminal vesicle and margin status] score), kallikrein panel, and individual kallikrein markers with the outcomes.

RESULTS

BCR and metastasis occurred in 67 and 30 patients, respectively. The median follow-up for patients who did not develop a BCR was 10.3 years (interquartile range = 8.2-11.8). In this high-risk group, neither Kattan risk, GPSM score, or the kallikrein panel model was associated with either outcome. However, after adjusting for Kattan risk and GPSM score, separately, preoperative intact PSA was associated with both outcomes while hK2 was associated with metastasis-free survival.

CONCLUSIONS

Conventional risk prediction tools were poor discriminators for risk of adverse outcomes after RP (Kattan risk and GPSM risk) in patients with very high-risk disease. Further studies are needed to define the role of individual kallikrein marker forms in the blood to predict adverse prostate cancer outcomes after RP in this high-risk setting.

Combining clinical and molecular data in regression prediction models: insights from a simulation study

Data integration, i.e. the use of different sources of information for data analysis, is becoming one of the most important topics in modern statistics. Especially in, but not limited to, biomedical applications, a relevant issue is the combination of low-dimensional (e.g. clinical data) and high-dimensional (e.g. molecular data such as gene expressions) data sources in a prediction model. Not only the different characteristics of the data, but also the complex correlation structure within and between the two data sources, pose challenging issues. In this paper, we investigate these issues via simulations, providing some useful insight into strategies to combine low- and high-dimensional data in a regression prediction model. In particular, we focus on the effect of the correlation structure on the results, while accounting for the influence of our specific choices in the design of the simulation study.

Using machine learning approaches to predict high-cost chronic obstructive pulmonary disease patients in China

The accurate identification and prediction of high-cost Chronic obstructive pulmonary disease (COPD) patients is important for addressing the economic burden of COPD. The objectives of this study were to use machine learning approaches to identify and predict potential high-cost patients and explore the key variables of the forecasting model, by comparing differences in the predictive performance of different variable sets. Machine learning approaches were used to estimate the medical costs of COPD patients using the Medical Insurance Data of a large city in western China. The prediction models used were logistic regression, random forest (RF), and extreme gradient boosting (XGBoost). All three models had good predictive performance. The XGBoost model outperformed the others. The areas under the ROC curve for Logistic Regression, RF and XGBoost were 0.787, 0.792 and 0.801. The precision and accuracy metrics indicated that the methods achieved correct and reliable results. The results of this study can be used by healthcare data analysts, policy makers, insurers, and healthcare planners to improve the delivery of health services.

Co-registered photoacoustic and ultrasound imaging of human colorectal cancer

<p>Colorectal cancer is the second most common malignancy diagnosed globally. Critical gaps exist in diagnostic and surveillance imaging modalities for colorectal neoplasia. Although prior studies have demonstrated the capability of photoacoustic imaging techniques to differentiate normal from neoplastic tissue in the gastrointestinal tract, evaluation of deep tissue with a fast speed and a large field of view remains limited. To investigate the ability of photoacoustic technology to image deeper tissue, we conducted a pilot study using a real-time co-registered photoacoustic tomography (PAT) and ultrasound (US) system. A total of 23 <italic>ex vivo</italic> human colorectal tissue samples were imaged immediately after surgical resection. Co-registered photoacoustic images of malignancies showed significantly increased PAT signal compared to normal regions of the same sample. The quantitative relative total hemoglobin (rHbT) concentration computed from four optical wavelengths, the spectral features, such as the mean spectral slope, and 0.5-MHz intercept extracted from PAT and US spectral data, and image features, such as the first- and second-order statistics along with the standard deviation of the mean radon transform of PAT images, have shown statistical significance between untreated colorectal tumors and the normal tissue. Using either a logistic regression model or a support vector machine, the best set of parameters of rHbT and PAT intercept has achieved area-under-the-curve (AUC) values of 0.97 and 0.95 for both training and testing data sets, respectively, for prediction of histologically confirmed invasive carcinoma.</p>.

Co-registered photoacoustic and ultrasound imaging of human colorectal cancer

<p>Colorectal cancer is the second most common malignancy diagnosed globally. Critical gaps exist in diagnostic and surveillance imaging modalities for colorectal neoplasia. Although prior studies have demonstrated the capability of photoacoustic imaging techniques to differentiate normal from neoplastic tissue in the gastrointestinal tract, evaluation of deep tissue with a fast speed and a large field of view remains limited. To investigate the ability of photoacoustic technology to image deeper tissue, we conducted a pilot study using a real-time co-registered photoacoustic tomography (PAT) and ultrasound (US) system. A total of 23 <italic>ex vivo</italic> human colorectal tissue samples were imaged immediately after surgical resection. Co-registered photoacoustic images of malignancies showed significantly increased PAT signal compared to normal regions of the same sample. The quantitative relative total hemoglobin (rHbT) concentration computed from four optical wavelengths, the spectral features, such as the mean spectral slope, and 0.5-MHz intercept extracted from PAT and US spectral data, and image features, such as the first- and second-order statistics along with the standard deviation of the mean radon transform of PAT images, have shown statistical significance between untreated colorectal tumors and the normal tissue. Using either a logistic regression model or a support vector machine, the best set of parameters of rHbT and PAT intercept has achieved area-under-the-curve (AUC) values of 0.97 and 0.95 for both training and testing data sets, respectively, for prediction of histologically confirmed invasive carcinoma.</p>.

Performance of SAPS II according to ICU length of stay: A Danish nationwide cohort study

BACKGROUND

Intensive care unit (ICU) severity scores use data available at admission or shortly thereafter. There are limited contemporary data on how the prognostic performance of these scores is affected by ICU length of stay (LOS).

METHODS

We conducted a nationwide cohort study using routinely collected health data from the Danish Intensive Care Database. We included adults with ICU admissions ≥24 hours between 1 January 2012 and 30 June 2016, who survived to ICU discharge and had valid ICU LOS and vital status data registered. We assessed discrimination of the Simplified Acute Physiology Score (SAPS) II for predicting mortality 90 days after ICU discharge, followed by recalibration of the model and assessment of standardized mortality ratios (SMRs) and calibration. Performance was assessed in the entire cohort and stratified by ICU LOS quartiles.

RESULTS

We included 44 523 patients. Increasing SAPS II was associated with increasing ICU LOS. Overall discrimination (area under the receiver-operating characteristics curve) of SAPS II was 0.70 (95% CI: 0.70-0.71), with decreasing discrimination from the first (0.75, 95% CI: 0.73-0.76) to the last (0.64, 95% CI: 0.63-0.65) ICU LOS quartile. SMRs were lower (less deaths) than expected in the first ICU LOS quartile and higher (more deaths) than expected in the last two ICU LOS quartiles. Calibration decreased with increasing ICU LOS.

CONCLUSIONS

We observed that discrimination and calibration of SAPS II decreased with increasing ICU LOS, and that this affected SMRs. These findings should be acknowledged when using SAPS II for clinical, research and administrative purposes.

Bayesian hierarchical statistics for traffic safety modelling and forecasting

Road traffic accidents (RTAs) represent a serious problem globally causing losses in many ways. Gulf Cooperation Council (GCC) countries have a high rate of RTAs compared to other high-income countries. In this study, a Bayesian hierarchical model was utilized for accident counts forecasting in Abu Dhabi, United Arab Emirates. This work will help traffic planners and decision makers to enhance road safety levels and decrease accident fatality rate. Accidents data along 5 years from 2008 to 2012 at 143 road sites in Abu Dhabi with 5,511 accidents were used. The proposed model considered a number of covariates; speed limit, type of road, number of lanes, type of area, weather, time, surface condition and seat belt usage. Five sites with the highest numbers of accidents were studied. Year 2012 was used to validate predictions. The model prediction accuracy was 72%.

Development and validation of an asthma exacerbation prediction model using electronic health record (EHR) data

Objective: Asthma exacerbations are associated with significant morbidity, mortality, and cost. Accurately identifying asthma patients at risk for exacerbation is essential. We sought to develop a risk prediction tool based on routinely collected data from electronic health records (EHRs).Methods: From a repository of EHRs data, we extracted structured data for gender, race, ethnicity, smoking status, use of asthma medications, environmental allergy testing BMI status, and Asthma Control Test scores (ACT). A subgroup of this population of patients with asthma that had available prescription fill data was identified, which formed the primary population for analysis. Asthma exacerbation was defined as asthma-related hospitalization, urgent/emergent visit or oral steroid use over a 12-month period. Univariable and multivariable statistical analysis was completed to identify factors associated with exacerbation. We developed and tested a risk prediction model based on the multivariable analysis.Results: We identified 37,675 patients with asthma. Of those, 1,787 patients with asthma and fill data were identified, and 979 (54.8%) of them experienced an exacerbation. In the multivariable analysis, smoking (OR = 1.69, CI: 1.08-2.64), allergy testing (OR = 2.40, CI: 1.54-3.73), obesity (OR = 1.66, CI: 1.29-2.12), and ACT score reflecting uncontrolled asthma (OR = 1.66, CI: 1.10-2.29) were associated with increased risk of exacerbation. The area-under-the-curve (AUC) of our model in a combined derivation and validation cohort was 0.67.Conclusion: Despite use of rigorous methodology, we were unable to produce a predictive model with an acceptable degree of accuracy and AUC to be clinically useful.

Sample size for binary logistic prediction models: Beyond events per variable criteria

Binary logistic regression is one of the most frequently applied statistical approaches for developing clinical prediction models. Developers of such models often rely on an Events Per Variable criterion (EPV), notably EPV ≥10, to determine the minimal sample size required and the maximum number of candidate predictors that can be examined. We present an extensive simulation study in which we studied the influence of EPV, events fraction, number of candidate predictors, the correlations and distributions of candidate predictor variables, area under the ROC curve, and predictor effects on out-of-sample predictive performance of prediction models. The out-of-sample performance (calibration, discrimination and probability prediction error) of developed prediction models was studied before and after regression shrinkage and variable selection. The results indicate that EPV does not have a strong relation with metrics of predictive performance, and is not an appropriate criterion for (binary) prediction model development studies. We show that out-of-sample predictive performance can better be approximated by considering the number of predictors, the total sample size and the events fraction. We propose that the development of new sample size criteria for prediction models should be based on these three parameters, and provide suggestions for improving sample size determination.

Comparative Analysis of Established Risk Scores and Novel Hemodynamic Metrics in Predicting Right Ventricular Failure in Left Ventricular Assist Device Patients

BACKGROUND

Right ventricular failure (RVF) portends poor outcomes after left ventricular assist device (LVAD) implantation. Although numerous RVF predictive models have been developed, there are few independent comparative analyses of these risk models.

METHODS AND RESULTS

RVF was defined as use of inotropes for >14 days, inhaled pulmonary vasodilators for >48 hours or unplanned right ventricular mechanical support postoperatively during the index hospitalization. Risk models were evaluated for the primary outcome of RVF by means of logistic regression and receiver operating characteristic curves. Among 93 LVAD patients with complete data from 2011 to 2016, the Michigan RVF score (C = 0.74 [95% CI 0.61-0.87]; P = .0004) was the only risk model to demonstrate significant discrimination for RVF, compared with newer risk scores (Utah, Pitt, EuroMACS). Among individual hemodynamic/echocardiographic metrics, preoperative right ventricular dysfunction (C = 0.72 [95% CI 0.58-0.85]; P = .0022) also demonstrated significant discrimination of RVF. The Michigan RVF score was also the best predictor of in-hospital mortality (C = 0.67 [95% CI 0.52-0.83]; P = .0319) and 3-year survival (Kaplan-Meier log-rank 0.0135).

CONCLUSIONS

In external validation analysis, the more established Michigan RVF score-which emphasizes preoperative hemodynamic instability and target end-organ dysfunction-performed best, albeit modestly, in predicting RVF and demonstrated association with in-hospital and long-term mortality.

Synthetic data method to incorporate external information into a current study

We consider the situation where there is a known regression model that can be used to predict an outcome, Y, from a set of predictor variables X. A new variable B is expected to enhance the prediction of Y. A dataset of size n containing Y, X and B is available, and the challenge is to build an improved model for Y|X,B that uses both the available individual level data and some summary information obtained from the known model for Y|X. We propose a synthetic data approach, which consists of creating m additional synthetic data observations, and then analyzing the combined dataset of size n+m to estimate the parameters of the Y|X, B model. This combined dataset of size n+m now has missing values of B form of the observations, and is analyzed using methods that can handle missing data (e.g. multiple imputation). We present simulation studies and illustrate the method using data from the Prostate Cancer Prevention Trial. Though the synthetic data method is applicable to a general regression context, to provide some justification, we show in two special cases that the asymptotic variance of the parameter estimates in the Y|X, B model are identical to those from an alternative constrained maximum likelihood estimation approach. This correspondence in special cases and the method's broad applicability makes it appealing for use across diverse scenarios.

Guided or self-guided internet-based cognitive-behavioural therapy (iCBT) for depression? Study protocol of an individual participant data network meta-analysis

INTRODUCTION

Although guided forms of internet-based cognitive-behavioural therapy (iCBT) result in a substantial reduction in depression, it seems that the most scalable way to deliver iCBT is without guidance. However, direct evidence on the comparison between guided and self-guided iCBT is scarce. Moreover, it is unclear which types of patients may benefit more from each of these two forms of iCBT. Network meta-analysis (NMA) using individual participant data (IPD) offers a way to assess the relative efficacy of multiple (>2) interventions. Moreover, it maximises our power to detect patient-level characteristics (covariates) that have an important effect on the efficacy of interventions. This protocol describes the procedures of an IPD-NMA, which aims at examining the relative efficacy of guided compared with self-guided iCBT and at identifying predictors and moderators of treatment outcome.

METHODS AND ANALYSIS

We will use an existing database on psychotherapies for adult depression to identify eligible studies. This database has been updated up to 1 January 2018, through literature searches in PubMed, Embase, PsycINFO and Cochrane Library. The outcome of this IPD-NMA is reduction in depressive symptoms severity. We will fit the model in a Bayesian setting. After fitting the model, we will report the relative treatment effects for different types of patients, and we will discuss the clinical implications of our findings. Based on the results from the IPD-NMA model, we will develop and validate a personalised prediction model, aiming to provide patient-level predictions about the effects of the interventions.

ETHICS AND DISSEMINATION

An ethical approval is not required for this study. The results will be published in a peer-review journal. These results will guide clinical decisions about the most efficient way to allocate iCBT resources, thereby increasing the scalability of this innovative therapeutic approach.

European cancer mortality predictions for the year 2019 with focus on breast cancer

BACKGROUND

To overcome the lag with which cancer statistics become available, we predicted numbers of deaths and rates from all cancers and selected cancer sites for 2019 in the European Union (EU).

MATERIALS AND METHODS

We retrieved cancer death certifications and population data from the World Health Organization and Eurostat databases for 1970-2014. We obtained estimates for 2019 with a linear regression on number of deaths over the most recent trend period identified by a logarithmic Poisson joinpoint regression model. We calculated the number of avoided deaths over the period 1989-2019.

RESULTS

We estimated about 1 410 000 cancer deaths in the EU for 2019, corresponding to age-standardized rates of 130.9/100 000 men (-5.9% since 2014) and 82.9 women (-3.6%). Lung cancer trends in women are predicted to increase 4.4% between 2014 and 2019, reaching a rate of 14.8. The projected rate for breast cancer was 13.4. Favourable trends for major neoplasms are predicted to continue, except for pancreatic cancer. Trends in breast cancer mortality were favourable in all six countries considered, except Poland. The falls were largest in women 50-69 (-16.4%), i.e. the age group covered by screening, but also seen at age 20-49 (-13.8%), while more modest at age 70-79 (-6.1%). As compared to the peak rate in 1988, over 5 million cancer deaths have been avoided in the EU over the 1989-2019 period. Of these, 440 000 were breast cancer deaths.

CONCLUSION

Between 2014 and 2019, cancer mortality will continue to fall in both sexes. Breast cancer rates will fall steadily, with about 35% decline in rates over the last three decades. This is likely due to reduced hormone replacement therapy use, improvements in screening, early diagnosis and treatment. Due to population ageing, however, the number of breast cancer deaths is not declining.

Comparison of revised Functional Capacity Index scores with Abbreviated Injury Scale 2008 scores in predicting 12-month severe trauma outcomes

INTRODUCTION

Anatomical injury as measured by the AIS often accounts for only a small proportion of variability in outcomes after injury. The predictive Functional Capacity Index (FCI) appended to the 2008 AIS claims to provide a widely available method of predicting 12-month function following injury.

OBJECTIVES

To determine the extent to which AIS-based and FCI-based scoring is able to add to a simple predictive model of 12-month function following severe injury.

METHODS

Adult trauma patients were drawn from the population-based Victorian State Trauma Registry. Major trauma and severely injured orthopaedic trauma patients were followed up via telephone interview including Glasgow Outcome Scale-Extended, the EQ-5D-3L and return to work status. A battery of AIS-based and FCI-based scores, and a simple count of AIS-coded injuries were added in turn to a base model using age and gender.

RESULTS

A total of 20 813 patients survived to 12 months and had at least one functional outcome recorded, representing 85% follow-up. Predictions using the base model varied substantially across outcome measures. Irrespective of the method used to classify the severity of injury, adding injury severity to the model significantly, but only slightly improved model fit. Across the outcomes evaluated, no method of injury severity assessment consistently outperformed any other.

CONCLUSIONS

Anatomical injury is a predictor of trauma outcome. However, injury severity as described by the FCI does not consistently improve discrimination, or even provide the best discrimination compared with AIS-based severity scores or a simple injury count.

Variability in the Chemical Composition and In Vitro Ruminal Fermentation of Olive Cake By-Products

The objective of this study was to determine the variability in the chemical composition and in vitro ruminal fermentation of olive cake (OC) by-products. Forty-two OC samples with different storage times (1⁻14 months) and processing (25 crude (COC), 9 exhausted (EOC) and 9 cyclone (CYOC)) were fermented in vitro with sheep ruminal fluid. Exhausted OC samples had a lower ether extract content than COC and CYOC (15.9, 110 and 157 g/kg dry matter (DM), respectively), but greater neutral detergent fiber (NDF; 645, 570 and 441 g/kg DM) and acid insoluble nitrogen (9.76, 8.10 and 8.05 g/kg DM) content. Exhausted OC had the greatest (p < 0.05) average gas production rate (AGPR), whereas the greatest fermented organic matter (FOM) was obtained for EOC and CYOC. The best single predictor of the AGPR was total sugars content (R² = 0.898), whereas NDF was the best one for FOM (R² = 0.767; p < 0.001). Statistical models using storage time as a predictor variable had lower accuracy and R² values than those from the chemical composition. In summary, the nutritive value of OC was highly dependent on its processing, but its ether extract content did not negatively affect ruminal fermentation parameters, which could be estimated from either carbohydrate composition or storage time.

Postoperative prognostic model for patients with clear cell renal cell carcinoma in a Chinese population

OBJECTIVES

To develop a predictive model for the oncological outcomes of clear cell renal cell carcinoma in a Chinese population.

METHODS

A retrospective study of 1108 patients with clear cell renal cell carcinoma who underwent nephrectomy or partial nephrectomy between January 2006 and December 2013 was carried out. Recurrence-free survival was calculated using Kaplan-Meier analysis. Differences between the groups were compared using the log-rank test. Cox proportional hazard regression was used to test associations between features and outcomes. The discriminative ability of the models was validated using Harrell's concordance index and bootstrapping.

RESULTS

Overall, 942 patients who met the inclusion criteria had been followed. The median follow-up period was 72 months (range 1-143 months). Multivariate analysis showed that age, Eastern Cooperative Oncology Group performance status, preoperative platelet count, neutrophil-to-lymphocyte ratio, tumor size, 2010 tumor stage (pT3 and pT4) and Fuhrman nuclear grade were independent risk factors affecting recurrence-free survival in clear cell renal cell carcinoma patients (P < 0.05). These factors were assigned to develop a new model. The patients were divided into three groups based on the risk of recurrence. The difference among the prognoses of patients in the three groups was statistically significant (P < 0.05). The concordance index for our new model and that for Leibovich's 2018 model were 0.791 and 0.750, respectively.

CONCLUSIONS

In the present study, the new model has a higher concordance index than does Leibovich's 2018 model of clear cell renal cell carcinoma in the Asian population, with no added pain for patients. This new model might be an appropriate risk stratification tool for clinical work.

A Prediction Model for Preoperative Risk Assessment in Endometrial Cancer Utilizing Clinical and Molecular Variables

The utility of comprehensive surgical staging in patients with low risk disease has been questioned. Thus, a reliable means of determining risk would be quite useful. The aim of our study was to create the best performing prediction model to classify endometrioid endometrial cancer (EEC) patients into low or high risk using a combination of molecular and clinical-pathological variables. We then validated these models with publicly available datasets. Analyses between low and high risk EEC were performed using clinical and pathological data, gene and miRNA expression data, gene copy number variation and somatic mutation data. Variables were selected to be included in the prediction model of risk using cross-validation analysis; prediction models were then constructed using these variables. Model performance was assessed by area under the curve (AUC). Prediction models were validated using appropriate datasets in The Cancer Genome Atlas (TCGA) and Gene Expression Omnibus (GEO) databases. A prediction model with only clinical variables performed at 88%. Integrating clinical and molecular data improved prediction performance up to 97%. The best prediction models included clinical, miRNA expression and/or somatic mutation data, and stratified pre-operative risk in EEC patients. Integrating molecular and clinical data improved the performance of prediction models to over 95%, resulting in potentially useful clinical tests.

Prediction Models to Control Aging Time in Red Wine

A combination of physical-chemical analysis has been used to monitor the aging of red wines from D.O. Toro (Spain). The changes in the chemical composition of wines that occur over the aging time can be used to distinguish between wine samples collected after one, four, seven and ten months of aging. Different computational models were used to develop a good authenticity tool to certify wines. In this research, different models have been developed: Artificial Neural Network models (ANNs), Support Vector Machine (SVM) and Random Forest (RF) models. The results obtained for the ANN model developed with sigmoidal function in the output neuron and the RF model permit us to determine the aging time, with an average absolute percentage deviation below 1%, so it can be concluded that these two models have demonstrated their capacity to predict the age of wine.

Neural network and logistic regression diagnostic prediction models for giant cell arteritis: development and validation

PURPOSE

To develop and validate neural network (NN) vs logistic regression (LR) diagnostic prediction models in patients with suspected giant cell arteritis (GCA). Design: Multicenter retrospective chart review.

METHODS

An audit of consecutive patients undergoing temporal artery biopsy (TABx) for suspected GCA was conducted at 14 international medical centers. The outcome variable was biopsy-proven GCA. The predictor variables were age, gender, headache, clinical temporal artery abnormality, jaw claudication, vision loss, diplopia, erythrocyte sedimentation rate, C-reactive protein, and platelet level. The data were divided into three groups to train, validate, and test the models. The NN model with the lowest false-negative rate was chosen. Internal and external validations were performed.

RESULTS

Of 1,833 patients who underwent TABx, there was complete information on 1,201 patients, 300 (25%) of whom had a positive TABx. On multivariable LR age, platelets, jaw claudication, vision loss, log C-reactive protein, log erythrocyte sedimentation rate, headache, and clinical temporal artery abnormality were statistically significant predictors of a positive TABx (P≤0.05). The area under the receiver operating characteristic curve/Hosmer-Lemeshow P for LR was 0.867 (95% CI, 0.794, 0.917)/0.119 vs NN 0.860 (95% CI, 0.786, 0.911)/0.805, with no statistically significant difference of the area under the curves (P=0.316). The misclassification rate/false-negative rate of LR was 20.6%/47.5% vs 18.1%/30.5% for NN. Missing data analysis did not change the results.

CONCLUSION

Statistical models can aid in the triage of patients with suspected GCA. Misclassification remains a concern, but cutoff values for 95% and 99% sensitivities are provided (https://goo.gl/THCnuU).

Sample size considerations and predictive performance of multinomial logistic prediction models

Multinomial Logistic Regression (MLR) has been advocated for developing clinical prediction models that distinguish between three or more unordered outcomes. We present a full‐factorial simulation study to examine the predictive performance of MLR models in relation to the relative size of outcome categories, number of predictors and the number of events per variable. It is shown that MLR estimated by Maximum Likelihood yields overfitted prediction models in small to medium sized data. In most cases, the calibration and overall predictive performance of the multinomial prediction model is improved by using penalized MLR. Our simulation study also highlights the importance of events per variable in the multinomial context as well as the total sample size. As expected, our study demonstrates the need for optimism correction of the predictive performance measures when developing the multinomial logistic prediction model. We recommend the use of penalized MLR when prediction models are developed in small data sets or in medium sized data sets with a small total sample size (ie, when the sizes of the outcome categories are balanced). Finally, we present a case study in which we illustrate the development and validation of penalized and unpenalized multinomial prediction models for predicting malignancy of ovarian cancer.

Performance of SAPS II according to ICU length of stay: Protocol for an observational study

BACKGROUND

Severity scores, including the Simplified Acute Physiology Score (SAPS) II, are widely used in the intensive care unit (ICU) to predict mortality outcomes using data from ICU admission or shortly hereafter. For patients with longer ICU length of stay (LOS), the predictive performance of admission-based severity scores may deteriorate compared to patients with shorter ICU LOS. This protocol and statistical analysis plan outlines a study that will assess the influence of ICU LOS on the performance of SAPS II for predicting 90-day post-ICU mortality.

METHODS

A Danish nationwide cohort study including adult (≥18 years) ICU patients admitted to a Danish ICU between 1 January 2012 and 30 June 2016. The study will be conducted using the Danish Intensive Care Database (DID), which contains data routinely, prospectively, and consecutively reported for all Danish ICU admissions. Discrimination of SAPS II for predicting 90-day post-ICU mortality will be assessed for the entire cohort and stratified according to ICU LOS. A first-level recalibration of SAPS II will be performed, and if adequate, standardised mortality ratios and calibration stratified according to ICU LOS will be reported.

CONCLUSIONS

The outlined large, nationwide cohort study will provide important, contemporary information about the influence of ICU LOS on severity score performance relevant for ICU clinicians, researchers, and administrators. Publication of the protocol and statistical analysis plan prior to study conduct ensures transparency, and limits the risk of publication bias, post hoc changes in analyses, and challenges with multiple comparisons.

Development and Applications of Prognostic Risk Models in the Management of Invasive Mold Disease

Prognostic models or risk scores are frequently used to aid individualize risk assessment for diseases with multiple, complex risk factors and diagnostic challenges. However, relatively little attention has been paid to the development of risk models for invasive mold diseases encountered in patients with hematological malignancies, despite a large body of epidemiological research. Herein we review recent studies that have described the development of prognostic models for mold disease, summarize our experience with the development and clinical use of one such model (BOSCORE), and discuss the potential impact of prognostic risk scores for individualized therapy, diagnostic and antifungal stewardship, as well as clinical and epidemiological research.

Feather and skin development of ostriches Struthio camelus

Information on feather and skin growth is important for the development of mathematical optimisation nutritional models for ostriches. Ostriches (n = 65) were subjected to a four-stage formulated growth diet programme (pre-starter, starter, grower and finisher), with declining protein and energy content. Nine birds were weighed, stunned, exsanguinated, defeathered, skinned and eviscerated at 1, 54, 84, 104, 115, 132 and 287 days of age. Feathers from four pre-selected locations on the body were harvested and weighed. The wet skin weight, wet unstretched skin size and wet unstretched crown size were measured at each slaughter stage. The live weight, feather and skin yields of the birds increased with age at slaughter, as did feather shaft diameter. Prediction models were developed to estimate the yield of the skin in terms of live weight and of empty body protein weight to aid in diet formulation. The allometry of feather growth was determined from total feather weight, as the maturation rates of the feathers differ from that of the ostrich body. Results from this study will aid in setting up a mathematical optimisation nutritional model for ostriches.

Predicting Forearm Physical Exposures During Computer Work Using Self-Reports, Software-Recorded Computer Usage Patterns, and Anthropometric and Workstation Measurements

Objectives

Alternative techniques to assess physical exposures, such as prediction models, could facilitate more efficient epidemiological assessments in future large cohort studies examining physical exposures in relation to work-related musculoskeletal symptoms. The aim of this study was to evaluate two types of models that predict arm-wrist-hand physical exposures (i.e. muscle activity, wrist postures and kinematics, and keyboard and mouse forces) during computer use, which only differed with respect to the candidate predicting variables; (i) a full set of predicting variables, including self-reported factors, software-recorded computer usage patterns, and worksite measurements of anthropometrics and workstation set-up (full models); and (ii) a practical set of predicting variables, only including the self-reported factors and software-recorded computer usage patterns, that are relatively easy to assess (practical models).

Methods

Prediction models were build using data from a field study among 117 office workers who were symptom-free at the time of measurement. Arm-wrist-hand physical exposures were measured for approximately two hours while workers performed their own computer work. Each worker's anthropometry and workstation set-up were measured by an experimenter, computer usage patterns were recorded using software and self-reported factors (including individual factors, job characteristics, computer work behaviours, psychosocial factors, workstation set-up characteristics, and leisure-time activities) were collected by an online questionnaire. We determined the predictive quality of the models in terms of R2 and root mean squared (RMS) values and exposure classification agreement to low-, medium-, and high-exposure categories (in the practical model only).

Results

The full models had R2 values that ranged from 0.16 to 0.80, whereas for the practical models values ranged from 0.05 to 0.43. Interquartile ranges were not that different for the two models, indicating that only for some physical exposures the full models performed better. Relative RMS errors ranged between 5% and 19% for the full models, and between 10% and 19% for the practical model. When the predicted physical exposures were classified into low, medium, and high, classification agreement ranged from 26% to 71%.

Conclusion

The full prediction models, based on self-reported factors, software-recorded computer usage patterns, and additional measurements of anthropometrics and workstation set-up, show a better predictive quality as compared to the practical models based on self-reported factors and recorded computer usage patterns only. However, predictive quality varied largely across different arm-wrist-hand exposure parameters. Future exploration of the relation between predicted physical exposure and symptoms is therefore only recommended for physical exposures that can be reasonably well predicted.

Building a Prediction Model for Radiographically Confirmed Pneumonia in Peruvian Children: From Symptoms to Imaging

Background

Community-acquired pneumonia remains the leading cause of death in children worldwide, and current diagnostic guidelines in resource-poor settings are neither sensitive nor specific. We sought to determine the ability to correctly diagnose radiographically confirmed clinical pneumonia when diagnostics tools were added to clinical signs and symptoms in a cohort of children with acute respiratory illnesses in Peru.

Methods

Children < 5 years of age with an acute respiratory illness presenting to a tertiary hospital in Lima, Peru, were enrolled. The ability to predict radiographically confirmed clinical pneumonia was assessed using logistic regression under four additive scenarios: clinical signs and symptoms only, addition of lung auscultation, addition of oxyhemoglobin saturation (Spo₂), and addition of lung ultrasound.

Results

Of 832 children (mean age, 21.3 months; 59% boys), 453 (54.6%) had clinical pneumonia and 221 (26.6%) were radiographically confirmed. Children with radiographically confirmed clinical pneumonia had lower average Spo₂ than those without (95.9% vs 96.6%, respectively; P < .01). The ability to correctly identify radiographically confirmed clinical pneumonia using clinical signs and symptoms was limited (area under the curve [AUC] = 0.62; 95% CI, 0.58-0.67) with a sensitivity of 66% (95% CI, 59%-73%) and specificity of 53% (95% CI, 49%-57%). The addition of lung auscultation improved classification (AUC = 0.73; 95% CI, 0.69-0.77) with a sensitivity of 75% (95% CI, 69%-81%) and specificity of 53% (95% CI, 49%-57%) for the presence of crackles. In contrast, the addition of Spo₂ did not improve classification (AUC = 0.73; 95% CI, 0.69-0.77) with a sensitivity of 40% (95% CI, 33%-47%) and specificity of 72% (95% CI, 68%-75%) for an Spo₂ ≤ 92%. Adding consolidation on lung ultrasound was associated with the largest improvement in classification (AUC = 0.85; 95% CI, 0.82-0.89) with a sensitivity of 55% (95% CI, 48%-63%) and specificity of 95% (95% CI, 93%-97%).

Conclusions

The addition of lung ultrasound and auscultation to clinical signs and symptoms improved the ability to correctly classify radiographically confirmed clinical pneumonia. Implementation of auscultation- and ultrasound-based diagnostic tools can be considered to improve diagnostic yield of pneumonia in resource-poor settings.

Review: Precision nutrition of ruminants: approaches, challenges and potential gains

A plethora of sensors and information technologies with applications to the precision nutrition of herbivores have been developed and continue to be developed. The nutritional processes start outside of the animal body with the available feed (quantity and quality) and continue inside it once the feed is consumed, degraded in the gastrointestinal tract and metabolised by organs and tissues. Finally, some nutrients are wasted via urination, defecation and gaseous emissions through breathing and belching whereas remaining nutrients ensure maintenance and production. Nowadays, several processes can be monitored in real-time using new technologies, but although these provide valuable data 'as is', further gains could be obtained using this information as inputs to nutrition simulation models to predict unmeasurable variables in real-time and to forecast outcomes of interest. Data provided by sensors can create synergies with simulation models and this approach has the potential to expand current applications. In addition, data provided by sensors could be used with advanced analytical techniques such as data fusion, optimisation techniques and machine learning to improve their value for applications in precision animal nutrition. The present paper reviews technologies that can monitor different nutritional processes relevant to animal production, profitability, environmental management and welfare. We discussed the model-data fusion approach in which data provided by sensor technologies can be used as input of nutrition simulation models in near-real time to produce more accurate, certain and timely predictions. We also discuss some examples that have taken this model-data fusion approach to complement the capabilities of both models and sensor data, and provided examples such as predicting feed intake and methane emissions. Challenges with automatising the nutritional management of individual animals include monitoring and predicting of the flow of nutrients including nutrient intake, quantity and composition of body growth and milk production, gestation, maintenance and physical activities at the individual animal level. We concluded that the livestock industries are already seeing benefits from the development of sensor and information technologies, and this benefit is expected to grow exponentially soon with the integration of nutrition simulation models and techniques for big data analysis. However, this approach may need re-evaluating or performing new empirical research in both fields of animal nutrition and simulation modelling to accommodate a new type of data provided by the sensor technologies.

A prospective evaluation of predictors of pain after arthroscopic rotator cuff repair: psychosocial factors have a stronger association than structural factors

HYPOTHESIS

We evaluated the correlation of preoperative factors with pain after arthroscopic rotator cuff repair (ARCR). We hypothesized that nonstructural factors, including metrics of psychological well-being and preoperative narcotic use, would correlate with higher pain levels postoperatively and that structural factors, such as tear size, would not be predictive.

METHODS

Ninety-three patients were prospectively enrolled and evenly distributed by tear size. Patient sex, age, occupation, smoking status, tear mechanism, tear characteristics on magnetic resonance imaging, visual analog scale (VAS) pain scores, narcotic usage, range of motion (ROM) by goniometry, and functional and psychological assessments through the American Shoulder and Elbow Surgeons (ASES) Standardized Shoulder Assessment Form, Simple Shoulder Test, Western Ontario Rotator Cuff Index (WORC), and RAND 36-item Short Form Survey questionnaires were obtained preoperatively. VAS scores and ROM were collected postoperatively at 2 weeks, 6 weeks, 3 months, 6 months, and 1 year. The ASES, SST, WORC, and RAND 36-item Short Form Survey questionnaires were repeated 1 year postoperatively.

RESULTS

The patients (54% men) were a mean age of 56.4 years. There were 68% traumatic tears, 11% smokers, and 13% used narcotics preoperatively. ROM, VAS, ASES, and WORC scores improved significantly from the preoperative to 1-year postoperative assessment. Correlating with increased pain scores at 1 year were preoperative narcotic use, higher preoperative VAS, and lower scores on the WORC index and emotion sections.

CONCLUSION

Our data show that the factors most predictive of persistent pain after ARCR are psychosocial characteristics, including poor performance on validated measures of emotional well-being. Demographic and tear-specific structural factors did not correlate with postoperative pain scores.

ISLES 2016 and 2017-Benchmarking Ischemic Stroke Lesion Outcome Prediction Based on Multispectral MRI

Performance of models highly depend not only on the used algorithm but also the data set it was applied to. This makes the comparison of newly developed tools to previously published approaches difficult. Either researchers need to implement others' algorithms first, to establish an adequate benchmark on their data, or a direct comparison of new and old techniques is infeasible. The Ischemic Stroke Lesion Segmentation (ISLES) challenge, which has ran now consecutively for 3 years, aims to address this problem of comparability. ISLES 2016 and 2017 focused on lesion outcome prediction after ischemic stroke: By providing a uniformly pre-processed data set, researchers from all over the world could apply their algorithm directly. A total of nine teams participated in ISLES 2015, and 15 teams participated in ISLES 2016. Their performance was evaluated in a fair and transparent way to identify the state-of-the-art among all submissions. Top ranked teams almost always employed deep learning tools, which were predominately convolutional neural networks (CNNs). Despite the great efforts, lesion outcome prediction persists challenging. The annotated data set remains publicly available and new approaches can be compared directly via the online evaluation system, serving as a continuing benchmark (www.isles-challenge.org).

Making IVF more effective through the evolution of prediction models: is prognosis the missing piece of the puzzle?

Assisted reproductive technology has evolved tremendously since the emergence of in vitro fertilization (IVF). In the course of the recent decade, there have been significant efforts in order to minimize multiple gestations, while improving percentages of singleton pregnancies and offering individualized services in IVF, in line with the trend of personalized medicine. Patients as well as clinicians and the entire IVF team benefit majorly from 'knowing what to expect' from an IVF cycle. Hereby, the question that has emerged is to what extent prognosis could facilitate toward the achievement of the above goal. In the current review, we present prediction models based on patients' characteristics and IVF data, as well as models based on embryo morphology and biomarkers during culture shaping a complication free and cost-effective personalized treatment. The starting point for the implementation of prediction models was initiated by the aspiration of moving toward optimal practice. Thus, prediction models could serve as useful tools that could safely set the expectations involved during this journey guiding and making IVF treatment more effective. The aim and scope of this review is to thoroughly present the evolution and contribution of prediction models toward an efficient IVF treatment.

ABBREVIATIONS

IVF: In vitro fertilization; ART: assisted reproduction techniques; BMI: body mass index; OHSS: ovarian hyperstimulation syndrome; eSET: elective single embryo transfer; ESHRE: European Society of Human Reproduction and Embryology; mtDNA: mitochondrial DNA; nDNA: nuclear DNA; ICSI: intracytoplasmic sperm injection; MBR: multiple birth rates; LBR: live birth rates; SART: Society for Assisted Reproductive Technology Clinic Outcome Reporting System; AFC: antral follicle count; GnRH: gonadotrophin releasing hormone; FSH: follicle stimulating hormone; LH: luteinizing hormone; AMH: anti-Müllerian hormone; DHEA: dehydroepiandrosterone; PCOS: polycystic ovarian syndrome; NPCOS: non-polycystic ovarian syndrome; CE: cost-effectiveness; CC: clomiphene citrate; ORT: ovarian reserve test; EU: embryo-uterus; DET: double embryo transfer; CES: Cumulative Embryo Score; GES: Graduated Embryo Score; CSS: Combined Scoring System; MSEQ: Mean Score of Embryo Quality; IMC: integrated morphology cleavage; EFNB2: ephrin-B2; CAMK1D: calcium/calmodulin-dependent protein kinase 1D; GSTA4: glutathione S-transferase alpha 4; GSR: glutathione reductase; PGR: progesterone receptor; AMHR2: anti-Müllerian hormone receptor 2; LIF: leukemia inhibitory factor; sHLA-G: soluble human leukocyte antigen G.

Predicting rehospitalization in patients treated with antipsychotics: a prospective observational study

BACKGROUND

Prediction of rehospitalization in patients treated with antipsychotics is important for identifying patients in need of additional support to prevent hospitalization. Our aim was to identify factors that predict rehospitalization in patients treated with antipsychotics at discharge from a psychiatric hospital.

METHODS

Adult patients suffering from schizophrenia, psychotic or bipolar I disorders who had been hospitalized in a psychiatric hospital for ⩾7 days and were treated with oral antipsychotics at discharge were included. The main outcome was rehospitalization within 6 months after discharge. A prediction model for rehospitalization was constructed including: patient/disease and medication characteristics, patients' beliefs about medicines, and healthcare-professional-rated assessment for all patients. The patients were stratified by diagnosis (schizophrenia and nonschizophrenia). Area under the receiver operating characteristic curve (AUC_ROC) was also assessed.

RESULTS

A total of 87 patients were included and 33.3% of them were rehospitalized within 6 months after discharge. The variables that predicted rehospitalization were duration of hospitalization, patients' attitude towards medicine use, and healthcare-professional-rated assessment with an AUC_ROC of 0.82. Rehospitalization for patients with schizophrenia could be predicted (AUC_ROC = 0.71) by the Global Assessment of Functioning score, age, and harm score. Rehospitalization was predicted (AUC_ROC = 0.73) for nonschizophrenia patients with, for example rehospitalization predicted by the nurse.

CONCLUSIONS

Rehospitalization was predicted by a combination of variables from the patient/disease and medication characteristics, patients' attitude towards medicine use, and healthcare-professional-rated assessment. These variables can be assessed relatively easily at discharge to predict rehospitalization within 6 months.

Prediction of enteric methane production, yield, and intensity in dairy cattle using an intercontinental database

Enteric methane (CH₄ ) production from cattle contributes to global greenhouse gas emissions. Measurement of enteric CH₄ is complex, expensive, and impractical at large scales; therefore, models are commonly used to predict CH₄ production. However, building robust prediction models requires extensive data from animals under different management systems worldwide. The objectives of this study were to (1) collate a global database of enteric CH₄ production from individual lactating dairy cattle; (2) determine the availability of key variables for predicting enteric CH₄ production (g/day per cow), yield [g/kg dry matter intake (DMI)], and intensity (g/kg energy corrected milk) and their respective relationships; (3) develop intercontinental and regional models and cross-validate their performance; and (4) assess the trade-off between availability of on-farm inputs and CH₄ prediction accuracy. The intercontinental database covered Europe (EU), the United States (US), and Australia (AU). A sequential approach was taken by incrementally adding key variables to develop models with increasing complexity. Methane emissions were predicted by fitting linear mixed models. Within model categories, an intercontinental model with the most available independent variables performed best with root mean square prediction error (RMSPE) as a percentage of mean observed value of 16.6%, 14.7%, and 19.8% for intercontinental, EU, and United States regions, respectively. Less complex models requiring only DMI had predictive ability comparable to complex models. Enteric CH₄ production, yield, and intensity prediction models developed on an intercontinental basis had similar performance across regions, however, intercepts and slopes were different with implications for prediction. Revised CH₄ emission conversion factors for specific regions are required to improve CH₄ production estimates in national inventories. In conclusion, information on DMI is required for good prediction, and other factors such as dietary neutral detergent fiber (NDF) concentration, improve the prediction. For enteric CH₄ yield and intensity prediction, information on milk yield and composition is required for better estimation.

Risk biomarkers enable precision in public health

Precision medicine uses biomarkers to diagnose disease. However, they can also be used to measure risk of disease. Thus, biomarkers herald a new addition to public health - Precision Public Health. We examine the implications. Risk biomarkers are identified by analyzing population cohorts. They constitute risk factors in mathematical 'Disease Risk Models'. The risk may be fixed as in a genetic biomarker or variable as in some protein biomarkers. They help monitor current risk of disease in an individual, thereby aiding efforts to reduce risk. In the UK, the NHS Health Check system is a universal system for assessing risk and for risk reduction. The system can now make use of modern biomarkers once appropriate infrastructure and governance are in place.

Prediction models for clinical outcome after a carotid revascularisation procedure: A systematic review

Introduction

Prediction models for clinical outcome after carotid artery stenting or carotid endarterectomy could aid physicians in estimating peri- and postprocedural risks in individual patients. We aimed to identify existing prediction models for short- and long-term outcome after carotid artery stenting or carotid endarterectomy in patients with symptomatic or asymptomatic carotid stenosis, and to summarise their most important predictors and predictive performance.

Patients and methods

We performed a systematic literature search for studies that developed a prediction model or risk score published until 22 December 2016. Eligible prediction models had to predict the risk of vascular events with at least one patient characteristic.

Results

We identified 37 studies that developed 46 prediction models. Thirty-four (74%) models were developed in carotid endarterectomy patients; 27 of these (59%) predicted short-term (in-hospital or within 30 days) risk. Most commonly predicted outcome was stroke or death (n = 12; 26%). Age (n = 31; 67%), diabetes mellitus (n = 21; 46%), heart failure (n = 16; 35%), and contralateral carotid stenosis ≥50% or occlusion (n = 16; 35%) were most commonly used as predictors. For 25 models (54%), it was unclear how missing data were handled; a complete case analysis was performed in 15 (33%) of the remaining 21 models. Twenty-eight (61%) models reported the full regression formula or risk score with risk classification. Twenty-one (46%) models were validated internally and 12 (26%) externally. Discriminative performance (c-statistic) ranged from 0.66 to 0.94 for models after carotid artery stenting and from 0.58 to 0.74 for models after carotid endarterectomy. The c-statistic ranged from 0.55 to 0.72 for the external validations.

Discussion

Age, diabetes mellitus, heart failure, and contralateral carotid stenosis ≥50% or occlusion were most often used as predictors in all models. Discriminative performance (c-statistic) was higher for prediction models after carotid artery stenting than after carotid endarterectomy.

Conclusion

The clinical usefulness of most prediction models for short- or long-term outcome after carotid artery stenting or carotid endarterectomy remains unclear because of incomplete reporting, methodological limitations, and lack of external validation.

Computational Tools for the Identification and Interpretation of Sequence Motifs in Immunopeptidomes

Recent advances in proteomics and mass-spectrometry have widely expanded the detectable peptide repertoire presented by major histocompatibility complex (MHC) molecules on the cell surface, collectively known as the immunopeptidome. Finely characterizing the immunopeptidome brings about important basic insights into the mechanisms of antigen presentation, but can also reveal promising targets for vaccine development and cancer immunotherapy. This report describes a number of practical and efficient approaches to analyze immunopeptidomics data, discussing the identification of meaningful sequence motifs in various scenarios and considering current limitations. Guidelines are provided for the filtering of false hits and contaminants, and to address the problem of motif deconvolution in cell lines expressing multiple MHC alleles, both for the MHC class I and class II systems. Finally, it is demonstrated how machine learning can be readily employed by non-expert users to generate accurate prediction models directly from mass-spectrometry eluted ligand data sets.

Big Data and Data Science in Critical Care

The digitalization of the health-care system has resulted in a deluge of clinical big data and has prompted the rapid growth of data science in medicine. Data science, which is the field of study dedicated to the principled extraction of knowledge from complex data, is particularly relevant in the critical care setting. The availability of large amounts of data in the ICU, the need for better evidence-based care, and the complexity of critical illness makes the use of data science techniques and data-driven research particularly appealing to intensivists. Despite the increasing number of studies and publications in the field, thus far there have been few examples of data science projects that have resulted in successful implementations of data-driven systems in the ICU. However, given the expected growth in the field, intensivists should be familiar with the opportunities and challenges of big data and data science. The present article reviews the definitions, types of algorithms, applications, challenges, and future of big data and data science in critical care.

Validity of activity monitors in wheelchair users: A systematic review

Assessing physical activity (PA) in manual wheelchair users (MWUs) is challenging because of their different movement patterns in comparison to the ambulatory population. The aim of this review was to investigate the validity of portable monitors in quantifying PA in MWUs. A systematic literature search was performed. The data source was full reports of validation and evaluation studies in peer-reviewed journals and conference proceedings. Eligible articles between January 1, 1999, and September 18, 2015, were identified in three databases: PubMed, Institute of Electrical and Electronics Engineers, and Scopus. A total of 164 articles (158 from the databases and 6 from the citation/reference tracking) were identified, and 29 met the eligibility criteria. Two investigators independently extracted the characteristics from each selected article following a predetermined protocol and completed seven summary tables describing the study characteristics and key outcomes. In the identified studies, the monitors were used to assess three types of PA measures: energy cost, user movement, and wheelchair movement. The customized algorithms/monitors did not estimate energy cost in MWUs as well as the commercial monitors did in the ambulatory population; however, they showed fair accuracy in measuring both wheelchair and user movements.

An analytic approach for interpretable predictive models in high-dimensional data in the presence of interactions with exposures

Predicting a phenotype and understanding which variables improve that prediction are two very challenging and overlapping problems in the analysis of high-dimensional (HD) data such as those arising from genomic and brain imaging studies. It is often believed that the number of truly important predictors is small relative to the total number of variables, making computational approaches to variable selection and dimension reduction extremely important. To reduce dimensionality, commonly used two-step methods first cluster the data in some way, and build models using cluster summaries to predict the phenotype. It is known that important exposure variables can alter correlation patterns between clusters of HD variables, that is, alter network properties of the variables. However, it is not well understood whether such altered clustering is informative in prediction. Here, assuming there is a binary exposure with such network-altering effects, we explore whether the use of exposure-dependent clustering relationships in dimension reduction can improve predictive modeling in a two-step framework. Hence, we propose a modeling framework called ECLUST to test this hypothesis, and evaluate its performance through extensive simulations. With ECLUST, we found improved prediction and variable selection performance compared to methods that do not consider the environment in the clustering step, or to methods that use the original data as features. We further illustrate this modeling framework through the analysis of three data sets from very different fields, each with HD data, a binary exposure, and a phenotype of interest. Our method is available in the eclust CRAN package.