Visualizing Risk Prediction Models

User-Centered Design and Implementation of an Interoperable FHIR Application for Pediatric Pneumonia Prognostication in a Randomized Trial

OBJECTIVES

 To support a pragmatic, electronic health record (EHR)-based randomized controlled trial, we applied user-centered design (UCD) principles, evidence-based risk communication strategies, and interoperable software architecture to design, test, and deploy a prognostic tool for children in emergency departments (EDs) with pneumonia.

METHODS

 Risk for severe in-hospital outcomes was estimated using a validated ordinal logistic regression model to classify pneumonia severity. To render the results usable for ED clinicians, we created an integrated SMART on Fast Healthcare Interoperability Resources (FHIR) web application built for interoperable use in two pediatric EDs using different EHR vendors: Epic and Cerner. We followed a UCD framework, including problem analysis and user research, conceptual design and early prototyping, user interface development, formative evaluation, and postdeployment summative evaluation.

RESULTS

 Problem analysis and user research from 39 clinicians and nurses revealed user preferences for risk aversion, accessibility, and timing of risk communication. Early prototyping and iterative design incorporated evidence-based design principles, including numeracy, risk framing, and best-practice visualization techniques. After rigorous unit and end-to-end testing, the application was successfully deployed in both EDs, which facilitated enrollment, randomization, model visualization, data capture, and reporting for trial purposes.

CONCLUSION

 The successful implementation of a custom application for pneumonia prognosis and clinical trial support in two health systems on different EHRs demonstrates the importance of UCD, adherence to modern clinical data standards, and rigorous testing. Key lessons included the need for understanding users' real-world needs, regular knowledge management, application maintenance, and the recognition that FHIR applications require careful configuration for interoperability.

Utilizing big data from electronic health records in pediatric clinical care

Big data has the capacity to transform both pediatric healthcare delivery and research, but its potential has yet to be fully realized. Curation of large multi-institutional datasets of high-quality data has allowed for significant advances in the timeliness of quality improvement efforts. Improved access to large datasets and computational power have also paved the way for the development of high-performing, data-driven decision support tools and precision medicine approaches. However, implementation of these approaches and tools into pediatric practice has been hindered by challenges in our ability to adequately capture the heterogeneity of the pediatric population as well as the nuanced complexities of pediatric diseases such as sepsis. Moreover, there are large gaps in knowledge and definitive evidence demonstrating the utility, usability, and effectiveness of these types of tools in pediatric practice, which presents significant challenges to provider willingness to leverage these solutions. The next wave of transformation for pediatric healthcare delivery and research through big data and sophisticated analytics will require focusing efforts on strategies to overcome cultural barriers to adoption and acceptance. IMPACT: Big data from EHRs can be used to drive improvement in pediatric clinical care. Clinical decision support, artificial intelligence, machine learning, and precision medicine can transform pediatric care using big data from the EHR. This article provides a review of barriers and enablers for the effective use of data analytics in pediatric clinical care using pediatric sepsis as a use case. The impact of this review is that it will inform influencers of pediatric care about the importance of current trends in data analytics and its use in improving outcomes of care through EHR-based strategies.

PredictCBC-2.0: a contralateral breast cancer risk prediction model developed and validated in ~ 200,000 patients

BACKGROUND

Prediction of contralateral breast cancer (CBC) risk is challenging due to moderate performances of the known risk factors. We aimed to improve our previous risk prediction model (PredictCBC) by updated follow-up and including additional risk factors.

METHODS

We included data from 207,510 invasive breast cancer patients participating in 23 studies. In total, 8225 CBC events occurred over a median follow-up of 10.2 years. In addition to the previously included risk factors, PredictCBC-2.0 included CHEK2 c.1100delC, a 313 variant polygenic risk score (PRS-313), body mass index (BMI), and parity. Fine and Gray regression was used to fit the model. Calibration and a time-dependent area under the curve (AUC) at 5 and 10 years were assessed to determine the performance of the models. Decision curve analysis was performed to evaluate the net benefit of PredictCBC-2.0 and previous PredictCBC models.

RESULTS

The discrimination of PredictCBC-2.0 at 10 years was higher than PredictCBC with an AUC of 0.65 (95% prediction intervals (PI) 0.56-0.74) versus 0.63 (95%PI 0.54-0.71). PredictCBC-2.0 was well calibrated with an observed/expected ratio at 10 years of 0.92 (95%PI 0.34-2.54). Decision curve analysis for contralateral preventive mastectomy (CPM) showed the potential clinical utility of PredictCBC-2.0 between thresholds of 4 and 12% 10-year CBC risk for BRCA1/2 mutation carriers and non-carriers.

CONCLUSIONS

Additional genetic information beyond BRCA1/2 germline mutations improved CBC risk prediction and might help tailor clinical decision-making toward CPM or alternative preventive strategies. Identifying patients who benefit from CPM, especially in the general breast cancer population, remains challenging.

Measures of Intracranial Injury Size Do Not Improve Clinical Decision Making for Children With Mild Traumatic Brain Injuries and Intracranial Injuries

BACKGROUND

When evaluating children with mild traumatic brain injuries (mTBIs) and intracranial injuries (ICIs), neurosurgeons intuitively consider injury size. However, the extent to which such measures (eg, hematoma size) improve risk prediction compared with the kids intracranial injury decision support tool for traumatic brain injury (KIIDS-TBI) model, which only includes the presence/absence of imaging findings, remains unknown.

OBJECTIVE

To determine the extent to which measures of injury size improve risk prediction for children with mild traumatic brain injuries and ICIs.

METHODS

We included children ≤18 years who presented to 1 of the 5 centers within 24 hours of TBI, had Glasgow Coma Scale scores of 13 to 15, and had ICI on neuroimaging. The data set was split into training (n = 1126) and testing (n = 374) cohorts. We used generalized linear modeling (GLM) and recursive partitioning (RP) to predict the composite of neurosurgery, intubation >24 hours, or death because of TBI. Each model's sensitivity/specificity was compared with the validated KIIDS-TBI model across 3 decision-making risk cutoffs (<1%, <3%, and <5% predicted risk).

RESULTS

The GLM and RP models included similar imaging variables (eg, epidural hematoma size) while the GLM model incorporated additional clinical predictors (eg, Glasgow Coma Scale score). The GLM (76%-90%) and RP (79%-87%) models showed similar specificity across all risk cutoffs, but the GLM model had higher sensitivity (89%-96% for GLM; 89% for RP). By comparison, the KIIDS-TBI model had slightly higher sensitivity (93%-100%) but lower specificity (27%-82%).

CONCLUSION

Although measures of ICI size have clear intuitive value, the tradeoff between higher specificity and lower sensitivity does not support the addition of such information to the KIIDS-TBI model.

Construction of Xinjiang metabolic syndrome risk prediction model based on interpretable models

Background

We aimed to construct simple and practical metabolic syndrome (MetS) risk prediction models based on the data of inhabitants of Urumqi and to provide a methodological reference for the prevention and control of MetS.

Methods

This is a cross-sectional study conducted in the Xinjiang Uygur Autonomous Region of China. We collected data from inhabitants of Urumqi from 2018 to 2019, including demographic characteristics, anthropometric indicators, living habits and family history. Resampling technology was used to preprocess the data imbalance problems, and then MetS risk prediction models were constructed based on logistic regression (LR) and decision tree (DT). In addition, nomograms and tree diagrams of DT were used to explain and visualize the model.

Results

Of the 25,542 participants included in the study, 3,267 (12.8%) were diagnosed with MetS, and 22,275 (87.2%) were diagnosed with non-MetS. Both the LR and DT models based on the random undersampling dataset had good AUROC values (0.846 and 0.913, respectively). The accuracy, sensitivity, specificity, and AUROC values of the DT model were higher than those of the LR model. Based on a random undersampling dataset, the LR model showed that exercises such as walking (OR=0.769) and running (OR= 0.736) were protective factors against MetS. Age 60 ~ 74 years (OR=1.388), previous diabetes (OR=8.902), previous hypertension (OR=2.830), fatty liver (OR=3.306), smoking (OR=1.541), high systolic blood pressure (OR=1.044), and high diastolic blood pressure (OR=1.072) were risk factors for MetS; the DT model had 7 depth layers and 18 leaves, with BMI as the root node of the DT being the most important factor affecting MetS, and the other variables in descending order of importance: SBP, previous diabetes, previous hypertension, DBP, fatty liver, smoking, and exercise.

Conclusions

Both DT and LR MetS risk prediction models have good prediction performance and their respective characteristics. Combining these two methods to construct an interpretable risk prediction model of MetS can provide methodological references for the prevention and control of MetS.

A novel LASSO-derived prognostic model predicting survival for non-small cell lung cancer patients with M1a diseases

Introduction

The current American Joint Committee on Cancer (AJCC) M1a staging of non‐small cell lung cancer (NSCLC) encompasses a wide disease spectrum, showing diverse prognosis.

Methods

Patients who diagnosed in an earlier period formed the training cohort, and those who diagnosed thereafter formed the validation cohort. Kaplan–Meier analysis was performed for the training cohort by dividing the M1a stage into three subgroups: (I) malignant pleural effusion (MPE) or malignant pericardial effusion (MPCE); (II) separate tumor nodules in contralateral lung (STCL); and (III) pleural tumor nodules on the ipsilateral lung (PTIL). Gender, age, histologic, N stage, grade, surgery for primary site, lymphadenectomy, M1a groups, and chemotherapy were selected as independent prognostic factors using the least absolute shrinkage and selection operator (LASSO) Cox regression analysis. And a nomogram was constructed using Cox hazard regression analysis. Accuracy and clinical practicability were separately tested by Harrell's concordance index, the receiver operating characteristic (ROC) curve, calibration plots, residual plot, the integrated discrimination improvement (IDI), net reclassification improvement (NRI), and decision curve analysis (DCA).

Results

The concordance index (0.661 for the training cohort and 0.688 for the validation cohort) and the area under the ROC curve (training cohort: 0.709 for 1‐year and 0.727 for 2‐year OS prediction; validation cohort: 0.737 for 1‐year and 0.734 for 2‐year OS prediction) indicated satisfactory discriminative ability of the nomogram. Calibration curve and DCA presented great prognostic accuracy, and clinical applicability. Its prognostic accuracy preceded the AJCC staging with evaluated NRI (1‐year: 0.327; 2‐year: 0.302) and IDI (1‐year: 0.138; 2‐year: 0.130).

Conclusion

Our study established a nomogram for the prediction of 1‐ and 2‐year OS in patients with NSCLC diagnosed with stage M1a, facilitating healthcare workers to accurately evaluate the individual survival of M1a NSCLC patients. The accuracy and clinical applicability of this nomogram were validated.

Novel nomogram-based integrated gonadotropin therapy individualization in in vitro fertilization/intracytoplasmic sperm injection: A modeling approach

OBJECTIVE

This study aimed to characterize a validated model for predicting oocyte retrieval in controlled ovarian stimulation (COS) and to construct model-based nomograms for assistance in clinical decision-making regarding the gonadotropin protocol and dose.

METHODS

This observational, retrospective, cohort study included 636 women with primary unexplained infertility and a normal menstrual cycle who were attempting assisted reproductive therapy for the first time. The enrolled women were split into an index group (n=497) for model building and a validation group (n=139). The primary outcome was absolute oocyte count. The dose-response relationship was tested using modified Poisson, negative binomial, hybrid Poisson-Emax, and linear models. The validation group was similarly analyzed, and its results were compared to that of the index group.

RESULTS

The Poisson model with the log-link function demonstrated superior predictive performance and precision (Akaike information criterion, 2,704; λ=8.27; relative standard error (λ)=2.02%). The covariate analysis included women's age (p<0.001), antral follicle count (p<0.001), basal follicle-stimulating hormone level (p<0.001), gonadotropin dose (p=0.042), and protocol type (p=0.002 and p<0.001 for short and antagonist protocols, respectively). The estimates from 500 bootstrap samples were close to those of the original model. The validation group showed model assessment metrics comparable to the index model. Based on the fitted model, a static nomogram was built to improve visualization. In addition, a dynamic electronic tool was created for convenience of use.

CONCLUSION

Based on our validated model, nomograms were constructed to help clinicians individualize the stimulation protocol and gonadotropin doses in COS cycles.

Development and validation of a lifestyle-based model for colorectal cancer risk prediction: the LiFeCRC score

BACKGROUND

Nutrition and lifestyle have been long established as risk factors for colorectal cancer (CRC). Modifiable lifestyle behaviours bear potential to minimize long-term CRC risk; however, translation of lifestyle information into individualized CRC risk assessment has not been implemented. Lifestyle-based risk models may aid the identification of high-risk individuals, guide referral to screening and motivate behaviour change. We therefore developed and validated a lifestyle-based CRC risk prediction algorithm in an asymptomatic European population.

METHODS

The model was based on data from 255,482 participants in the European Prospective Investigation into Cancer and Nutrition (EPIC) study aged 19 to 70 years who were free of cancer at study baseline (1992-2000) and were followed up to 31 September 2010. The model was validated in a sample comprising 74,403 participants selected among five EPIC centres. Over a median follow-up time of 15 years, there were 3645 and 981 colorectal cancer cases in the derivation and validation samples, respectively. Variable selection algorithms in Cox proportional hazard regression and random survival forest (RSF) were used to identify the best predictors among plausible predictor variables. Measures of discrimination and calibration were calculated in derivation and validation samples. To facilitate model communication, a nomogram and a web-based application were developed.

RESULTS

The final selection model included age, waist circumference, height, smoking, alcohol consumption, physical activity, vegetables, dairy products, processed meat, and sugar and confectionary. The risk score demonstrated good discrimination overall and in sex-specific models. Harrell's C-index was 0.710 in the derivation cohort and 0.714 in the validation cohort. The model was well calibrated and showed strong agreement between predicted and observed risk. Random survival forest analysis suggested high model robustness. Beyond age, lifestyle data led to improved model performance overall (continuous net reclassification improvement = 0.307 (95% CI 0.264-0.352)), and especially for young individuals below 45 years (continuous net reclassification improvement = 0.364 (95% CI 0.084-0.575)).

CONCLUSIONS

LiFeCRC score based on age and lifestyle data accurately identifies individuals at risk for incident colorectal cancer in European populations and could contribute to improved prevention through motivating lifestyle change at an individual level.

A qualitative research framework for the design of user-centered displays of explanations for machine learning model predictions in healthcare

BACKGROUND

There is an increasing interest in clinical prediction tools that can achieve high prediction accuracy and provide explanations of the factors leading to increased risk of adverse outcomes. However, approaches to explaining complex machine learning (ML) models are rarely informed by end-user needs and user evaluations of model interpretability are lacking in the healthcare domain. We used extended revisions of previously-published theoretical frameworks to propose a framework for the design of user-centered displays of explanations. This new framework served as the basis for qualitative inquiries and design review sessions with critical care nurses and physicians that informed the design of a user-centered explanation display for an ML-based prediction tool.

METHODS

We used our framework to propose explanation displays for predictions from a pediatric intensive care unit (PICU) in-hospital mortality risk model. Proposed displays were based on a model-agnostic, instance-level explanation approach based on feature influence, as determined by Shapley values. Focus group sessions solicited critical care provider feedback on the proposed displays, which were then revised accordingly.

RESULTS

The proposed displays were perceived as useful tools in assessing model predictions. However, specific explanation goals and information needs varied by clinical role and level of predictive modeling knowledge. Providers preferred explanation displays that required less information processing effort and could support the information needs of a variety of users. Providing supporting information to assist in interpretation was seen as critical for fostering provider understanding and acceptance of the predictions and explanations. The user-centered explanation display for the PICU in-hospital mortality risk model incorporated elements from the initial displays along with enhancements suggested by providers.

CONCLUSIONS

We proposed a framework for the design of user-centered displays of explanations for ML models. We used the proposed framework to motivate the design of a user-centered display of an explanation for predictions from a PICU in-hospital mortality risk model. Positive feedback from focus group participants provides preliminary support for the use of model-agnostic, instance-level explanations of feature influence as an approach to understand ML model predictions in healthcare and advances the discussion on how to effectively communicate ML model information to healthcare providers.

Prediction and clinical utility of a contralateral breast cancer risk model

BACKGROUND

Breast cancer survivors are at risk for contralateral breast cancer (CBC), with the consequent burden of further treatment and potentially less favorable prognosis. We aimed to develop and validate a CBC risk prediction model and evaluate its applicability for clinical decision-making.

METHODS

We included data of 132,756 invasive non-metastatic breast cancer patients from 20 studies with 4682 CBC events and a median follow-up of 8.8 years. We developed a multivariable Fine and Gray prediction model (PredictCBC-1A) including patient, primary tumor, and treatment characteristics and BRCA1/2 germline mutation status, accounting for the competing risks of death and distant metastasis. We also developed a model without BRCA1/2 mutation status (PredictCBC-1B) since this information was available for only 6% of patients and is routinely unavailable in the general breast cancer population. Prediction performance was evaluated using calibration and discrimination, calculated by a time-dependent area under the curve (AUC) at 5 and 10 years after diagnosis of primary breast cancer, and an internal-external cross-validation procedure. Decision curve analysis was performed to evaluate the net benefit of the model to quantify clinical utility.

RESULTS

In the multivariable model, BRCA1/2 germline mutation status, family history, and systemic adjuvant treatment showed the strongest associations with CBC risk. The AUC of PredictCBC-1A was 0.63 (95% prediction interval (PI) at 5 years, 0.52-0.74; at 10 years, 0.53-0.72). Calibration-in-the-large was -0.13 (95% PI: -1.62-1.37), and the calibration slope was 0.90 (95% PI: 0.73-1.08). The AUC of Predict-1B at 10 years was 0.59 (95% PI: 0.52-0.66); calibration was slightly lower. Decision curve analysis for preventive contralateral mastectomy showed potential clinical utility of PredictCBC-1A between thresholds of 4-10% 10-year CBC risk for BRCA1/2 mutation carriers and non-carriers.

CONCLUSIONS

We developed a reasonably calibrated model to predict the risk of CBC in women of European-descent; however, prediction accuracy was moderate. Our model shows potential for improved risk counseling, but decision-making regarding contralateral preventive mastectomy, especially in the general breast cancer population where limited information of the mutation status in BRCA1/2 is available, remains challenging.

Nomogram construction for predicting survival of patients with non-small cell lung cancer with malignant pleural or pericardial effusion based on SEER analysis of 10,268 patients

Determining the accurate outcome of patients with non-small cell lung cancer (NSCLC) and malignant pleural effusion (MPE) or malignant pleural pericardial effusion (MPCE) at the initial diagnosis remains a challenge. The aim of the present study was to develop an effective nomogram for individualized estimation of overall survival in these patients. Patients diagnosed between January 2010 and December 2015 were selected from the Surveillance, Epidemiology, and End Results (SEER) database. Age, race, sex, grade, histology, laterality, stage and status of MPE or MPCE at initial diagnosis were included as covariates. Several survival models were created and the performance of each was evaluated. The most effective model was then validated by internal bootstrap resampling and by using an independent external cohort. A nomogram was created based on this survival model and the predictive accuracy of the nomogram was evaluated by calibration plots. Data from 10,268 patients with lung cancer with MPE or MPCE at initial diagnosis were collected. The multivariate analysis with a lognormal model suggested that age, race, sex, histology, stage and status of MPE or MPCE at initial diagnosis were significant independent factors to predict survival. A nomogram was constructed based on the lognormal survival model, which showed the best performance. The concordance index of the survival model in the SEER cohort was 0.736. Both internal and external validation showed an acceptable level of agreement between the nomogram-predicted survival probability and actual survival. The nomogram of the present study based on a large cohort from the SEER database may improve prognostic prediction of patients with NSCLC with MPE or MPCE at initial diagnosis, and allow physicians to make appropriate decisions for disease management of their patients.

Visualising statistical models using dynamic nomograms

Translational Statistics proposes to promote the use of Statistics within research and improve the communication of statistical findings in an accurate and accessible manner to diverse audiences. When statistical models become more complex, it becomes harder to evaluate the role of explanatory variables on the response. For example, the interpretation and communication of the effect of predictors in regression models where interactions or smoothing splines are included can be challenging. Informative graphical representations of statistical models play a critical translational role; static nomograms are one such useful tool to visualise statistical models. In this paper, we propose the use of dynamic nomogram as a translational tool which can accommodate models of increased complexity. In theory, all models appearing in the literature could be accompanied by the corresponding dynamic nomogram to translate models in an informative manner. The R package presented will facilitate this communication for a variety of linear and non-linear models.

Methodological standards for the development and evaluation of clinical prediction rules: a review of the literature

Clinical prediction rules (CPRs) that predict the absolute risk of a clinical condition or future outcome for individual patients are abundant in the medical literature; however, systematic reviews have demonstrated shortcomings in the methodological quality and reporting of prediction studies. To maximise the potential and clinical usefulness of CPRs, they must be rigorously developed and validated, and their impact on clinical practice and patient outcomes must be evaluated. This review aims to present a comprehensive overview of the stages involved in the development, validation and evaluation of CPRs, and to describe in detail the methodological standards required at each stage, illustrated with examples where appropriate. Important features of the study design, statistical analysis, modelling strategy, data collection, performance assessment, CPR presentation and reporting are discussed, in addition to other, often overlooked aspects such as the acceptability, cost-effectiveness and longer-term implementation of CPRs, and their comparison with clinical judgement. Although the development and evaluation of a robust, clinically useful CPR is anything but straightforward, adherence to the plethora of methodological standards, recommendations and frameworks at each stage will assist in the development of a rigorous CPR that has the potential to contribute usefully to clinical practice and decision-making and have a positive impact on patient care.

Functional form estimation using oblique projection matrices for LS-SVM regression models

Kernel regression models have been used as non-parametric methods for fitting experimental data. However, due to their non-parametric nature, they belong to the so-called "black box" models, indicating that the relation between the input variables and the output, depending on the kernel selection, is unknown. In this paper we propose a new methodology to retrieve the relation between each input regressor variable and the output in a least squares support vector machine (LS-SVM) regression model. The method is based on oblique subspace projectors (ObSP), which allows to decouple the influence of input regressors on the output by including the undesired variables in the null space of the projection matrix. Such functional relations are represented by the nonlinear transformation of the input regressors, and their subspaces are estimated using appropriate kernel evaluations. We exploit the properties of ObSP in order to decompose the output of the obtained regression model as a sum of the partial nonlinear contributions and interaction effects of the input variables, we called this methodology Nonlinear ObSP (NObSP). We compare the performance of the proposed algorithm with the component selection and smooth operator (COSSO) for smoothing spline ANOVA models. We use as benchmark 2 toy examples and a real life regression model using the concrete strength dataset from the UCI machine learning repository. We showed that NObSP is able to outperform COSSO, producing stable estimations of the functional relations between the input regressors and the output, without the use of prior-knowledge. This methodology can be used in order to understand the functional relations between the inputs and the output in a regression model, retrieving the physical interpretation of the regression models.

Reaping the Benefits and Avoiding the Risks: Unrealistic Optimism in the Health Domain

People's perceptions of benefits and risks play a key role in their acceptance or rejection of medical interventions, yet these perceptions may be poorly calibrated. This online study with N = 373 adults aged 19-76 years focused on unrealistic optimism in the health domain. Participants indicated how likely they were to experience benefits and risks associated with medical conditions and completed objective and subjective numeracy scales. Participants exhibited optimistic views about the likelihood of experiencing the benefits and the side effects of treatment options described in the scenarios. Objective and subjective numeracy were not associated with more accurate ratings. Moreover, participants' underestimation of the risks was significantly greater than their overestimation of the benefits. From an applied perspective, these results suggest that clinicians may need to ensure that patients do not underestimate risks of medical interventions, and that they convey realistic expectations about the benefits that can be obtained with certain procedures.

Poor performance of clinical prediction models: the harm of commonly applied methods

OBJECTIVE

To evaluate limitations of common statistical modeling approaches in deriving clinical prediction models and explore alternative strategies.

STUDY DESIGN AND SETTING

A previously published model predicted the likelihood of having a mutation in germline DNA mismatch repair genes at the time of diagnosis of colorectal cancer. This model was based on a cohort where 38 mutations were found among 870 participants, with validation in an independent cohort with 35 mutations. The modeling strategy included stepwise selection of predictors from a pool of over 37 candidate predictors and dichotomization of continuous predictors. We simulated this strategy in small subsets of a large contemporary cohort (2,051 mutations among 19,866 participants) and made comparisons to other modeling approaches. All models were evaluated according to bias and discriminative ability (concordance index, c) in independent data.

RESULTS

We found over 50% bias for five of six originally selected predictors, unstable model specification, and poor performance at validation (median c = 0.74). A small validation sample hampered stable assessment of performance. Model prespecification based on external knowledge and using continuous predictors led to better performance (c = 0.836 and c = 0.852 with 38 and 2,051 events respectively).

CONCLUSION

Prediction models perform poorly if based on small numbers of events and developed with common but suboptimal statistical approaches. Alternative modeling strategies to best exploit available predictive information need wider implementation, with collaborative research to increase sample sizes.

The Need for Accurate Risk Prediction Models for Road Mapping, Shared Decision Making and Care Planning for the Elderly with Advanced Chronic Kidney Disease

As people age, chronic kidney disease becomes more common, but it rarely leads to end-stage kidney disease. When it does, the choice between dialysis and conservative care can be daunting, as much depends on life expectancy and personal expectations of medical care. Shared decision making implies adequately informing patients about their options, and facilitating deliberation of the available information, such that decisions are tailored to the individual's values and preferences. Accurate estimations of one's risk of progression to end-stage kidney disease and death with or without dialysis are essential for shared decision making to be effective. Formal risk prediction models can help, provided they are externally validated, well-calibrated and discriminative; include unambiguous and measureable variables; and come with readily applicable equations or scores. Reliable, externally validated risk prediction models for progression of chronic kidney disease to end-stage kidney disease or mortality in frail elderly with or without chronic kidney disease are scant. Within this paper, we discuss a number of promising models, highlighting both the strengths and limitations physicians should understand for using them judiciously, and emphasize the need for external validation over new development for further advancing the field.

Explaining Support Vector Machines: A Color Based Nomogram

Problem setting

Support vector machines (SVMs) are very popular tools for classification, regression and other problems. Due to the large choice of kernels they can be applied with, a large variety of data can be analysed using these tools. Machine learning thanks its popularity to the good performance of the resulting models. However, interpreting the models is far from obvious, especially when non-linear kernels are used. Hence, the methods are used as black boxes. As a consequence, the use of SVMs is less supported in areas where interpretability is important and where people are held responsible for the decisions made by models.

Objective

In this work, we investigate whether SVMs using linear, polynomial and RBF kernels can be explained such that interpretations for model-based decisions can be provided. We further indicate when SVMs can be explained and in which situations interpretation of SVMs is (hitherto) not possible. Here, explainability is defined as the ability to produce the final decision based on a sum of contributions which depend on one single or at most two input variables.

Results

Our experiments on simulated and real-life data show that explainability of an SVM depends on the chosen parameter values (degree of polynomial kernel, width of RBF kernel and regularization constant). When several combinations of parameter values yield the same cross-validation performance, combinations with a lower polynomial degree or a larger kernel width have a higher chance of being explainable.

Conclusions

This work summarizes SVM classifiers obtained with linear, polynomial and RBF kernels in a single plot. Linear and polynomial kernels up to the second degree are represented exactly. For other kernels an indication of the reliability of the approximation is presented. The complete methodology is available as an R package and two apps and a movie are provided to illustrate the possibilities offered by the method.