Risk prediction models for acute kidney injury following major noncardiac surgery: systematic review

BACKGROUND

Acute kidney injury (AKI) is a serious complication of major noncardiac surgery. Risk prediction models for AKI following noncardiac surgery may be useful for identifying high-risk patients to target with prevention strategies.

METHODS

We conducted a systematic review of risk prediction models for AKI following major noncardiac surgery. MEDLINE, EMBASE, BIOSIS Previews and Web of Science were searched for articles that (i) developed or validated a prediction model for AKI following major noncardiac surgery or (ii) assessed the impact of a model for predicting AKI following major noncardiac surgery that has been implemented in a clinical setting.

RESULTS

We identified seven models from six articles that described a risk prediction model for AKI following major noncardiac surgeries. Three studies developed prediction models for AKI requiring renal replacement therapy following liver transplantation, three derived prediction models for AKI based on the Risk, Injury, Failure, Loss of kidney function, End-stage kidney disease (RIFLE) criteria following liver resection and one study developed a prediction model for AKI following major noncardiac surgical procedures. The final models included between 4 and 11 independent variables, and c-statistics ranged from 0.79 to 0.90. None of the models were externally validated.

CONCLUSIONS

Risk prediction models for AKI after major noncardiac surgery are available; however, these models lack validation, studies of clinical implementation and impact analyses. Further research is needed to develop, validate and study the clinical impact of such models before broad clinical uptake.

Development and Validation of a Clinical Risk-Assessment Tool Predictive of All-Cause Mortality

In clinical settings, the diagnosis of medical conditions is often aided by measurement of various serum biomarkers through the use of laboratory tests. These biomarkers provide information about different aspects of a patient's health and overall function of multiple organ systems. We have developed a statistical procedure that condenses the information from a variety of health biomarkers into a composite index, which could be used as a risk score for predicting all-cause mortality. It could also be viewed as a holistic measure of overall physiological health status. This health status metric is computed as a function of standardized values of each biomarker measurement, weighted according to their empirically determined relative strength of association with mortality. The underlying risk model was developed using the biomonitoring and mortality data of a large sample of US residents obtained from the National Health and Nutrition Examination Survey (NHANES) and the National Death Index (NDI). Biomarker concentration levels were standardized using spline-based Cox regression models, and optimization algorithms were used to estimate the weights. The predictive accuracy of the tool was optimized by bootstrap aggregation. We also demonstrate how stacked generalization, a machine learning technique, can be used for further enhancement of the prediction power. The index was shown to be highly predictive of all-cause mortality and long-term outcomes for specific health conditions. It also exhibited a robust association with concurrent chronic conditions, recent hospital utilization, and current health status as assessed by self-rated health.

Prognostic Tools for Early Mortality in Hemorrhagic Stroke: Systematic Review and Meta-Analysis

Background and Purpose

Several risk scores have been developed to predict mortality in intracerebral hemorrhage (ICH). We aimed to systematically determine the performance of published prognostic tools.

Methods

We searched MEDLINE and EMBASE for prognostic models (published between 2004 and April 2014) used in predicting early mortality (<6 months) after ICH. We evaluated the discrimination performance of the tools through a random-effects meta-analysis of the area under the receiver operating characteristic curve (AUC) or c-statistic. We evaluated the following components of the study validity: study design, collection of prognostic variables, treatment pathways, and missing data.

Results

We identified 11 articles (involving 41,555 patients) reporting on the accuracy of 12 different tools for predicting mortality in ICH. Most studies were either retrospective or post-hoc analyses of prospectively collected data; all but one produced validation data. The Hemphill-ICH score had the largest number of validation cohorts (9 studies involving 3,819 patients) within our systematic review and showed good performance in 4 countries, with a pooled AUC of 0.80 [95% confidence interval (CI)=0.77-0.85]. We identified several modified versions of the Hemphill-ICH score, with the ICH-Grading Scale (GS) score appearing to be the most promising variant, with a pooled AUC across four studies of 0.87 (95% CI=0.84-0.90). Subgroup testing found statistically significant differences between the AUCs obtained in studies involving Hemphill-ICH and ICH-GS scores (p=0.01).

Conclusions

Our meta-analysis evaluated the performance of 12 ICH prognostic tools and found greater supporting evidence for 2 models (Hemphill-ICH and ICH-GS), with generally good performance overall.

Indoor tanning and the MC1R genotype: risk prediction for basal cell carcinoma risk in young people

Basal cell carcinoma (BCC) incidence is increasing, particularly in young people, and can be associated with significant morbidity and treatment costs. To identify young individuals at risk of BCC, we assessed existing melanoma or overall skin cancer risk prediction models and built a novel risk prediction model, with a focus on indoor tanning and the melanocortin 1 receptor gene, MC1R. We evaluated logistic regression models among 759 non-Hispanic whites from a case-control study of patients seen between 2006 and 2010 in New Haven, Connecticut. In our data, the adjusted area under the receiver operating characteristic curve (AUC) for a model by Han et al. (Int J Cancer. 2006;119(8):1976-1984) with 7 MC1R variants was 0.72 (95% confidence interval (CI): 0.66, 0.78), while that by Smith et al. (J Clin Oncol. 2012;30(15 suppl):8574) with MC1R and indoor tanning had an AUC of 0.69 (95% CI: 0.63, 0.75). Our base model had greater predictive ability than existing models and was significantly improved when we added ever-indoor tanning, burns from indoor tanning, and MC1R (AUC = 0.77, 95% CI: 0.74, 0.81). Our early-onset BCC risk prediction model incorporating MC1R and indoor tanning extends the work of other skin cancer risk prediction models, emphasizes the value of both genotype and indoor tanning in skin cancer risk prediction in young people, and should be validated with an independent cohort.

Assessing calibration of multinomial risk prediction models

Calibration, that is, whether observed outcomes agree with predicted risks, is important when evaluating risk prediction models. For dichotomous outcomes, several tools exist to assess different aspects of model calibration, such as calibration-in-the-large, logistic recalibration, and (non-)parametric calibration plots. We aim to extend these tools to prediction models for polytomous outcomes. We focus on models developed using multinomial logistic regression (MLR): outcome Y with k categories is predicted using k - 1 equations comparing each category i (i = 2, … ,k) with reference category 1 using a set of predictors, resulting in k - 1 linear predictors. We propose a multinomial logistic recalibration framework that involves an MLR fit where Y is predicted using the k - 1 linear predictors from the prediction model. A non-parametric alternative may use vector splines for the effects of the linear predictors. The parametric and non-parametric frameworks can be used to generate multinomial calibration plots. Further, the parametric framework can be used for the estimation and statistical testing of calibration intercepts and slopes. Two illustrative case studies are presented, one on the diagnosis of malignancy of ovarian tumors and one on residual mass diagnosis in testicular cancer patients treated with cisplatin-based chemotherapy. The risk prediction models were developed on data from 2037 and 544 patients and externally validated on 1107 and 550 patients, respectively. We conclude that calibration tools can be extended to polytomous outcomes. The polytomous calibration plots are particularly informative through the visual summary of the calibration performance.

Development and validation of risk models and molecular diagnostics to permit personalized management of cancer

Despite the advances made in cancer management over the past few decades, improvements in cancer diagnosis and prognosis are still poor, highlighting the need for individualized strategies. Toward this goal, risk prediction models and molecular diagnostic tools have been developed, tailoring each step of risk assessment from diagnosis to treatment and clinical outcomes based on the individual's clinical, epidemiological, and molecular profiles. These approaches hold increasing promise for delivering a new paradigm to maximize the efficiency of cancer surveillance and efficacy of treatment. However, they require stringent study design, methodology development, comprehensive assessment of biomarkers and risk factors, and extensive validation to ensure their overall usefulness for clinical translation. In the current study, the authors conducted a systematic review using breast cancer as an example and provide general guidelines for risk prediction models and molecular diagnostic tools, including development, assessment, and validation.

Impact of correlation on predictive ability of biomarkers

In this paper, we investigate how the correlation structure of independent variables affects the discrimination of risk prediction model. Using multivariate normal data and binary outcome, we prove that zero correlation among predictors is often detrimental for discrimination in a risk prediction model and negatively correlated predictors with positive effect sizes are beneficial. A very high multiple R-squared from regressing the new predictor on the old ones can also be beneficial. As a practical guide to new variable selection, we recommend to select predictors that have negative correlation with the risk score based on the existing variables. This step is easy to implement even when the number of new predictors is large. We illustrate our results by using real-life Framingham data suggesting that the conclusions hold outside of normality. The findings presented in this paper might be useful for preliminary selection of potentially important predictors, especially is situations where the number of predictors is large.

Recalibration of the Gail model for predicting invasive breast cancer risk in Spanish women: a population-based cohort study

The Gail model for predicting the absolute risk of invasive breast cancer has been validated extensively in US populations, but its performance in the international setting remains uncertain. We evaluated the predictive accuracy of the Gail model in 54,649 Spanish women aged 45-68 years who were free of breast cancer at the 1996-1998 baseline mammographic examination in the population-based Navarre Breast Cancer Screening Program. Incident cases of invasive breast cancer and competing deaths were ascertained until the end of 2005 (average follow-up of 7.7 years) through linkage with population-based cancer and mortality registries. The Gail model was tested for calibration and discrimination in its original form and after recalibration to the lower breast cancer incidence and risk factor prevalence in the study cohort, and compared through cross-validation with a Navarre model fully developed from this cohort. The original Gail model overpredicted significantly the 835 cases of invasive breast cancer observed in the cohort (ratio of expected to observed cases 1.46, 95 % CI 1.36-1.56). The recalibrated Gail model was well calibrated overall (expected-to-observed ratio 1.00, 95 % CI 0.94-1.07), but it tended to underestimate risk for women in low-risk quintiles and to overestimate risk in high-risk quintiles (P = 0.01). The Navarre model showed good cross-validated calibration overall (expected-to-observed ratio 0.98, 95 % CI 0.92-1.05) and in different cohort subsets. The Navarre and Gail models had modest cross-validated discrimination indexes of 0.542 (95 % CI 0.521-0.564) and 0.544 (95 % CI 0.523-0.565), respectively. Although the original Gail model cannot be applied directly to populations with different underlying rates of invasive breast cancer, it can readily be recalibrated to provide unbiased estimates of absolute risk in such populations. Nevertheless, its limited discrimination ability at the individual level highlights the need to develop extended models with additional strong risk factors.

Common genetic variants in prostate cancer risk prediction--results from the NCI Breast and Prostate Cancer Cohort Consortium (BPC3)

BACKGROUND

One of the goals of personalized medicine is to generate individual risk profiles that could identify individuals in the population that exhibit high risk. The discovery of more than two-dozen independent single-nucleotide polymorphism markers in prostate cancer has raised the possibility for such risk stratification. In this study, we evaluated the discriminative and predictive ability for prostate cancer risk models incorporating 25 common prostate cancer genetic markers, family history of prostate cancer, and age.

METHODS

We fit a series of risk models and estimated their performance in 7,509 prostate cancer cases and 7,652 controls within the National Cancer Institute Breast and Prostate Cancer Cohort Consortium (BPC3). We also calculated absolute risks based on SEER incidence data.

RESULTS

The best risk model (C-statistic = 0.642) included individual genetic markers and family history of prostate cancer. We observed a decreasing trend in discriminative ability with advancing age (P = 0.009), with highest accuracy in men younger than 60 years (C-statistic = 0.679). The absolute ten-year risk for 50-year-old men with a family history ranged from 1.6% (10th percentile of genetic risk) to 6.7% (90th percentile of genetic risk). For men without family history, the risk ranged from 0.8% (10th percentile) to 3.4% (90th percentile).

CONCLUSIONS

Our results indicate that incorporating genetic information and family history in prostate cancer risk models can be particularly useful for identifying younger men that might benefit from prostate-specific antigen screening.

IMPACT

Although adding genetic risk markers improves model performance, the clinical utility of these genetic risk models is limited.

Applying the Lorenz curve to disease risk to optimize health benefits under cost constraints

This paper shows how the Lorenz curve can be used, together with models of disease risk, to allocate scarce resources so as to optimize a health benefit. Consider the example of breast cancer mortality. If there were sufficient resources to provide all women with mammograms, a certain maximal number of lives could be saved. Suppose, however, that only a fraction of that amount of money is available for prevention activities. Suppose that a questionnaire could be given to assess a woman's risk of dying of breast cancer. Depending on the amount of money available, on the ratio of the cost of a questionnaire to the cost of a mammogram, and on the Lorenz curve of the distribution of risks of breast cancer mortality, I calculate the proportion of women who should be given questionnaires, the proportion of women given the questionnaires who should be given mammograms because they have high risks, and the proportion of women not given questionnaires who should be assigned to receive mammograms at random so as to maximize the number of lives saved.