A systematic review of breast cancer incidence risk prediction models with meta-analysis of their performance

Application of deep neural survival networks to the development of risk prediction models for diabetes mellitus, hypertension, and dyslipidemia

OBJECTIVES

: Although numerous risk prediction models have been proposed, few such models have been developed using neural network-based survival analysis. We developed risk prediction models for three cardiovascular disease risk factors (diabetes mellitus, hypertension, and dyslipidemia) among a working-age population in Japan using DeepSurv, a deep feed-forward neural network.

METHODS

: Data were obtained from the Japan Epidemiology Collaboration on Occupational Health Study. A total of 51 258, 44 197, and 31 452 individuals were included in the development of risk models for diabetes mellitus, hypertension, and dyslipidemia, respectively; two-thirds of whom were used to develop prediction models, and the rest were used to validate the models. We compared the performances of DeepSurv-based models with those of prediction models based on the Cox proportional hazards model.

RESULTS

: The area under the receiver-operating characteristic curve was 0.878 [95% confidence interval (CI) = 0.864-0.892] for diabetes mellitus, 0.835 (95% CI = 0.826-0.845) for hypertension, and 0.826 (95% CI = 0.817-0.835) for dyslipidemia. Compared with the Cox proportional hazards-based models, the DeepSurv-based models had better reclassification performance [diabetes mellitus: net reclassification improvement (NRI) = 0.474, P  ≤ 0.001; hypertension: NRI = 0.194, P  ≤ 0.001; dyslipidemia: NRI = 0.397, P  ≤ 0.001] and discrimination performance [diabetes mellitus: integrated discrimination improvement (IDI) = 0.013, P  ≤ 0.001; hypertension: IDI = 0.007, P  ≤ 0.001; and dyslipidemia: IDI = 0.043, P  ≤ 0.001].

CONCLUSION

: This study suggests that DeepSurv has the potential to improve the performance of risk prediction models for cardiovascular disease risk factors.

An advanced machine learning method for simultaneous breast cancer risk prediction and risk ranking in Chinese population: A prospective cohort and modeling study

BACKGROUND

Breast cancer (BC) risk-stratification tools for Asian women that are highly accurate and can provide improved interpretation ability are lacking. We aimed to develop risk-stratification models to predict long- and short-term BC risk among Chinese women and to simultaneously rank potential non-experimental risk factors.

METHODS

The Breast Cancer Cohort Study in Chinese Women, a large ongoing prospective dynamic cohort study, includes 122,058 women aged 25-70 years from the eastern part of China. We developed multiple machine-learning risk prediction models using parametric models (penalized logistic regression, bootstrap, and ensemble learning), which were the short-term ensemble penalized logistic regression (EPLR) risk prediction model and the ensemble penalized long-term (EPLT) risk prediction model to estimate BC risk. The models were assessed based on calibration and discrimination, and following this assessment, they were externally validated in new study participants from 2017 to 2020.

RESULTS

The AUC values of the short-term EPLR risk prediction model were 0.800 for the internal validation and 0.751 for the external validation set. For the long-term EPLT risk prediction model, the area under the receiver operating characteristic curve was 0.692 and 0.760 in internal and external validations, respectively. The net reclassification improvement index of the EPLT relative to the Gail and the Han Chinese Breast Cancer Prediction Model (HCBCP) models for external validation was 0.193 and 0.233, respectively, indicating that the EPLT model has higher classification accuracy.

CONCLUSIONS

We developed the EPLR and EPLT models to screen populations with a high risk of developing BC. These can serve as useful tools to aid in risk-stratified screening and BC prevention.

Cumulative Genetic Score and C9orf72 Repeat Status Independently Contribute to Amyotrophic Lateral Sclerosis Risk in 2 Case-Control Studies

Background and Objectives

Most patients with amyotrophic lateral sclerosis (ALS) lack a monogenic mutation. This study evaluates ALS cumulative genetic risk in an independent Michigan and Spanish replication cohort using polygenic scores.

Methods

Participant samples from University of Michigan were genotyped and assayed for the chromosome 9 open reading frame 72 hexanucleotide expansion. Final cohort size was 219 ALS and 223 healthy controls after genotyping and participant filtering. Polygenic scores excluding the C9 region were generated using an independent ALS genome-wide association study (20,806 cases, 59,804 controls). Adjusted logistic regression and receiver operating characteristic curves evaluated the association and classification between polygenic scores and ALS status, respectively. Population attributable fractions and pathway analyses were conducted. An independent Spanish study sample (548 cases, 2,756 controls) was used for replication.

Results

Polygenic scores constructed from 275 single-nucleotide variation (SNV) had the best model fit in the Michigan cohort. An SD increase in ALS polygenic score associated with 1.28 (95% CI 1.04-1.57) times higher odds of ALS with area under the curve of 0.663 vs a model without the ALS polygenic score (p value = 1 × 10-6). The population attributable fraction of the highest 20th percentile of ALS polygenic scores, relative to the lowest 80th percentile, was 4.1% of ALS cases. Genes annotated to this polygenic score enriched for important ALS pathomechanisms. Meta-analysis with the Spanish study, using a harmonized 132 single nucleotide variation polygenic score, yielded similar logistic regression findings (odds ratio: 1.13, 95% CI 1.04-1.23).

Discussion

ALS polygenic scores can account for cumulative genetic risk in populations and reflect disease-relevant pathways. If further validated, this polygenic score will inform future ALS risk models.

Understanding the contribution of lifestyle in breast cancer risk prediction: a systematic review of models applicable to Europe

BACKGROUND

Breast cancer (BC) is a significant health concern among European women, with the highest prevalence rates among all cancers. Existing BC prediction models account for major risks such as hereditary, hormonal and reproductive factors, but research suggests that adherence to a healthy lifestyle can reduce the risk of developing BC to some extent. Understanding the influence and predictive role of lifestyle variables in current risk prediction models could help identify actionable, modifiable, targets among high-risk population groups.

PURPOSE

To systematically review population-based BC risk prediction models applicable to European populations and identify lifestyle predictors and their corresponding parameter values for a better understanding of their relative contribution to the prediction of incident BC.

METHODS

A systematic review was conducted in PubMed, Embase and Web of Science from January 2000 to August 2021. Risk prediction models were included if (i) developed and/or validated in adult cancer-free women in Europe, (ii) based on easily ascertained information, and (iii) reported models' final predictors. To investigate further the comparability of lifestyle predictors across models, estimates were standardised into risk ratios and visualised using forest plots.

RESULTS

From a total of 49 studies, 33 models were developed and 22 different existing models, mostly from Gail (22 studies) and Tyrer-Cuzick and co-workers (12 studies) were validated or modified for European populations. Family history of BC was the most frequently included predictor (31 models), while body mass index (BMI) and alcohol consumption (26 and 21 models, respectively) were the lifestyle predictors most often included, followed by smoking and physical activity (7 and 6 models respectively). Overall, for lifestyle predictors, their modest predictive contribution was greater for riskier lifestyle levels, though highly variable model estimates across different models.

CONCLUSIONS

Given the increasing BC incidence rates in Europe, risk models utilising readily available risk factors could greatly aid in widening the population coverage of screening efforts, while the addition of lifestyle factors could help improving model performance and serve as intervention targets of prevention programmes.

Genetic and lifestyle factors for breast cancer risk assessment in Southeast China

BACKGROUND

Despite the rising incidence and mortality of breast cancer among women in China, there are currently few predictive models for breast cancer in the Chinese population and with low accuracy. This study aimed to identify major genetic and life-style risk factors in a Chinese population for potential application in risk assessment models.

METHODS

A case-control study in southeast China was conducted including 1321 breast cancer patients and 2045 controls during 2013-2016, in which the data were randomly divided into a training set and a test set on a 7:3 scale. The association between genetic and life-style factors and breast cancer was examined using logistic regression models. Using AUC curves, we also compared the performance of the logistic model to machine learning models, namely LASSO regression model and support vector machine (SVM), and the scores calculated from CKB, Gail and Tyrer-Cuzick models in the test set.

RESULTS

Among all factors considered, the best model was achieved when polygenetic risk score, lifestyle, and reproductive factors were considered jointly in the logistic regression model (AUC = 0.73; 95% CI: 0.70-0.77). The models created in this study performed better than those using scores calculated from the CKB, Gail, and Tyrer-Cuzick models. However, the logistic model and machine learning models did not significantly differ from one another.

CONCLUSION

In summary, we have found genetic and lifestyle risk predictors for breast cancer with moderate discrimination, which might provide reference for breast cancer screening in southeast China. Further population-based studies are needed to validate the model for future applications in personalized breast cancer screening programs.

Predicting suicide risk in 137,112 people with severe mental illness in Finland: external validation of the Oxford Mental Illness and Suicide tool (OxMIS)

Oxford Mental Illness and Suicide tool (OxMIS) is a standardised, scalable, and transparent instrument for suicide risk assessment in people with severe mental illness (SMI) based on 17 sociodemographic, criminal history, familial, and clinical risk factors. However, alongside most prediction models in psychiatry, external validations are currently lacking. We utilised a Finnish population sample of all persons diagnosed by mental health services with SMI (schizophrenia-spectrum and bipolar disorders) between 1996 and 2017 (n = 137,112). To evaluate the performance of OxMIS, we initially calculated the predicted 12-month suicide risk for each individual by weighting risk factors by effect sizes reported in the original OxMIS prediction model and converted to a probability. This probability was then used to assess the discrimination and calibration of the OxMIS model in this external sample. Within a year of assessment, 1.1% of people with SMI (n = 1475) had died by suicide. The overall discrimination of the tool was good, with an area under the curve of 0.70 (95% confidence interval: 0.69-0.71). The model initially overestimated suicide risks in those with elevated predicted risks of >5% over 12 months (Harrell's E_max = 0.114), which applied to 1.3% (n = 1780) of the cohort. However, when we used a 5% maximum predicted suicide risk threshold as is recommended clinically, the calibration was excellent (ICI = 0.002; E_max = 0.005). Validating clinical prediction tools using routinely collected data can address research gaps in prediction psychiatry and is a necessary step to translating such models into clinical practice.

Barriers to Implementation of Breast Cancer Risk Assessment: The Health Care Team Perspective

PURPOSE

To assess health care professionals' perceptions of barriers to the utilization of breast cancer risk assessment tools in the public health setting through a series of one-on-one interviews with health care team members.

METHODS

We conducted a cross-sectional qualitative study consisting of one-on-one semistructured telephone interviews with health care team members in the public health setting in the state of Tennessee between May 2020 and October 2020. An iterative inductive-deductive approach was used for qualitative analysis of interview data, resulting in the development of a conceptual framework to depict influences of provider behavior in the utilization of breast cancer risk assessment.

RESULTS

A total of 24 interviews were completed, and a framework of influences of provider behavior in the utilization of breast cancer risk assessment was developed. Participants identified barriers to the utilization of breast cancer risk assessment (knowledge and understanding of risk assessment tools, workflow challenges, and availability of personnel); patient-level barriers as perceived by health care team members (psychological, economic, educational, and environmental); and strategies to increase the utilization of breast cancer risk assessment at the provider level (leadership buy-in, training, supportive policies, and incentives) and patient level (improved communication and better understanding of patients' perceived cancer risk and severity of cancer).

CONCLUSIONS

Understanding barriers to implementation of breast cancer risk assessment and strategies to overcome these barriers as perceived by health care team members offers an opportunity to improve implementation of risk assessment and to identify a racially, geographically, and socioeconomically diverse population of young women at high risk for breast cancer.

Development and validation of a nomogram to predict Chinese breast cancer risk based on clinical serum biomarkers

Background: This study investigated and compared clinical serum biomarkers and developed a diagnostic nomogram for breast cancer. Methods: A total of 1224 breast cancer and 1280 healthy controls were enrolled. Univariate and multivariate analyses were performed to identify factors and a nomogram was developed. Discrimination, accuracy and clinical utility values were evaluated by receiver operating characteristic, Hosmer-Lemeshow, calibration plots, decision curve analysis and clinical impact plots. Results: carcinoembryonic antigen, CA125, CA153, lymphocyte-to-monocyte ratio, platelet-to-lymphocyte ratio, fibrinogen and platelet distributing width were effectively identified to predict breast cancer. The nomogram showed the area under the curve of 0.708 and 0.710 in the training and validation set. Calibration plots, Hosmer-Lemeshow, decision curve analysis and clinical impact plots confirmed great accuracy and clinical utility. Conclusion: We developed and validated a nomogram that is effectively used for risk prediction of Chinese breast cancer.

Black Women's Perspectives on Breast Cancer Risk Assessment

PURPOSE

The aim of this study was to gather the perspectives of Black women on breast cancer risk assessment through a series of one-on-one interviews.

METHODS

The authors conducted a cross-sectional qualitative study consisting of one-on-one semistructured telephone interviews with Black women in Tennessee between September 2020 and November 2020. Guided by the Health Belief Model, qualitative analysis of interview data was performed in an iterative inductive and deductive approach and resulted in the development of a conceptual framework to depict influences on a woman's decision to engage with breast cancer risk assessment.

RESULTS

A total of 37 interviews were completed, and a framework of influences on a woman's decision to engage in breast cancer risk assessment was developed. Study participants identified several emerging themes regarding women's perspectives on breast cancer risk assessment and potential influences on women's decisions to engage with risk assessment. Much of women's decision context was based on risk appraisal (perceived severity of cancer and susceptibility of cancer), emotions (fear and trust), and perceived risks and benefits of having risk assessment. The decision was further influenced by modifiers such as communication, the risk assessment protocol, access to health care, knowledge, and health status. Perceived challenges to follow-up if identified as high risk also influenced women's decisions to pursue risk assessment.

CONCLUSIONS

Black women in this study identified several barriers to engagement with breast cancer risk assessment. Efforts to overcome these barriers and increase the use of breast cancer risk assessment can potentially serve as a catalyst to address existing breast cancer disparities. Continued work is needed to develop patient-centric strategies to overcome identified barriers.

Breast Cancer Risk Assessment Tools for Stratifying Women into Risk Groups: A Systematic Review

BACKGROUND

The benefits and harms of breast screening may be better balanced through a risk-stratified approach. We conducted a systematic review assessing the accuracy of questionnaire-based risk assessment tools for this purpose.

METHODS

Population: asymptomatic women aged ≥40 years; Intervention: questionnaire-based risk assessment tool (incorporating breast density and polygenic risk where available); Comparison: different tool applied to the same population; Primary outcome: breast cancer incidence; Scope: external validation studies identified from databases including Medline and Embase (period 1 January 2008-20 July 2021). We assessed calibration (goodness-of-fit) between expected and observed cancers and compared observed cancer rates by risk group. Risk of bias was assessed with PROBAST.

RESULTS

Of 5124 records, 13 were included examining 11 tools across 15 cohorts. The Gail tool was most represented (n = 11), followed by Tyrer-Cuzick (n = 5), BRCAPRO and iCARE-Lit (n = 3). No tool was consistently well-calibrated across multiple studies and breast density or polygenic risk scores did not improve calibration. Most tools identified a risk group with higher rates of observed cancers, but few tools identified lower-risk groups across different settings. All tools demonstrated a high risk of bias.

CONCLUSION

Some risk tools can identify groups of women at higher or lower breast cancer risk, but this is highly dependent on the setting and population.

Development of an Artificial Intelligence-Based Breast Cancer Detection Model by Combining Mammograms and Medical Health Records

BACKGROUND

Artificial intelligence (AI)-based computational models that analyze breast cancer have been developed for decades. The present study was implemented to investigate the accuracy and efficiency of combined mammography images and clinical records for breast cancer detection using machine learning and deep learning classifiers.

METHODS

This study was verified using 731 images from 357 women who underwent at least one mammogram and had clinical records for at least six months before mammography. The model was trained on mammograms and clinical variables to discriminate benign and malignant lesions. Multiple pre-trained deep CNN models to detect cancer in mammograms, including X-ception, VGG16, ResNet-v2, ResNet50, and CNN3 were employed. Machine learning models were constructed using k-nearest neighbor (KNN), support vector machine (SVM), random forest (RF), Artificial Neural Network (ANN), and gradient boosting machine (GBM) in the clinical dataset.

RESULTS

The detection performance obtained an accuracy of 84.5% with a specificity of 78.1% at a sensitivity of 89.7% and an AUC of 0.88. When trained on mammography image data alone, the result achieved a slightly lower score than the combined model (accuracy, 72.5% vs. 84.5%, respectively).

CONCLUSIONS

A breast cancer-detection model combining machine learning and deep learning models was performed in this study with a satisfactory result, and this model has potential clinical applications.

Screening Programs for Breast Cancer: Toward Individualized, Risk-Adapted Strategies of Early Detection

Early detection of breast cancer (BC) comprises two approaches: screening of asymptomatic women in a specified target population at risk (usually a target age range for women at average risk), and early diagnosis for women with BC signs and symptoms. Screening for BC is a key health intervention for early detection. While population-based screening programs have been implemented for age-selected women, the pivotal clinical trials have not addressed the global utility nor the improvement of screening performance by utilizing more refined parameters for patient eligibility, such as individualized risk stratification. In addition, with the exception of the subset of women known to carry germline pathogenetic mutations in (high- or moderately-penetrant) cancer predisposition genes, such as BRCA1 and BRCA2, there has been less success in outreach and service provision for the unaffected relatives of women found to carry a high-risk mutation (i.e., "cascade testing") as it is in these individuals for whom such actionable information can result in cancers (and/or cancer deaths) being averted. Moreover, even in the absence of clinical cancer genetics services, as is the case for the immediate and at least near-term in most countries globally, the capacity to stratify the risk of an individual to develop BC has existed for many years, is available for free online at various sites/platforms, and is increasingly being validated for non-Caucasian populations. Ultimately, a precision approach to BC screening is largely missing. In the present chapter, we aim to address the concept of risk-adapted screening of BC, in multiple facets, and understand if there is a value in the implementation of adapted screening strategies in selected women, outside the established screening prescriptions, in the terms of age-range, screening modality and schedules of imaging.

Prediction of breast cancer risk among women of the Mariana Islands: the BRISK retrospective case-control study

OBJECTIVES

To develop a breast cancer risk prediction model for Chamorro and Filipino women of the Mariana Islands and compare its performance to that of the Breast Cancer Risk Assessment Tool (BCRAT).

DESIGN

Case-control study.

SETTING

Clinics/facilities and other community-based settings on Guam and Saipan (Northern Mariana Islands).

PARTICIPANTS

245 women (87 breast cancer cases and 158 controls) of Chamorro or Filipino ethnicity, age 25-80 years, with no prior history of cancer (other than skin cancer), residing on Guam or Saipan for at least 5 years.

PRIMARY AND SECONDARY OUTCOME MEASURES

Breast cancer risk models were constructed using combinations of exposures previously identified to affect breast cancer risk in this population, population breast cancer incidence rates and all-cause mortality rates for Guam.

RESULTS

Models using ethnic-specific relative risks performed better than those with relative risks estimated from all women. The model with the best performance among both ethnicities (the Breast Cancer Risk Model (BRISK) model; area under the receiver operating characteristic curve (AUC): 0.64 and 0.67 among Chamorros and Filipinos, respectively) included age at menarche, age at first live birth, number of relatives with breast cancer and waist circumference. The 10-year breast cancer risk predicted by the BRISK model was 1.28% for Chamorros and 0.89% for Filipinos. Performance of the BCRAT was modest among both Chamorros (AUC: 0.60) and Filipinos (AUC: 0.55), possibly due to incomplete information on BCRAT risk factors.

CONCLUSIONS

The ability to develop breast cancer risk models for Mariana Islands women is constrained by the small population size and limited availability of health services and data. Nonetheless, we have demonstrated that breast cancer risk prediction models with adequate discriminatory performance can be built for small populations such as in the Mariana Islands. Anthropometry, in particular waist circumference, was important for estimating breast cancer risk in this population.

Postpartum depression: a developed and validated model predicting individual risk in new mothers

Postpartum depression (PPD) is a serious condition associated with potentially tragic outcomes, and in an ideal world PPDs should be prevented. Risk prediction models have been developed in psychiatry estimating an individual's probability of developing a specific condition, and recently a few models have also emerged within the field of PPD research, although none are implemented in clinical care. For the present study we aimed to develop and validate a prediction model to assess individualized risk of PPD and provide a tentative template for individualized risk calculation offering opportunities for additional external validation of this tool. Danish population registers served as our data sources and PPD was defined as recorded contact to a psychiatric treatment facility (ICD-10 code DF32-33) or redeemed antidepressant prescriptions (ATC code N06A), resulting in a sample of 6,402 PPD cases (development sample) and 2,379 (validation sample). Candidate predictors covered background information including cohabitating status, age, education, and previous psychiatric episodes in index mother (Core model), additional variables related to pregnancy and childbirth (Extended model), and further health information about the mother and her family (Extended+ model). Results indicated our recalibrated Extended model with 14 variables achieved highest performance with satisfying calibration and discrimination. Previous psychiatric history, maternal age, low education, and hyperemesis gravidarum were the most important predictors. Moving forward, external validation of the model represents the next step, while considering who will benefit from preventive PPD interventions, as well as considering potential consequences from false positive and negative test results, defined through different threshold values.

Identifying women with increased risk of breast cancer and implementing risk-reducing strategies and supplemental imaging

Breast cancer (BC) is the second most common cancer in women, affecting 1 in 8 women in the United States (12.5%) in their lifetime. However, some women have a higher lifetime risk of BC because of genetic and lifestyle factors, mammographic breast density, and reproductive and hormonal factors. Because BC risk is variable, screening and prevention strategies should be individualized after considering patient-specific risk factors. Thus, health care professionals need to be able to assess risk profiles, identify high-risk women, and individualize screening and prevention strategies through a shared decision-making process. In this article, we review the risk factors for BC, risk-assessment models that identify high-risk patients, and preventive medications and lifestyle modifications that may decrease risk. We also discuss the benefits and limitations of various supplemental screening methods.

Reporting of prognostic clinical prediction models based on machine learning methods in oncology needs to be improved

OBJECTIVE

Evaluate the completeness of reporting of prognostic prediction models developed using machine learning methods in the field of oncology.

STUDY DESIGN AND SETTING

We conducted a systematic review, searching the MEDLINE and Embase databases between 01/01/2019 and 05/09/2019, for non-imaging studies developing a prognostic clinical prediction model using machine learning methods (as defined by primary study authors) in oncology. We used the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement to assess the reporting quality of included publications. We described overall reporting adherence of included publications and by each section of TRIPOD.

RESULTS

Sixty-two publications met the inclusion criteria. 48 were development studies and 14 were development with validation studies. 152 models were developed across all publications. Median adherence to TRIPOD reporting items was 41% [range: 10%-67%] and at least 50% adherence was found in 19% (n=12/62) of publications. Adherence was lower in development only studies (median: 38% [range: 10%-67%]); and higher in development with validation studies (median: 49% [range: 33%-59%]).

CONCLUSION

Reporting of clinical prediction models using machine learning in oncology is poor and needs urgent improvement, so readers and stakeholders can appraise the study methods, understand study findings, and reduce research waste.

Distribution of Estimated Lifetime Breast Cancer Risk Among Women Undergoing Screening Mammography

OBJECTIVE. Supplemental screening breast MRI is recommended for women with an estimated lifetime risk of breast cancer of greater than 20-25%. The performance of risk prediction models varies for each individual and across groups of women. The present study investigates the concordance of three breast cancer risk prediction models among women presenting for screening mammography. SUBJECTS AND METHODS. In this prospective study, we calculated the estimated lifetime risk of breast cancer using the modified Gail, Tyrer-Cuzick version 7, and BRCAPRO models for each woman who presented for screening mammography. Per American Cancer Society guidelines, for each woman the risk was categorized as less than 20% or 20% or greater as well as less than 25% or 25% or greater with use of each model. Venn diagrams were constructed to evaluate concordance across models. The McNemar test was used to test differences in risk group allocations between models, with p ≤ .05 considered to denote statistical significance. RESULTS. Of 3503 screening mammography patients who underwent risk stratification, 3219 (91.9%) were eligible for risk estimation using all three models. Using at least one model, 440 (13.7%) women had a lifetime risk of 20% or greater, including 390 women (12.1%) according to the Tyrer-Cuzick version 7 model, 18 (0.6%) according to the BRCAPRO model, and 141 (4.4%) according to the modified Gail model. Six women (0.2%) had a risk of 20% or greater according to all three models. Women were significantly more likely to be classified as having a high lifetime breast cancer risk by the Tyrer-Cuzick version 7 model compared with the modified Gail model, with thresholds of 20% or greater (odds ratio, 6.4; 95% CI, 4.7-8.7) or 25% or greater (odds ratio, 7.4; 95% CI, 4.7-11.9) used for both models. CONCLUSION. To identify women with a high lifetime breast cancer risk, practices should use estimates of lifetime breast cancer risk derived from multiple risk prediction models.

Prediction of Incident Cancers in the Lifelines Population-Based Cohort

Simple Summary

The accurate prediction of incident cancers could be relevant to understanding and reducing cancer incidence. The aim of this study was to develop machine learning (ML) models that could predict an incident diagnosis of cancer. Data were available for 116,188 cancer-free participants and 4232 incident cancer cases. The main outcome was an incident cancer (excluding skin cancer) during follow-up assessment in a population-based cohort. The performance of three ML algorithms was evaluated using supervised binary classification to identify incident cancers among participants. An overall area under the receiver operator curve (AUC) < 0.75 was obtained; the highest AUC was for prostate cancer AUC > 0.80. Linear and non-linear ML algorithms including socioeconomic, lifestyle, and clinical variables produced a moderate predictive performance of incident cancers in the Lifelines cohort.

Abstract

Cancer incidence is rising, and accurate prediction of incident cancers could be relevant to understanding and reducing cancer incidence. The aim of this study was to develop machine learning (ML) models that could predict an incident diagnosis of cancer. Participants without any history of cancer within the Lifelines population-based cohort were followed for a median of 7 years. Data were available for 116,188 cancer-free participants and 4232 incident cancer cases. At baseline, socioeconomic, lifestyle, and clinical variables were assessed. The main outcome was an incident cancer during follow-up (excluding skin cancer), based on linkage with the national pathology registry. The performance of three ML algorithms was evaluated using supervised binary classification to identify incident cancers among participants. Elastic net regularization and Gini index were used for variables selection. An overall area under the receiver operator curve (AUC) <0.75 was obtained, the highest AUC value was for prostate cancer (random forest AUC = 0.82 (95% CI 0.77–0.87), logistic regression AUC = 0.81 (95% CI 0.76–0.86), and support vector machines AUC = 0.83 (95% CI 0.78–0.88), respectively); age was the most important predictor in these models. Linear and non-linear ML algorithms including socioeconomic, lifestyle, and clinical variables produced a moderate predictive performance of incident cancers in the Lifelines cohort.

Acceptability and potential impact on uptake of using different risk stratification approaches to determine eligibility for screening: A population-based survey

BACKGROUND

Using risk stratification approaches to determine eligibility has the potential to improve efficiency of screening.

OBJECTIVES

To compare the public acceptability and potential impact on uptake of using different approaches to determine eligibility for screening.

DESIGN

An online population-based survey of 668 adults in the UK aged 45-79 including a series of scenarios in the context of a potential kidney cancer screening programme in which eligibility was determined by age, sex, age and sex combined, a simple risk score (age, sex, body mass index, smoking status), a complex risk score additionally incorporating family history and lifestyle, or a genetic risk score.

OUTCOME MEASURES

We used multi-level ordinal logistic regression to compare acceptability and potential uptake within individuals and multivariable ordinal logistic regression differences between individuals.

RESULTS

Using sex, age and sex, or the simple risk score were less acceptable than age (P < .0001). All approaches were less acceptable to women than men. Over 70% were comfortable waiting until they were older if the complex risk score or genetics indicated a low risk. If told they were high risk, 85% would be more likely to take up screening. Being told they were low risk had no overall influence on uptake.

CONCLUSIONS

Varying the starting age of screening based on estimated risk from models incorporating phenotypic or genetic risk factors would be acceptable to most individuals and may increase uptake.

PATIENT OR PUBLIC CONTRIBUTION

Two members of the public contributed to the development of the survey and have commented on this paper.

Assessing Risk of Breast Cancer: A Review of Risk Prediction Models

Accurate and individualized breast cancer risk assessment can be used to guide personalized screening and prevention recommendations. Existing risk prediction models use genetic and nongenetic risk factors to provide an estimate of a woman's breast cancer risk and/or the likelihood that she has a BRCA1 or BRCA2 mutation. Each model is best suited for specific clinical scenarios and may have limited applicability in certain types of patients. For example, the Breast Cancer Risk Assessment Tool, which identifies women who would benefit from chemoprevention, is readily accessible and user-friendly but cannot be used in women under 35 years of age or those with prior breast cancer or lobular carcinoma in situ. Emerging research on deep learning-based artificial intelligence (AI) models suggests that mammographic images contain risk indicators that could be used to strengthen existing risk prediction models. This article reviews breast cancer risk factors, describes the appropriate use, strengths, and limitations of each risk prediction model, and discusses the emerging role of AI for risk assessment.

Discriminatory Accuracy of the Gail Model for Breast Cancer Risk Assessment among Iranian Women

Background:

The Gail model is the most well-known tool for breast cancer risk assessment worldwide. Although it was validated in various Western populations, inconsistent results were reported from Asian populations. We used data from a large case-control study and evaluated the discriminatory accuracy of the Gail model for breast cancer risk assessment among the Iranian female population.

Methods:

We used data from 942 breast cancer patients and 975 healthy controls at the Cancer Institute of Iran, Tehran, Iran, in 2016. We refitted the Gail model to our case-control data (the IR-Gail model). We compared the discriminatory power of the IR-Gail with the original Gail model, using ROC curve analyses and estimation of the area under the ROC curve (AUC).

Results:

Except for the history of biopsies that showed an extremely high relative risk (OR=9.1), the observed ORs were similar to the estimates observed in Gail’s study. Incidence rates of breast cancer were extremely lower in Iran than in the USA, leading to a lower average absolute risk among the Iranian population (2.78, ±SD 2.45). The AUC was significantly improved after refitting the model, but it remained modest (0.636 vs. 0.627, ΔAUC = 0.009, bootstrapped P=0.008). We reported that the cut-point of 1.67 suggested in the Gail study did not discriminate between breast cancer patients and controls among the Iranian female population.

Conclusion:

Although the coefficients from the local study improved the discriminatory accuracy of the model, it remained modest. Cohort studies are warranted to evaluate the validity of the model for Iranian women.

Gastric Cancer Risk Prediction Using an Epidemiological Risk Assessment Model and Polygenic Risk Score

Simple Summary

Risk prediction models incorporate various established risk factors to estimate individual risk specifically in cancer. These models additionally include biological or genetic risk factors to assess cancer risk more accurately. The polygenic risk score (PRS) combines the effects of multiple single-nucleotide polymorphisms (SNPs) that are associated with disease; its discrimination ability was assessed both alone and when used in combination with conventional risk prediction models. As few studies have evaluated the combination of genetic variants to identify high risk population of gastric cancer (GC), and we examined the performance of a GC risk assessment model in combination with SNPs as a PRS in consideration of Helicobacter pylori (H. pylori) infection status. Such a combination improves the identification of a GC-susceptible population among people with H. pylori infection.

Abstract

We investigated the performance of a gastric cancer (GC) risk assessment model in combination with single-nucleotide polymorphisms (SNPs) as a polygenic risk score (PRS) in consideration of Helicobacter pylori (H. pylori) infection status. Six SNPs identified from genome-wide association studies and a marginal association with GC in the study population were included in the PRS. Discrimination of the GC risk assessment model, PRS, and the combination of the two (PRS-GCS) were examined regarding incremental risk and the area under the receiver operating characteristic curve (AUC), with grouping according to H. pylori infection status. The GC risk assessment model score showed an association with GC, irrespective of H. pylori infection. Conversely, the PRS exhibited an association only for those with H. pylori infection. The PRS did not discriminate GC in those without H. pylori infection, whereas the GC risk assessment model showed a modest discrimination. Among individuals with H. pylori infection, discrimination by the GC risk assessment model and the PRS were comparable, with the PRS-GCS combination resulting in an increase in the AUC of 3%. In addition, the PRS-GCS classified more patients and fewer controls at the highest score quintile in those with H. pylori infection. Overall, the PRS-GCS improved the identification of a GC-susceptible population of people with H. pylori infection. In those without H. pylori infection, the GC risk assessment model was better at identifying the high-risk group.

Novel approach to meta-analysis of tests and clinical prediction rules with three or more risk categories

OBJECTIVE

Multichotomous tests have three or more outcome or risk categories, and can provide richer information and a better fit with clinical decision-making than dichotomous tests. Our objective is to present a fully developed approach to the meta-analysis of multichotomous clinical prediction rules (CPRs) and tests, including meta-analysis of stratum specific likelihood ratios.

STUDY DESIGN

We have developed a novel approach to the meta-analysis of likelihood ratios for multichotomous tests that avoids the need to dichotomise outcome categories, and demonstrate its application to a sample CPR. We also review previously reported approaches to the meta-analysis of the area under the receiver operating characteristic curve (AUROCC) and meta-analysis of a measure of calibration (observed:expected) for multichotomous tests or CPRs.

RESULTS

Using data from 10 studies of the Cancer of the Prostate Risk Assessment (CAPRA) risk score for prostate cancer recurrence, we calculated summary estimates of the likelihood ratios for low, moderate and high risk groups of 0.40 (95% CI 0.32 to 0.49), 1.24 (95% CI 0.99 to 1.55) and 4.47 (95% CI 3.21 to 6.23), respectively. Applying the summary estimates of the likelihood ratios for each risk group to the overall prevalence of cancer recurrence in a population allows one to estimate the likelihood of recurrence for each risk group in that population.

CONCLUSION

An approach to meta-analysis of multichotomous tests or CPRs is presented. A spreadsheet for data preparation and code for R and Stata are provided for other researchers to download and use. Combined with summary estimates of the AUROCC and calibration, this is a comprehensive strategy for meta-analysis of multichotomous tests and CPRs.

The accuracy of breast cancer risk self-assessment does not correlate with knowledge about breast cancer and knowledge and attitudes towards primary chemoprevention

The increase of breast cancer (BC) incidence has drawn attention to BC risk as means of reducing mortality and morbidity of the disease. The aim of this study was to determine the accuracy of BC risk perception, evaluate factors that affect risk perception and assess the correlation between BC risk perception and attitudes towards BC chemoprevention. A cross-sectional study included total of 258 women with average and high-risk for BC according to the Breast Cancer Risk Assessment Tool (BCRAT). All data were collected by face-to-face interview by three trained 6th year medical school students using a 54-item questionnaire. Each participant's actual BC risk was compared to a perceived risk and the accuracy of the BC risk self-assessment was determined. 72% of high-risk women underestimated their BC risk (p < 0.001). One third of subjects with a family history of BC have also underestimated their own risk (p = 0.002). Women who responded to screening mammography were more informed about BC risk factors (p = 0.001). General knowledge about BC chemoprevention was surprisingly low, regardless of the accuracy of BC risk self-assessment. High-risk women appear to be unrealistically optimistic, since there was a significant difference between the accuracy of self-perceived risk and the objective BC risk.

Inclusion of a gene-environment interaction between alcohol consumption and the aldehyde dehydrogenase 2 genotype in a risk prediction model for upper aerodigestive tract cancer in Japanese men

The well-known gene-environment interaction between alcohol consumption and aldehyde dehydrogenase 2 (ALDH2) genotype in upper aerodigestive tract cancer risk may improve our ability to identify high-risk subjects. Here, we developed and validated risk prediction models for this cancer in Japanese men and evaluated whether adding the gene-environment interaction to the model improved the predictive performance. We developed two case-cohort datasets in the Japan Public Health Center-based Prospective Study: one from subjects in the baseline survey for model development (108 cases and 4049 subcohort subjects) and the second from subjects in the 5-year follow-up survey for model validation (31 cases and 1527 subcohort subjects). We developed an environmental model including age, smoking status, and alcohol consumption, and a gene-environment interaction model including age, smoking status, and the combination of alcohol consumption and the ALDH2 genotype. We found a statistically significant gene-environment interaction for alcohol consumption and the ALDH2 genotype. The c-index for the gene-environment interaction model (0.71) was slightly higher than that for the environmental model (0.67). The values of integrated discrimination improvement and net reclassification improvement for the gene-environment interaction model were also slightly higher than those for the environmental model. Goodness-of-fit tests suggested that the models were well calibrated. Results from external model validation by the 5-year follow-up survey were consistent with those from the model development by the baseline survey. The addition of a gene-environment interaction to a lifestyle-based model might improve the performance to estimate the probability of developing upper aerodigestive tract cancer for Japanese men.

Comparative Validation of Breast Cancer Risk Prediction Models and Projections for Future Risk Stratification

BACKGROUND

External validation of risk models is critical for risk-stratified breast cancer prevention. We used the Individualized Coherent Absolute Risk Estimation (iCARE) as a flexible tool for risk model development and comparative model validation and to make projections for population risk stratification.

METHODS

Performance of two recently developed models, one based on the Breast and Prostate Cancer Cohort Consortium analysis (iCARE-BPC3) and another based on a literature review (iCARE-Lit), were compared with two established models (Breast Cancer Risk Assessment Tool and International Breast Cancer Intervention Study Model) based on classical risk factors in a UK-based cohort of 64 874 white non-Hispanic women (863 patients) age 35-74 years. Risk projections in a target population of US white non-Hispanic women age 50-70 years assessed potential improvements in risk stratification by adding mammographic breast density (MD) and polygenic risk score (PRS).

RESULTS

The best calibrated models were iCARE-Lit (expected to observed number of cases [E/O] = 0.98, 95% confidence interval [CI] = 0.87 to 1.11) for women younger than 50 years, and iCARE-BPC3 (E/O = 1.00, 95% CI = 0.93 to 1.09) for women 50 years or older. Risk projections using iCARE-BPC3 indicated classical risk factors can identify approximately 500 000 women at moderate to high risk (>3% 5-year risk) in the target population. Addition of MD and a 313-variant PRS is expected to increase this number to approximately 3.5 million women, and among them, approximately 153 000 are expected to develop invasive breast cancer within 5 years.

CONCLUSIONS

iCARE models based on classical risk factors perform similarly to or better than BCRAT or IBIS in white non-Hispanic women. Addition of MD and PRS can lead to substantial improvements in risk stratification. However, these integrated models require independent prospective validation before broad clinical applications.

Prediction of Lifetime and 10-Year Risk of Cancer in Individual Patients With Established Cardiovascular Disease

Background

Cardiovascular disease (CVD) and cancer share many common risk factors; patients with CVD also may be at risk of developing cancer.

Objectives

The aim of this study was to derive and externally validate prediction models for the estimation of lifetime and 10-year risk for total, colorectal, and lung cancer in patients with established CVD.

Methods

Data from patients with established CVD from the UCC-SMART cohort (N = 7,280) were used for model development, and from the CANTOS trial (N = 9,322) for model validation. Predictors were selected based on previously published cancer risk scores, clinical availability, and presence in the derivation dataset. Fine and Gray competing risk-adjusted lifetime models were developed for the outcomes total, colorectal, and lung cancer.

Results

Selected predictors were age, sex, smoking, weight, height, alcohol use, antiplatelet use, diabetes, and C-reactive protein. External calibration for the 4-year risk of lung, colorectal, and total cancer was reasonable in our models, as was discrimination with C-statistics of 0.74, 0.64, and 0.63, respectively. Median predicted lifetime and 10-year risks in CANTOS were 26% (range 1% to 52%) and 13% (range 1% to 31%) for total cancer; 4% (range 0% to 13%) and 2% (range 0% to 6%) for colorectal cancer; and 5% (range 0% to 37%) and 2% (range 0% to 24%) for lung cancer.

Conclusions

Lifetime and 10-year risk of total, colorectal, and lung cancer can be estimated reasonably well in patients with established CVD with readily available clinical predictors. With additional study, these tools could be used in clinical practice to further aid in the emphasis of healthy lifestyle changes and to guide thresholds for targeted diagnostics and screening.

Predicting breast cancer risk using interacting genetic and demographic factors and machine learning

Breast cancer (BC) is a multifactorial disease and the most common cancer in women worldwide. We describe a machine learning approach to identify a combination of interacting genetic variants (SNPs) and demographic risk factors for BC, especially factors related to both familial history (Group 1) and oestrogen metabolism (Group 2), for predicting BC risk. This approach identifies the best combinations of interacting genetic and demographic risk factors that yield the highest BC risk prediction accuracy. In tests on the Kuopio Breast Cancer Project (KBCP) dataset, our approach achieves a mean average precision (mAP) of 77.78 in predicting BC risk by using interacting genetic and Group 1 features, which is better than the mAPs of 74.19 and 73.65 achieved using only Group 1 features and interacting SNPs, respectively. Similarly, using interacting genetic and Group 2 features yields a mAP of 78.00, which outperforms the system based on only Group 2 features, which has a mAP of 72.57. Furthermore, the gene interaction maps built from genes associated with SNPs that interact with demographic risk factors indicate important BC-related biological entities, such as angiogenesis, apoptosis and oestrogen-related networks. The results also show that demographic risk factors are individually more important than genetic variants in predicting BC risk.

Machine learning-based lifetime breast cancer risk reclassification compared with the BOADICEA model: impact on screening recommendations

Background

The clinical utility of machine-learning (ML) algorithms for breast cancer risk prediction and screening practices is unknown. We compared classification of lifetime breast cancer risk based on ML and the BOADICEA model. We explored the differences in risk classification and their clinical impact on screening practices.

Methods

We used three different ML algorithms and the BOADICEA model to estimate lifetime breast cancer risk in a sample of 112,587 individuals from 2481 families from the Oncogenetic Unit, Geneva University Hospitals. Performance of algorithms was evaluated using the area under the receiver operating characteristic (AU-ROC) curve. Risk reclassification was compared for 36,146 breast cancer-free women of ages 20–80. The impact on recommendations for mammography surveillance was based on the Swiss Surveillance Protocol.

Results

The predictive accuracy of ML-based algorithms (0.843 ≤ AU-ROC ≤ 0.889) was superior to BOADICEA (AU-ROC = 0.639) and reclassified 35.3% of women in different risk categories. The largest reclassification (20.8%) was observed in women characterised as ‘near population’ risk by BOADICEA. Reclassification had the largest impact on screening practices of women younger than 50.

Conclusion

ML-based reclassification of lifetime breast cancer risk occurred in approximately one in three women. Reclassification is important for younger women because it impacts clinical decision- making for the initiation of screening.

Familial risk of breast cancer by dynamic, accumulative, and static definitions of family history

BACKGROUND

Familial breast cancer risk studies usually overlook the dynamic nature of family history.

METHODS

The authors assessed the effect of incorporating the timing of cancer diagnosis events into the assessment of familial risks of breast cancer in first-degree and second-degree relatives in a nationwide cohort study of 5,099,172 women (follow-up was between 1958-2015). Family history was assessed using 3 approaches: 1) as a static variable (ever having a relative with breast cancer); 2) as accumulative history; and 3) as a dynamic variable (time-dependent variable).

RESULTS

For women aged <50 years, familial risk was mostly higher when family history was assessed as a dynamic variable compared with using a static or accumulative family history. For example, the cumulative risk of receiving a breast cancer diagnosis until age 50 years for women with a history of breast cancer in 1 first-degree relative was 2.6% (95% CI, 2.5%-2.7%) using the static method, 2.4% (95% CI, 2.3%-2.4%) using the accumulative method, and 3.1% (95% CI, 3.0%-3.2%) using the dynamic method. Relative risk in women aged <50 years with a breast cancer diagnosis in a sister was 1.40-fold (95% CI, 1.31-fold to 1.48-fold) using the static method, 1.66-fold (95% CI, 1.57-fold to 1.76-fold) using the accumulative method, and 2.28-fold (95% CI, 2.07-fold to 2.51-fold) using the dynamic method.

CONCLUSIONS

The results of the current study demonstrated that assessing family history as static, accumulative, or dynamic results in different familial risk estimates. The answer as to which method to use for family history assessment depends on the implications of the study, with the dynamic method appearing to be better suited for risk stratification studies, the accumulative method being the most convenient in practice and the least favored for risk prediction, and the static method being suitable for etiological impact and risk attribution studies.

Early warning scores for detecting deterioration in adult hospital patients: systematic review and critical appraisal of methodology

OBJECTIVE

To provide an overview and critical appraisal of early warning scores for adult hospital patients.

DESIGN

Systematic review.

DATA SOURCES

Medline, CINAHL, PsycInfo, and Embase until June 2019.

ELIGIBILITY CRITERIA FOR STUDY SELECTION

Studies describing the development or external validation of an early warning score for adult hospital inpatients.

RESULTS

13 171 references were screened and 95 articles were included in the review. 11 studies were development only, 23 were development and external validation, and 61 were external validation only. Most early warning scores were developed for use in the United States (n=13/34, 38%) and the United Kingdom (n=10/34, 29%). Death was the most frequent prediction outcome for development studies (n=10/23, 44%) and validation studies (n=66/84, 79%), with different time horizons (the most frequent was 24 hours). The most common predictors were respiratory rate (n=30/34, 88%), heart rate (n=28/34, 83%), oxygen saturation, temperature, and systolic blood pressure (all n=24/34, 71%). Age (n=13/34, 38%) and sex (n=3/34, 9%) were less frequently included. Key details of the analysis populations were often not reported in development studies (n=12/29, 41%) or validation studies (n=33/84, 39%). Small sample sizes and insufficient numbers of event patients were common in model development and external validation studies. Missing data were often discarded, with just one study using multiple imputation. Only nine of the early warning scores that were developed were presented in sufficient detail to allow individualised risk prediction. Internal validation was carried out in 19 studies, but recommended approaches such as bootstrapping or cross validation were rarely used (n=4/19, 22%). Model performance was frequently assessed using discrimination (development n=18/22, 82%; validation n=69/84, 82%), while calibration was seldom assessed (validation n=13/84, 15%). All included studies were rated at high risk of bias.

CONCLUSIONS

Early warning scores are widely used prediction models that are often mandated in daily clinical practice to identify early clinical deterioration in hospital patients. However, many early warning scores in clinical use were found to have methodological weaknesses. Early warning scores might not perform as well as expected and therefore they could have a detrimental effect on patient care. Future work should focus on following recommended approaches for developing and evaluating early warning scores, and investigating the impact and safety of using these scores in clinical practice.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42017053324.

Towards a more precise and individualized assessment of breast cancer risk

Many clinically based models are available for breast cancer risk assessment; however, these models are not particularly useful at the individual level, despite being designed with that intent. There is, therefore, a significant need for improved, precise individualized risk assessment. In this Research Perspective, we highlight commonly used clinical risk assessment models and recent scientific advances to individualize risk assessment using precision biomarkers. Genome-wide association studies have identified >100 single nucleotide polymorphisms (SNPs) associated with breast cancer risk, and polygenic risk scores (PRS) have been developed by several groups using this information. The ability of a PRS to improve risk assessment is promising; however, validation in both genetically and ethnically diverse populations is needed. Additionally, novel classes of biomarkers, such as microRNAs, may capture clinically relevant information based on epigenetic regulation of gene expression. Our group has recently identified a circulating-microRNA signature predictive of long-term breast cancer in a prospective cohort of high-risk women. While progress has been made, the importance of accurate risk assessment cannot be understated. Precision risk assessment will identify those women at greatest risk of developing breast cancer, thus avoiding overtreatment of women at average risk and identifying the most appropriate candidates for chemoprevention or surgical prevention.

A risk prediction model for the development of subsequent primary melanoma in a population-based cohort

BACKGROUND

Guidelines for follow-up of patients with melanoma are based on limited evidence.

OBJECTIVES

To guide skin surveillance, we developed a risk prediction model for subsequent primary melanomas, using demographic, phenotypical, histopathological, sun exposure and genomic risk factors.

METHODS

Using Cox regression frailty models, we analysed data for 2613 primary melanomas from 1266 patients recruited to the population-based Genes, Environment and Melanoma study in New South Wales, Australia, with a median of 14 years' follow-up via the cancer registry. Discrimination and calibration were assessed.

RESULTS

The median time to diagnosis of a subsequent primary melanoma decreased with each new primary melanoma. The final model included 12 risk factors. Harrell's C-statistic was 0·73 [95% confidence interval (CI) 0·68-0·77], 0·65 (95% CI 0·62-0·68) and 0·65 (95% CI 0·61-0·69) for predicting second, third and fourth primary melanomas, respectively. The risk of a subsequent primary melanoma was 4·75 times higher (95% CI 3·87-5·82) for the highest vs. the lowest quintile of the risk score. The mean absolute risk of a subsequent primary melanoma within 5 years was 8·0 ± SD 4.1% after the first primary melanoma, and 46·8 ± 15·0% after the second, but varied substantially by risk score.

CONCLUSIONS

The risk of developing a subsequent primary melanoma varies considerably between individuals and is particularly high for those with two or more primary melanomas. The risk prediction model and its associated nomograms enable estimation of the absolute risk of subsequent primary melanoma, on the basis of on an individual's risk factors, and can be used to tailor surveillance intensity, communicate risk and provide patient education. What's already known about this topic? Current guidelines for the frequency and length of follow-up to detect new primary melanomas in patients with one or more previous primary melanomas are based on limited evidence. People with one or more primary melanomas have, on average, a higher risk of developing another primary invasive melanoma, compared with the general population, but an accurate way of estimating individual risk is needed. What does this study add? We provide a comprehensive risk prediction model for subsequent primary melanomas, using data from 1266 participants with melanoma (2613 primary melanomas), over a median 14 years' follow-up. The model includes 12 risk factors comprising demographic, phenotypical, histopathological and genomic factors, and sun exposure. It enables estimation of the absolute risk of subsequent primary melanomas, and can be used to tailor surveillance intensity, communicate individual risk and provide patient education.

The Global Challenge to Prevent Breast Cancer: Surfacing New Ideas to Accelerate Prevention Research

Despite increases in screening and advances in treatment, breast cancer continues to be the most common cancer and cause of cancer deaths among women worldwide, and breast cancer rates have remained steady for decades. A new focus on population-level primary prevention is needed to tackle this disease at the most fundamental level. Unfortunately, only a small fraction of breast cancer research funds currently go to prevention. The California Breast Cancer Research Program (CBCRP) seeks to change this. In order to accelerate breast cancer primary prevention efforts, in 2018, CBCRP launched the Global Challenge to Prevent Breast Cancer, a prize competition to foster and disseminate new and innovative prevention research ideas. This Special Issue highlights the results of the Global Challenge and other CBCRP primary prevention efforts.

Risk-Based Screening Mammography for Women Aged <40: Outcomes From the National Mammography Database

OBJECTIVE

There is insufficient large-scale evidence for screening mammography in women <40 years at elevated risk. This study compares risk-based screening of women aged 30 to 39 with risk factors versus women aged 40 to 49 without risk factors in the National Mammography Database (NMD).

METHODS

This retrospective, HIPAA-compliant, institutional review board-exempt study analyzed data from 150 NMD mammography facilities in 31 states. Patients were stratified by 5-year age intervals, availability of prior mammograms, and specific risk factors for breast cancer: family history of breast cancer, personal history of breast cancer, and dense breasts. Four screening performance metrics were calculated for each age and risk group: recall rate (RR), cancer detection rate (CDR), and positive predictive values for biopsy recommended (PPV₂) and biopsy performed (PPV₃).

RESULTS

Data from 5,986,131 screening mammograms performed between January 2008 and December 2015 in 2,647,315 women were evaluated. Overall, mean CDR was 3.69 of 1,000 (95% confidence interval: 3.64-3.74), RR was 9.89% (9.87%-9.92%), PPV₂ was 20.1% (19.9%-20.4%), and PPV₃ was 28.2% (27.0%-28.5%). Women aged 30 to 34 and 35 to 39 had similar CDR, RR, and PPVs, with the presence of the three evaluated risk factors associated with significantly higher CDR. Moreover, compared with a population currently recommended for screening mammography in the United States (aged 40-49 at average risk), incidence screening (at least one prior screening examination) of women aged 30 to 39 with the three evaluated risk factors has similar cancer detection rates and recall rates.

DISCUSSION

Women with one or more of these three specific risk factors likely benefit from screening commencing at age 30 instead of age 40.

Predicting the risk of stroke among patients with type 2 diabetes: a systematic review and meta-analysis of C-statistics

OBJECTIVE

Stroke is a major cause of disability and death worldwide. People with diabetes are at a twofold to fivefold increased risk for stroke compared with people without diabetes. This study systematically reviews the literature on available stroke prediction models specifically developed or validated in patients with diabetes and assesses their predictive performance through meta-analysis.

DESIGN

Systematic review and meta-analysis.

DATA SOURCES

A detailed search was performed in MEDLINE, PubMed and EMBASE (from inception to 22 April 2019) to identify studies describing stroke prediction models.

ELIGIBILITY CRITERIA

All studies that developed stroke prediction models in populations with diabetes were included.

DATA EXTRACTION AND SYNTHESIS

Two reviewers independently identified eligible articles and extracted data. Random effects meta-analysis was used to obtain a pooled C-statistic.

RESULTS

Our search retrieved 26 202 relevant papers and finally yielded 38 stroke prediction models, of which 34 were specifically developed for patients with diabetes and 4 were developed in general populations but validated in patients with diabetes. Among the models developed in those with diabetes, 9 reported their outcome as stroke, 23 reported their outcome as composite cardiovascular disease (CVD) where stroke was a component of the outcome and 2 did not report stroke initially as their outcome but later were validated for stroke as the outcome in other studies. C-statistics varied from 0.60 to 0.92 with a median C-statistic of 0.71 (for stroke as the outcome) and 0.70 (for stroke as part of a composite CVD outcome). Seventeen models were externally validated in diabetes populations with a pooled C-statistic of 0.68.

CONCLUSIONS

Overall, the performance of these diabetes-specific stroke prediction models was not satisfactory. Research is needed to identify and incorporate new risk factors into the model to improve models' predictive ability and further external validation of the existing models in diverse population to improve generalisability.

Machine learning techniques for personalized breast cancer risk prediction: comparison with the BCRAT and BOADICEA models

BACKGROUND

Comprehensive breast cancer risk prediction models enable identifying and targeting women at high-risk, while reducing interventions in those at low-risk. Breast cancer risk prediction models used in clinical practice have low discriminatory accuracy (0.53-0.64). Machine learning (ML) offers an alternative approach to standard prediction modeling that may address current limitations and improve accuracy of those tools. The purpose of this study was to compare the discriminatory accuracy of ML-based estimates against a pair of established methods-the Breast Cancer Risk Assessment Tool (BCRAT) and Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm (BOADICEA) models.

METHODS

We quantified and compared the performance of eight different ML methods to the performance of BCRAT and BOADICEA using eight simulated datasets and two retrospective samples: a random population-based sample of U.S. breast cancer patients and their cancer-free female relatives (N = 1143), and a clinical sample of Swiss breast cancer patients and cancer-free women seeking genetic evaluation and/or testing (N = 2481).

RESULTS

Predictive accuracy (AU-ROC curve) reached 88.28% using ML-Adaptive Boosting and 88.89% using ML-random forest versus 62.40% with BCRAT for the U.S. population-based sample. Predictive accuracy reached 90.17% using ML-adaptive boosting and 89.32% using ML-Markov chain Monte Carlo generalized linear mixed model versus 59.31% with BOADICEA for the Swiss clinic-based sample.

CONCLUSIONS

There was a striking improvement in the accuracy of classification of women with and without breast cancer achieved with ML algorithms compared to the state-of-the-art model-based approaches. High-accuracy prediction techniques are important in personalized medicine because they facilitate stratification of prevention strategies and individualized clinical management.

A Case-Control Study to Add Volumetric or Clinical Mammographic Density into the Tyrer-Cuzick Breast Cancer Risk Model

Background

Accurate breast cancer risk assessment for women attending routine screening is needed to guide screening and preventive interventions. We evaluated the accuracy of risk predictions from both visual and volumetric mammographic density combined with the Tyrer-Cuzick breast cancer risk model.

Methods

A case-control study (474 patient participants and 2243 healthy control participants) of women aged 40–79 years was performed using self-reported classical risk factors. Breast density was measured by using automated volumetric software and Breast Imaging and Reporting Data System (BI-RADS) density categories. Odds ratios (95% CI) were estimated by using logistic regression, adjusted for age, demographic factors, and 10-year risk from the Tyrer-Cuzick model, for a change from the 25^th to 75^th percentile of the adjusted percent density distribution in control participants (IQ-OR).

Results

After adjustment for classical risk factors in the Tyrer-Cuzick model, age, and body mass index (BMI), BI-RADS density had an IQ-OR of 1.55 (95% CI = 1.33 to 1.80) compared with 1.40 (95% CI = 1.21 to 1.60) for volumetric percent density. Fibroglandular volume (IQ-OR = 1.28, 95% CI = 1.12 to 1.47) was a weaker predictor than was BI-RADS density (P_diff = 0.014) or volumetric percent density (P_diff = 0.065). In this setting, 4.8% of women were at high risk (8% + 10-year risk), using the Tyrer-Cuzick model without density, and 7.1% (BI-RADS) compared with 6.8% (volumetric) when combined with density.

Conclusion

The addition of volumetric and visual mammographic density measures to classical risk factors improves risk stratification. A combined risk could be used to guide precision medicine, through risk-adapted screening and prevention strategies.

Comparison of Questionnaire-Based Breast Cancer Prediction Models in the Nurses' Health Study

BACKGROUND

The Gail model and the model developed by Tyrer and Cuzick are two questionnaire-based approaches with demonstrated ability to predict development of breast cancer in a general population.

METHODS

We compared calibration, discrimination, and net reclassification of these models, using data from questionnaires sent every 2 years to 76,922 participants in the Nurses' Health Study between 1980 and 2006, with 4,384 incident invasive breast cancers identified by 2008 (median follow-up, 24 years; range, 1-28 years). In a random one third sample of women, we also compared the performance of these models with predictions from the Rosner-Colditz model estimated from the remaining participants.

RESULTS

Both the Gail and Tyrer-Cuzick models showed evidence of miscalibration (Hosmer-Lemeshow P < 0.001 for each) with notable (P < 0.01) overprediction in higher-risk women (2-year risk above about 1%) and underprediction in lower-risk women (risk below about 0.25%). The Tyrer-Cuzick model had slightly higher C-statistics both overall (P < 0.001) and in age-specific comparisons than the Gail model (overall C, 0.63 for Tyrer-Cuzick vs. 0.61 for the Gail model). Evaluation of net reclassification did not favor either model. In the one third sample, the Rosner-Colditz model had better calibration and discrimination than the other two models. All models had C-statistics <0.60 among women ages ≥70 years.

CONCLUSIONS

Both the Gail and Tyrer-Cuzick models had some ability to discriminate breast cancer cases and noncases, but have limitations in their model fit.

IMPACT

Refinements may be needed to questionnaire-based approaches to predict breast cancer in older and higher-risk women.

Empirical evidence of the impact of study characteristics on the performance of prediction models: a meta-epidemiological study

OBJECTIVES

To empirically assess the relation between study characteristics and prognostic model performance in external validation studies of multivariable prognostic models.

DESIGN

Meta-epidemiological study.

DATA SOURCES AND STUDY SELECTION

On 16 October 2018, we searched electronic databases for systematic reviews of prognostic models. Reviews from non-overlapping clinical fields were selected if they reported common performance measures (either the concordance (c)-statistic or the ratio of observed over expected number of events (OE ratio)) from 10 or more validations of the same prognostic model.

DATA EXTRACTION AND ANALYSES

Study design features, population characteristics, methods of predictor and outcome assessment, and the aforementioned performance measures were extracted from the included external validation studies. Random effects meta-regression was used to quantify the association between the study characteristics and model performance.

RESULTS

We included 10 systematic reviews, describing a total of 224 external validations, of which 221 reported c-statistics and 124 OE ratios. Associations between study characteristics and model performance were heterogeneous across systematic reviews. C-statistics were most associated with variation in population characteristics, outcome definitions and measurement and predictor substitution. For example, validations with eligibility criteria comparable to the development study were associated with higher c-statistics compared with narrower criteria (difference in logit c-statistic 0.21(95% CI 0.07 to 0.35), similar to an increase from 0.70 to 0.74). Using a case-control design was associated with higher OE ratios, compared with using data from a cohort (difference in log OE ratio 0.97(95% CI 0.38 to 1.55), similar to an increase in OE ratio from 1.00 to 2.63).

CONCLUSIONS

Variation in performance of prognostic models across studies is mainly associated with variation in case-mix, study designs, outcome definitions and measurement methods and predictor substitution. Researchers developing and validating prognostic models should realise the potential influence of these study characteristics on the predictive performance of prognostic models.

Predicting the risk of locoregional recurrence after early breast cancer: an external validation of the Dutch INFLUENCE-nomogram with clinical cancer registry data from Germany

Purpose

Follow-up after breast cancer treatment aims for an early detection of locoregional breast cancer recurrences (LRR) to improve the patients’ outcome. By estimating individual’s 5-year recurrence-risks, the Dutch INFLUENCE-nomogram can assist health professionals and patients in developing personalized risk-based follow-up pathways. The objective of this study is to validate the prediction tool on non-Dutch patients.

Material and methods

Data for this external validation derive from a large clinical cancer registry in southern Germany, covering a population of 1.1 million. Patients with curative resection of early-stage breast cancer, diagnosed between 2000 and 2012, were included in the analysis (n = 6520). For each of them, an individual LRR-risk was estimated by the INFLUENCE-nomogram. Its predictive ability was tested by comparing estimated and observed LRR-probabilities using the Hosmer–Lemeshow goodness-of-fit test and C-statistics.

Results

In the German validation-cohort, 2.8% of the patients developed an LRR within 5 years after primary surgery (n = 184). While the INFLUENCE-nomogram generally underestimates the actual LRR-risk of the German patients (p < 0.001), its discriminative ability is comparable to the one observed in the original Dutch modeling-cohort (C-statistic German validation-cohort: 0.73, CI 0.69–0.77 vs. C-statistic Dutch modeling-cohort: 0.71, CI 0.69–0.73). Similar results were obtained in most of the subgroup analyses stratified by age, type of surgery and intrinsic biological subtypes.

Conclusion

The outcomes of this external validation underline the generalizability of the INFLUENCE-nomogram beyond the Dutch population. The model performance could be enhanced in future by incorporating additional risk factors for LRR.

Modeling risk assessment for breast cancer in symptomatic women: a Saudi Arabian study

BACKGROUND

Despite the continuing increase in the breast cancer incidence rate among Saudi Arabian women, no breast cancer risk-prediction model is available in this population. The aim of this research was to develop a risk-assessment tool to distinguish between high risk and low risk of breast cancer in a sample of Saudi women who were screened for breast cancer.

METHODS

A retrospective chart review was conducted on symptomatic women who underwent breast mass biopsies between September 8, 2015 and November 8, 2017 at King Abdulaziz Medical City, Riyadh, Saudi Arabia.

RESULTS

A total of 404 (63.8%) malignant breast biopsies and 229 (36.2%) benign breast biopsies were analyzed. Women ≥40 years old (aOR: 6.202, CI 3.497-11.001, P=0.001), hormone-replacement therapy (aOR 24.365, 95% CI 8.606-68.987, P=0.001), postmenopausal (aOR 3.058, 95% CI 1.861-5.024, P=0.001), and with a family history of breast cancer (aOR 2.307, 95% CI 1.142-4.658, P=0.020) were independently associated with an increased risk of breast cancer. This model showed an acceptable fit and had area under the receiver-operating characteristic curve of 0.877 (95% CI 0.851-0.903), with optimism-corrected area under the curve of 0.865.

CONCLUSION

The prediction model developed in this study has a high ability in predicting increased breast cancer risk in our facility. Combining information on age, use of hormone therapy, postmenopausal status, and family history of breast cancer improved the degree of discriminatory accuracy of breast cancer prediction. Our risk model may assist in initiating population-screening programs and prompt clinical decision making to manage cases and prevent unfavorable outcomes.

Development and Validation of Lifestyle-Based Models to Predict Incidence of the Most Common Potentially Preventable Cancers

BACKGROUND

Most risk models for cancer are either specific to individual cancers or include complex or predominantly non-modifiable risk factors.

METHODS

We developed lifestyle-based models for the five cancers for which the most cases are potentially preventable through lifestyle change in the UK (lung, colorectal, bladder, kidney, and esophageal for men and breast, lung, colorectal, endometrial, and kidney for women). We selected lifestyle risk factors from the European Code against Cancer and obtained estimates of relative risks from meta-analyses of observational studies. We used mean values for risk factors from nationally representative samples and mean 10-year estimated absolute risks from routinely available sources. We then assessed the performance of the models in 23,768 participants in the EPIC-Norfolk cohort who had no history of the five selected cancers at baseline.

RESULTS

In men, the combined risk model showed good discrimination [AUC, 0.71; 95% confidence interval (CI), 0.69-0.73] and calibration. Discrimination was lower in women (AUC, 0.59; 95% CI, 0.57-0.61), but calibration was good. In both sexes, the individual models for lung cancer had the highest AUCs (0.83; 95% CI, 0.80-0.85 for men and 0.82; 95% CI, 0.76-0.87 for women). The lowest AUCs were for breast cancer in women and kidney cancer in men.

CONCLUSIONS

The discrimination and calibration of the models are both reasonable, with the discrimination for individual cancers comparable or better than many other published risk models.

IMPACT

These models could be used to demonstrate the potential impact of lifestyle change on risk of cancer to promote behavior change.

PROBAST: A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies: Explanation and Elaboration

Prediction models in health care use predictors to estimate for an individual the probability that a condition or disease is already present (diagnostic model) or will occur in the future (prognostic model). Publications on prediction models have become more common in recent years, and competing prediction models frequently exist for the same outcome or target population. Health care providers, guideline developers, and policymakers are often unsure which model to use or recommend, and in which persons or settings. Hence, systematic reviews of these studies are increasingly demanded, required, and performed. A key part of a systematic review of prediction models is examination of risk of bias and applicability to the intended population and setting. To help reviewers with this process, the authors developed PROBAST (Prediction model Risk Of Bias ASsessment Tool) for studies developing, validating, or updating (for example, extending) prediction models, both diagnostic and prognostic. PROBAST was developed through a consensus process involving a group of experts in the field. It includes 20 signaling questions across 4 domains (participants, predictors, outcome, and analysis). This explanation and elaboration document describes the rationale for including each domain and signaling question and guides researchers, reviewers, readers, and guideline developers in how to use them to assess risk of bias and applicability concerns. All concepts are illustrated with published examples across different topics. The latest version of the PROBAST checklist, accompanying documents, and filled-in examples can be downloaded from www.probast.org.

Prognostic tools for cardiovascular disease in patients with type 2 diabetes: A systematic review and meta-analysis of C-statistics

BACKGROUND

Diabetes is associated with an increased risk for cardiovascular diseases (CVD). Risk prediction models are tools widely used to identify individuals at particularly high-risk of adverse events. Many CVD risk prediction models have been developed but their accuracy and consistency vary.

OBJECTIVE

This study reviews the literature on available CVD risk prediction models specifically developed or validated in patients with diabetes and performs a meta-analysis of C-statistics to assess and compare their predictive performance.

METHODS

The online databases and manual reference checks of all identified relevant publications were searched.

RESULTS

Fifteen CVD prediction models developed for patients with diabetes and 11 models developed in a general population but later validated in diabetes patients were identified. Meta-analysis of C-statistics showed an overall pooled C-statistic of 0.67 and 0.64 for validated models developed in diabetes patients and in general populations respectively. This small difference in the C-statistic suggests that CVD risk prediction for diabetes patients depends little on the population the model was developed in (p = 0.068).

CONCLUSIONS

The discriminative ability of diabetes-specific CVD prediction models were modest. Improvements in the predictive ability of these models are required to understand both short and long-term risk before implementation into clinical practice.

Clinicians' use of breast cancer risk assessment tools according to their perceived importance of breast cancer risk factors: an international survey

The BOADICEA breast cancer (BC) risk assessment model and its associated Web Application v3 (BWA) tool are being extended to incorporate additional genetic and non-genetic BC risk factors. From an online survey through the BOADICEA website and UK, Dutch, French and Swedish national genetic societies, we explored the relationships between the usage frequencies of the BWA and six other common BC risk assessment tools and respondents' perceived importance of BC risk factors. Respondents (N = 443) varied in age, country and clinical seniority but comprised mainly genetics health professionals (82%) and BWA users (93%). Oncology professionals perceived reproductive, hormonal (exogenous) and lifestyle BC risk factors as more important in BC risk assessment compared to genetics professionals (p values < 0.05 to 0.0001). BWA was used more frequently by respondents who gave high weight to breast tumour pathology and low weight to personal BC history as BC risk factors. BWA use was positively related to the weight given to hormonal BC risk factors. The importance attributed to lifestyle and BMI BC risk factors was not associated with the use of BWA or any of the other tools. Next version of the BWA encompassing additional BC risk factors will facilitate more comprehensive BC risk assessment in genetics and oncology practice.