Health equity assessment of machine learning performance (HEAL): a framework and dermatology AI model case study

Background

Artificial intelligence (AI) has repeatedly been shown to encode historical inequities in healthcare. We aimed to develop a framework to quantitatively assess the performance equity of health AI technologies and to illustrate its utility via a case study.

Methods

Here, we propose a methodology to assess whether health AI technologies prioritise performance for patient populations experiencing worse outcomes, that is complementary to existing fairness metrics. We developed the Health Equity Assessment of machine Learning performance (HEAL) framework designed to quantitatively assess the performance equity of health AI technologies via a four-step interdisciplinary process to understand and quantify domain-specific criteria, and the resulting HEAL metric. As an illustrative case study (analysis conducted between October 2022 and January 2023), we applied the HEAL framework to a dermatology AI model. A set of 5420 teledermatology cases (store-and-forward cases from patients of 20 years or older, submitted from primary care providers in the USA and skin cancer clinics in Australia), enriched for diversity in age, sex and race/ethnicity, was used to retrospectively evaluate the AI model's HEAL metric, defined as the likelihood that the AI model performs better for subpopulations with worse average health outcomes as compared to others. The likelihood that AI performance was anticorrelated to pre-existing health outcomes was estimated using bootstrap methods as the probability that the negated Spearman's rank correlation coefficient (i.e., "R") was greater than zero. Positive values of R suggest that subpopulations with poorer health outcomes have better AI model performance. Thus, the HEAL metric, defined as p (R >0), measures how likely the AI technology is to prioritise performance for subpopulations with worse average health outcomes as compared to others (presented as a percentage below). Health outcomes were quantified as disability-adjusted life years (DALYs) when grouping by sex and age, and years of life lost (YLLs) when grouping by race/ethnicity. AI performance was measured as top-3 agreement with the reference diagnosis from a panel of 3 dermatologists per case.

Findings

Across all dermatologic conditions, the HEAL metric was 80.5% for prioritizing AI performance of racial/ethnic subpopulations based on YLLs, and 92.1% and 0.0% respectively for prioritizing AI performance of sex and age subpopulations based on DALYs. Certain dermatologic conditions were significantly associated with greater AI model performance compared to a reference category of less common conditions. For skin cancer conditions, the HEAL metric was 73.8% for prioritizing AI performance of age subpopulations based on DALYs.

Interpretation

Analysis using the proposed HEAL framework showed that the dermatology AI model prioritised performance for race/ethnicity, sex (all conditions) and age (cancer conditions) subpopulations with respect to pre-existing health disparities. More work is needed to investigate ways of promoting equitable AI performance across age for non-cancer conditions and to better understand how AI models can contribute towards improving equity in health outcomes.

Funding

Google LLC.

Predicting lymph node metastasis from primary tumor histology and clinicopathologic factors in colorectal cancer using deep learning

BACKGROUND

Presence of lymph node metastasis (LNM) influences prognosis and clinical decision-making in colorectal cancer. However, detection of LNM is variable and depends on a number of external factors. Deep learning has shown success in computational pathology, but has struggled to boost performance when combined with known predictors.

METHODS

Machine-learned features are created by clustering deep learning embeddings of small patches of tumor in colorectal cancer via k-means, and then selecting the top clusters that add predictive value to a logistic regression model when combined with known baseline clinicopathological variables. We then analyze performance of logistic regression models trained with and without these machine-learned features in combination with the baseline variables.

RESULTS

The machine-learned extracted features provide independent signal for the presence of LNM (AUROC: 0.638, 95% CI: [0.590, 0.683]). Furthermore, the machine-learned features add predictive value to the set of 6 clinicopathologic variables in an external validation set (likelihood ratio test, p < 0.00032; AUROC: 0.740, 95% CI: [0.701, 0.780]). A model incorporating these features can also further risk-stratify patients with and without identified metastasis (p < 0.001 for both stage II and stage III).

CONCLUSION

This work demonstrates an effective approach to combine deep learning with established clinicopathologic factors in order to identify independently informative features associated with LNM. Further work building on these specific results may have important impact in prognostication and therapeutic decision making for LNM. Additionally, this general computational approach may prove useful in other contexts.

Artificial intelligence for diagnosis and Gleason grading of prostate cancer: the PANDA challenge

Artificial intelligence (AI) has shown promise for diagnosing prostate cancer in biopsies. However, results have been limited to individual studies, lacking validation in multinational settings. Competitions have been shown to be accelerators for medical imaging innovations, but their impact is hindered by lack of reproducibility and independent validation. With this in mind, we organized the PANDA challenge-the largest histopathology competition to date, joined by 1,290 developers-to catalyze development of reproducible AI algorithms for Gleason grading using 10,616 digitized prostate biopsies. We validated that a diverse set of submitted algorithms reached pathologist-level performance on independent cross-continental cohorts, fully blinded to the algorithm developers. On United States and European external validation sets, the algorithms achieved agreements of 0.862 (quadratically weighted κ, 95% confidence interval (CI), 0.840-0.884) and 0.868 (95% CI, 0.835-0.900) with expert uropathologists. Successful generalization across different patient populations, laboratories and reference standards, achieved by a variety of algorithmic approaches, warrants evaluating AI-based Gleason grading in prospective clinical trials.

Comparative analysis of machine learning approaches to classify tumor mutation burden in lung adenocarcinoma using histopathology images

Both histologic subtypes and tumor mutation burden (TMB) represent important biomarkers in lung cancer, with implications for patient prognosis and treatment decisions. Typically, TMB is evaluated by comprehensive genomic profiling but this requires use of finite tissue specimens and costly, time-consuming laboratory processes. Histologic subtype classification represents an established component of lung adenocarcinoma histopathology, but can be challenging and is associated with substantial inter-pathologist variability. Here we developed a deep learning system to both classify histologic patterns in lung adenocarcinoma and predict TMB status using de-identified Hematoxylin and Eosin (H&E) stained whole slide images. We first trained a convolutional neural network to map histologic features across whole slide images of lung cancer resection specimens. On evaluation using an external data source, this model achieved patch-level area under the receiver operating characteristic curve (AUC) of 0.78–0.98 across nine histologic features. We then integrated the output of this model with clinico-demographic data to develop an interpretable model for TMB classification. The resulting end-to-end system was evaluated on 172 held out cases from TCGA, achieving an AUC of 0.71 (95% CI 0.63–0.80). The benefit of using histologic features in predicting TMB is highlighted by the significant improvement this approach offers over using the clinical features alone (AUC of 0.63 [95% CI 0.53–0.72], p = 0.002). Furthermore, we found that our histologic subtype-based approach achieved performance similar to that of a weakly supervised approach (AUC of 0.72 [95% CI 0.64–0.80]). Together these results underscore that incorporating histologic patterns in biomarker prediction for lung cancer provides informative signals, and that interpretable approaches utilizing these patterns perform comparably with less interpretable, weakly supervised approaches.

Determining breast cancer biomarker status and associated morphological features using deep learning

Background

Breast cancer management depends on biomarkers including estrogen receptor, progesterone receptor, and human epidermal growth factor receptor 2 (ER/PR/HER2). Though existing scoring systems are widely used and well-validated, they can involve costly preparation and variable interpretation. Additionally, discordances between histology and expected biomarker findings can prompt repeat testing to address biological, interpretative, or technical reasons for unexpected results.

Methods

We developed three independent deep learning systems (DLS) to directly predict ER/PR/HER2 status for both focal tissue regions (patches) and slides using hematoxylin-and-eosin-stained (H&E) images as input. Models were trained and evaluated using pathologist annotated slides from three data sources. Areas under the receiver operator characteristic curve (AUCs) were calculated for test sets at both a patch-level (>135 million patches, 181 slides) and slide-level (n = 3274 slides, 1249 cases, 37 sites). Interpretability analyses were performed using Testing with Concept Activation Vectors (TCAV), saliency analysis, and pathologist review of clustered patches.

Results

The patch-level AUCs are 0.939 (95%CI 0.936–0.941), 0.938 (0.936–0.940), and 0.808 (0.802–0.813) for ER/PR/HER2, respectively. At the slide level, AUCs are 0.86 (95%CI 0.84–0.87), 0.75 (0.73–0.77), and 0.60 (0.56–0.64) for ER/PR/HER2, respectively. Interpretability analyses show known biomarker-histomorphology associations including associations of low-grade and lobular histology with ER/PR positivity, and increased inflammatory infiltrates with triple-negative staining.

Conclusions

This study presents rapid breast cancer biomarker estimation from routine H&E slides and builds on prior advances by prioritizing interpretability of computationally learned features in the context of existing pathological knowledge.

Development and Validation of a Deep Learning Algorithm for Gleason Grading of Prostate Cancer From Biopsy Specimens

Question

How does a deep learning system for assessing prostate biopsy specimens compare with interpretations determined by specialists in urologic pathology and by general pathologists?

Findings

In a validation data set of 752 biopsy specimens obtained from 2 independent medical laboratories and a tertiary teaching hospital, this study found that rate of agreement with subspecialists was significantly higher for the deep learning system than it was for a cohort of general pathologists.

Meaning

The deep learning system warrants evaluation as an assistive tool for improving prostate cancer diagnosis and treatment decisions, especially where subspecialist expertise is unavailable.

Importance

For prostate cancer, Gleason grading of the biopsy specimen plays a pivotal role in determining case management. However, Gleason grading is associated with substantial interobserver variability, resulting in a need for decision support tools to improve the reproducibility of Gleason grading in routine clinical practice.

Objective

To evaluate the ability of a deep learning system (DLS) to grade diagnostic prostate biopsy specimens.

Design, Setting, and Participants

The DLS was evaluated using 752 deidentified digitized images of formalin-fixed paraffin-embedded prostate needle core biopsy specimens obtained from 3 institutions in the United States, including 1 institution not used for DLS development. To obtain the Gleason grade group (GG), each specimen was first reviewed by 2 expert urologic subspecialists from a multi-institutional panel of 6 individuals (years of experience: mean, 25 years; range, 18-34 years). A third subspecialist reviewed discordant cases to arrive at a majority opinion. To reduce diagnostic uncertainty, all subspecialists had access to an immunohistochemical-stained section and 3 histologic sections for every biopsied specimen. Their review was conducted from December 2018 to June 2019.

Main Outcomes and Measures

The frequency of the exact agreement of the DLS with the majority opinion of the subspecialists in categorizing each tumor-containing specimen as 1 of 5 categories: nontumor, GG1, GG2, GG3, or GG4-5. For comparison, the rate of agreement of 19 general pathologists’ opinions with the subspecialists’ majority opinions was also evaluated.

Results

For grading tumor-containing biopsy specimens in the validation set (n = 498), the rate of agreement with subspecialists was significantly higher for the DLS (71.7%; 95% CI, 67.9%-75.3%) than for general pathologists (58.0%; 95% CI, 54.5%-61.4%) (P < .001). In subanalyses of biopsy specimens from an external validation set (n = 322), the Gleason grading performance of the DLS remained similar. For distinguishing nontumor from tumor-containing biopsy specimens (n = 752), the rate of agreement with subspecialists was 94.3% (95% CI, 92.4%-95.9%) for the DLS and similar at 94.7% (95% CI, 92.8%-96.3%) for general pathologists (P = .58).

Conclusions and Relevance

In this study, the DLS showed higher proficiency than general pathologists at Gleason grading prostate needle core biopsy specimens and generalized to an independent institution. Future research is necessary to evaluate the potential utility of using the DLS as a decision support tool in clinical workflows and to improve the quality of prostate cancer grading for therapy decisions.

Evaluation of the Use of Combined Artificial Intelligence and Pathologist Assessment to Review and Grade Prostate Biopsies

IMPORTANCE

Expert-level artificial intelligence (AI) algorithms for prostate biopsy grading have recently been developed. However, the potential impact of integrating such algorithms into pathologist workflows remains largely unexplored.

OBJECTIVE

To evaluate an expert-level AI-based assistive tool when used by pathologists for the grading of prostate biopsies.

DESIGN, SETTING, AND PARTICIPANTS

This diagnostic study used a fully crossed multiple-reader, multiple-case design to evaluate an AI-based assistive tool for prostate biopsy grading. Retrospective grading of prostate core needle biopsies from 2 independent medical laboratories in the US was performed between October 2019 and January 2020. A total of 20 general pathologists reviewed 240 prostate core needle biopsies from 240 patients. Each pathologist was randomized to 1 of 2 study cohorts. The 2 cohorts reviewed every case in the opposite modality (with AI assistance vs without AI assistance) to each other, with the modality switching after every 10 cases. After a minimum 4-week washout period for each batch, the pathologists reviewed the cases for a second time using the opposite modality. The pathologist-provided grade group for each biopsy was compared with the majority opinion of urologic pathology subspecialists.

EXPOSURE

An AI-based assistive tool for Gleason grading of prostate biopsies.

MAIN OUTCOMES AND MEASURES

Agreement between pathologists and subspecialists with and without the use of an AI-based assistive tool for the grading of all prostate biopsies and Gleason grade group 1 biopsies.

RESULTS

Biopsies from 240 patients (median age, 67 years; range, 39-91 years) with a median prostate-specific antigen level of 6.5 ng/mL (range, 0.6-97.0 ng/mL) were included in the analyses. Artificial intelligence-assisted review by pathologists was associated with a 5.6% increase (95% CI, 3.2%-7.9%; P < .001) in agreement with subspecialists (from 69.7% for unassisted reviews to 75.3% for assisted reviews) across all biopsies and a 6.2% increase (95% CI, 2.7%-9.8%; P = .001) in agreement with subspecialists (from 72.3% for unassisted reviews to 78.5% for assisted reviews) for grade group 1 biopsies. A secondary analysis indicated that AI assistance was also associated with improvements in tumor detection, mean review time, mean self-reported confidence, and interpathologist agreement.

CONCLUSIONS AND RELEVANCE

In this study, the use of an AI-based assistive tool for the review of prostate biopsies was associated with improvements in the quality, efficiency, and consistency of cancer detection and grading.

Closing the translation gap: AI applications in digital pathology

Recent advances in artificial intelligence show tremendous promise to improve the accuracy, reproducibility, and availability of medical diagnostics across a number of medical subspecialities. This is especially true in the field of digital pathology, which has recently witnessed a surge in publications describing state-of-the-art performance for machine learning models across a wide range of diagnostic applications. Nonetheless, despite this promise, there remain significant gaps in translating applications for any of these technologies into actual clinical practice. In this review, we will first give a brief overview of the recent progress in applying AI to digitized pathology images, focusing on how these tools might be applied in clinical workflows in the near term to improve the accuracy and efficiency of pathologists. Then we define and describe in detail the various factors that need to be addressed in order to successfully close the "translation gap" for AI applications in digital pathology.

Abstract 2096: A deep learning system to predict disease-specific survival in stage II and stage III colorectal cancer

Accurate prognosis in colorectal cancer can have important implications for clinical management. Here, we develop a deep learning system (DLS) to first identify invasive cancer and then directly predict disease specific survival (DSS) for stage II and stage III colorectal cancer using only digitized histopathology whole-slide images. The DLS was trained using slides from 1173 stage II and 1266 stage III cases (18,304 total slides) and was evaluated on a held-out test set of 601 stage II and 638 stage III cases (9,340 total slides). The area under the receiver operating characteristic curve (AUC) for 5-year DSS prediction was 68.0 for stage II (95% CI 62.2-73.1) and 65.5 for stage III (95% CI 61.1-70.0). For stage II, 5-year DSS was 64% for DLS-predicted high-risk cases versus 89% for DLS-predicted low-risk cases (upper and lower risk quartiles; p&lt;0.001, log rank test). For stage III, 5-year DSS was 35% for DLS-predicted high-risk cases versus 66% for DLS-predicted low-risk cases (upper and lower risk quartiles; p&lt;0.001, log rank test). In a multivariable Cox model, the DLS prediction remained significantly associated with DSS after adjusting for T-category, N-category, age, gender, tumor grade, and lymphovascular invasion (stage II: adjusted hazard ratio 1.55, 95% CI 1.33-1.81, p&lt;0.0001; stage III: adjusted hazard ratio 1.35, 95% CI (1.21-1.51), p&lt;0.0001). Finally, a combined proportional-hazards model using the DLS along with baseline clinicopathologic information provided better risk prediction than the DLS or baseline information alone, increasing 5-year AUC over the baseline-only model by 8.9 points (95% CI 3.9-13.6) and 5.3 points (95% CI 2.3-8.4) for stages II and III, respectively. Taken together, these findings demonstrate that the DLS provides significant prognostic value and risk stratification in both stage II and stage III colorectal cancer, and can be combined with known risk features to further improve prognostic accuracy. This represents novel work to train a DLS to directly predict patient outcomes using whole-slide images and weakly supervised learning. The ability to use non-annotated slides as input has important implications for possible clinical applications and the features learned by the model may also help to identify new prognosis-associated morphologic factors in colorectal cancer. Additional work is ongoing to confirm the utility of these findings, such as validation in additional datasets and interpretability experiments to better understand the features learned by the DLS for these predictions.

Citation Format: Ellery Wulczyn, David F. Steiner, Melissa Moran, Markus Plass, Robert Reihs, Heimo Mueller, Apaar Sadhwani, Yuannan Cai, Isabelle Flament, Po-Hsuan Cameron Chen, Yun Liu, Martin C. Stumpe, Zhaoyang Xu, Kurt Zatloukal, Craig H. Mermel. A deep learning system to predict disease-specific survival in stage II and stage III colorectal cancer [abstract]. In: Proceedings of the Annual Meeting of the American Association for Cancer Research 2020; 2020 Apr 27-28 and Jun 22-24. Philadelphia (PA): AACR; Cancer Res 2020;80(16 Suppl):Abstract nr 2096.

Current and future applications of artificial intelligence in pathology: a clinical perspective

During the last decade, a dramatic rise in the development and application of artificial intelligence (AI) tools for use in pathology services has occurred. This trend is often expected to continue and reshape the field of pathology in the coming years. The deployment of computational pathology and applications of AI tools can be considered as a paradigm shift that will change pathology services, making them more efficient and capable of meeting the needs of this era of precision medicine. Despite the success of AI models, the translational process from discovery to clinical applications has been slow. The gap between self-contained research and clinical environment may be too wide and has been largely neglected. In this review, we cover the current and prospective applications of AI in pathology. We examine its applications in diagnosis and prognosis, and we offer insights for considerations that could improve clinical applicability of these tools. Then, we discuss its potential to improve workflow efficiency, and its benefits in pathologist education. Finally, we review the factors that could influence adoption in clinical practices and the associated regulatory processes.

Deep learning-based survival prediction for multiple cancer types using histopathology images

Providing prognostic information at the time of cancer diagnosis has important implications for treatment and monitoring. Although cancer staging, histopathological assessment, molecular features, and clinical variables can provide useful prognostic insights, improving risk stratification remains an active research area. We developed a deep learning system (DLS) to predict disease specific survival across 10 cancer types from The Cancer Genome Atlas (TCGA). We used a weakly-supervised approach without pixel-level annotations, and tested three different survival loss functions. The DLS was developed using 9,086 slides from 3,664 cases and evaluated using 3,009 slides from 1,216 cases. In multivariable Cox regression analysis of the combined cohort including all 10 cancers, the DLS was significantly associated with disease specific survival (hazard ratio of 1.58, 95% CI 1.28–1.70, p<0.0001) after adjusting for cancer type, stage, age, and sex. In a per-cancer adjusted subanalysis, the DLS remained a significant predictor of survival in 5 of 10 cancer types. Compared to a baseline model including stage, age, and sex, the c-index of the model demonstrated an absolute 3.7% improvement (95% CI 1.0–6.5) in the combined cohort. Additionally, our models stratified patients within individual cancer stages, particularly stage II (p = 0.025) and stage III (p<0.001). By developing and evaluating prognostic models across multiple cancer types, this work represents one of the most comprehensive studies exploring the direct prediction of clinical outcomes using deep learning and histopathology images. Our analysis demonstrates the potential for this approach to provide significant prognostic information in multiple cancer types, and even within specific pathologic stages. However, given the relatively small number of cases and observed clinical events for a deep learning task of this type, we observed wide confidence intervals for model performance, thus highlighting that future work will benefit from larger datasets assembled for the purposes for survival modeling.

Artificial intelligence in digital breast pathology: Techniques and applications

Breast cancer is the most common cancer and second leading cause of cancer-related death worldwide. The mainstay of breast cancer workup is histopathological diagnosis - which guides therapy and prognosis. However, emerging knowledge about the complex nature of cancer and the availability of tailored therapies have exposed opportunities for improvements in diagnostic precision. In parallel, advances in artificial intelligence (AI) along with the growing digitization of pathology slides for the primary diagnosis are a promising approach to meet the demand for more accurate detection, classification and prediction of behaviour of breast tumours. In this article, we cover the current and prospective uses of AI in digital pathology for breast cancer, review the basics of digital pathology and AI, and outline outstanding challenges in the field.

Whole-Slide Image Focus Quality: Automatic Assessment and Impact on AI Cancer Detection

Background

Digital pathology enables remote access or consults and powerful image analysis algorithms. However, the slide digitization process can create artifacts such as out-of-focus (OOF). OOF is often only detected on careful review, potentially causing rescanning, and workflow delays. Although scan time operator screening for whole-slide OOF is feasible, manual screening for OOF affecting only parts of a slide is impractical.

Methods

We developed a convolutional neural network (ConvFocus) to exhaustively localize and quantify the severity of OOF regions on digitized slides. ConvFocus was developed using our refined semi-synthetic OOF data generation process and evaluated using seven slides spanning three different tissue and three different stain types, each of which were digitized using two different whole-slide scanner models ConvFocus's predictions were compared with pathologist-annotated focus quality grades across 514 distinct regions representing 37,700 35 μm × 35 μm image patches, and 21 digitized "z-stack" WSIs that contain known OOF patterns.

Results

When compared to pathologist-graded focus quality, ConvFocus achieved Spearman rank coefficients of 0.81 and 0.94 on two scanners and reproduced the expected OOF patterns from z-stack scanning. We also evaluated the impact of OOF on the accuracy of a state-of-the-art metastatic breast cancer detector and saw a consistent decrease in performance with increasing OOF.

Conclusions

Comprehensive whole-slide OOF categorization could enable rescans before pathologist review, potentially reducing the impact of digitization focus issues on the clinical workflow. We show that the algorithm trained on our semi-synthetic OOF data generalizes well to real OOF regions across tissue types, stains, and scanners. Finally, quantitative OOF maps can flag regions that might otherwise be misclassified by image analysis algorithms, preventing OOF-induced errors.

An augmented reality microscope with real-time artificial intelligence integration for cancer diagnosis

The microscopic assessment of tissue samples is instrumental for the diagnosis and staging of cancer, and thus guides therapy. However, these assessments demonstrate considerable variability and many regions of the world lack access to trained pathologists. Though artificial intelligence (AI) promises to improve the access and quality of healthcare, the costs of image digitization in pathology and difficulties in deploying AI solutions remain as barriers to real-world use. Here we propose a cost-effective solution: the augmented reality microscope (ARM). The ARM overlays AI-based information onto the current view of the sample in real time, enabling seamless integration of AI into routine workflows. We demonstrate the utility of ARM in the detection of metastatic breast cancer and the identification of prostate cancer, with latency compatible with real-time use. We anticipate that the ARM will remove barriers towards the use of AI designed to improve the accuracy and efficiency of cancer diagnosis.

Similar image search for histopathology: SMILY

The increasing availability of large institutional and public histopathology image datasets is enabling the searching of these datasets for diagnosis, research, and education. Although these datasets typically have associated metadata such as diagnosis or clinical notes, even carefully curated datasets rarely contain annotations of the location of regions of interest on each image. As pathology images are extremely large (up to 100,000 pixels in each dimension), further laborious visual search of each image may be needed to find the feature of interest. In this paper, we introduce a deep-learning-based reverse image search tool for histopathology images: Similar Medical Images Like Yours (SMILY). We assessed SMILY's ability to retrieve search results in two ways: using pathologist-provided annotations, and via prospective studies where pathologists evaluated the quality of SMILY search results. As a negative control in the second evaluation, pathologists were blinded to whether search results were retrieved by SMILY or randomly. In both types of assessments, SMILY was able to retrieve search results with similar histologic features, organ site, and prostate cancer Gleason grade compared with the original query. SMILY may be a useful general-purpose tool in the pathologist's arsenal, to improve the efficiency of searching large archives of histopathology images, without the need to develop and implement specific tools for each application.

Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer

For prostate cancer patients, the Gleason score is one of the most important prognostic factors, potentially determining treatment independent of the stage. However, Gleason scoring is based on subjective microscopic examination of tumor morphology and suffers from poor reproducibility. Here we present a deep learning system (DLS) for Gleason scoring whole-slide images of prostatectomies. Our system was developed using 112 million pathologist-annotated image patches from 1226 slides, and evaluated on an independent validation dataset of 331 slides. Compared to a reference standard provided by genitourinary pathology experts, the mean accuracy among 29 general pathologists was 0.61 on the validation set. The DLS achieved a significantly higher diagnostic accuracy of 0.70 (p = 0.002) and trended towards better patient risk stratification in correlations to clinical follow-up data. Our approach could improve the accuracy of Gleason scoring and subsequent therapy decisions, particularly where specialist expertise is unavailable. The DLS also goes beyond the current Gleason system to more finely characterize and quantitate tumor morphology, providing opportunities for refinement of the Gleason system itself.

Abstract LB-055: Clinical acquired resistance to RAF inhibitor combinations in BRAF-mutant colorectal cancer through MAPK pathway alterations

BRAF V600E mutations occur in ∼10% of colorectal cancer (CRC), and are associated with poor prognosis. RAF inhibition alone has not been an effective treatment in BRAF-mutant (BRAFm) CRC patients, with response rates of only 5%, due to persistence of MAPK signaling. Combined RAF/EGFR, RAF/MEK, or RAF/MEK/EGFR inhibitors have produced improved efficacy in BRAFm CRC patients, yet ultimately resistance develops after an initial treatment response. Understanding the mechanisms of clinical acquired resistance that arise to RAF inhibitor combinations in BRAFm CRC patients may lead to valuable opportunities to overcome resistance and prolong clinical response.

We sought to identify clinically relevant mechanisms of acquired resistance to RAF inhibitor combinations by obtaining tumor biopsies from BRAFm CRC patients upon disease progression, after initial response to RAF/EGFR or RAF/MEK inhibitor combinations. Matched pre-treatment, post-progression, and normal DNA were analyzed by whole exome sequencing (WES) and RNA-seq.

In one BRAFm CRC patient with prolonged stable disease on a RAF/EGFR combination, WES identified KRAS amplification in a progressing lesion. FISH confirmed the presence of KRAS amplification in the post-progression biopsy, and RNA-seq revealed KRAS transcript overexpression. Interestingly, in resistant clones generated from BRAFm CRC cell lines selected with either RAF/EGFR or RAF/MEK inhibitors, KRAS exon 2 mutations were identified. Either KRAS amplification or KRAS mutation led to sustained MAPK pathway activity and cross-resistance to either RAF/EGFR or RAF/MEK inhibitor combinations.

In a second patient with a minor response to a RAF/EGFR inhibitor combination, BRAF amplification was identified in a progressing lesion, which was confirmed by FISH and was not present in a pre-treatment biopsy of the same lesion. BRAF amplification led to cross-resistance to the BRAF/MEK inhibitor combination.

In a third patient with a minor response to a RAF/MEK inhibitor combination, WES identified the presence of an ARAF Q489L mutation and a MEK1 F53L mutation in a single progressing lesion, suggesting possible intra-lesional heterogeneity of acquired resistance mechanisms. However, utilizing a cell line derived from the patient's post-progression biopsy, we found that 30 out of 30 single-cell clones harbored both the ARAF and MEK1 mutations, and that MEK1 F53L seemed to function as the primary driver of acquired resistance in these resistant cells. MEK1 F53L expression markedly abrogated the ability of RAF/MEK and RAF/EGFR inhibitor combinations to suppress MAPK signaling.

Despite developing resistance to upstream MAPK pathway inhibitors, we found that each of the acquired resistance mechanisms we detected remained sensitive to ERK inhibition, which could effectively suppress MAPK signaling. Our findings demonstrate the central importance of MAPK pathway activity in BRAFm CRC, and highlight the critical need for MAPK pathway inhibition in the prevention of disease progression. Additionally, our work indicates ERK inhibitors may be valuable additions to future therapeutic combinations for BRAFm CRC patients. Further efforts to understand acquired resistance mechanisms will be vital to developing novel therapeutic strategies to overcome resistance and extend clinical benefit in this lethal CRC subtype.

Citation Format: Leanne G. Ahronian, Erin M. Sennott, Eliezer M. Van Allen, Nikhil Wagle, Eunice L. Kwak, Jason E. Faris, Jason T. Godfrey, Koki Nishimura, Kerry D. Lynch, Craig H. Mermel, Elizabeth L. Lockerman, Anuj Kalsy, Joseph M. Gurski, Samira Bahl, Kristin Anderka, Lisa M. Green, Niall J. Lennon, Tiffany G. Huynh, Mari Mino-Kenudson, Gad Getz, Dora Dias-Santagata, A. John Iafrate, Jeffrey A. Engelman, Levi A. Garraway, Ryan B. Corcoran. Clinical acquired resistance to RAF inhibitor combinations in BRAF-mutant colorectal cancer through MAPK pathway alterations. [abstract]. In: Proceedings of the 106th Annual Meeting of the American Association for Cancer Research; 2015 Apr 18-22; Philadelphia, PA. Philadelphia (PA): AACR; Cancer Res 2015;75(15 Suppl):Abstract nr LB-055. doi:10.1158/1538-7445.AM2015-LB-055

RB loss in resistant EGFR mutant lung adenocarcinomas that transform to small-cell lung cancer

Tyrosine kinase inhibitors are effective treatments for non-small-cell lung cancers (NSCLCs) with epidermal growth factor receptor (EGFR) mutations. However, relapse typically occurs after an average of 1 year of continuous treatment. A fundamental histological transformation from NSCLC to small-cell lung cancer (SCLC) is observed in a subset of the resistant cancers, but the molecular changes associated with this transformation remain unknown. Analysis of tumour samples and cell lines derived from resistant EGFR mutant patients revealed that Retinoblastoma (RB) is lost in 100% of these SCLC transformed cases, but rarely in those that remain NSCLC. Further, increased neuroendocrine marker and decreased EGFR expression as well as greater sensitivity to BCL2 family inhibition are observed in resistant SCLC transformed cancers compared with resistant NSCLCs. Together, these findings suggest that this subset of resistant cancers ultimately adopt many of the molecular and phenotypic characteristics of classical SCLC.

Resistance to tyrosine kinase inhibitors occurs in treatments of non-small-cell lung cancers (NSCLCs) with EGFR mutations but the mechanisms underlying this acquired resistance are unknown. Here the authors examine the molecular changes that occur in resistant cancers that transition from NSCLC to small-cell lung cancer phenotype and implicate loss of retinoblastoma in this process.

Clinical Acquired Resistance to RAF Inhibitor Combinations in BRAF-Mutant Colorectal Cancer through MAPK Pathway Alterations

BRAF mutations occur in approximately 10% of colorectal cancers. Although RAF inhibitor monotherapy is highly effective in BRAF-mutant melanoma, response rates in BRAF-mutant colorectal cancer are poor. Recent clinical trials of combined RAF/EGFR or RAF/MEK inhibition have produced improved efficacy, but patients ultimately develop resistance. To identify molecular alterations driving clinical acquired resistance, we performed whole-exome sequencing on paired pretreatment and postprogression tumor biopsies from patients with BRAF-mutant colorectal cancer treated with RAF inhibitor combinations. We identified alterations in MAPK pathway genes in resistant tumors not present in matched pretreatment tumors, including KRAS amplification, BRAF amplification, and a MEK1 mutation. These alterations conferred resistance to RAF/EGFR or RAF/MEK combinations through sustained MAPK pathway activity, but an ERK inhibitor could suppress MAPK activity and overcome resistance. Identification of MAPK pathway reactivating alterations upon clinical acquired resistance underscores the MAPK pathway as a critical target in BRAF-mutant colorectal cancer and suggests therapeutic options to overcome resistance.

SIGNIFICANCE

RAF inhibitor combinations represent promising approaches in clinical development for BRAF-mutant colorectal cancer. Initial characterization of clinical acquired resistance mechanisms to these regimens identified several MAPK pathway alterations driving resistance by reactivating MAPK signaling, highlighting the critical dependence of BRAF-mutant colorectal cancers on MAPK signaling and offering potential strategies to overcome resistance.

Age-related clonal hematopoiesis associated with adverse outcomes

BACKGROUND

The incidence of hematologic cancers increases with age. These cancers are associated with recurrent somatic mutations in specific genes. We hypothesized that such mutations would be detectable in the blood of some persons who are not known to have hematologic disorders.

METHODS

We analyzed whole-exome sequencing data from DNA in the peripheral-blood cells of 17,182 persons who were unselected for hematologic phenotypes. We looked for somatic mutations by identifying previously characterized single-nucleotide variants and small insertions or deletions in 160 genes that are recurrently mutated in hematologic cancers. The presence of mutations was analyzed for an association with hematologic phenotypes, survival, and cardiovascular events.

RESULTS

Detectable somatic mutations were rare in persons younger than 40 years of age but rose appreciably in frequency with age. Among persons 70 to 79 years of age, 80 to 89 years of age, and 90 to 108 years of age, these clonal mutations were observed in 9.5% (219 of 2300 persons), 11.7% (37 of 317), and 18.4% (19 of 103), respectively. The majority of the variants occurred in three genes: DNMT3A, TET2, and ASXL1. The presence of a somatic mutation was associated with an increase in the risk of hematologic cancer (hazard ratio, 11.1; 95% confidence interval [CI], 3.9 to 32.6), an increase in all-cause mortality (hazard ratio, 1.4; 95% CI, 1.1 to 1.8), and increases in the risks of incident coronary heart disease (hazard ratio, 2.0; 95% CI, 1.2 to 3.4) and ischemic stroke (hazard ratio, 2.6; 95% CI, 1.4 to 4.8).

CONCLUSIONS

Age-related clonal hematopoiesis is a common condition that is associated with increases in the risk of hematologic cancer and in all-cause mortality, with the latter possibly due to an increased risk of cardiovascular disease. (Funded by the National Institutes of Health and others.).

The 2013 symposium on pathology data integration and clinical decision support and the current state of field

Background:

Pathologists and informaticians are becoming increasingly interested in electronic clinical decision support for pathology, laboratory medicine and clinical diagnosis. Improved decision support may optimize laboratory test selection, improve test result interpretation and permit the extraction of enhanced diagnostic information from existing laboratory data. Nonetheless, the field of pathology decision support is still developing. To facilitate the exchange of ideas and preliminary studies, we convened a symposium entitled: Pathology data integration and clinical decision support.

Methods:

The symposium was held at the Massachusetts General Hospital, on May 10, 2013. Participants were selected to represent diverse backgrounds and interests and were from nine different institutions in eight different states.

Results:

The day included 16 plenary talks and three panel discussions, together covering four broad areas. Summaries of each presentation are included in this manuscript.

Conclusions:

A number of recurrent themes emerged from the symposium. Among the most pervasive was the dichotomy between diagnostic data and diagnostic information, including the opportunities that laboratories may have to use electronic systems and algorithms to convert the data they generate into more useful information. Differences between human talents and computer abilities were described; well-designed symbioses between humans and computers may ultimately optimize diagnosis. Another key theme related to the unique needs and challenges in providing decision support for genomics and other emerging diagnostic modalities. Finally, many talks relayed how the barriers to bringing decision support toward reality are primarily personnel, political, infrastructural and administrative challenges rather than technological limitations.

Discovery and saturation analysis of cancer genes across 21 tumour types

Although a few cancer genes are mutated in a high proportion of tumours of a given type (>20%), most are mutated at intermediate frequencies (2-20%). To explore the feasibility of creating a comprehensive catalogue of cancer genes, we analysed somatic point mutations in exome sequences from 4,742 human cancers and their matched normal-tissue samples across 21 cancer types. We found that large-scale genomic analysis can identify nearly all known cancer genes in these tumour types. Our analysis also identified 33 genes that were not previously known to be significantly mutated in cancer, including genes related to proliferation, apoptosis, genome stability, chromatin regulation, immune evasion, RNA processing and protein homeostasis. Down-sampling analysis indicates that larger sample sizes will reveal many more genes mutated at clinically important frequencies. We estimate that near-saturation may be achieved with 600-5,000 samples per tumour type, depending on background mutation frequency. The results may help to guide the next stage of cancer genomics.

Pan-cancer patterns of somatic copy number alteration

Determining how somatic copy number alterations (SCNAs) promote cancer is an important goal. We characterized SCNA patterns in 4,934 cancers from The Cancer Genome Atlas Pan-Cancer data set. Whole-genome doubling, observed in 37% of cancers, was associated with higher rates of every other type of SCNA, TP53 mutations, CCNE1 amplifications and alterations of the PPP2R complex. SCNAs that were internal to chromosomes tended to be shorter than telomere-bounded SCNAs, suggesting different mechanisms underlying their generation. Significantly recurrent focal SCNAs were observed in 140 regions, including 102 without known oncogene or tumor suppressor gene targets and 50 with significantly mutated genes. Amplified regions without known oncogenes were enriched for genes involved in epigenetic regulation. When levels of genomic disruption were accounted for, 7% of region pairs were anticorrelated, and these regions tended to encompass genes whose proteins physically interact, suggesting related functions. These results provide insights into mechanisms of generation and functional consequences of cancer-related SCNAs.

Mutational heterogeneity in cancer and the search for new cancer-associated genes

Major international projects are underway that are aimed at creating a comprehensive catalogue of all the genes responsible for the initiation and progression of cancer. These studies involve the sequencing of matched tumour-normal samples followed by mathematical analysis to identify those genes in which mutations occur more frequently than expected by random chance. Here we describe a fundamental problem with cancer genome studies: as the sample size increases, the list of putatively significant genes produced by current analytical methods burgeons into the hundreds. The list includes many implausible genes (such as those encoding olfactory receptors and the muscle protein titin), suggesting extensive false-positive findings that overshadow true driver events. We show that this problem stems largely from mutational heterogeneity and provide a novel analytical methodology, MutSigCV, for resolving the problem. We apply MutSigCV to exome sequences from 3,083 tumour-normal pairs and discover extraordinary variation in mutation frequency and spectrum within cancer types, which sheds light on mutational processes and disease aetiology, and in mutation frequency across the genome, which is strongly correlated with DNA replication timing and also with transcriptional activity. By incorporating mutational heterogeneity into the analyses, MutSigCV is able to eliminate most of the apparent artefactual findings and enable the identification of genes truly associated with cancer.

Systematic interrogation of 3q26 identifies TLOC1 and SKIL as cancer drivers

3q26 is frequently amplified in several cancer types with a common amplified region containing 20 genes. To identify cancer driver genes in this region, we interrogated the function of each of these genes by loss- and gain-of-function genetic screens. Specifically, we found that TLOC1 (SEC62) was selectively required for the proliferation of cell lines with 3q26 amplification. Increased TLOC1 expression induced anchorage-independent growth, and a second 3q26 gene, SKIL (SNON), facilitated cell invasion in immortalized human mammary epithelial cells. Expression of both TLOC1 and SKIL induced subcutaneous tumor growth. Proteomic studies showed that TLOC1 binds to DDX3X, which is essential for TLOC1-induced transformation and affected protein translation. SKIL induced invasion through upregulation of SLUG (SNAI2) expression. Together, these studies identify TLOC1 and SKIL as driver genes at 3q26 and more broadly suggest that cooperating genes may be coamplified in other regions with somatic copy number gain.

SIGNIFICANCE

These studies identify TLOC1 and SKIL as driver genes in 3q26. These observations provide evidence that regions of somatic copy number gain may harbor cooperating genes of different but complementary functions.

Prognostically relevant gene signatures of high-grade serous ovarian carcinoma

Because of the high risk of recurrence in high-grade serous ovarian carcinoma (HGS-OvCa), the development of outcome predictors could be valuable for patient stratification. Using the catalog of The Cancer Genome Atlas (TCGA), we developed subtype and survival gene expression signatures, which, when combined, provide a prognostic model of HGS-OvCa classification, named "Classification of Ovarian Cancer" (CLOVAR). We validated CLOVAR on an independent dataset consisting of 879 HGS-OvCa expression profiles. The worst outcome group, accounting for 23% of all cases, was associated with a median survival of 23 months and a platinum resistance rate of 63%, versus a median survival of 46 months and platinum resistance rate of 23% in other cases. Associating the outcome prediction model with BRCA1/BRCA2 mutation status, residual disease after surgery, and disease stage further optimized outcome classification. Ovarian cancer is a disease in urgent need of more effective therapies. The spectrum of outcomes observed here and their association with CLOVAR signatures suggests variations in underlying tumor biology. Prospective validation of the CLOVAR model in the context of additional prognostic variables may provide a rationale for optimal combination of patient and treatment regimens.

Detection of preanalytic laboratory testing errors using a statistically guided protocol

Preanalytic laboratory testing errors are often difficult to identify. We demonstrate how laboratories can integrate statistical models with clinical judgment to develop protocols for preanalytic error detection. Specifically, we developed a protocol to identify spuriously elevated glucose values resulting from improper "line draws" or related phlebotomy errors. Using a decision tree-generating algorithm and an annotated set of training data, we generated decision trees to classify critically elevated glucose results as "real" or "spurious" based on available laboratory parameters. Decision trees revealed that a 30-day patient-specific average glucose concentration lower than 186.3 mg/dL (10.3 mmol/L), a current glucose concentration higher than 663 mg/dL (37 mmol/L), and an anion gap lower than 16.5 mEq/L (16.5 mmol/L) suggested a spurious result. We then used the results from the decision tree analysis to inform the implementation of a clinical protocol that significantly improved the laboratory's identification of spurious results. Similar approaches may be useful in developing protocols to identify other errors or to assist in clinical interpretation of results.

Recurrent hemizygous deletions in cancers may optimize proliferative potential

Tumors exhibit numerous recurrent hemizygous focal deletions that contain no known tumor suppressors and are poorly understood. To investigate whether these regions contribute to tumorigenesis, we searched genetically for genes with cancer-relevant properties within these hemizygous deletions. We identified STOP and GO genes, which negatively and positively regulate proliferation, respectively. STOP genes include many known tumor suppressors, whereas GO genes are enriched for essential genes. Analysis of their chromosomal distribution revealed that recurring deletions preferentially overrepresent STOP genes and underrepresent GO genes. We propose a hypothesis called the cancer gene island model, whereby gene islands encompassing high densities of STOP genes and low densities of GO genes are hemizygously deleted to maximize proliferative fitness through cumulative haploinsufficiencies. Because hundreds to thousands of genes are hemizygously deleted per tumor, this mechanism may help to drive tumorigenesis across many cancer types.

GISTIC2.0 facilitates sensitive and confident localization of the targets of focal somatic copy-number alteration in human cancers

We describe methods with enhanced power and specificity to identify genes targeted by somatic copy-number alterations (SCNAs) that drive cancer growth. By separating SCNA profiles into underlying arm-level and focal alterations, we improve the estimation of background rates for each category. We additionally describe a probabilistic method for defining the boundaries of selected-for SCNA regions with user-defined confidence. Here we detail this revised computational approach, GISTIC2.0, and validate its performance in real and simulated datasets.

ERG rearrangement is specific to prostate cancer and does not occur in any other common tumor

Identification of specific somatic gene alterations is crucial for the insight into the development, progression, and clinical behavior of individual cancer types. The recently discovered recurrent ERG rearrangement in prostate cancer might represent a prostate cancer-specific alteration that has not been systematically assessed in tumors other than prostate cancer. Aim of this study was to assess, whether the ERG rearrangement and the distinct deletion site between TMPRSS2 and ERG, both predominantly resulting in a TMPRSS2-ERG fusion, occur in tumors other than prostate cancer. We assessed 54 different tumor types (2942 samples in total) for their ERG rearrangement status by fluorescence in situ hybridization (FISH). To calibrate, we analyzed 285 prostate cancer samples for the ERG rearrangement frequency. Additionally, we interrogated a high-resolution single nucleotide polymorphism (SNP) data set across 3131 cancer specimens (26 tumor types) for copy number alterations. None of the 54 different tumor types assessed by FISH harbored an ERG rearrangement, whereas the prostate cancer samples revealed an ERG rearrangement in 49.5% of cases. Furthermore, within the 26 tumor types assessed for copy number alterations by SNP, the distinct deletion site between TMPRSS2 and ERG (21q22.2-3) was detectable exclusively in prostate cancer. Although Ewing's sarcoma and AML have known rearrangements rarely involving ERG, we hypothesize that the ERG rearrangement as well as the distinct deletion site on 21q22.2-3 between TMPRSS2 and ERG are prostate-cancer-specific genomic alterations. These observations provide further insight into the oncogenesis of prostate cancer and might be critical for the development of ERG rearrangement assessment as a clinical tool.

The landscape of somatic copy-number alteration across human cancers

A powerful way to discover key genes playing causal roles in oncogenesis is to identify genomic regions that undergo frequent alteration in human cancers. Here, we report high-resolution analyses of somatic copy-number alterations (SCNAs) from 3131 cancer specimens, belonging largely to 26 histological types. We identify 158 regions of focal SCNA that are altered at significant frequency across multiple cancer types, of which 122 cannot be explained by the presence of a known cancer target gene located within these regions. Several gene families are enriched among these regions of focal SCNA, including the BCL2 family of apoptosis regulators and the NF-κB pathway. We show that cancer cells harboring amplifications surrounding the MCL1 and BCL2L1 anti-apoptotic genes depend upon expression of these genes for survival. Finally, we demonstrate that a large majority of SCNAs identified in individual cancer types are present in multiple cancer types.

SOX2 is an amplified lineage-survival oncogene in lung and esophageal squamous cell carcinomas

Lineage-survival oncogenes are activated by somatic DNA alterations in cancers arising from the cell lineages in which these genes play a role in normal development. Here we show that a peak of genomic amplification on chromosome 3q26.33 found in squamous cell carcinomas (SCCs) of the lung and esophagus contains the transcription factor gene SOX2, which is mutated in hereditary human esophageal malformations, is necessary for normal esophageal squamous development, promotes differentiation and proliferation of basal tracheal cells and cooperates in induction of pluripotent stem cells. SOX2 expression is required for proliferation and anchorage-independent growth of lung and esophageal cell lines, as shown by RNA interference experiments. Furthermore, ectopic expression of SOX2 here cooperated with FOXE1 or FGFR2 to transform immortalized tracheobronchial epithelial cells. SOX2-driven tumors show expression of markers of both squamous differentiation and pluripotency. These characteristics identify SOX2 as a lineage-survival oncogene in lung and esophageal SCC.

Lin28 promotes transformation and is associated with advanced human malignancies

Multiple members of the let-7 family of miRNAs are often repressed in human cancers1,2, thereby promoting oncogenesis by de-repressing the targets K-Ras, c-Myc, and HMGA2 3,4. However, the mechanism by which let-7 miRNAs are coordinately repressed is unclear. The RNA-binding proteins Lin28 and Lin28B block let-7 precursors from being processed to mature miRNAs5–8, suggesting that over-expression of Lin28/Lin28B might promote malignancy via repression of let-7. Here we show that LIN28 and LIN28B are over-expressed in primary human tumors and human cancer cell lines (overall frequency ∼15%), and that over-expression is linked to repression of let-7 family miRNAs and de-repression of let-7 targets. Lin28/Lin28B facilitate cellular transformation in vitro, and over-expression is associated with advanced disease across multiple tumor types. Our work provides a mechanism for the coordinate repression of let-7 miRNAs observed in a subset of human cancers, and associates activation of LIN28/LIN28B with poor clinical prognosis.

Src family kinases are important negative regulators of G-CSF-dependent granulopoiesis

Granulocyte colony-stimulating factor (G-CSF) is the principal cytokine regulating granulopoiesis. Truncation mutations of the G-CSF receptor (G-CSFR) are associated with the development of acute myeloid leukemia in patients with severe congenital neutropenia. Although increased proliferative signaling by a representative G-CSFR truncation mutation (termed d715) has been documented, the molecular basis for this hyperproliferative phenotype has not been fully characterized. Given the accumulating evidence implicating Src family kinases in the transduction of cytokine receptor signals, the role of these kinases in the regulation of G-CSF signaling was examined. We show that Hck and Lyn, Src family kinases expressed in myeloid cells, are negative regulators of granulopoiesis that act at distinct stages of granulocytic differentiation. Whereas Hck regulates the G-CSF-induced proliferation of granulocytic precursors, Lyn regulates the production of myeloid progenitors. Interestingly, d715 G-CSFR myeloid progenitors were resistant to the growth-stimulating effect of treatment with a Src kinase inhibitor. Together, these data establish Lyn and Hck as key negative regulators of granulopoiesis and raise the possibility that loss of Src family kinase activation by the d715 G-CSFR may contribute to its hyperproliferative phenotype.