Free digital image analysis software helps to resolve equivocal scores in HER2 immunohistochemistry

The Performance and Clinical Applicability of HER2 Digital Image Analysis in Breast Cancer: A Systematic Review

This systematic review aims to address the research gap in the performance of computational algorithms for the digital image analysis of HER2 images in clinical settings. While numerous studies have explored various aspects of these algorithms, there is a lack of comprehensive evaluation regarding their effectiveness in real-world clinical applications. We conducted a search of the Web of Science and PubMed databases for studies published from 31 December 2013 to 30 June 2024, focusing on performance effectiveness and components such as dataset size, diversity and source, ground truth, annotation, and validation methods. The study was registered with PROSPERO (CRD42024525404). Key questions guiding this review include the following: How effective are current computational algorithms at detecting HER2 status in digital images? What are the common validation methods and dataset characteristics used in these studies? Is there standardization of algorithm evaluations of clinical applications that can improve the clinical utility and reliability of computational tools for HER2 detection in digital image analysis? We identified 6833 publications, with 25 meeting the inclusion criteria. The accuracy rate with clinical datasets varied from 84.19% to 97.9%. The highest accuracy was achieved on the publicly available Warwick dataset at 98.8% in synthesized datasets. Only 12% of studies used separate datasets for external validation; 64% of studies used a combination of accuracy, precision, recall, and F1 as a set of performance measures. Despite the high accuracy rates reported in these studies, there is a notable absence of direct evidence supporting their clinical application. To facilitate the integration of these technologies into clinical practice, there is an urgent need to address real-world challenges and overreliance on internal validation. Standardizing study designs on real clinical datasets can enhance the reliability and clinical applicability of computational algorithms in improving the detection of HER2 cancer.

Revolutionizing Pathology with Artificial Intelligence: Innovations in Immunohistochemistry

Artificial intelligence (AI) is a reality of our times, and it has been successfully implemented in all fields, including medicine. As a relatively new domain, all efforts are directed towards creating algorithms applicable in most medical specialties. Pathology, as one of the most important areas of interest for precision medicine, has received significant attention in the development and implementation of AI algorithms. This focus is especially important for achieving accurate diagnoses. Moreover, immunohistochemistry (IHC) serves as a complementary diagnostic tool in pathology. It can be further augmented through the application of deep learning (DL) and machine learning (ML) algorithms for assessing and analyzing immunohistochemical markers. Such advancements can aid in delineating targeted therapeutic approaches and prognostic stratification. This article explores the applications and integration of various AI software programs and platforms used in immunohistochemical analysis. It concludes by highlighting the application of these technologies to pathologies such as breast, prostate, lung, melanocytic proliferations, and hematologic conditions. Additionally, it underscores the necessity for further innovative diagnostic algorithms to assist physicians in the diagnostic process.

Immunohistochemical assessment of HER2 low breast cancer: interobserver reproducibility and correlation with digital image analysis

PURPOSE

The recent findings from the DESTINY-Breast04 trial highlighted the clinical importance of distinguishing between HER2 immunohistochemistry (IHC) scores 0 and 1 + in metastatic breast cancer (BC). However, pathologist interpretation of HER2 IHC scoring is subjective, and standardized methodology is needed. We evaluated the consistency of HER2 IHC scoring among pathologists and the accuracy of digital image analysis (DIA) in interpreting HER2 IHC staining in cases of HER2-low BC.

METHODS

Fifty whole-slide biopsies of BC with HER2 IHC staining were evaluated, comprising 25 cases originally reported as IHC score 0 and 25 as 1 +. These slides were digitally scanned. Six pathologists with breast expertise independently reviewed and scored the scanned images, and DIA was applied. Agreement among pathologists and concordance between pathologist scores and DIA results were statistically analyzed using Kendall coefficient of concordance (W) tests.

RESULTS

Substantial agreement among at least five of the six pathologists was found for 18 of the score 0 cases (72%) and 15 of the score 1 + cases (60%), indicating excellent interobserver agreement (W = 0.828). DIA scores were highly concordant with pathologist scores in 96% of cases (47/49), indicating excellent concordance (W = 0.959).

CONCLUSION

Although breast subspecialty pathologists were relatively consistent in evaluating BC with HER2 IHC scores of 0 and 1 +, DIA may be a reliable supplementary tool to enhance the standardization and quantification of HER2 IHC assessment, especially in challenging cases where results may be ambiguous (i.e., scores 0-1 +). These findings hold promise for improving the accuracy and consistency of HER2 testing.

Early Breast Cancer Risk Assessment: Integrating Histopathology with Artificial Intelligence

Effective risk assessment in early breast cancer is essential for informed clinical decision-making, yet consensus on defining risk categories remains challenging. This paper explores evolving approaches in risk stratification, encompassing histopathological, immunohistochemical, and molecular biomarkers alongside cutting-edge artificial intelligence (AI) techniques. Leveraging machine learning, deep learning, and convolutional neural networks, AI is reshaping predictive algorithms for recurrence risk, thereby revolutionizing diagnostic accuracy and treatment planning. Beyond detection, AI applications extend to histological subtyping, grading, lymph node assessment, and molecular feature identification, fostering personalized therapy decisions. With rising cancer rates, it is crucial to implement AI to accelerate breakthroughs in clinical practice, benefiting both patients and healthcare providers. However, it is important to recognize that while AI offers powerful automation and analysis tools, it lacks the nuanced understanding, clinical context, and ethical considerations inherent to human pathologists in patient care. Hence, the successful integration of AI into clinical practice demands collaborative efforts between medical experts and computational pathologists to optimize patient outcomes.

Novel approach to HER2 quantification using phosphor-integrated dots in human breast invasive cancer microarray

HER2 expression in breast cancer is evaluated to select patients for anti-HER2 therapy. With the advent of newly approved HER2-targeted drugs for low HER2 expression breast cancer, more solid evidence on the whole spectrum of HER2 expression is needed. In this study, we quantitatively assessed HER2 expression from the whole core by combining high-intensity phosphor-integrated dot (PID) immunostaining and whole slide imaging (WSI) analysis. Two types of staining were performed using a 170-core tissue microarray of invasive breast cancer. First, HER2 was stained by immunohistochemistry (IHC), and IHC scores were determined by two practicing pathologists according to the ASCO/CAP HER2 guideline. Second, HER2 was stained with PID, and tentative PID scores were determined by quantitative analysis. The results show that PID can numerically classify HER2 expression status into scores 3+, 2+, 1+, and 0. The HER2 value quantified by PID strongly correlated with the 3, 3'-diaminobenzidine (DAB) IHC score determined by pathologists (R2 = 0.93). PID IHC score 1+ cases included both DAB IHC score 1+ and 0 cases, and low HER2 expression cases appeared to be often evaluated as DAB IHC score 0. Therefore, digital image analysis by PID and WSI can help stratify HER2 IHC. It may also help classify low HER2 expression.

High HER2 Intratumoral Heterogeneity Is a Predictive Factor for Poor Prognosis in Early-Stage and Locally Advanced HER2-Positive Breast Cancer

PURPOSE

Breast cancer tumors frequently have intratumoral heterogeneity (ITH). Tumors with high ITH cause therapeutic resistance and have human epidermal growth factor receptor 2 (HER2) heterogeneity in response to HER2-targeted therapies. This study aimed to investigate whether high HER2 heterogeneity levels were clinically related to a poor prognosis for HER2-targeted adjuvant therapy resistance in primary breast cancers.

METHODS

This study included patients with primary breast cancer (n = 251) treated with adjuvant HER2-targeted therapies. HER2 heterogeneity was manifested by the shape of HER2 fluorescence in situ hybridization amplification (FISH) distributed histograms with the HER2 gene copy number within a tumor sample. Each tumor was classified into a biphasic grade graph (high heterogeneity [HH]) group or a monophasic grade graph (low heterogeneity [LH]) group based on heterogeneity. Both groups were evaluated for disease-free survival (DFS) and overall survival (OS) for a median of ten years of annual follow-up.

RESULTS

Of 251 patients with HER2-positive breast cancer, 46 (18.3%) and 205 (81.7%) were classified into the HH and LH groups, respectively. The HH group had more distant metastases and a poorer prognosis than the LH group (DFS: p < 0.001 (HH:63% vs. LH:91% at 10 years) and for the OS: p = 0.012 (HH:78% vs. LH:95% at 10 years).

CONCLUSIONS

High HER2 heterogeneity is a poor prognostic factor in patients with HER2-positive breast cancer. A novel approach to heterogeneity, which is manifested by the shape of HER2 FISH distributions, might be clinically useful in the prognosis prediction of patients after HER2 adjuvant therapy.

Applications of Artificial Intelligence in Breast Pathology

CONTEXT.—

Increasing implementation of whole slide imaging together with digital workflow and advances in computing capacity enable the use of artificial intelligence (AI) in pathology, including breast pathology. Breast pathologists often face a significant workload, with diagnosis complexity, tedious repetitive tasks, and semiquantitative evaluation of biomarkers. Recent advances in developing AI algorithms have provided promising approaches to meet the demand in breast pathology.

OBJECTIVE.—

To provide an updated review of AI in breast pathology. We examined the success and challenges of current and potential AI applications in diagnosing and grading breast carcinomas and other pathologic changes, detecting lymph node metastasis, quantifying breast cancer biomarkers, predicting prognosis and therapy response, and predicting potential molecular changes.

DATA SOURCES.—

We obtained data and information by searching and reviewing literature on AI in breast pathology from PubMed and based our own experience.

CONCLUSIONS.—

With the increasing application in breast pathology, AI not only assists in pathology diagnosis to improve accuracy and reduce pathologists' workload, but also provides new information in predicting prognosis and therapy response.

HER2 Intratumoral Heterogeneity in Breast Cancer, an Evolving Concept

Amplification and/or overexpression of human epidermal growth factor receptor 2 (HER2) in breast cancer is associated with an adverse prognosis. The introduction of anti-HER2 targeted therapy has dramatically improved the clinical outcomes of patients with HER2-positive breast cancer. Unfortunately, a significant number of patients eventually relapse and develop distant metastasis. HER2 intratumoral heterogeneity (ITH) has been reported to be associated with poor prognosis in patients with anti-HER2 targeted therapies and was proposed to be a potential mechanism for anti-HER2 resistance. In this review, we described the current definition, common types of HER2 ITH in breast cancer, the challenge in interpretation of HER2 status in cases showing ITH and the clinical applications of anti-HER2 agents in breast cancer showing heterogeneous HER2 expression. Digital image analysis has emerged as an objective and reproducible scoring method and its role in the assessment of HER2 status with ITH remains to be demonstrated.

The Role of Artificial Intelligence in Accurate Interpretation of HER2 Immunohistochemical Scores 0 and 1+ in Breast Cancer

The new human epidermal growth factor receptor (HER)2-targeting antibody-drug conjugate offers the opportunity to treat patients with HER2-low breast cancer. Distinguishing HER2 immunohistochemical (IHC) scores of 0 and 1+ is not only critical but also challenging owing to HER2 heterogeneity and variability of observers. In this study, we aimed to increase the interpretation accuracy and consistency of HER2 IHC 0 and 1+ evaluation through assistance from an artificial intelligence (AI) algorithm. In addition, we examined the value of our AI algorithm in evaluating HER2 IHC scores in tumors with heterogeneity. AI-assisted interpretation consisted of AI algorithms and an augmenting reality module with a microscope. Fifteen pathologists (5 junior, 5 midlevel, and 5 senior) participated in this multi-institutional 2-round ring study that included 246 infiltrating duct carcinoma cases that were not otherwise specified. In round 1, pathologists analyzed 246 HER2 IHC slides by microscope without AI assistance. After a 2-week washout period, the pathologists read the same slides with AI algorithm assistance and rendered the definitive results by adjusting to the AI algorithm. The accuracy of interpretation accuracy with AI assistance (0.93 vs 0.80), thereby the evaluation precision of HER2 0 and the recall of HER2 1+. In addition, the AI algorithm improved the total consistency (intraclass correlation coefficient = 0.542-0.812), especially in HER2 1+ cases. In cases with heterogeneity, accuracy improved significantly (0.68 to 0.89) and to a similar level as in cases without heterogeneity (accuracy, 0.97). Both accuracy and consistency improved more for junior pathologists than those for the midlevel and senior pathologists. To the best of our knowledge, this is the first study to show that the accuracy and consistency of HER2 IHC 0 and 1+ evaluation and the accuracy of HER2 IHC evaluation in breast cancers with heterogeneity can be significantly improved using AI-assisted interpretation.

Digital image analysis and assisted reading of the HER2 score display reduced concordance: pitfalls in the categorisation of HER2-low breast cancer

AIMS

Digital image analysis (DIA) is used increasingly as an assisting tool to evaluate biomarkers, including human epidermal growth factor receptor 2 (HER2) in invasive breast cancer (BC). DIA can assist pathologists in HER2 evaluation by presenting quantitative information about the HER2 staining in APP assisted reading (AR). Concurrently, the HER2-low category (HER2-1+/2+ without HER2 gene amplification) has gained prominence due to newly developed antibody-drug conjugates. However, major inter- and intraobserver variability have been observed for the entity. The present quality assurance study investigated the concordance between DIA and AR in clinical use, especially concerning the HER2-low category.

METHODS AND RESULTS

HER2 immunohistochemistry (IHC) in 761 tumours from 727 patients was evaluated in tissue microarray (TMA) cores by DIA (Visiopharm HER2-CONNECT) and AR. Overall concordance between HER2-scores were 73% (n = 552, weighted-κ: 0.66), and 88% (n = 669, weighted-κ: 0.70), when combining HER2-0/1+. A total of 205 scores were discordant by one category, while four were discordant by two categories. A heterogeneous HER2 pattern was relatively common in the discordant cases and a pitfall in the categorisation of HER2-low BC. AR more commonly reassigned a lower HER2 score (from HER2-1+ to HER2-0) within the HER2-low subgroup (n = 624) compared with DIA.

CONCLUSION

DIA and AR display moderate agreement with heterogeneous and aberrant staining, representing a source of discordance and a pitfall in the evaluation of HER2.

Determining HER2 Status by Artificial Intelligence: An Investigation of Primary, Metastatic, and HER2 Low Breast Tumors

The expression of human epidermal growth factor receptor 2 (HER2) protein or gene transcripts is critical for therapeutic decision making in breast cancer. We examined the performance of a digitalized and artificial intelligence (AI)-assisted workflow for HER2 status determination in accordance with the American Society of Clinical Oncology (ASCO)/College of Pathologists (CAP) guidelines. Our preliminary cohort consisted of 495 primary breast carcinomas, and our study cohort included 67 primary breast carcinomas and 30 metastatic deposits, which were evaluated for HER2 status by immunohistochemistry (IHC) and in situ hybridization (ISH). Three practicing breast pathologists independently assessed and scored slides, building the ground truth. Following a washout period, pathologists were provided with the results of the AI digital image analysis (DIA) and asked to reassess the slides. Both rounds of assessment from the pathologists were compared to the AI results and ground truth for each slide. We observed an overall HER2 positivity rate of 15% in our study cohort. Moderate agreement (Cohen's κ 0.59) was observed between the ground truth and AI on IHC, with most discrepancies occurring between 0 and 1+ scores. Inter-observer agreement amongst pathologists was substantial (Fleiss´ κ 0.77) and pathologists' agreement with AI scores was 80.6%. Substantial agreement of the AI with the ground truth (Cohen´s κ 0.80) was detected on ISH-stained slides, and the accuracy of AI was similar for the primary and metastatic tumors. We demonstrated the feasibility of a combined HER2 IHC and ISH AI workflow, with a Cohen's κ of 0.94 when assessed in accordance with the ASCO/CAP recommendations.

Current Biological, Pathological and Clinical Landscape of HER2-Low Breast Cancer

HER2-low breast cancer (BC) is a newly defined subset of HER2-negative BC that has HER2 immunohistochemical (IHC) score of 1+ or score of 2+/in situ hybridization (ISH) negative phenotype. Recent clinical trials have demonstrated significant clinical benefits of novel HER2 directing antibody-drug conjugates (ADCs) in treating this group of tumors. Trastuzumab-deruxtecan (T-Dxd), a HER2-directing ADC was recently approved by the U.S. Food and Drug Administration as the first targeted therapy to treat HER2-low BC. However, HER2-low BC is still not well characterized clinically and pathologically. This review aims to update the current biological, pathological and clinical landscape of HER2-low BC based on the English literature published in the past two years and to propose the future directions on clinical management, pathology practice, and translational research in this subset of BC. We hope it would help better understand the tumor biology of HER2-low BC and the current efforts for identifying and treating this newly recognized targetable group of BC.

Update on prognostic and predictive biomarkers of breast cancer

Breast cancer represents a heterogeneous group of human cancer at both histological and molecular levels. Estrogen receptor (ER), progesterone receptor (PR) and human epidermal growth factor receptor 2 (HER2) are the most commonly used biomarkers in clinical practice for making treatment plans for breast cancer patients by oncologists. Recently, PD-L1 testing plays an important role for immunotherapy for triple-negative breast cancer. With the increased understanding of the molecular characterization of breast cancer and the emergence of novel targeted therapies, more potential biomarkers are needed for the development of more personalized treatments. In this review, we summarized several main prognostic and predictive biomarkers in breast cancer at genomic, transcriptomic and proteomic levels, including hormone receptors, HER2, Ki67, multiple gene expression assays, PD-L1 testing, mismatch repair deficiency/microsatellite instability, tumor mutational burden, PIK3CA, ESR1 andNTRK and briefly introduced the roles of digital imaging analysis in breast biomarker evaluation.

Quantitative Image Analysis as an Adjunct to Manual Scoring of ER, PgR, and HER2 in Invasive Breast Carcinoma

OBJECTIVES

Biomarker expression evaluation for estrogen receptor (ER), progesterone receptor (PgR), and human epidermal growth factor receptor 2 (HER2) is an essential prognostic and predictive parameter for breast cancer and critical for guiding hormonal and neoadjuvant therapy. This study compared quantitative image analysis (QIA) with pathologists' scoring for ER, PgR, and HER2.

METHODS

A retrospective analysis was undertaken of 1,367 invasive breast carcinomas, including all histopathology subtypes, for which ER, PgR, and HER2 were analyzed by manual scoring and QIA. The resulting scores were compared, and in a subset of HER2 cases (n = 373, 26%), scores were correlated with available fluorescence in situ hybridization (FISH) results.

RESULTS

Concordance between QIA and manual scores for ER, PgR, and HER2 was 93%, 96%, and 90%, respectively. Discordant cases had low positive scores (1%-10%) for ER (n = 33), were due to nonrepresentative region selection (eg, ductal carcinoma in situ) or tumor heterogeneity for PgR (n = 43), and were of one-step difference (negative to equivocal, equivocal to positive, or vice versa) for HER2 (n = 90). Among HER2 cases where FISH results were available, only four (1.0%) showed discordant QIA and FISH results.

CONCLUSIONS

QIA is a computer-aided diagnostic support tool for pathologists. It significantly improves ER, PgR, and HER2 scoring standardization. QIA demonstrated excellent concordance with pathologists' scores. To avoid pitfalls, pathologist oversight of representative region selection is recommended.

Unusual Patterns of HER2 Expression in Breast Cancer: Insights and Perspectives

The biomarker human epidermal growth factor receptor-2 (HER2) has represented the best example of successful targeted therapy in breast cancer patients. Based on the concept of "oncogene addiction," we have learnt how to identify patients likely benefitting from anti-HER2 agents. Since HER2 gene amplification leads to marked overexpression of the HER2 receptors on the cell membrane, immunohistochemistry with clinically validated antibodies and scoring system based on intensity and completeness of the membranous expression constitute the screening method to separate negative (score 0/1+) and positive (score 3+) carcinomas and to identify those tumours with complete yet only moderate HER2 expression (score 2+, equivocal carcinomas), which need to be investigated further in terms of gene status to confirm the presence of a loop of oncogene addiction. This process has demanded quality controls and led to recommendations by Scientific Societies, which pathologists routinely need to follow to guarantee reproducibility. In this review, we will span from the description of classical HER2 evaluation to the discussion of those scenarios in which HER2 expression is unusual and/or difficult to define. We will dissect HER2 heterogeneity, HER2 conversion from primary to relapsed/metastatic breast cancer, and we will introduce the new category of HER2-low breast carcinomas.

Digital Image Analysis-Based Evaluation of Claudin-1 and Claudin-7 Delocalization in Cutaneous Squamous Cell Carcinoma and in Its Precancerous State

Accumulating evidence has revealed that delocalization of the transmembrane proteins, Claudin-1 and Claudin-7, to the cytoplasm and/or nucleus occurs in various tumors. However, their subcellular distribution in terms of the membrane, cytoplasm, and nucleus and relationship with signaling pathways have not been elucidated during carcinogenesis. We first determined the expression of these proteins in the membrane, cytoplasm, and nucleus using ImageJ software and automatically collected the immunohistochemical quantification of dysplasia (actinic keratosis (AK)), carcinoma in situ (CIS; Bowen’s disease (BD)), and invasive cutaneous squamous cell carcinoma (SCC) for digital image analysis (DIA). The activity of p-ERK, p-AKT, and p-mTOR and their correlation with subcellular Claudin-1 and Claudin-7 were also performed. Finally, we validated Claudin-1 and Claudin-7 delocalization at the cytoplasm and nucleus in cultured human normal keratinocytes and cutaneous SCC cells. Claudin-1 and Claudin-7 were delocalized as revealed by membranous, cytoplasmic, and nuclear staining in sun-exposed skin, AK, BD, and SCC. In BD, both membranous and cytoplasmic Claudin-1 (nuclear Claudin-1 decrease but no significant difference) were higher than AK, while Claudin-7 almost had the opposite situation. In SCC, cytoplasmic and nuclear Claudin-1 (membranous Claudin-1 no significant difference) was lower than in AK and sun-exposed skin, while Claudin-7 had higher membranous and cytoplasmic but lower nuclear expression. Moreover, p-AKT and p-mTOR (but not p-ERK) were downregulated in the SCC. Subcellular Claudin-1 and Claudin-7 were not only correlated with each other, but also correlated with p-ERK in BD and p-AKT and p-mTOR in SCC. Together, these results imply the delocalization of Claudin-1 and Claudin-7 and their correlation with MAPK/ERK and PI3K-AKT-mTOR signaling pathways in tumorigenesis and infiltration in cutaneous SCC.

Intratumoral Heterogeneity and Immune Response Indicators to Predict Overall Survival in a Retrospective Study of HER2-Borderline (IHC 2+) Breast Cancer Patients

Breast cancer (BC) categorized as human epidermal growth factor receptor 2 (HER2) borderline [2+ by immunohistochemistry (IHC 2+)] presents challenges for the testing, frequently obscured by intratumoral heterogeneity (ITH). This leads to difficulties in therapy decisions. We aimed to establish prognostic models of overall survival (OS) of these patients, which take into account spatial aspects of ITH and tumor microenvironment by using hexagonal tiling analytics of digital image analysis (DIA). In particular, we assessed the prognostic value of Immunogradient indicators at the tumor–stroma interface zone (IZ) as a feature of antitumor immune response. Surgical excision samples stained for estrogen receptor (ER), progesterone receptor (PR), Ki67, HER2, and CD8 from 275 patients with HER2 IHC 2+ invasive ductal BC were used in the study. DIA outputs were subsampled by HexT for ITH quantification and tumor microenvironment extraction for Immunogradient indicators. Multiple Cox regression revealed HER2 membrane completeness (HER2 MC) (HR: 0.18, p = 0.0007), its spatial entropy (HR: 0.37, p = 0.0341), and ER contrast (HR: 0.21, p = 0.0449) as independent predictors of better OS, with worse OS predicted by pT status (HR: 6.04, p = 0.0014) in the HER2 non-amplified patients. In the HER2-amplified patients, HER2 MC contrast (HR: 0.35, p = 0.0367) and CEP17 copy number (HR: 0.19, p = 0.0035) were independent predictors of better OS along with worse OS predicted by pN status (HR: 4.75, p = 0.0018). In the non-amplified tumors, three Immunogradient indicators provided the independent prognostic value: CD8 density in the tumor aspect of the IZ and CD8 center of mass were associated with better OS (HR: 0.23, p = 0.0079 and 0.14, p = 0.0014, respectively), and CD8 density variance along the tumor edge predicted worse OS (HR: 9.45, p = 0.0002). Combining these three computational indicators of the CD8 cell spatial distribution within the tumor microenvironment augmented prognostic stratification of the patients. In the HER2-amplified group, CD8 cell density in the tumor aspect of the IZ was the only independent immune response feature to predict better OS (HR: 0.22, p = 0.0047). In conclusion, we present novel prognostic models, based on computational ITH and Immunogradient indicators of the IHC biomarkers, in HER2 IHC 2+ BC patients.

Quantitative Image Analysis for Tissue Biomarker Use: A White Paper From the Digital Pathology Association

Tissue biomarkers have been of increasing utility for scientific research, diagnosing disease, and treatment response prediction. There has been a steady shift away from qualitative assessment toward providing more quantitative scores for these biomarkers. The application of quantitative image analysis has thus become an indispensable tool for in-depth tissue biomarker interrogation in these contexts. This white paper reviews current technologies being employed for quantitative image analysis, their application and pitfalls, regulatory framework demands, and guidelines established for promoting their safe adoption in clinical practice.

Can AI-assisted microscope facilitate breast HER2 interpretation? A multi-institutional ring study

The level of human epidermal growth factor receptor-2 (HER2) protein and gene expression in breast cancer is an essential factor in judging the prognosis of breast cancer patients. Several investigations have shown high intraobserver and interobserver variability in the evaluation of HER2 staining by visual examination. In this study, we aim to propose an artificial intelligence (AI)-assisted microscope to improve the HER2 assessment accuracy and reliability. Our AI-assisted microscope was equipped with a conventional microscope with a cell-level classification-based HER2 scoring algorithm and an augmented reality module to enable pathologists to obtain AI results in real time. We organized a three-round ring study of 50 infiltrating duct carcinoma not otherwise specified (NOS) cases without neoadjuvant treatment, and recruited 33 pathologists from 6 hospitals. In the first ring study (RS1), the pathologists read 50 HER2 whole-slide images (WSIs) through an online system. After a 2-week washout period, they read the HER2 slides using a conventional microscope in RS2. After another 2-week washout period, the pathologists used our AI microscope for assisted interpretation in RS3. The consistency and accuracy of HER2 assessment by the AI-assisted microscope were significantly improved (p < 0.001) over those obtained using a conventional microscope and online WSI. Specifically, our AI-assisted microscope improved the precision of immunohistochemistry (IHC) 3 + and 2 + scoring while ensuring the recall of fluorescent in situ hybridization (FISH)-positive results in IHC 2 + . Also, the average acceptance rate of AI for all pathologists was 0.90, demonstrating that the pathologists agreed with most AI scoring results.

Quantitative digital image analysis of somatostatin receptor 2 immunohistochemistry in pancreatic neuroendocrine tumors

Immunohistochemical analysis of somatostatin receptor 2 (SSTR2) provides important information regarding the potential therapeutic efficacy of somatostatin analogues (SSAs) in patients with neuroendocrine tumors. HER2 scoring has been proposed to interpret SSTR2 immunoreactivity but their reproducibility was relatively low because of its intrinsic subjective nature. Digital image analysis (DIA) has recently been proposed as an objective and more precise method of evaluating immunoreactivity. Therefore, in this study, we used DIA for analyzing SSTR2 immunoreactivity in pancreatic neuroendocrine tumors (PanNETs) to obtain its H score and "(%) strong positive cells" and compared the results with those of manually obtained HER2 scores. Membranous SSTR2 immunoreactivity evaluated by DIA was calculated by two scales as: "Membrane Optical Density" and "Minimum Membrane Completeness". PanNETs with HER2 score of > 2 demonstrated the highest concordance with results of "(%) strong positive cells" obtained by DIA when "Minimum Membrane Completeness" was tentatively set at 80%. The SSTR2 immunoreactivity, evaluated based on all scoring systems, was different between grades G1 and G2 in insulinoma but not in non-functional PanNETs. DIA provided reproducible results of SSTR2 immunoreactivity in PanNETs and yielded important information as to the potential application of SSAs.

Dissecting the Business Case for Adoption and Implementation of Digital Pathology: A White Paper from the Digital Pathology Association

We believe the switch to a digital pathology (DP) workflow is imminent and it is essential to understand the economic implications of conversion. Many aspects of the adoption of DP will be disruptive and have a direct financial impact, both in short term costs, such as investment in equipment and personnel, and long term revenue potential, such as improved productivity and novel tests. The focus of this whitepaper is to educate pathologists, laboratorians and other stakeholders about the business and monetary considerations of converting to a digital pathology workflow. The components of a DP business plan will be thoroughly summarized, and guidance will be provided on how to build a case for adoption and implementation as well as a roadmap for transitioning from an analog to a digital pathology workflow in various laboratory settings. It is important to clarify that this publication is not intended to list prices although some financials will be mentioned as examples. The authors encourage readers who are evaluating conversion to a DP workflow to use this paper as a foundational guide for conducting a thorough and complete assessment while incorporating in current market pricing. Contributors to this paper analyzed peer-reviewed literature and data collected from various institutions, some of which are mentioned. Digital pathology will change the way we practice through facilitating patient access to expert pathology services and enabling image analysis tools and assays to aid in diagnosis, prognosis, risk stratification and therapeutic selection. Together, they will result in the delivery of valuable information from which to make better decisions and improve the health of patients.

Digital scoring of EpCAM and slug expression as prognostic markers in head and neck squamous cell carcinomas

Head and neck squamous cell carcinomas (HNSCCs) have poor clinical outcome owing to therapy resistance and frequent recurrences that are among others attributable to tumor cells in partial epithelial‐to‐mesenchymal transition (pEMT). We compared side‐by‐side software‐based and visual quantification of immunohistochemistry (IHC) staining of epithelial marker EpCAM and EMT regulator Slug in n = 102 primary HNSCC to assess optimal analysis protocols. IHC scores incorporated expression levels and percentages of positive cells. Digital and visual evaluation of membrane‐associated EpCAM yielded correlating scorings, whereas visual evaluation of nuclear Slug resulted in significantly higher overall scores. Multivariable Cox proportional hazard analysis defined the median EpCAM expression levels resulting from visual quantification as an independent prognostic factor of overall survival. Slug expression levels resulting from digital quantification were an independent prognostic factor of recurrence‐free survival, locoregional recurrence‐free survival, and disease‐specific survival. Hence, we propose to use visual assessment for the membrane‐associated EpCAM protein, whereas nuclear protein Slug assessment was more accurate following digital measurement.

Accuracy of Digital Image Analysis (DIA) of Borderline Human Epidermal Growth Factor Receptor (HER2) Immunohistochemistry in Invasive Ductal Carcinoma

Background: Digital image analysis (DIA), used to extract information from pathology slides, provides better precision and no limitation regarding different interpretations by observers. Objectives: The present study aimed at evaluating the accuracy of DIA in the interpretation of borderline (2+) human epidermal growth factor receptor 2 (HER2) immunohistochemistry (IHC) slides of invasive ductal carcinoma of the breast. Methods: Sixty pathology samples with invasive ductal carcinoma of the breast were extracted based on HER2 (2+) and their fluorescence in situ hybridization (FISH), and chromogenic in situ hybridization (CISH) responses (as reference standard). The slides were digitized and, then, two pathologists examined the slides and documented diagnosis. DIA was performed by a free web application. Results: Totally, 307 digital images with 298 megabytes volume were extracted. The accuracy, sensitivity, and specificity values of DIA were 86 %, 46.1 %, and 97.8 %, respectively, with 8 false-negative cases. There was moderate agreement between the pathologist 1 (kappa = 0.42) and pathologist 2 (kappa = 0.41) with DIA. Conclusions: DIA had good accuracy and could be used for the interpretation of borderline HER2 IHC method in invasive ductal carcinoma.

Color standardization and optimization in whole slide imaging

Whole slide imaging (WSI) has various uses, including the development of decision support systems, image analysis, education, conferences, and remote diagnostics. It is also used to develop artificial intelligence using machine learning methods. In the clinical setting, however, many issues have hindered the implementation of WSI. These issues are becoming more important as WSI is gaining wider use in clinical practice, particularly with the implementation of artificial intelligence in pathological diagnosis. One of the most important issues is the standardization of color for WSI, which is an important component of digital pathology. In this paper, we review the major factors of color variation and how to evaluate and modify color variation to establish color standardization. There are five major reasons for color variation, which include specimen thickness, staining, scanner, viewer, and display. Recognizing that the color is not standardized is the first step towards standardization, and it is difficult to ascertain whether the appropriate color of the WSI is displayed at the reviewers' end.

A Validation Study of Human Epidermal Growth Factor Receptor 2 Immunohistochemistry Digital Imaging Analysis and its Correlation with Human Epidermal Growth Factor Receptor 2 Fluorescence In situ Hybridization Results in Breast Carcinoma

Background:

The Visiopharm human epidermal growth factor receptor 2 (HER2) digital imaging analysis (DIA) algorithm assesses digitized HER2 immunohistochemistry (IHC) by measuring cell membrane connectivity. We aimed to validate this algorithm for clinical use by comparing with pathologists’ scoring and correlating with HER2 fluorescence in situ hybridization (FISH) results.

Materials and Methods:

The study cohort consisted of 612 consecutive invasive breast carcinoma specimens including 395 biopsies and 217 resections. HER2 IHC slides were scanned using Philips IntelliSite Scanners, and the digital images were analyzed using Visiopharm HER2-CONNECT App to obtain the connectivity values (0–1) and scores (0, 1+, 2+, and 3+). HER2 DIA scores were compared with Pathologists’ manual scores, and HER2 connectivity values were correlated with HER2 FISH results.

Results:

The concordance between HER2 DIA scores and pathologists’ scores was 87.3% (534/612). All discordant cases (n = 78) were only one-step discordant (negative to equivocal, equivocal to positive, or vice versa). Five cases (0.8%) showed discordant HER2 IHC DIA and HER2 FISH results, but all these cases had relatively low HER2 copy numbers (between 4 and 6). HER2 IHC connectivity showed significantly better correlation with HER2 copy number than HER2/CEP17 ratio.

Conclusions:

HER2 IHC DIA demonstrates excellent concordance with pathologists’ scores and accurately discriminates between HER2 FISH positive and negative cases. HER2 IHC connectivity has better correlation with HER2 copy number than HER2/CEP17 ratio, suggesting HER2 copy number may be more important in predicting HER2 protein expression, and response to anti-HER2-targeted therapy.

Quantitative digital imaging analysis of HER2 immunohistochemistry predicts the response to anti-HER2 neoadjuvant chemotherapy in HER2-positive breast carcinoma

PURPOSE

Patients with HER2-positive breast cancer commonly receive anti-HER2 neoadjuvant chemotherapy and pathologic complete response (pCR) can be achieved in up to half of the patients. HER2 protein expression detected by immunohistochemistry (IHC) can be quantified using digital imaging analysis (DIA) as a value of membranous connectivity. We aimed to investigate the association HER2 IHC DIA quantitative results with response to anti-HER2 neoadjuvant chemotherapy.

METHODS

Digitized HER2 IHC whole slide images were analyzed using Visiopharm HER2-CONNECT to obtain quantitative HER2 membranous connectivity from a cohort of 153 HER2+ invasive breast carcinoma cases treated with anti-HER2 neoadjuvant chemotherapy (NAC). HER2 connectivity and other factors including age, histologic grade, ER, PR, and HER2 fluorescence in situ hybridization (FISH) were analyzed for association with the response to anti-HER2 NAC.

RESULTS

Eighty-three cases (54.2%) had pCR, while 70 (45.8%) showed residual tumor. Younger age, negative ER/PR, higher HER2 DIA connectivity, higher HER2 FISH ratio and copy number were significantly associated with pCR in univariate analysis. Multivariate analysis demonstrated only age, HER2 DIA connectivity, PR negativity, and HER2 copy number was significantly associated with pCR, whereas HER2 DIA connectivity had the strongest association.

CONCLUSIONS

HER2 IHC DIA connectivity is the most important factor predicting pCR to anti-HER2 neoadjuvant chemotherapy in patients with HER2-positive breast cancer.

HER2 immunohistochemistry inter-observer reproducibility in 205 cases of invasive breast carcinoma additionally tested by ISH

Assessment of HER2 biomarker in invasive breast carcinoma patients allows a specific therapeutic approach. Clinical guidelines indicate immunohistochemistry (IHC) and in situ hybridization (ISH) to test HER2, however both have drawbacks which results in low reproducibility of results especially in equivocal cases. Our main objective is to quantify inter-observer IHC reproducibility and cross it with the ISH result. Our series includes 205 invasive breast carcinoma cases sent for ISH retest from 14 hospitals, 5 observers to assess the IHC and 2 observers for the ISH of each case. We found that the observers only achieve an absolute agreement for IHC in 1 out of 3 cases. The inter-observer concordance for IHC is low (0.2 ≤ k ≤ 0.4) or moderate (0.41 ≤ k ≤ 0.6). In ISH positive cases the concordance for IHC is higher than in the ISH negative cases. In conclusion, the study shows low and moderate IHC inter-observer concordance, finding the more worrying values among the ISH negative cases which are the most part of this particular sample. Subjective interpretation of the techniques, among other factors, has negative impact in HER2 evaluation. To offset this limitation we have checked that reaching a consensus from different observers for HER2 IHC assessment improves the results.

What is the added value of digital image analysis of HER2 immunohistochemistry in breast cancer in clinical practice? A study with multiple platforms

AIMS

We aimed to compare digital image analysis (DIA) of human epidermal growth factor receptor 2 (HER2) immunohistochemistry (IHC) in breast cancer by two platforms: (i) to validate DIA against standard diagnostics; and (ii) to evaluate the added value of DIA in clinical practice.

METHODS AND RESULTS

HER2 IHC and in-situ hybridisation (ISH) were performed on 152 consecutive invasive breast carcinomas. IHC scores were determined with DIA using two independent platforms. Manual scoring was performed by two independent observers. HER2 status was considered positive in 3+ and ISH-positive 2+ cases. HER2 status using DIA was compared to HER2 status with standard diagnostics (manual scoring with ISH in 2+ cases). Interplatform agreement of IHC scores was 'moderate' (linear weighted κ = 0.58), agreement between manual scoring and platform A was 'moderate' (κ = 0.60) and between manual scoring and platform B 'almost perfect' (κ = 0.85). Compared to manual scoring, DIA resulted in a reduction of 2+ cases from 17.1 to 1.3% with platform A and from 17.1 to 15.8% with platform B. However, compared to standard diagnostics, there were three false-negative cases with DIA using platform A [81.3% sensitivity, 100% specificity, 100% positive predictive value (PPV), 97.8% negative predictive value (NPV)]. Sensitivity, specificity, PPV and NPV were 100% with DIA using platform B.

CONCLUSIONS

DIA of HER2 IHC is a valid tool in determining HER2 status in breast carcinoma. Algorithms in different platforms can behave differently, and optimal calibration is essential. In clinical practice, DIA offers an objective alternative to manual scoring, but a reduction in 2+ cases could result in loss of sensitivity.

Image Analysis of HER2 Immunohistochemical Staining of Surgical Breast Cancer Specimens

PURPOSE

Trastuzumab is an effective treatment for human epidermal growth factor receptor 2 (HER2)-amplified breast cancers. We sought to develop a simple protocol for HER2 image analysis of breast cancer specimens.

MATERIALS AND METHODS

In a preliminary test, we found that at least 1000 tumor cells need to be examined in the most strongly stained areas. Next, we evaluated the clinical usefulness of this established protocol of image analysis in 555 breast cancer patients. Results of the HER2 immunohistochemical (IHC) staining were compared between manual scoring and image analysis.

RESULTS

The HER2 IHC results obtained by the image analysis method correlated well with those obtained by the manual scoring method (Cohen's kappa=0.830). Using the HER2 silver in situ hybridization (SISH) results as a gold standard, sensitivity values were 72.1% for manual scoring and 74.0% for image analysis; specificity values were 96.2% for manual scoring and 94.7% for image analysis; and accuracy values were 91.7% for manual scoring and 90.8% for image analysis. McNemar's test was applied to the results, and there were no statistically significant differences in sensitivity and specificity between the positive (p=0.688) and negative (p=0.118) SISH groups.

CONCLUSION

HER2 image analysis results were similar to those obtained via the manual scoring method, indicating that the use of image analysis can reduce assessment time and effort. We suggest that image analysis-based evaluation of 1000 tumor cells in the most strongly IHC-stained area, regardless of stroma content, is sufficient for determining HER2 expression levels in breast cancer specimens.

Microvascular fractal dimension predicts prognosis and response to chemotherapy in glioblastoma: an automatic image analysis study

The microvascular profile has been included in the WHO glioma grading criteria. Nevertheless, microvessels in gliomas of the same WHO grade, e.g., WHO IV glioblastoma (GBM), exhibit heterogeneous and polymorphic morphology, whose possible clinical significance remains to be determined. In this study, we employed a fractal geometry-derived parameter, microvascular fractal dimension (mvFD), to quantify microvessel complexity and developed a home-made macro in Image J software to automatically determine mvFD from the microvessel-stained immunohistochemical images of GBM. We found that mvFD effectively quantified the morphological complexity of GBM microvasculature. Furthermore, high mvFD favored the survival of GBM patients as an independent prognostic indicator and predicted a better response to chemotherapy of GBM patients. When investigating the underlying relations between mvFD and tumor growth by deploying Ki67/mvFD as an index for microvasculature-normalized tumor proliferation, we discovered an inverse correlation between mvFD and Ki67/mvFD. Furthermore, mvFD inversely correlated with the expressions of a glycolytic marker, LDHA, which indicated poor prognosis of GBM patients. Conclusively, we developed an automatic approach for mvFD measurement, and demonstrated that mvFD could predict the prognosis and response to chemotherapy of GBM patients.

Physical basis of the 'magnification rule' for standardized Immunohistochemical scoring of HER2 in breast and gastric cancer

Background

Detection of HER2/neu receptor overexpression and/or amplification is a prerequisite for efficient anti-HER2 treatment of breast and gastric carcinomas. Immunohistochemistry (IHC) of the HER2 protein is the most common screening test, thus precise and reproducible IHC-scoring is of utmost importance. Interobserver variance still is a problem; in particular in gastric carcinomas the reliable differentiation of IHC scores 2+ and 1+ is challenging.

Herein we describe the physical basis of what we called the ‘magnification rule’: Different microscope objectives are employed to reproducibly subdivide the continuous spectrum of IHC staining intensities into distinct categories (1+, 2+, 3+).

Methods

HER2-IHC was performed on 120 breast cancer biopsy specimens (n = 40 per category). Width and color-intensity of membranous DAB chromogen precipitates were measured by whole-slide scanning and digital morphometry. Image-analysis data were related to semi-quantitative manual scoring according to the magnification rule and to the optical properties of the employed microscope objectives.

Results

The semi-quantitative manual HER2-IHC scores are correlated to color-intensity measured by image-analysis and to the width of DAB-precipitates. The mean widths ±standard deviations of precipitates were: IHC-score 1+, 0.64 ± 0.1 μm; score 2+, 1.0 ± 0.23 μm; score 3+, 2.14 ± 0.4 μm. The width of precipitates per category matched the optical resolution of the employed microscope objective lenses: Approximately 0.4 μm (40×), 1.0 μm (10×) and 2.0 μm (5×).

Conclusions

Perceived intensity, width of the DAB chromogen precipitate, and absolute color-intensity determined by image-analysis are linked. These interrelations form the physical basis of the ‘magnification rule’: 2+ precipitates are too narrow to be observed with 5× microscope objectives, 1+ precipitates are too narrow for 10× objectives. Thus, the rule uses the optical resolution windows of standard diagnostic microscope objectives to derive the width of the DAB-precipitates. The width is in turn correlated with color-intensity. Hereby, the more or less subjective estimation of IHC scores based only on the staining-intensity is replaced by a quasi-morphometric measurement. The principle seems universally applicable to immunohistochemical stainings of membrane-bound biomarkers that require an intensity-dependent scoring.

Electronic supplementary material

The online version of this article (10.1186/s13000-018-0696-x) contains supplementary material, which is available to authorized users.

Whole-Slide Imaging for Esophageal Adenocarcinoma

Whole-slide imaging revolutionizes the field of pathology especially in the areas of facilitation of research, long-term storages, exchange of information, and image analysis. In this process, a scanning equipment (scanner) scans the whole glass slide into a digital file. In research in esophageal adenocarcinoma or other cancers, whole-slide imaging could help in production of high-resolution images for studying and sharing of research information, assessment of tissue microarray slides as well as allowing digital image analysis of the tissue information such as level of staining (e.g., HER2) in a more efficient and objective manner. In this chapter, we will elaborate the concepts, advantages, barriers, and the operations of whole-slide imaging scanning.

Digital image analysis of HER2 immunohistochemistry in gastric- and oesophageal adenocarcinoma: a validation study on biopsies and surgical specimens

AIMS

To test the validity of diagnostics incorporating digital image analysis (DIA) for human epidermal growth factor 2 (HER2) immunohistochemistry (IHC) in gastro-oesophageal adenocarcinomas, as an alternative to current standard diagnostics using manual scoring.

METHODS AND RESULTS

We included 319 consecutive gastro-oesophageal adenocarcinomas (232 biopsies and 87 surgical specimens). DIA was applied to determine HER2 IHC classification, using both standard breast cancer (BC) and modified gastro-oesophageal cancer (GEC) cut-offs. Consensus manual scores were established by four independent observers. Chromogenic in-situ hybridization (CISH) was performed on all 2+ cases by manual scoring, DIA or both. HER2 status was considered positive in 3+ and CISH-positive 2+ cases. Overall agreement between DIA and consensus manual scores was 76.5% (weighted κ = 0.66, BC cut-offs) and 85.6% (weighted κ = 0.80, GEC cut-offs). Agreement was similar for biopsies and surgical specimens. All disagreement occurred in the manual IHC equivocal cases. DIA resulted in a reduction of 2+ cases: 75.8% with BC cut-offs and 46.5% with GEC cut-offs. HER2 status was positive in 48 cases (15%) with standard diagnostics and DIA using GEC cut-offs, and 46 cases (14.4%) using BC cut-offs (all with CISH in 2+ cases). Considering standard diagnostics as a reference, DIA showed 93.8% sensitivity and 99.6% specificity (BC cut-offs) or 97.9% sensitivity and 99.6% specificity (GEC cut-offs).

CONCLUSIONS

DIA is a reliable and feasible alternative to manual HER2 IHC scoring in gastro-oesophageal adenocarcinoma, both in biopsies and surgical specimens, leading to a reduction of 2+ cases for which subsequent ISH testing is required.

Clinical Applications of Whole-slide Imaging in Anatomic Pathology

The development of whole-slide imaging has paved the way for digitizing of glass slides that are the basis for surgical pathology. This transformative technology has changed the landscape in research applications and education but despite its tremendous potential, its adoption for clinical use has been slow. We review the various niche applications that initiated awareness of this technology, provide examples of clinical use cases, and discuss the requirements and challenges for full adoption in clinical diagnosis. The opportunities for applications of image analysis tools in a workflow will be changed by integration of whole-slide imaging into routine diagnosis.

Colour in digital pathology: a review

Colour is central to the practice of pathology because of the use of coloured histochemical and immunohistochemical stains to visualize tissue features. Our reliance upon histochemical stains and light microscopy has evolved alongside a wide variation in slide colour, with little investigation into the implications of colour variation. However, the introduction of the digital microscope and whole-slide imaging has highlighted the need for further understanding and control of colour. This is because the digitization process itself introduces further colour variation which may affect diagnosis, and image analysis algorithms often use colour or intensity measures to detect or measure tissue features. The US Food and Drug Administration have released recent guidance stating the need to develop a method of controlling colour reproduction throughout the digitization process in whole-slide imaging for primary diagnostic use. This comprehensive review introduces applied basic colour physics and colour interpretation by the human visual system, before discussing the importance of colour in pathology. The process of colour calibration and its application to pathology are also included, as well as a summary of the current guidelines and recommendations regarding colour in digital pathology.

Computer-aided prognosis on breast cancer with hematoxylin and eosin histopathology images: A review

With the advance of digital pathology, image analysis has begun to show its advantages in information analysis of hematoxylin and eosin histopathology images. Generally, histological features in hematoxylin and eosin images are measured to evaluate tumor grade and prognosis for breast cancer. This review summarized recent works in image analysis of hematoxylin and eosin histopathology images for breast cancer prognosis. First, prognostic factors for breast cancer based on hematoxylin and eosin histopathology images were summarized. Then, usual procedures of image analysis for breast cancer prognosis were systematically reviewed, including image acquisition, image preprocessing, image detection and segmentation, and feature extraction. Finally, the prognostic value of image features and image feature–based prognostic models was evaluated. Moreover, we discussed the issues of current analysis, and some directions for future research.