Development of a diagnostic test set to assess agreement in breast pathology: practical application of the Guidelines for Reporting Reliability and Agreement Studies (GRRAS)

From Image to Diagnosis: Characterizing Sources of Error in Histopathologic Interpretation

An accurate histopathologic diagnosis on surgical biopsy material is necessary for the clinical management of patients and has important implications for research, clinical trial design/enrollment, and public health education. This study used a mixed methods approach to isolate sources of diagnostic error while residents and attending pathologists interpreted digitized breast biopsy slides. Ninety participants, including pathology residents and attending physicians at major United States medical centers reviewed a set of 14 digitized whole-slide images of breast biopsies. Each case had a consensus-defined diagnosis and critical region of interest (cROI) representing the most significant pathology on the slide. Participants were asked to view unmarked digitized slides, draw their participant region of interest (pROI), describe its features, and render a diagnosis. Participants' review behavior was tracked using case viewer software and an eye-tracking device. Diagnostic accuracy was calculated in comparison to the consensus diagnosis. We measured the frequency of errors emerging during 4 interpretive phases: (1) detecting the cROI, (2) recognizing its relevance, (3) using the correct terminology to describe findings in the pROI, and (4) making a diagnostic decision. According to eye-tracking data, trainees and attending pathologists were very likely (∼94% of the time) to find the cROI when inspecting a slide. However, trainees were less likely to consider the cROI relevant to their diagnosis. Pathology trainees (41% of cases) were more likely to use incorrect terminology to describe pROI features than attending pathologists (21% of cases). Failure to accurately describe features was the only factor strongly associated with an incorrect diagnosis. Identifying where errors emerge in the interpretive and/or descriptive process and working on building organ-specific feature recognition and verbal fluency in describing those features are critical steps for achieving competency in diagnostic decision making.

Zoom behavior during visual search modulates pupil diameter and reflects adaptive control states

Adaptive gain theory proposes that the dynamic shifts between exploration and exploitation control states are modulated by the locus coeruleus-norepinephrine system and reflected in tonic and phasic pupil diameter. This study tested predictions of this theory in the context of a societally important visual search task: the review and interpretation of digital whole slide images of breast biopsies by physicians (pathologists). As these medical images are searched, pathologists encounter difficult visual features and intermittently zoom in to examine features of interest. We propose that tonic and phasic pupil diameter changes during image review may correspond to perceived difficulty and dynamic shifts between exploration and exploitation control states. To examine this possibility, we monitored visual search behavior and tonic and phasic pupil diameter while pathologists (N = 89) interpreted 14 digital images of breast biopsy tissue (1,246 total images reviewed). After viewing the images, pathologists provided a diagnosis and rated the level of difficulty of the image. Analyses of tonic pupil diameter examined whether pupil dilation was associated with pathologists' difficulty ratings, diagnostic accuracy, and experience level. To examine phasic pupil diameter, we parsed continuous visual search data into discrete zoom-in and zoom-out events, including shifts from low to high magnification (e.g., 1× to 10×) and the reverse. Analyses examined whether zoom-in and zoom-out events were associated with phasic pupil diameter change. Results demonstrated that tonic pupil diameter was associated with image difficulty ratings and zoom level, and phasic pupil diameter showed constriction upon zoom-in events, and dilation immediately preceding a zoom-out event. Results are interpreted in the context of adaptive gain theory, information gain theory, and the monitoring and assessment of physicians' diagnostic interpretive processes.

Pathologist pupil dilation reflects experience level and difficulty in diagnosing medical images

Purpose: Digital whole slide imaging allows pathologists to view slides on a computer screen instead of under a microscope. Digital viewing allows for real-time monitoring of pathologists' search behavior and neurophysiological responses during the diagnostic process. One particular neurophysiological measure, pupil diameter, could provide a basis for evaluating clinical competence during training or developing tools that support the diagnostic process. Prior research shows that pupil diameter is sensitive to cognitive load and arousal, and it switches between exploration and exploitation of a visual image. Different categories of lesions in pathology pose different levels of challenge, as indicated by diagnostic disagreement among pathologists. If pupil diameter is sensitive to the perceived difficulty in diagnosing biopsies, eye-tracking could potentially be used to identify biopsies that may benefit from a second opinion. Approach: We measured case onset baseline-corrected (phasic) and uncorrected (tonic) pupil diameter in 90 pathologists who each viewed and diagnosed 14 digital breast biopsy cases that cover the diagnostic spectrum from benign to invasive breast cancer. Pupil data were extracted from the beginning of viewing and interpreting of each individual case. After removing 122 trials ( < 10 % ) with poor eye-tracking quality, 1138 trials remained. We used multiple linear regression with robust standard error estimates to account for dependent observations within pathologists. Results: We found a positive association between the magnitude of phasic dilation and subject-centered difficulty ratings and between the magnitude of tonic dilation and untransformed difficulty ratings. When controlling for case diagnostic category, only the tonic-difficulty relationship persisted. Conclusions: Results suggest that tonic pupil dilation may indicate overall arousal differences between pathologists as they interpret biopsy cases and could signal a need for additional training, experience, or automated decision aids. Phasic dilation is sensitive to characteristics of biopsies that tend to elicit higher difficulty ratings and could indicate a need for a second opinion.

Deep Learning Approaches in Histopathology

The revolution of artificial intelligence and its impacts on our daily life has led to tremendous interest in the field and its related subtypes: machine learning and deep learning. Scientists and developers have designed machine learning- and deep learning-based algorithms to perform various tasks related to tumor pathologies, such as tumor detection, classification, grading with variant stages, diagnostic forecasting, recognition of pathological attributes, pathogenesis, and genomic mutations. Pathologists are interested in artificial intelligence to improve the diagnosis precision impartiality and to minimize the workload combined with the time consumed, which affects the accuracy of the decision taken. Regrettably, there are already certain obstacles to overcome connected to artificial intelligence deployments, such as the applicability and validation of algorithms and computational technologies, in addition to the ability to train pathologists and doctors to use these machines and their willingness to accept the results. This review paper provides a survey of how machine learning and deep learning methods could be implemented into health care providers' routine tasks and the obstacles and opportunities for artificial intelligence application in tumor morphology.

More scanning, but not zooming, is associated with diagnostic accuracy in evaluating digital breast pathology slides

Diagnoses of medical images can invite strikingly diverse strategies for image navigation and visual search. In computed tomography screening for lung nodules, distinct strategies, termed scanning and drilling, relate to both radiologists' clinical experience and accuracy in lesion detection. Here, we examined associations between search patterns and accuracy for pathologists (N = 92) interpreting a diverse set of breast biopsy images. While changes in depth in volumetric images reveal new structures through movement in the z-plane, in digital pathology changes in depth are associated with increased magnification. Thus, "drilling" in radiology may be more appropriately termed "zooming" in pathology. We monitored eye-movements and navigation through digital pathology slides to derive metrics of how quickly the pathologists moved through XY (scanning) and Z (zooming) space. Prior research on eye-movements in depth has categorized clinicians as either "scanners" or "drillers." In contrast, we found that there was no reliable association between a clinician's tendency to scan or zoom while examining digital pathology slides. Thus, in the current work we treated scanning and zooming as continuous predictors rather than categorizing as either a "scanner" or "zoomer." In contrast to prior work in volumetric chest images, we found significant associations between accuracy and scanning rate but not zooming rate. These findings suggest fundamental differences in the relative value of information types and review behaviors across two image formats. Our data suggest that pathologists gather critical information by scanning on a given plane of depth, whereas radiologists drill through depth to interrogate critical features.

Classifying Breast Histopathology Images with a Ductal Instance-Oriented Pipeline

In this study, we propose the Ductal Instance-Oriented Pipeline (DIOP) that contains a duct-level instance segmentation model, a tissue-level semantic segmentation model, and three-levels of features for diagnostic classification. Based on recent advancements in instance segmentation and the Mask RCNN model, our duct-level segmenter tries to identify each ductal individual inside a microscopic image; then, it extracts tissue-level information from the identified ductal instances. Leveraging three levels of information obtained from these ductal instances and also the histopathology image, the proposed DIOP outperforms previous approaches (both feature-based and CNN-based) in all diagnostic tasks; for the four-way classification task, the DIOP achieves comparable performance to general pathologists in this unique dataset. The proposed DIOP only takes a few seconds to run in the inference time, which could be used interactively on most modern computers. More clinical explorations are needed to study the robustness and generalizability of this system in the future.

An analysis of key indicators of reproducibility in radiology

BACKGROUND

Given the central role of radiology in patient care, it is important that radiological research is grounded in reproducible science. It is unclear whether there is a lack of reproducibility or transparency in radiologic research.

PURPOSE

To analyze published radiology literature for the presence or lack of key indicators of reproducibility.

METHODS

This cross-sectional retrospective study was performed by conducting a search of the National Library of Medicine (NLM) for publications contained within journals in the field of radiology. Our inclusion criteria were being MEDLINE indexed, written in English, and published from January 1, 2014, to December 31, 2018. We randomly sampled 300 publications for this study. A pilot-tested Google form was used to record information from the publications regarding indicators of reproducibility. Following peer-review, we extracted data from an additional 200 publications in an attempt to reproduce our initial results. The additional 200 publications were selected from the list of initially randomized publications.

RESULTS

Our initial search returned 295,543 records, from which 300 were randomly selected for analysis. Of these 300 records, 294 met inclusion criteria and 6 did not. Among the empirical publications, 5.6% (11/195, [3.0-8.3]) contained a data availability statement, 0.51% (1/195) provided clear documented raw data, 12.0% (23/191, [8.4-15.7]) provided a materials availability statement, 0% provided analysis scripts, 4.1% (8/195, [1.9-6.3]) provided a pre-registration statement, 2.1% (4/195, [0.4-3.7]) provided a protocol statement, and 3.6% (7/195, [1.5-5.7]) were pre-registered. The validation study of the 5 key indicators of reproducibility-availability of data, materials, protocols, analysis scripts, and pre-registration-resulted in 2 indicators (availability of protocols and analysis scripts) being reproduced, as they fell within the 95% confidence intervals for the proportions from the original sample. However, materials' availability and pre-registration proportions from the validation sample were lower than what was found in the original sample.

CONCLUSION

Our findings demonstrate key indicators of reproducibility are missing in the field of radiology. Thus, the ability to reproduce studies contained in radiology publications may be problematic and may have potential clinical implications.

Assessment of Machine Learning of Breast Pathology Structures for Automated Differentiation of Breast Cancer and High-Risk Proliferative Lesions

IMPORTANCE

Following recent US Food and Drug Administration approval, adoption of whole slide imaging in clinical settings may be imminent, and diagnostic accuracy, particularly among challenging breast biopsy specimens, may benefit from computerized diagnostic support tools.

OBJECTIVE

To develop and evaluate computer vision methods to assist pathologists in diagnosing the full spectrum of breast biopsy samples, from benign to invasive cancer.

DESIGN, SETTING, AND PARTICIPANTS

In this diagnostic study, 240 breast biopsies from Breast Cancer Surveillance Consortium registries that varied by breast density, diagnosis, patient age, and biopsy type were selected, reviewed, and categorized by 3 expert pathologists as benign, atypia, ductal carcinoma in situ (DCIS), and invasive cancer. The atypia and DCIS cases were oversampled to increase statistical power. High-resolution digital slide images were obtained, and 2 automated image features (tissue distribution feature and structure feature) were developed and evaluated according to the consensus diagnosis of the expert panel. The performance of the automated image analysis methods was compared with independent interpretations from 87 practicing US pathologists. Data analysis was performed between February 2017 and February 2019.

MAIN OUTCOMES AND MEASURES

Diagnostic accuracy defined by consensus reference standard of 3 experienced breast pathologists.

RESULTS

The accuracy of machine learning tissue distribution features, structure features, and pathologists for classification of invasive cancer vs noninvasive cancer was 0.94, 0.91, and 0.98, respectively; the accuracy of classification of atypia and DCIS vs benign tissue was 0.70, 0.70, and 0.81, respectively; and the accuracy of classification of DCIS vs atypia was 0.83, 0.85, and 0.80, respectively. The sensitivity of both machine learning features was lower than that of the pathologists for the invasive vs noninvasive classification (tissue distribution feature, 0.70; structure feature, 0.49; pathologists, 0.84) but higher for the classification of atypia and DCIS vs benign cases (tissue distribution feature, 0.79; structure feature, 0.85; pathologists, 0.72) and the classification of DCIS vs atypia (tissue distribution feature, 0.88; structure feature, 0.89; pathologists, 0.70). For the DCIS vs atypia classification, the specificity of the machine learning feature classification was similar to that of the pathologists (tissue distribution feature, 0.78; structure feature, 0.80; pathologists, 0.82).

CONCLUSION AND RELEVANCE

The computer-based automated approach to interpreting breast pathology showed promise, especially as a diagnostic aid in differentiating DCIS from atypical hyperplasia.

Characterizing Diagnostic Search Patterns in Digital Breast Pathology: Scanners and Drillers

Following a baseline demographic survey, 87 pathologists interpreted 240 digital whole slide images of breast biopsy specimens representing a range of diagnostic categories from benign to atypia, ductal carcinoma in situ, and invasive cancer. A web-based viewer recorded pathologists' behaviors while interpreting a subset of 60 randomly selected and randomly ordered slides. To characterize diagnostic search patterns, we used the viewport location, time stamp, and zoom level data to calculate four variables: average zoom level, maximum zoom level, zoom level variance, and scanning percentage. Two distinct search strategies were confirmed: scanning is characterized by panning at a constant zoom level, while drilling involves zooming in and out at various locations. Statistical analysis was applied to examine the associations of different visual interpretive strategies with pathologist characteristics, diagnostic accuracy, and efficiency. We found that females scanned more than males, and age was positively correlated with scanning percentage, while the facility size was negatively correlated. Throughout 60 cases, the scanning percentage and total interpretation time per slide decreased, and these two variables were positively correlated. The scanning percentage was not predictive of diagnostic accuracy. Increasing average zoom level, maximum zoom level, and zoom variance were correlated with over-interpretation.

Breast Cancer Prognostic Factors in the Digital Era: Comparison of Nottingham Grade using Whole Slide Images and Glass Slides

Background:

To assess reproducibility and accuracy of overall Nottingham grade and component scores using digital whole slide images (WSIs) compared to glass slides.

Methods:

Two hundred and eight pathologists were randomized to independently interpret 1 of 4 breast biopsy sets using either glass slides or digital WSI. Each set included 5 or 6 invasive carcinomas (22 total invasive cases). Participants interpreted the same biopsy set approximately 9 months later following a second randomization to WSI or glass slides. Nottingham grade, including component scores, was assessed on each interpretation, providing 2045 independent interpretations of grade. Overall grade and component scores were compared between pathologists (interobserver agreement) and for interpretations by the same pathologist (intraobserver agreement). Grade assessments were compared when the format (WSI vs. glass slides) changed or was the same for the two interpretations.

Results:

Nottingham grade intraobserver agreement was highest using glass slides for both interpretations (73%, 95% confidence interval [CI]: 68%, 78%) and slightly lower but not statistically different using digital WSI for both interpretations (68%, 95% CI: 61%, 75%; P= 0.22). The agreement was lowest when the format changed between interpretations (63%, 95% CI: 59%, 68%). Interobserver agreement was significantly higher (P < 0.001) using glass slides versus digital WSI (68%, 95% CI: 66%, 70% versus 60%, 95% CI: 57%, 62%, respectively). Nuclear pleomorphism scores had the lowest inter- and intra-observer agreement. Mitotic scores were higher on glass slides in inter- and intra-observer comparisons.

Conclusions:

Pathologists’ intraobserver agreement (reproducibility) is similar for Nottingham grade using glass slides or WSI. However, slightly lower agreement between pathologists suggests that verification of grade using digital WSI may be more challenging.

Detection and classification of cancer in whole slide breast histopathology images using deep convolutional networks

Generalizability of algorithms for binary cancer vs. no cancer classification is unknown for clinically more significant multi-class scenarios where intermediate categories have different risk factors and treatment strategies. We present a system that classifies whole slide images (WSI) of breast biopsies into five diagnostic categories. First, a saliency detector that uses a pipeline of four fully convolutional networks, trained with samples from records of pathologists' screenings, performs multi-scale localization of diagnostically relevant regions of interest in WSI. Then, a convolutional network, trained from consensus-derived reference samples, classifies image patches as non-proliferative or proliferative changes, atypical ductal hyperplasia, ductal carcinoma in situ, and invasive carcinoma. Finally, the saliency and classification maps are fused for pixel-wise labeling and slide-level categorization. Experiments using 240 WSI showed that both saliency detector and classifier networks performed better than competing algorithms, and the five-class slide-level accuracy of 55% was not statistically different from the predictions of 45 pathologists. We also present example visualizations of the learned representations for breast cancer diagnosis.

Complexities of perceived and actual performance in pathology interpretation: A comparison of cutaneous melanocytic skin and breast interpretations

BACKGROUND

Little is known about how pathologists process differences between actual and perceived interpretations.

OBJECTIVE

To compare perceived and actual diagnostic agreement before and after educational interventions.

METHODS

Pathologists interpreted test sets of skin and/or breast specimens that included benign, atypical, in situ and invasive lesions. Interventions involved self-directed learning, one skin and one breast, that showed pathologists how their interpretations compared to a reference diagnoses. Prior to the educational intervention, participants estimated how their interpretations would compare to the reference diagnoses. After the intervention, participants estimated their overall agreement with the reference diagnoses. Perceived and actual agreements were compared.

RESULTS

For pathologists interpreting skin, mean actual agreement was 52.4% and overall pre- and postinterventional mean perceived agreement was 72.9% vs 54.2%, an overestimated mean difference of 20.5% (95% confidence interval [CI] 17.2% to 24.0%) and 1.8% (95% CI -0.5% to 4.1%), respectively. For pathologists interpreting breast, mean actual agreement was 75.9% and overall pre- and postinterventional mean perceived agreement was 81.4% vs 76.9%, an overestimation of 5.5% (95% CI 3.0% to 8.0%) and 1.0% (95% CI 0.0% to 2.0%), respectively.

CONCLUSIONS

Pathologists interpreting breast tissue had improved comprehension of their performance after the intervention compared to pathologists interpreting skin lesions.

Localization of Diagnostically Relevant Regions of Interest in Whole Slide Images: a Comparative Study

Whole slide digital imaging technology enables researchers to study pathologists' interpretive behavior as they view digital slides and gain new understanding of the diagnostic medical decision-making process. In this study, we propose a simple yet important analysis to extract diagnostically relevant regions of interest (ROIs) from tracking records using only pathologists' actions as they viewed biopsy specimens in the whole slide digital imaging format (zooming, panning, and fixating). We use these extracted regions in a visual bag-of-words model based on color and texture features to predict diagnostically relevant ROIs on whole slide images. Using a logistic regression classifier in a cross-validation setting on 240 digital breast biopsy slides and viewport tracking logs of three expert pathologists, we produce probability maps that show 74 % overlap with the actual regions at which pathologists looked. We compare different bag-of-words models by changing dictionary size, visual word definition (patches vs. superpixels), and training data (automatically extracted ROIs vs. manually marked ROIs). This study is a first step in understanding the scanning behaviors of pathologists and the underlying reasons for diagnostic errors.

Multi-Instance Multi-Label Learning for Multi-Class Classification of Whole Slide Breast Histopathology Images

Digital pathology has entered a new era with the availability of whole slide scanners that create the high-resolution images of full biopsy slides. Consequently, the uncertainty regarding the correspondence between the image areas and the diagnostic labels assigned by pathologists at the slide level, and the need for identifying regions that belong to multiple classes with different clinical significances have emerged as two new challenges. However, generalizability of the state-of-the-art algorithms, whose accuracies were reported on carefully selected regions of interest (ROIs) for the binary benign versus cancer classification, to these multi-class learning and localization problems is currently unknown. This paper presents our potential solutions to these challenges by exploiting the viewing records of pathologists and their slide-level annotations in weakly supervised learning scenarios. First, we extract candidate ROIs from the logs of pathologists' image screenings based on different behaviors, such as zooming, panning, and fixation. Then, we model each slide with a bag of instances represented by the candidate ROIs and a set of class labels extracted from the pathology forms. Finally, we use four different multi-instance multi-label learning algorithms for both slide-level and ROI-level predictions of diagnostic categories in whole slide breast histopathology images. Slide-level evaluation using 5-class and 14-class settings showed average precision values up to 81% and 69%, respectively, under different weakly labeled learning scenarios. ROI-level predictions showed that the classifier could successfully perform multi-class localization and classification within whole slide images that were selected to include the full range of challenging diagnostic categories.

Characteristics associated with requests by pathologists for second opinions on breast biopsies

AIMS

Second opinions in pathology improve patient safety by reducing diagnostic errors, leading to more appropriate clinical treatment decisions. Little objective data are available regarding the factors triggering a request for second opinion despite second opinion consultations being part of the diagnostic system of pathology. Therefore we sought to assess breast biopsy cases and interpreting pathologists characteristics associated with second opinion requests.

METHODS

Collected pathologist surveys and their interpretations of 60 test set cases were used to explore the relationships between case characteristics, pathologist characteristics and case perceptions, and requests for second opinions. Data were evaluated by logistic regression and generalised estimating equations.

RESULTS

115 pathologists provided 6900 assessments; pathologists requested second opinions on 70% (4827/6900) of their assessments 36% (1731/4827) of these would not have been required by policy. All associations between case characteristics and requesting second opinions were statistically significant, including diagnostic category, breast density, biopsy type, and number of diagnoses noted per case. Exclusive of institutional policies, pathologists wanted second opinions most frequently for atypia (66%) and least frequently for invasive cancer (20%). Second opinion rates were higher when the pathologist had lower assessment confidence, in cases with higher perceived difficulty, and cases with borderline diagnoses.

CONCLUSIONS

Pathologists request second opinions for challenging cases, particularly those with atypia, high breast density, core needle biopsies, or many co-existing diagnoses. Further studies should evaluate whether the case characteristics identified in this study could be used as clinical criteria to prompt system-level strategies for mandating second opinions.

The diagnostic challenge of low-grade ductal carcinoma in situ

BACKGROUND

Diagnostic agreement among pathologists is 84% for ductal carcinoma in situ (DCIS). Studies of interpretive variation according to grade are limited.

METHODS

A national sample of 115 pathologists interpreted 240 breast pathology test set cases in the Breast Pathology Study and their interpretations were compared to expert consensus interpretations. We assessed agreement of pathologists' interpretations with a consensus reference diagnosis of DCIS dichotomised into low- and high-grade lesions. Generalised estimating equations were used in logistic regression models of rates of under- and over-interpretation of DCIS by grade.

RESULTS

We evaluated 2097 independent interpretations of DCIS (512 low-grade DCIS and 1585 high-grade DCIS). Agreement with reference diagnoses was 46% (95% confidence interval [CI] 42-51) for low-grade DCIS and 83% (95% CI 81-86) for high-grade DCIS. The proportion of reference low-grade DCIS interpretations over-interpreted by pathologists (i.e. categorised as either high-grade DCIS or invasive cancer) was 23% (95% CI 19-28); 30% (95% CI 26-34) were interpreted as a lower diagnostic category (atypia or benign proliferative). Reference high-grade DCIS was under-interpreted in 14% (95% CI 12-16) of observations and only over-interpreted 3% (95% CI 2-4).

CONCLUSION

Grade is a major factor when examining pathologists' variability in diagnosing DCIS, with much lower agreement for low-grade DCIS cases compared to high-grade. These findings support the hypothesis that low-grade DCIS poses a greater interpretive challenge than high-grade DCIS, which should be considered when developing DCIS management strategies.

Surgical implications and variability in the use of the flat epithelial atypia diagnosis on breast biopsy specimens

OBJECTIVES

Flat epithelial atypia (FEA) is a relatively new diagnostic term with uncertain clinical significance for surgical management. Any implied risk of invasive breast cancer associated with FEA is contingent upon diagnostic reproducibility, yet little is known regarding its use.

MATERIALS AND METHODS

Pathologists in the Breast Pathology Study interpreted one of four 60-case test sets, one slide per case, constructed from 240 breast biopsy specimens. An electronic data form with standardized diagnostic categories was used; participants were instructed to indicate all diagnoses present. We assessed participants' use of FEA as a diagnostic term within: 1) each test set; 2) 72 cases classified by reference as benign without FEA; and 3) six cases classified by reference as FEA. 115 pathologists participated, providing 6900 total independent assessments.

RESULTS

Notation of FEA ranged from 0% to 35% of the cases interpreted, with most pathologists noting FEA on 4 or more test cases. At least one participant noted FEA in 34 of the 72 benign non-FEA cases. For the 6 reference FEA cases, participant agreement with the case reference FEA diagnosis ranged from 17% to 52%; diagnoses noted by participating pathologists for these FEA cases included columnar cell hyperplasia, usual ductal hyperplasia, atypical lobular hyperplasia, and atypical ductal hyperplasia.

CONCLUSIONS

We observed wide variation in the diagnosis of FEA among U.S. pathologists. This suggests that perceptions of diagnostic criteria and any implied risk associated with FEA may also vary. Surgical excision following a core biopsy diagnosis of FEA should be reconsidered and studied further.

A Randomized Study Comparing Digital Imaging to Traditional Glass Slide Microscopy for Breast Biopsy and Cancer Diagnosis

Background:

Digital whole slide imaging may be useful for obtaining second opinions and is used in many countries. However, the U.S. Food and Drug Administration requires verification studies.

Methods:

Pathologists were randomized to interpret one of four sets of breast biopsy cases during two phases, separated by ≥9 months, using glass slides or digital format (sixty cases per set, one slide per case, n = 240 cases). Accuracy was assessed by comparing interpretations to a consensus reference standard. Intraobserver reproducibility was assessed by comparing the agreement of interpretations on the same cases between two phases. Estimated probabilities of confirmation by a reference panel (i.e., predictive values) were obtained by incorporating data on the population prevalence of diagnoses.

Results:

Sixty-five percent of responding pathologists were eligible, and 252 consented to randomization; 208 completed Phase I (115 glass, 93 digital); and 172 completed Phase II (86 glass, 86 digital). Accuracy was slightly higher using glass compared to digital format and varied by category: invasive carcinoma, 96% versus 93% (P = 0.04); ductal carcinoma in situ (DCIS), 84% versus 79% (P < 0.01); atypia, 48% versus 43% (P = 0.08); and benign without atypia, 87% versus 82% (P < 0.01). There was a small decrease in intraobserver agreement when the format changed compared to when glass slides were used in both phases (P = 0.08). Predictive values for confirmation by a reference panel using glass versus digital were: invasive carcinoma, 98% and 97% (not significant [NS]); DCIS, 70% and 57% (P = 0.007); atypia, 38% and 28% (P = 0.002); and benign without atypia, 97% and 96% (NS).

Conclusions:

In this large randomized study, digital format interpretations were similar to glass slide interpretations of benign and invasive cancer cases. However, cases in the middle of the spectrum, where more inherent variability exists, may be more problematic in digital format. Future studies evaluating the effect these findings exert on clinical practice and patient outcomes are required.

Accuracy is in the eyes of the pathologist: The visual interpretive process and diagnostic accuracy with digital whole slide images

Digital whole slide imaging is an increasingly common medium in pathology, with application to education, telemedicine, and rendering second opinions. It has also made it possible to use eye tracking devices to explore the dynamic visual inspection and interpretation of histopathological features of tissue while pathologists review cases. Using whole slide images, the present study examined how a pathologist's diagnosis is influenced by fixed case-level factors, their prior clinical experience, and their patterns of visual inspection. Participating pathologists interpreted one of two test sets, each containing 12 digital whole slide images of breast biopsy specimens. Cases represented four diagnostic categories as determined via expert consensus: benign without atypia, atypia, ductal carcinoma in situ (DCIS), and invasive cancer. Each case included one or more regions of interest (ROIs) previously determined as of critical diagnostic importance. During pathologist interpretation we tracked eye movements, viewer tool behavior (zooming, panning), and interpretation time. Models were built using logistic and linear regression with generalized estimating equations, testing whether variables at the level of the pathologists, cases, and visual interpretive behavior would independently and/or interactively predict diagnostic accuracy and efficiency. Diagnostic accuracy varied as a function of case consensus diagnosis, replicating earlier research. As would be expected, benign cases tended to elicit false positives, and atypia, DCIS, and invasive cases tended to elicit false negatives. Pathologist experience levels, case consensus diagnosis, case difficulty, eye fixation durations, and the extent to which pathologists' eyes fixated within versus outside of diagnostic ROIs, all independently or interactively predicted diagnostic accuracy. Higher zooming behavior predicted a tendency to over-interpret benign and atypia cases, but not DCIS cases. Efficiency was not predicted by pathologist- or visual search-level variables. Results provide new insights into the medical interpretive process and demonstrate the complex interactions between pathologists and cases that guide diagnostic decision-making. Implications for training, clinical practice, and computer-aided decision aids are considered.

Histological features associated with diagnostic agreement in atypical ductal hyperplasia of the breast: illustrative cases from the B-Path study

AIMS

This study examined the case-specific characteristics associated with interobserver diagnostic agreement in atypical ductal hyperplasia (ADH) of the breast.

METHODS AND RESULTS

Seventy-two test set cases with a consensus diagnosis of ADH from the B-Path study were evaluated. Cases were scored for 17 histological features, which were then correlated with the participant agreement with the consensus ADH diagnosis. Participating pathologists' perceptions of case difficulty, borderline features or whether they would obtain a second opinion were also examined for associations with agreement. Of the 2070 participant interpretations of the 72 consensus ADH cases, 48% were scored by participants as difficult and 45% as borderline between two diagnoses; the presence of both of these features was significantly associated with increased agreement (P < 0.001). A second opinion would have been obtained in 80% of interpretations, and this was associated with increased agreement (P < 0.001). Diagnostic agreement ranged from 10% to 89% on a case-by-case basis. Cases with papillary lesions, cribriform architecture and obvious cytological monotony were associated with higher agreement. Lower agreement rates were associated with solid or micropapillary architecture, borderline cytological monotony, or cases without a diagnostic area that was obvious on low power.

CONCLUSIONS

The results of this study suggest that pathologists frequently recognize the challenge of ADH cases, with some cases being more prone to diagnostic variability. In addition, there are specific histological features associated with diagnostic agreement on ADH cases. Multiple example images from cases in this test set are provided to serve as educational illustrations of these challenges.

Region of interest identification and diagnostic agreement in breast pathology

A pathologist's accurate interpretation relies on identifying relevant histopathological features. Little is known about the precise relationship between feature identification and diagnostic decision making. We hypothesized that greater overlap between a pathologist's selected diagnostic region of interest (ROI) and a consensus derived ROI is associated with higher diagnostic accuracy. We developed breast biopsy test cases that included atypical ductal hyperplasia (n=80); ductal carcinoma in situ (n=78); and invasive breast cancer (n=22). Benign cases were excluded due to the absence of specific abnormalities. Three experienced breast pathologists conducted an independent review of the 180 digital whole slide images, established a reference consensus diagnosis and marked one or more diagnostic ROIs for each case. Forty-four participating pathologists independently diagnosed and marked ROIs on the images. Participant diagnoses and ROI were compared with consensus reference diagnoses and ROI. Regression models tested whether percent overlap between participant ROI and consensus reference ROI predicted diagnostic accuracy. Each of the 44 participants interpreted 39-50 cases for a total of 1972 individual diagnoses. Percent ROI overlap with the expert reference ROI was higher in pathologists who self-reported academic affiliation (69 vs 65%, P=0.002). Percent overlap between participants' ROI and consensus reference ROI was then classified into ordinal categories: 0, 1-33, 34-65, 66-99 and 100% overlap. For each incremental change in the ordinal percent ROI overlap, diagnostic agreement increased by 60% (OR 1.6, 95% CI (1.5-1.7), P<0.001) and the association remained significant even after adjustment for other covariates. The magnitude of the association between ROI overlap and diagnostic agreement increased with increasing diagnostic severity. The findings indicate that pathologists are more likely to converge with an expert reference diagnosis when they identify an overlapping diagnostic image region, suggesting that future computer-aided detection systems that highlight potential diagnostic regions could be a helpful tool to improve accuracy and education.

Pupil diameter changes reflect difficulty and diagnostic accuracy during medical image interpretation

Background

No automated methods exist to objectively monitor and evaluate the diagnostic process while physicians review computerized medical images. The present study tested whether using eye tracking to monitor tonic and phasic pupil dynamics may prove valuable in tracking interpretive difficulty and predicting diagnostic accuracy.

Methods

Pathologists interpreted digitized breast biopsies varying in diagnosis and rated difficulty, while pupil diameter was monitored. Tonic diameter was recorded during the entire duration of interpretation, and phasic diameter was examined when the eyes fixated on a pre-determined diagnostic region during inspection.

Results

Tonic pupil diameter was higher with increasing rated difficulty levels of cases. Phasic diameter was interactively influenced by case difficulty and the eventual agreement with consensus diagnosis. More difficult cases produced increases in pupil diameter, but only when the pathologists’ diagnoses were ultimately correct. All results were robust after adjusting for the potential impact of screen brightness on pupil diameter.

Conclusions

Results contribute new understandings of the diagnostic process, theoretical positions regarding locus coeruleus-norepinephrine system function, and suggest novel approaches to monitoring, evaluating, and guiding medical image interpretation.

Identifying and processing the gap between perceived and actual agreement in breast pathology interpretation

We examined how pathologists' process their perceptions of how their interpretations on diagnoses for breast pathology cases agree with a reference standard. To accomplish this, we created an individualized self-directed continuing medical education program that showed pathologists interpreting breast specimens how their interpretations on a test set compared with a reference diagnosis developed by a consensus panel of experienced breast pathologists. After interpreting a test set of 60 cases, 92 participating pathologists were asked to estimate how their interpretations compared with the standard for benign without atypia, atypia, ductal carcinoma in situ and invasive cancer. We then asked pathologists their thoughts about learning about differences in their perceptions compared with actual agreement. Overall, participants tended to overestimate their agreement with the reference standard, with a mean difference of 5.5% (75.9% actual agreement; 81.4% estimated agreement), especially for atypia and were least likely to overestimate it for invasive breast cancer. Non-academic affiliated pathologists were more likely to more closely estimate their performance relative to academic affiliated pathologists (77.6 vs 48%; P=0.001), whereas participants affiliated with an academic medical center were more likely to underestimate agreement with their diagnoses compared with non-academic affiliated pathologists (40 vs 6%). Before the continuing medical education program, nearly 55% (54.9%) of participants could not estimate whether they would overinterpret the cases or underinterpret them relative to the reference diagnosis. Nearly 80% (79.8%) reported learning new information from this individualized web-based continuing medical education program, and 23.9% of pathologists identified strategies they would change their practice to improve. In conclusion, when evaluating breast pathology specimens, pathologists do a good job of estimating their diagnostic agreement with a reference standard, but for atypia cases, pathologists tend to overestimate diagnostic agreement. Many participants were able to identify ways to improve.

Evaluation of 12 strategies for obtaining second opinions to improve interpretation of breast histopathology: simulation study

OBJECTIVE

 To evaluate the potential effect of second opinions on improving the accuracy of diagnostic interpretation of breast histopathology.

DESIGN

 Simulation study.

SETTING

 12 different strategies for acquiring independent second opinions.

PARTICIPANTS

 Interpretations of 240 breast biopsy specimens by 115 pathologists, one slide for each case, compared with reference diagnoses derived by expert consensus.

MAIN OUTCOME MEASURES

 Misclassification rates for individual pathologists and for 12 simulated strategies for second opinions. Simulations compared accuracy of diagnoses from single pathologists with that of diagnoses based on pairing interpretations from first and second independent pathologists, where resolution of disagreements was by an independent third pathologist. 12 strategies were evaluated in which acquisition of second opinions depended on initial diagnoses, assessment of case difficulty or borderline characteristics, pathologists' clinical volumes, or whether a second opinion was required by policy or desired by the pathologists. The 240 cases included benign without atypia (10% non-proliferative, 20% proliferative without atypia), atypia (30%), ductal carcinoma in situ (DCIS, 30%), and invasive cancer (10%). Overall misclassification rates and agreement statistics depended on the composition of the test set, which included a higher prevalence of difficult cases than in typical practice.

RESULTS

 Misclassification rates significantly decreased (P<0.001) with all second opinion strategies except for the strategy limiting second opinions only to cases of invasive cancer. The overall misclassification rate decreased from 24.7% to 18.1% when all cases received second opinions (P<0.001). Obtaining both first and second opinions from pathologists with a high volume (≥10 breast biopsy specimens weekly) resulted in the lowest misclassification rate in this test set (14.3%, 95% confidence interval 10.9% to 18.0%). Obtaining second opinions only for cases with initial interpretations of atypia, DCIS, or invasive cancer decreased the over-interpretation of benign cases without atypia from 12.9% to 6.0%. Atypia cases had the highest misclassification rate after single interpretation (52.2%), remaining at more than 34% in all second opinion scenarios.

CONCLUSION

 Second opinions can statistically significantly improve diagnostic agreement for pathologists' interpretations of breast biopsy specimens; however, variability in diagnosis will not be completely eliminated, especially for breast specimens with atypia.

Variability in Pathologists' Interpretations of Individual Breast Biopsy Slides: A Population Perspective

BACKGROUND

The effect of physician diagnostic variability on accuracy at a population level depends on the prevalence of diagnoses.

OBJECTIVE

To estimate how diagnostic variability affects accuracy from the perspective of a U.S. woman aged 50 to 59 years having a breast biopsy.

DESIGN

Applied probability using Bayes' theorem.

SETTING

B-Path (Breast Pathology) Study comparing pathologists' interpretations of a single biopsy slide versus a reference consensus interpretation from 3 experts.

PARTICIPANTS

115 practicing pathologists (6900 total interpretations from 240 distinct cases).

MEASUREMENTS

A single representative slide from each of the 240 cases was used to estimate the proportion of biopsies with a diagnosis that would be verified if the same slide were interpreted by a reference group of 3 expert pathologists. Probabilities of confirmation (predictive values) were estimated using B-Path Study results and prevalence of biopsy diagnoses for women aged 50 to 59 years in the Breast Cancer Surveillance Consortium.

RESULTS

Overall, if 1 representative slide were used per case, 92.3% (95% CI, 91.4% to 93.1%) of breast biopsy diagnoses would be verified by reference consensus diagnoses, with 4.6% (CI, 3.9% to 5.3%) overinterpreted and 3.2% (CI, 2.7% to 3.6%) underinterpreted. Verification of invasive breast cancer and benign without atypia diagnoses is highly probable; estimated predictive values were 97.7% (CI, 96.5% to 98.7%) and 97.1% (CI, 96.7% to 97.4%), respectively. Verification is less probable for atypia (53.6% overinterpreted and 8.6% underinterpreted) and ductal carcinoma in situ (DCIS) (18.5% overinterpreted and 11.8% underinterpreted).

LIMITATIONS

Estimates are based on a testing situation with 1 slide used per case and without access to second opinions. Population-adjusted estimates may differ for women from other age groups, unscreened women, or women in different practice settings.

CONCLUSION

This analysis, based on interpretation of a single breast biopsy slide per case, predicts a low likelihood that a diagnosis of atypia or DCIS would be verified by a reference consensus diagnosis. This diagnostic grey zone should be considered in clinical management decisions in patients with these diagnoses.

PRIMARY FUNDING SOURCE

National Cancer Institute.

The Influence of Disease Severity of Preceding Clinical Cases on Pathologists' Medical Decision Making

BACKGROUND

Medical decision making may be influenced by contextual factors. We evaluated whether pathologists are influenced by disease severity of recently observed cases.

METHODS

Pathologists independently interpreted 60 breast biopsy specimens (one slide per case; 240 total cases in the study) in a prospective randomized observational study. Pathologists interpreted the same cases in 2 phases, separated by a washout period of >6 months. Participants were not informed that the cases were identical in each phase, and the sequence was reordered randomly for each pathologist and between phases. A consensus reference diagnosis was established for each case by 3 experienced breast pathologists. Ordered logit models examined the effect the pathologists' diagnoses on the preceding case or the 5 preceding cases had on their diagnosis for the subsequent index case.

RESULTS

Among 152 pathologists, 49 provided interpretive data in both phases I and II, 66 from only phase I, and 37 from phase II only. In phase I, pathologists were more likely to indicate a more severe diagnosis than the reference diagnosis when the preceding case was diagnosed as ductal carcinoma in situ (DCIS) or invasive cancer (proportional odds ratio [POR], 1.28; 95% confidence interval [CI], 1.15-1.42). Results were similar when considering the preceding 5 cases and for the pathologists in phase II who interpreted the same cases in a different order compared with phase I (POR, 1.17; 95% CI, 1.05-1.31).

CONCLUSION

Physicians appear to be influenced by the severity of previously interpreted test cases. Understanding types and sources of diagnostic bias may lead to improved assessment of accuracy and better patient care.

Medical malpractice concerns and defensive medicine: a nationwide survey of breast pathologists

OBJECTIVES

"Assurance behaviors" in medical practice involve providing additional services of marginal or no medical value to avoid adverse outcomes, deter patients from filing malpractice claims, or ensure that legal standards of care were met. The extent to which concerns about medical malpractice influence assurance behaviors of pathologists interpreting breast specimens is unknown.

METHODS

Breast pathologists (n = 252) enrolled in a nationwide study completed an online survey of attitudes regarding malpractice and perceived alterations in interpretive behavior due to concerns of malpractice. Associations between pathologist characteristics and the impact of malpractice concerns on personal and colleagues' assurance behaviors were determined by χ(2) and logistic regression analysis.

RESULTS

Most participants reported using one or more assurance behaviors due to concerns about medical malpractice for both their personal (88%) and colleagues' (88%) practices, including ordering additional stains, recommending additional surgical sampling, obtaining second reviews, or choosing the more severe diagnosis for borderline cases. Nervousness over breast pathology was positively associated with assurance behavior and remained statistically significant in a multivariable logistic regression model (odds ratio, 2.5; 95% confidence interval, 1.0-6.1; P = .043).

CONCLUSIONS

Practicing US breast pathologists report exercising defensive medicine by using assurance behaviors due to malpractice concerns.

Digitized whole slides for breast pathology interpretation: current practices and perceptions

Digital whole slide imaging (WSI) is an emerging technology for pathology interpretation; however, little is known about pathologists' practice patterns or perceptions regarding WSI. A national sample (N = 252) of pathologists from New Hampshire, Vermont, Washington, Oregon, Arizona, Alaska, Maine, and Minnesota were surveyed in this cross-sectional study (2011-2013). The survey included questions on pathologists' experience, WSI practice patterns, and perceptions using a six-point Likert scale. Agreement was summarized with descriptive statistics to characterize pathologists' use and perceptions of WSI. The majority of participating pathologists were males (63%) between 40 and 59 years of age (70%) and not affiliated with an academic medical center (72%). Experience with WSI was reported by 49%. Types of use reported included CME/board exams/teaching (28%), tumor board/clinical conference (22%), archival purposes (6%), consultative diagnosis (4%), research (4%), and other uses (12%). Most respondents (79%) agreed that accurate diagnoses can be made with this technology, and that WSI is useful for obtaining a second opinion (88%). However, 78% of pathologists agreed that digital slides are too slow for routine clinical interpretation. Fifty-nine percent agreed that the benefits of WSI outweigh concerns. The respondents were equally split as to whether they would like to adopt WSI (51%) or not (49%). About half of pathologists reported experience with the WSI technology, largely for CME, licensure/board exams, and teaching. Positive perceptions regarding WSI slightly outweigh negative perceptions. Understanding practice patterns with WSI as dissemination advances may facilitate concordance of perceptions with adoption of the technology.

Diagnostic concordance among pathologists interpreting breast biopsy specimens

IMPORTANCE

A breast pathology diagnosis provides the basis for clinical treatment and management decisions; however, its accuracy is inadequately understood.

OBJECTIVES

To quantify the magnitude of diagnostic disagreement among pathologists compared with a consensus panel reference diagnosis and to evaluate associated patient and pathologist characteristics.

DESIGN, SETTING, AND PARTICIPANTS

Study of pathologists who interpret breast biopsies in clinical practices in 8 US states.

EXPOSURES

Participants independently interpreted slides between November 2011 and May 2014 from test sets of 60 breast biopsies (240 total cases, 1 slide per case), including 23 cases of invasive breast cancer, 73 ductal carcinoma in situ (DCIS), 72 with atypical hyperplasia (atypia), and 72 benign cases without atypia. Participants were blinded to the interpretations of other study pathologists and consensus panel members. Among the 3 consensus panel members, unanimous agreement of their independent diagnoses was 75%, and concordance with the consensus-derived reference diagnoses was 90.3%.

MAIN OUTCOMES AND MEASURES

The proportions of diagnoses overinterpreted and underinterpreted relative to the consensus-derived reference diagnoses were assessed.

RESULTS

Sixty-five percent of invited, responding pathologists were eligible and consented to participate. Of these, 91% (N = 115) completed the study, providing 6900 individual case diagnoses. Compared with the consensus-derived reference diagnosis, the overall concordance rate of diagnostic interpretations of participating pathologists was 75.3% (95% CI, 73.4%-77.0%; 5194 of 6900 interpretations). Among invasive carcinoma cases (663 interpretations), 96% (95% CI, 94%-97%) were concordant, and 4% (95% CI, 3%-6%) were underinterpreted; among DCIS cases (2097 interpretations), 84% (95% CI, 82%-86%) were concordant, 3% (95% CI, 2%-4%) were overinterpreted, and 13% (95% CI, 12%-15%) were underinterpreted; among atypia cases (2070 interpretations), 48% (95% CI, 44%-52%) were concordant, 17% (95% CI, 15%-21%) were overinterpreted, and 35% (95% CI, 31%-39%) were underinterpreted; and among benign cases without atypia (2070 interpretations), 87% (95% CI, 85%-89%) were concordant and 13% (95% CI, 11%-15%) were overinterpreted. Disagreement with the reference diagnosis was statistically significantly higher among biopsies from women with higher (n = 122) vs lower (n = 118) breast density on prior mammograms (overall concordance rate, 73% [95% CI, 71%-75%] for higher vs 77% [95% CI, 75%-80%] for lower, P < .001), and among pathologists who interpreted lower weekly case volumes (P < .001) or worked in smaller practices (P = .034) or nonacademic settings (P = .007).

CONCLUSIONS AND RELEVANCE

In this study of pathologists, in which diagnostic interpretation was based on a single breast biopsy slide, overall agreement between the individual pathologists' interpretations and the expert consensus-derived reference diagnoses was 75.3%, with the highest level of concordance for invasive carcinoma and lower levels of concordance for DCIS and atypia. Further research is needed to understand the relationship of these findings with patient management.

Demographic and practice characteristics of pathologists who enjoy breast tissue interpretation

Physician attributes, job satisfaction and confidence in clinical skills are associated with enhanced performance and better patient outcomes. We surveyed 252 pathologists to evaluate associations between enjoyment of breast pathology, demographic/clinical characteristics and diagnostic performance. Diagnostic performance was determined by comparing pathologist assessments of a set of 60 cases with consensus assessments of the same cases made by a panel of experienced pathologists. Eighty-three percent of study participants reported enjoying breast pathology. Pathologists who enjoy breast interpretation were more likely to review ≥10 cases/week (p = 0.003), report breast interpretation expertise (p = 0.013) and have high levels of confidence interpreting breast pathology (p < 0.001). These pathologists were less likely to report that the field was challenging (p < 0.001) and that breast cases make them more nervous than other types of pathology (p < 0.001). Enjoyment was not associated with diagnostic performance. Millions of women undergo breast biopsy annually, thus it is reassuring that although nearly a fifth of practicing pathologists who interpret breast tissue report not enjoying the field, precision is not impacted.

A framework for evaluating diagnostic discordance in pathology discovered during research studies

CONTEXT

Little is known about the frequency of discordant diagnoses identified during research.

OBJECTIVE

To describe diagnostic discordance identified during research and apply a newly designed research framework for investigating discordance.

DESIGN

Breast biopsy cases (N = 407) from registries in Vermont and New Hampshire were independently reviewed by a breast pathology expert. The following research framework was developed to assess those cases: (1) compare the expert review and study database diagnoses, (2) determine the clinical significance of diagnostic discordance, (3) identify and correct data errors and verify the existence of true diagnostic discrepancies, (4) consider the impact of borderline cases, and (5) determine the notification approach for verified disagreements.

RESULTS

Initial overall discordance between the original diagnosis recorded in our research database and a breast pathology expert was 32.2% (131 of 407). This was reduced to less than 10% after following the 5-step research framework. Detailed review identified 12 cases (2.9%) with data errors (2 in the underlying pathology registry, 3 with incomplete slides sent for expert review, and 7 with data abstraction errors). After excluding the cases with data errors, 38 cases (9.6%) among the remaining 395 had clinically meaningful discordant diagnoses (κ = 0.82; SE, 0.04; 95% confidence interval, 0.76-0.87). Among these 38 cases, 20 (53%) were considered borderline between 2 diagnoses by either the original pathologist or the expert. We elected to notify the pathology registries and facilities regarding discordant diagnoses.

CONCLUSIONS

Understanding the types and sources of diagnostic discordance uncovered in research studies may lead to improved scientific data and better patient care.

Eye movements as an index of pathologist visual expertise: a pilot study

A pilot study examined the extent to which eye movements occurring during interpretation of digitized breast biopsy whole slide images (WSI) can distinguish novice interpreters from experts, informing assessments of competency progression during training and across the physician-learning continuum. A pathologist with fellowship training in breast pathology interpreted digital WSI of breast tissue and marked the region of highest diagnostic relevance (dROI). These same images were then evaluated using computer vision techniques to identify visually salient regions of interest (vROI) without diagnostic relevance. A non-invasive eye tracking system recorded pathologists’ (N = 7) visual behavior during image interpretation, and we measured differential viewing of vROIs versus dROIs according to their level of expertise. Pathologists with relatively low expertise in interpreting breast pathology were more likely to fixate on, and subsequently return to, diagnostically irrelevant vROIs relative to experts. Repeatedly fixating on the distracting vROI showed limited value in predicting diagnostic failure. These preliminary results suggest that eye movements occurring during digital slide interpretation can characterize expertise development by demonstrating differential attraction to diagnostically relevant versus visually distracting image regions. These results carry both theoretical implications and potential for monitoring and evaluating student progress and providing automated feedback and scanning guidance in educational settings.

Development of an electronic breast pathology database in a community health system

Background:

Health care systems rely on electronic patient data, yet access to breast tissue pathology results continues to depend on interpreting dictated free-text reports.

Objective:

The objective was to develop a method to electronically search and categorize pathologic diagnoses of patients’ breast tissue specimens from dictated free-text pathology reports in a large health system for multiple users including clinicians.

Design:

A database integrating existing patient-level administrative and clinical information for breast cancer screening and diagnostic services and a web-based application for comprehensive searching of pathology reports were developed by a health system team led by pathologists. The Breast Pathology Assessment Tool and Hierarchy for Diagnosis (BPATH-Dx) provided search terms and guided electronic transcription of diagnoses from text fields on breast pathology clinical reports to standardized categories.

Approach:

Breast pathology encounters in the pathology database were matched with administrative data for 7332 women with breast tissue specimens obtained from an initial procedure in the health system from January 1, 2008 to December 31, 2011. Sequential queries of the pathology text based on BPATH-Dx categorized biopsies according to their worst pathological diagnosis, as is standard practice. Diagnoses ranged from invasive breast cancer (23.3%), carcinoma in situ (7.8%), atypical lesions (6.39%), proliferative lesions without atypia (27.9%), and nonproliferative lesions (34.7%), and were further classified into subcategories. A random sample of 5% of reports that were manually reviewed indicated 97.5% agreement.

Conclusions:

Sequential queries of free-text pathology reports guided by a standardized assessment tool in conjunction with a web-based search application provide an efficient and reproducible approach to accessing nonmalignant breast pathology diagnoses. This method advances the use of pathology data and electronic health records to improve health care quality, patient care, outcomes, and research.

Second opinion in breast pathology: policy, practice and perception

AIMS

To assess the laboratory policies, pathologists' clinical practice and perceptions about the value of second opinions for breast pathology cases among pathologists practising in the USA.

METHODS

Cross-sectional data were collected from 252 pathologists who interpret breast specimens in eight states using a web-based survey. Descriptive statistics were used to characterise findings.

RESULTS

Most participants had >10 years of experience interpreting breast specimens (64%), were not affiliated with academic centres (73%) and were not considered experts by their peers (79%). Laboratory policies mandating second opinions varied by diagnosis: invasive cancer 65%; ductal carcinoma in situ (DCIS) 56%; atypical ductal hyperplasia 36% and other benign cases 33%. 81% obtained second opinions in the absence of policies. Participants believed they improve diagnostic accuracy (96%) and protect from malpractice suits (83%), and were easy to obtain, did not take too much time and did not make them look less adequate. The most common (60%) approach to resolving differences between the first and second opinion is to ask for a third opinion, followed by reaching a consensus.

CONCLUSIONS

Laboratory-based second opinion policies vary for breast pathology but are most common for invasive cancer and DCIS cases. Pathologists have favourable attitudes towards second opinions, adhere to policies and obtain them even when policies are absent. Those without a formal policy may benefit from supportive clinical practices and systems that help obtain second opinions.

Understanding diagnostic variability in breast pathology: lessons learned from an expert consensus review panel

AIMS

To gain a better understanding of the reasons for diagnostic variability, with the aim of reducing the phenomenon.

METHODS AND RESULTS

In preparation for a study on the interpretation of breast specimens (B-PATH), a panel of three experienced breast pathologists reviewed 336 cases to develop consensus reference diagnoses. After independent assessment, cases coded as diagnostically discordant were discussed at consensus meetings. By the use of qualitative data analysis techniques, transcripts of 16 h of consensus meetings for a subset of 201 cases were analysed. Diagnostic variability could be attributed to three overall root causes: (i) pathologist-related; (ii) diagnostic coding/study methodology-related; and (iii) specimen-related. Most pathologist-related root causes were attributable to professional differences in pathologists' opinions about whether the diagnostic criteria for a specific diagnosis were met, most frequently in cases of atypia. Diagnostic coding/study methodology-related root causes were primarily miscategorizations of descriptive text diagnoses, which led to the development of a standardized electronic diagnostic form (BPATH-Dx). Specimen-related root causes included artefacts, limited diagnostic material, and poor slide quality. After re-review and discussion, a consensus diagnosis could be assigned in all cases.

CONCLUSIONS

Diagnostic variability is related to multiple factors, but consensus conferences, standardized electronic reporting formats and comments on suboptimal specimen quality can be used to reduce diagnostic variability.

Actualizing Personalized Healthcare for Women through Connected Data Systems: Breast Cancer Screening and Diagnosis

BACKGROUND

Healthcare organizations have invested in electronic patient data systems, yet use of health data to optimize personalized care has been limited.

PRIMARY STUDY OBJECTIVE

To develop and pilot an integrated source of health system data related to breast healthcare.

METHODS/DESIGN

This study is a quality improvement project. Patient-level data from multiple internal sources were identified, mapped to a common data model, linked, and validated to create a breast healthcare-specific data mart. Linkages were based on matching algorithms using patient identifiers to group data from the same patient. Data definitions, a data dictionary, and indicators for quality and benchmarking aligned with standardized measures. Clinical pathways were developed to outline the patient populations, data elements, decision points, and outcomes for specific conditions.

SETTING

Electronic data sources in a community-based health system in the United States.

PARTICIPANTS

Women receiving breast cancer screening, prevention, and diagnosis services.

MAIN OUTCOME MEASURES

Distribution of mammography examinations and pathologic results of breast biopsies.

RESULTS

From 2008 to 2011, 200768 screening and 50200 diagnostic mammograms were obtained; rates varied by age over time. Breast biopsies for 7332 women indicated 23.3% with invasive breast cancer, 6.7% with ductal carcinoma in situ, and 70.0% with nonmalignant diagnoses that would not have been further differentiated by administrative codes alone.

LIMITATIONS

Evaluation of validity and efficiency and additional tracking of clinical outcomes are needed.

CONCLUSIONS

The creation of a patient-centered data system by connecting and integrating disparate data sources within a large health system allows customized analyses of data and improves capacity for clinical decision making and personalized healthcare.