Initial Experience With a Novel Pre-sign-out Quality Assurance Tool for Review of Random Surgical Pathology Diagnoses in a Subspecialty-based University Practice

Error Reduction and Diagnostic Concordance in Breast Pathology

Errors in anatomic pathology can result in patients receiving inappropriate treatment and poor patient outcomes. Policies and procedures are necessary to decrease error and improve diagnostic concordance. Breast pathology may be more prone to diagnostic errors than other surgical pathology subspecialties due to inherit borderline diagnostic categories such as atypical ductal hyperplasia and low-grade ductal carcinoma in situ. Mandatory secondary review of internal and outside referral cases before treatment is effective in reducing diagnostic errors and improving concordance. Assessment of error through amendment/addendum tracking, implementing an incident reporting system, and multidisciplinary tumor boards can establish procedures to prevent future error.

Augmented Human Intelligence and Automated Diagnosis in Flow Cytometry for Hematologic Malignancies

OBJECTIVES

Clinical flow cytometry is laborious, time-consuming, and expensive given the need for data review by highly trained personnel such as technologists and pathologists as well as the significant number of normal cases. Given these issues, automation in analysis and diagnosis holds the key to major efficiency gains. The objective was to design an automated pipeline for the diagnosis of B-cell malignancies in flow cytometry and evaluate its performance against our standard clinical diagnostic flow cytometry process.

METHODS

Using 3,417 cases of peripheral blood data over 6 months from our 10-color B-cell screening tube, we used a newly described method for feature extraction and dimensionality reduction called UMAP on the raw flow cytometry data followed by random forest classification to classify cases without gating on specific population.

RESULTS

Our automated classifier was able to achieve greater than 95% accuracy in diagnosing all B-cell malignancies, and even better performance for specific malignancies for which the panel was designed, such as chronic lymphocytic leukemia. By adjusting classifier cutoffs, 100% sensitivity could be achieved with an albeit low 14% specificity. Hypothetically, this would allow 11% of the cases to be autoverified without human intervention.

CONCLUSIONS

These results suggest that a clinical implementation of this pipeline can greatly assist in quality control, improve turnaround time, and decrease staff workloads.

An artificial intelligence algorithm for prostate cancer diagnosis in whole slide images of core needle biopsies: a blinded clinical validation and deployment study

BACKGROUND

There is high demand to develop computer-assisted diagnostic tools to evaluate prostate core needle biopsies (CNBs), but little clinical validation and a lack of clinical deployment of such tools. We report here on a blinded clinical validation study and deployment of an artificial intelligence (AI)-based algorithm in a pathology laboratory for routine clinical use to aid prostate diagnosis.

METHODS

An AI-based algorithm was developed using haematoxylin and eosin (H&E)-stained slides of prostate CNBs digitised with a Philips scanner, which were divided into training (1 357 480 image patches from 549 H&E-stained slides) and internal test (2501 H&E-stained slides) datasets. The algorithm provided slide-level scores for probability of cancer, Gleason score 7-10 (vs Gleason score 6 or atypical small acinar proliferation [ASAP]), Gleason pattern 5, and perineural invasion and calculation of cancer percentage present in CNB material. The algorithm was subsequently validated on an external dataset of 100 consecutive cases (1627 H&E-stained slides) digitised on an Aperio AT2 scanner. In addition, the AI tool was implemented in a pathology laboratory within routine clinical workflow as a second read system to review all prostate CNBs. Algorithm performance was assessed with area under the receiver operating characteristic curve (AUC), specificity, and sensitivity, as well as Pearson's correlation coefficient (Pearson's r) for cancer percentage.

FINDINGS

The algorithm achieved an AUC of 0·997 (95% CI 0·995 to 0·998) for cancer detection in the internal test set and 0·991 (0·979 to 1·00) in the external validation set. The AUC for distinguishing between a low-grade (Gleason score 6 or ASAP) and high-grade (Gleason score 7-10) cancer diagnosis was 0·941 (0·905 to 0·977) and the AUC for detecting Gleason pattern 5 was 0·971 (0·943 to 0·998) in the external validation set. Cancer percentage calculated by pathologists and the algorithm showed good agreement (r=0·882, 95% CI 0·834 to 0·915; p<0·0001) with a mean bias of -4·14% (-6·36 to -1·91). The algorithm achieved an AUC of 0·957 (0·930 to 0·985) for perineural invasion. In routine practice, the algorithm was used to assess 11 429 H&E-stained slides pertaining to 941 cases leading to 90 Gleason score 7-10 alerts and 560 cancer alerts. 51 (9%) cancer alerts led to additional cuts or stains being ordered, two (4%) of which led to a third opinion request. We report on the first case of missed cancer that was detected by the algorithm.

INTERPRETATION

This study reports the successful development, external clinical validation, and deployment in clinical practice of an AI-based algorithm to accurately detect, grade, and evaluate clinically relevant findings in digitised slides of prostate CNBs.

FUNDING

Ibex Medical Analytics.

Integrated Pathology Informatics Enables High-Quality Personalized and Precision Medicine: Digital Pathology and Beyond

CONTEXT

- The critical role of pathology in diagnosis, prognosis, and prediction demands high-quality subspecialty diagnostics that integrates information from multiple laboratories.

OBJECTIVE

- To identify key requirements and to establish a systematic approach to providing high-quality pathology in a health care system that is responsible for services across a large geographic area.

DESIGN

- This report focuses on the development of a multisite pathology informatics platform to support high-quality surgical pathology and hematopathology using a sophisticated laboratory information system and whole slide imaging for histology and immunohistochemistry, integrated with ancillary tools, including electron microscopy, flow cytometry, cytogenetics, and molecular diagnostics.

RESULTS

- These tools enable patients in numerous geographic locations access to a model of subspecialty pathology that allows reporting of every specimen by the right pathologist at the right time. The use of whole slide imaging for multidisciplinary case conferences enables better communication among members of patient care teams. The system encourages data collection using a discrete data synoptic reporting module, has implemented documentation of quality assurance activities, and allows workload measurement, providing examples of additional benefits that can be gained by this electronic approach to pathology.

CONCLUSION

- This approach builds the foundation for accurate big data collection and high-quality personalized and precision medicine.

Quality Assurance in Breast Pathology: Lessons Learned From a Review of Amended Reports

Context.—A review of amended pathology reports provides valuable information regarding defects in the surgical pathology process.

Objective.—To review amended breast pathology reports with emphasis placed on interpretative errors and their mechanisms of detection.

Design.—All amended pathology reports for breast surgical specimens for a 5-year period at a large academic medical center were retrospectively identified and classified based on an established taxonomy.

Results.—Of 12 228 breast pathology reports, 122 amended reports were identified. Most (88 cases; 72%) amendments were due to noninterpretative errors, including 58 report defects, 12 misidentifications, and 3 specimen defects. A few (34 cases; 27.9%) were classified as misinterpretations, including 14 major diagnostic changes (11.5% of all amendments). Among major changes, there were cases of missed microinvasion or small foci of invasion, missed micrometastasis, atypical ductal hyperplasia overcalled as ductal carcinoma in situ, ductal carcinoma in situ involving sclerosing adenosis mistaken for invasive carcinoma, lymphoma mistaken for invasive carcinoma, and amyloidosis misdiagnosed as fat necrosis. Nine major changes were detected at interpretation of receptor studies and were not associated with clinical consequences. Three cases were associated with clinical consequences, and of note, the same pathologist interpreted the corresponding receptor studies.

Conclusions.—Review of amended reports was a useful method for identifying error frequencies, types, and methods of detection. Any time that a case is revisited for ancillary studies or other reasons, it is an opportunity for the surgical pathologist to reconsider one's own or another's diagnosis.

Enhancing and Customizing Laboratory Information Systems to Improve/Enhance Pathologist Workflow

Optimizing pathologist workflow can be difficult because it is affected by many variables. Surgical pathologists must complete many tasks that culminate in a final pathology report. Several software systems can be used to enhance/improve pathologist workflow. These include voice recognition software, pre-sign-out quality assurance, image utilization, and computerized provider order entry. Recent changes in the diagnostic coding and the more prominent role of centralized electronic health records represent potential areas for increased ways to enhance/improve the workflow for surgical pathologists. Additional unforeseen changes to the pathologist workflow may accompany the introduction of whole-slide imaging technology to the routine diagnostic work.

Interpretive Diagnostic Error Reduction in Surgical Pathology and Cytology: Guideline From the College of American Pathologists Pathology and Laboratory Quality Center and the Association of Directors of Anatomic and Surgical Pathology

CONTEXT

Additional reviews of diagnostic surgical and cytology cases have been shown to detect diagnostic discrepancies.

OBJECTIVE

To develop, through a systematic review of the literature, recommendations for the review of pathology cases to detect or prevent interpretive diagnostic errors.

DESIGN

The College of American Pathologists Pathology and Laboratory Quality Center in association with the Association of Directors of Anatomic and Surgical Pathology convened an expert panel to develop an evidence-based guideline to help define the role of case reviews in surgical pathology and cytology. A literature search was conducted to gather data on the review of cases in surgical pathology and cytology.

RESULTS

The panel drafted 5 recommendations, with strong agreement from open comment period participants ranging from 87% to 93%. The recommendations are: (1) anatomic pathologists should develop procedures for the review of selected pathology cases to detect disagreements and potential interpretive errors; (2) anatomic pathologists should perform case reviews in a timely manner to avoid having a negative impact on patient care; (3) anatomic pathologists should have documented case review procedures that are relevant to their practice setting; (4) anatomic pathologists should continuously monitor and document the results of case reviews; and (5) if pathology case reviews show poor agreement within a defined case type, anatomic pathologists should take steps to improve agreement.

CONCLUSIONS

Evidence exists that case reviews detect errors; therefore, the expert panel recommends that anatomic pathologists develop procedures for the review of pathology cases to detect disagreements and potential interpretive errors, in order to improve the quality of patient care.

Enhancing and Customizing Laboratory Information Systems to Improve/Enhance Pathologist Workflow

Optimizing pathologist workflow can be difficult because it is affected by many variables. Surgical pathologists must complete many tasks that culminate in a final pathology report. Several software systems can be used to enhance/improve pathologist workflow. These include voice recognition software, pre-sign-out quality assurance, image utilization, and computerized provider order entry. Recent changes in the diagnostic coding and the more prominent role of centralized electronic health records represent potential areas for increased ways to enhance/improve the workflow for surgical pathologists. Additional unforeseen changes to the pathologist workflow may accompany the introduction of whole-slide imaging technology to the routine diagnostic work.

Role of Informatics in Patient Safety and Quality Assurance

Quality assurance encompasses monitoring daily processes for accurate, timely, and complete reports in surgical pathology. Quality assurance also includes implementation of policies and procedures that prevent or detect errors in a timely manner. This article presents uses of informatics in quality assurance. Three main foci are critical to the general improvement of diagnostic surgical pathology. First is the application of informatics to specimen identification with lean methods for real-time statistical control of specimen receipt and processing. Second is the development of case reviews before sign-out. Third is the development of information technology in communication of results to assure treatment in a timely manner.

Can digital pathology result in cost savings? A financial projection for digital pathology implementation at a large integrated health care organization

Background:

Digital pathology offers potential improvements in workflow and interpretive accuracy. Although currently digital pathology is commonly used for research and education, its clinical use has been limited to niche applications such as frozen sections and remote second opinion consultations. This is mainly due to regulatory hurdles, but also to a dearth of data supporting a positive economic cost-benefit. Large scale adoption of digital pathology and the integration of digital slides into the routine anatomic/surgical pathology “slide less” clinical workflow will occur only if digital pathology will offer a quantifiable benefit, which could come in the form of more efficient and/or higher quality care.

Aim:

As a large academic-based health care organization expecting to adopt digital pathology for primary diagnosis upon its regulatory approval, our institution estimated potential operational cost savings offered by the implementation of an enterprise-wide digital pathology system (DPS).

Methods:

Projected cost savings were calculated for the first 5 years following implementation of a DPS based on operational data collected from the pathology department. Projected savings were based on two factors: (1) Productivity and lab consolidation savings; and (2) avoided treatment costs due to improvements in the accuracy of cancer diagnoses among nonsubspecialty pathologists. Detailed analyses of incremental treatment costs due to interpretive errors, resulting in either a false positive or false negative diagnosis, was performed for melanoma and breast cancer and extrapolated to 10 other common cancers.

Results:

When phased in over 5-years, total cost savings based on anticipated improvements in pathology productivity and histology lab consolidation were estimated at $12.4 million for an institution with 219,000 annual accessions. The main contributing factors to these savings were gains in pathologist clinical full-time equivalent capacity impacted by improved pathologist productivity and workload distribution. Expanding the current localized specialty sign-out model to an enterprise-wide shared general/subspecialist sign-out model could potentially reduce costs of incorrect treatment by $5.4 million. These calculations were based on annual over and under treatment costs for breast cancer and melanoma estimated to be approximately $26,000 and $11,000/case, respectively, and extrapolated to $21,500/case for other cancer types.

Conclusions:

The projected 5-year total cost savings for our large academic-based health care organization upon fully implementing a DPS was approximately $18 million. If the costs of digital pathology acquisition and implementation do not exceed this value, the return on investment becomes attractive to hospital administrators. Furthermore, improved patient outcome enabled by this technology strengthens the argument supporting adoption of an enterprise-wide DPS.

A medical staff peer review system in a public teaching hospital--an internal quality improvement tool

Peer review of the quality of care of the medical staff in a healthcare delivery system, properly executed and utilized, can bring about changes that improve the quality and safety of patient care, enhance clinical performance, and augment physician education. Although all healthcare facilities are mandated to conduct peer reviews, the process of how it is conducted, reported, and utilized varies widely. In 2007, our institution, a large public teaching acute care facility, developed and implemented an electronic Medical Staff Peer Review System (MS-PRS) that replaced the existing paper-based system and created a centralized database for all peer review activities. Despite limited resources and mounting known challenges, we have developed and implemented a system that includes 100% mortality reviews, an ongoing random review for reappointment and operative procedures, and morbidity peer reviews. Parallel to the 4-year implementation of the system, we observed a steady, significant downward trend in the medical malpractice claim rate, which can be attributable in part to the implementation of MS-PRS. In this paper, we share our experiences in the development, outcomes, challenges encountered, and lessons learned from MS-PRS and provide our recommendations to similar institutions for the development of such a system.

Use of a laboratory information system driven tool for pre-signout quality assurance of random cytopathology reports

BACKGROUND

Quality assurance (QA) programs in cytopathology laboratories in the USA currently primarily involve the review of Pap tests per clinical laboratory improvement amendments of 1988 federal regulations. A pre-signout quality assurance tool (PQAT) at our institution allows the laboratory information system (LIS) to also automatically and randomly select an adjustable percentage of non-gynecological cytopathology cases for review before release of the final report. The aim of this study was to review our experience and the effectiveness of this novel PQAT tool in cytology.

MATERIALS AND METHODS

Software modifications in the existing LIS application (CoPathPlus, Cerner) allow for the random QA of 8% of cases prior to signout. Selected cases are assigned to a second QA cytopathologist for review and all agreement and disagreements tracked. Detected errors are rectified before the case is signed out. Data from cases selected for PQAT over an 18-month period were collected and analyzed.

RESULTS

The total number of non-gynecological cases selected for QA review was 1339 (7.45%) out of 17,967 cases signed out during this time period. Most (1304) cases (97.4%) had an agreement in diagnosis. In 2.6% of cases, there were disagreements, including 34 minor and only 1 major disagreement. Average turnaround time of cases selected for review was not significantly altered.

CONCLUSION

The PQAT provides a prospective QA mechanism in non-gynecological cytopathology to prevent diagnostic errors from occurring. This LIS-driven tool allows for peer review and corrective action to be taken prior to reporting without delaying turnaround time, thereby improving patient safety.