Prostatic pathology reporting in the UK: development of a national external quality assurance scheme

Estimating the Irreducible Uncertainty in Visual Diagnosis: Statistical Modeling of Skill Using Response Models

BACKGROUND

For the representative problem of prostate cancer grading, we sought to simultaneously model both the continuous nature of the case spectrum and the decision thresholds of individual pathologists, allowing quantitative comparison of how they handle cases at the borderline between diagnostic categories.

METHODS

Experts and pathology residents each rated a standardized set of prostate cancer histopathological images on the International Society of Urological Pathologists (ISUP) scale used in clinical practice. They diagnosed 50 histologic cases with a range of malignancy, including intermediate cases in which clear distinction was difficult. We report a statistical model showing the degree to which each individual participant can separate the cases along the latent decision spectrum.

RESULTS

The slides were rated by 36 physicians in total: 23 ISUP pathologists and 13 residents. As anticipated, the cases showed a full continuous range of diagnostic severity. Cases ranged along a logit scale consistent with the consensus rating (Consensus ISUP 1: mean -0.93 [95% confidence interval {CI} -1.10 to -0.78], ISUP 2: -0.19 logits [-0.27 to -0.12]; ISUP 3: 0.56 logits [0.06-1.06]; ISUP 4 1.24 logits [1.10-1.38]; ISUP 5: 1.92 [1.80-2.04]). The best raters were able to meaningfully discriminate between all 5 ISUP categories, showing intercategory thresholds that were quantifiably precise and meaningful.

CONCLUSIONS

We present a method that allows simultaneous quantification of both the confusability of a particular case and the skill with which raters can distinguish the cases.

IMPLICATIONS

The technique generalizes beyond the current example to other clinical situations in which a diagnostician must impose an ordinal rating on a biological spectrum.

HIGHLIGHTS

Question: How can we quantify skill in visual diagnosis for cases that sit at the border between 2 ordinal categories-cases that are inherently difficult to diagnose?Findings: In this analysis of pathologists and residents rating prostate biopsy specimens, decision-aligned response models are calculated that show how pathologists would be likely to classify any given case on the diagnostic spectrum. Decision thresholds are shown to vary in their location and precision.Significance: Improving on traditional measures such as kappa and receiver-operating characteristic curves, this specialization of item response models allows better individual feedback to both trainees and pathologists, including better quantification of acceptable decision variation.

Low-risk prostate lesions: An evidence review to inform discussion on losing the "cancer" label

BACKGROUND

Active surveillance (AS) mitigates harms from overtreatment of low-risk prostate lesions. Recalibration of diagnostic thresholds to redefine which prostate lesions are considered "cancer" and/or adopting alternative diagnostic labels could increase AS uptake and continuation.

METHODS

We searched PubMed and EMBASE to October 2021 for evidence on: (1) clinical outcomes of AS, (2) subclinical prostate cancer at autopsy, (3) reproducibility of histopathological diagnosis, and (4) diagnostic drift. Evidence is presented via narrative synthesis.

RESULTS

AS: one systematic review (13 studies) of men undergoing AS found that prostate cancer-specific mortality was 0%-6% at 15 years. There was eventual termination of AS and conversion to treatment in 45%-66% of men. Four additional cohort studies reported very low rates of metastasis (0%-2.1%) and prostate cancer-specific mortality (0%-0.1%) over follow-up to 15 years. Overall, AS was terminated without medical indication in 1%-9% of men. Subclinical reservoir: 1 systematic review (29 studies) estimated that the subclinical cancer prevalence was 5% at <30 years, and increased nonlinearly to 59% by >79 years. Four additional autopsy studies (mean age: 54-72 years) reported prevalences of 12%-43%. Reproducibility: 1 recent well-conducted study found high reproducibility for low-risk prostate cancer diagnosis, but this was more variable in 7 other studies. Diagnostic drift: 4 studies provided consistent evidence of diagnostic drift, with the most recent (published 2020) reporting that 66% of cases were upgraded and 3% were downgraded when using contemporary diagnostic criteria compared to original diagnoses (1985-1995).

CONCLUSIONS

Evidence collated may inform discussion of diagnostic changes for low-risk prostate lesions.

Interobserver Variability in the Diagnosis of High-Grade Prostatic Intraepithelial Neoplasia in a Tertiary Hospital in Northern Jordan

Prostate intraepithelial neoplasia is described as a precursor lesion to prostatic adenocarcinoma. High-grade prostate intraepithelial neoplasia (HGPIN) is classified as both grade 2 and 3 prostate intraepithelial neoplasia due to inconsistency between pathologists' findings. In our study, we assessed the interobserver variability in the diagnosis of HGPIN among genitourinary and nongenitourinary pathologists. All cases with prostate adenocarcinoma diagnosis on needle core biopsy, radical prostatectomy, and transurethral resection of prostate (TURP) between the years 2005 and 2014 were included. In total, 191 prostate cancer cases were included: 109 needle core biopsies, 45 radical prostatectomies, and 37 TURP. All were independently reviewed by 2 urologic pathologists for the presence of HGPIN. High-grade prostate intraepithelial neoplasia was diagnosed in 65 cases (34%), among which the lesion was recognized by the reporting pathologists in 36 (55%) of the cases and was missed in 29 (45%) of the cases with a κ coefficient of 0.53. There was a moderate interobserver agreement in the diagnosis of HGPIN. Consultation with genitourinary pathologist can improve HGPIN diagnosis.

Gleason drift in the NIHR ProtecT study

AIMS

There is increasing evidence of Gleason score (GS) drift in prostatic core biopsies during the last two decades. The ProtecT study is a randomized controlled study and provides an excellent cohort to study the effect of time, prostate-specific antigen (PSA) level, perineural invasion, tumour length and age on GS.

METHODS AND RESULTS

The ProtecT study recruited men in the United Kingdom between 1999 and 2010. The Gleason scores were grouped into four categories ≤ 3 + 3, 3 + 4, 4 + 3 and ≥ 4 + 4 for analysis. Data from England between 2000 and 2012 were also available. A total of 3282 biopsies containing cancer were analysed. For each year of the ProtecT study, the odds of being diagnosed with a higher GS category increased by 4.9%. Higher GS was also associated with perineural invasion, increasing tumour length, age and PSA level. While biopsy GS from England was incomplete, it also showed a marked decrease in GS five and six tumours during the same period.

CONCLUSION

There was GS drift from 3 + 3 to 3 + 4 with time in the ProtecT study, but there appeared to be no significant change in percentage of GS 4 + 3 or higher. This drift was less dramatic when compared to GS in the rest of England.

Measuring interobserver variation in a pathology EQA scheme using weighted κ for multiple readers

Background

A Urological Pathology External Quality Assurance (EQA) Scheme in the UK has reported observer variation in the diagnosis and grading of adenocarcinoma in prostatic biopsies using basic κ statistics, which rate all disagreements equally.

Aim

The aim of this study is to use customised weighting schemes to report κ statistics that reflect the closeness of interobserver agreement in the prostate EQA scheme.

Methods

A total of 83, 114 and 116 pathologists took part, respectively, in three web-based circulations and were classified as either expert or other readers. For analyses of diagnosis, there were 10, 8 and 8 cases in the three circulations, respectively. For analyses of Gleason Sum Score, only invasive cases were included, leaving 5, 5 and 6 cases, respectively. Analyses were conducted using customised weighting schemes with ‘pairwise-weighted’ κ for multiple readers.

Results

Analysis of diagnosis for all circulations and all readers gave a composite κ value of 0.86 and pairwise-weighted κ (κp–w) value of 0.91, both regarded as ‘almost perfect’ agreement. This was due to the high proportion of responses that showed partial agreement. Analysis of Gleason Sum Score gave κ=0.38 and κp–w=0.58 over all circulations and all readers, indicating that discrepancies occur at the boundary between adjacent grades and may not be as clinically significant as suggested by composite κ.

Conclusion

Weighted κ show higher levels of agreement than previously reported as they have the advantage of applying weighting, which reflects the relative importance of different types of discordance in diagnosis or grading. Agreement on grading remained low.

A practical application of analysing weighted kappa for panels of experts and EQA schemes in pathology

Background

Kappa statistics are frequently used to analyse observer agreement for panels of experts and External Quality Assurance (EQA) schemes and generally treat all disagreements as total disagreement. However, the differences between ordered categories may not be of equal importance (eg, the difference between grades 1 vs 2 compared with 1 vs 3). Weighted kappa can be used to adjust for this when comparing a small number of readers, but this has not as yet been applied to the large number of readers typical of a national EQA scheme.

Aim

To develop and validate a method for applying weighted kappa to a large number of readers within the context of a real dataset: the UK National Urological Pathology EQA Scheme for prostatic biopsies.

Methods

Data on Gleason grade recorded by 19 expert readers were extracted from the fixed text responses of 20 cancer cases from four circulations of the EQA scheme. Composite kappa, currently used to compute an unweighted kappa for large numbers of readers, was compared with the mean kappa for all pairwise combinations of readers. Weighted kappa generalised for multiple readers was compared with the newly developed ‘pairwise-weighted’ kappa.

Results

For unweighted analyses, the median increase from composite to pairwise kappa was 0.006 (range −0.005 to +0.052). The difference between the pairwise-weighted kappa and generalised weighted kappa for multiple readers never exceeded ±0.01.

Conclusion

Pairwise-weighted kappa is a suitable and highly accurate approximation to weighted kappa for multiple readers.

Invasive size is an independent predictor of survival in pulmonary adenocarcinoma

Current classification of pulmonary adenocarcinoma includes noninvasive bronchioloalveolar carcinoma, mixed subtype adenocarcinoma, and several patterns of invasive carcinoma. The extent of invasion in mixed subtype adenocarcinoma is variable, and prior studies suggest that estimates of extent of desmoplasia or invasion and gross tumor size are predictors of survival. Pathologic review of 178 consecutive primary lung adenocarcinoma resections from 1997 to 2000 was performed blinded to outcome. Lymph node metastases were not present in adenocarcinomas with less then 0.6 cm of invasion. In multivariate analysis and in strata adjusted for stage, measurement of linear extent of invasion was significantly associated with survival whereas gross size measurement alone was not. Significant differences in median survival were observed when patients were divided into noninvasive, microinvasive (<0.6 cm invasion), and invasive subcategories. In conclusion, among lung adenocarcinomas, histologic assessment of invasive growth may provide valuable prognostic information, and tumors with invasion under 0.6 cm have a more indolent clinical course after resection.

[Active surveillance of localized prostate cancer. Significance of prostate core needle biopsies]

Today, more than 80% of men diagnosed with prostate cancer (PCA) by PSA screening do not die from the sequelae of their disease. About 70% present with early, organ-confined cancer and almost half of them are small (<5 cm(3)) without evidence of progression over years (insignificant PCA). It is assumed that screening brings the diagnosis of PCA forward by about 9 years and that in almost one third of these cases immediate radical prostatectomy or radiotherapy would result in overtreatment. Thus, the treatment strategy of "active surveillance" with selective but delayed intervention for patients with organ-confined PCA could be an attractive alternative to the known curative therapy options. However, a prerequisite of such a therapeutic approach would be a precise identification of patients at high risk for cancer progression. Careful work-up of prostate core needle biopsies including improved pre-embedding preparation and detailed interpretation are of the utmost importance. A Gleason score < or =6 and tumor in only one or two cores are considered predictive of organ-confined cancer. Pathologists should concentrate on correct Gleason scoring in core needle biopsies and identification of lesions that exclude a patient from active surveillance.