The effect of an e-learning module on grading variation of (pre)malignant breast lesions

Use of the ISUP e-learning module improves interrater reliability in prostate cancer grading

AIMS

Prostate cancer (PCa) grading is an important prognostic parameter, but is subject to considerable observer variation. Previous studies have shown that interobserver variability decreases after participants were trained using an e-learning module. However, since the publication of these studies, grading of PCa has been enhanced by adopting the International Society of Urological Pathology (ISUP) 2014 grading classification. This study investigates the effect of training on interobserver variability of PCa grading, using the ISUP Education web e-learning on Gleason grading.

METHODS

The ISUP Education Prostate Test B Module was distributed among Dutch pathologists. The module uses images graded by the ISUP consensus panel consisting of 24 expert uropathologists. Participants graded the same 10 images before and after e-learning. We included those who completed the tests before and after training. We evaluated variation in PCa grading in a fully crossed study design, using linearly weighted kappa values for each pathologist, comparing them to other pathologists and to the ISUP consensus panel. We analysed the improvement in median weighted kappas before and after training, using Wilcoxon's signed rank-test.

RESULTS

We included 42 pathologists. Inter-rater reliability between pathologists improved from 0.70 before training to 0.74 after training (p=0.01). When compared with the ISUP consensus panel, five pathologists improved significantly, whereas the kappa of one pathologist was significantly lower after training. All pathologists who improved significantly, graded with less than substantial agreement before training.

CONCLUSIONS

ISUP Prostate Test B e-learning reduces variability in PCa grading. E-learning is a cost-effective method for standardisation of pathology.

Interlaboratory Gleason grading variation affects treatment: a Dutch historic cohort study in 30 509 patients with prostate cancer

AIM

Substantial variation in Gleason grading (GG) of prostate cancer (PCa) exists between Dutch pathology laboratories. This study investigates its impact on treatment strategies.

METHODS

Pathology reports of prostate needle biopsies and clinical data of patients with PCa diagnosed between 2017 and 2019 were retrieved from the Dutch nationwide network and registry of histopathology and cytopathology and The Netherlands Cancer Registry. We investigated the impact of grading variation on treatment strategy for patients whose grade was decisive in treatment choice. First, we evaluated the effect of grading practice (low, average or high grading) on active treatment (AT) versus active surveillance in patients with prostate-specific antigen (PSA) <10 ng/mL and cT1c/cT2a disease. Second, we assessed the association of grading practice with performance of pelvic lymph node dissection (PLND) in patients with PSA 10-20 ng/mL or cT2b disease. We used multivariable logistic regression to analyse the relation between laboratories' grading practices and AT or PLND.

RESULTS

We included 30 509 patients. GG was decisive in treatment strategy for 11 925 patients (39%). AT was performed significantly less often in patients diagnosed by laboratories that graded lower than average (OR=0.77, 95% CI 0.68 to 0.88). Conversely, patients received AT significantly more often when diagnosed in high-grading laboratories versus average-grading laboratories (OR=1.21, 95% CI 1.03 to1.43). PLND was performed significantly less often in patients diagnosed by low-grading versus average-grading laboratories (OR=0.66, 95% CI 0.48 to 0.90).

CONCLUSION

Our study shows that the odds that a patient undergoes AT or PLND, depends on laboratories' grading practices in a substantial number of patients. This likely influences patient prognosis and outcome, necessitating standardisation of GG to prevent suboptimal patient outcome.

Learning Outcomes and Educational Effectiveness of E-Learning as a Continuing Professional Development Intervention for Practicing Surgeons and Proceduralists: A Systematic Review

BACKGROUND

Electronic learning (e-Learning) has become a prevalent mode of delivering medical education. We aimed to determine the learning outcomes and educational effectiveness of e-Learning as a continuing professional development (CPD) intervention for practicing surgeons and proceduralists.

METHODS

We searched MEDLINE databases and included studies reporting learning outcomes of e-learning CPD interventions for practicing surgeons and physicians performing technical procedures. We excluded articles only studying surgical trainees and those not reporting learning outcomes. Two reviewers independently screened, extracted data, and assessed study quality using the Critical Appraisal Skills Programme (CASP) tools. Learning outcomes and educational effectiveness were categorized using Moore's Outcomes Framework (PROSPERO: CRD42022333523).

RESULTS

Of 1307 identified articles, 12 were included- 9 cohort studies, one randomized controlled trial and 2 qualitative studies, with a total of 2158 participants. Eight studies were rated as moderate, five as strong, and 2 as weak in study quality. E-Learning CPD interventions included web-based modules, image recognition, videos, a repository of videos and schematics, and an online journal club. Seven studies reported participants' satisfaction with the e-Learning interventions (Moore's Level 2), 4 reported improvements in participants' declarative knowledge (Level 3a), 1 reported improvements in procedural knowledge (Level 3b) and five reported improvements in participants' procedural competence in an educational setting (Level 4). No studies demonstrated improvements in participants' workplace-based performance, the health of patients, or community health (Levels 5-7).

CONCLUSIONS

E-Learning as a CPD educational intervention is associated with high satisfaction and improvements in knowledge and procedural competencies of practicing surgeons and proceduralists in an educational setting. Future research is required to investigate whether e-Learning is associated with higher-level learning outcomes.

Detection of grey zones in inter-rater agreement studies

BACKGROUND

In inter-rater agreement studies, the assessment behaviour of raters can be influenced by their experience, training levels, the degree of willingness to take risks, and the availability of clear guidelines for the assessment. When the assessment behaviour of raters differentiates for some levels of an ordinal classification, a grey zone occurs between the corresponding adjacent cells to these levels around the main diagonal of the table. A grey zone introduces a negative bias to the estimate of the agreement level between the raters. In that sense, it is crucial to detect the existence of a grey zone in an agreement table.

METHODS

In this study, a framework composed of a metric and the corresponding threshold is developed to identify grey zones in an agreement table. The symmetry model and Cohen's kappa are used to define the metric, and the threshold is based on a nonlinear regression model. A numerical study is conducted to assess the accuracy of the developed framework. Real data examples are provided to illustrate the use of the metric and the impact of identifying a grey zone.

RESULTS

The sensitivity and specificity of the proposed framework are shown to be very high under moderate, substantial, and near-perfect agreement levels for [Formula: see text] and [Formula: see text] tables and sample sizes greater than or equal to 100 and 50, respectively. Real data examples demonstrate that when a grey zone is detected in the table, it is possible to report a notably higher level of agreement in the studies.

CONCLUSIONS

The accuracy of the proposed framework is sufficiently high; hence, it provides practitioners with a precise way to detect the grey zones in agreement tables.

Significant Inter- and Intralaboratory Variation in Gleason Grading of Prostate Cancer: A Nationwide Study of 35,258 Patients in The Netherlands

Simple Summary

Gleason grading of prostate cancer is essential for treatment strategies and patient prognosis. Previous studies showed grading variation between pathologists when grading prostate cancer. Our study analyzed the presence and extent of grading variation between and within pathology laboratories in The Netherlands. In our nationwide retrospective study, we analyzed prostate needle biopsy reports of 35,258 patients in The Netherlands graded by 40 pathology laboratories. We found a considerable variation between and within pathology laboratories, as over half of the laboratories graded significantly different from the national mean. This likely affects treatment strategy and prognosis assessment of prostate cancer patients.

Abstract

Purpose: Our aim was to analyze grading variation between pathology laboratories and between pathologists within individual laboratories using nationwide real-life data. Methods: We retrieved synoptic (n = 13,397) and narrative (n = 29,377) needle biopsy reports from the Dutch Pathology Registry and prostate-specific antigen values from The Netherlands Cancer Registration for prostate cancer patients diagnosed between January 2017 and December 2019. We determined laboratory-specific proportions per histologic grade and unadjusted odds ratios (ORs) for International Society of Urological Pathologists Grades 1 vs. 2–5 for 40 laboratories due to treatment implications for higher grades. Pathologist-specific proportions were determined for 21 laboratories that consented to this part of analysis. The synoptic reports of 21 laboratories were used for analysis of case-mix correction for PSA, age, year of diagnosis, number of biopsies and positive cores. Results: A total of 38,321 reports of 35,258 patients were included. Grade 1 ranged between 19.7% and 44.3% per laboratory (national mean = 34.1%). Out of 40 laboratories, 22 (55%) reported a significantly deviant OR, ranging from 0.48 (95% confidence interval (CI) 0.39–0.59) to 1.54 (CI 1.22–1.93). Case-mix correction was performed for 10,294 reports, altering the status of 3/21 (14%) laboratories, but increasing the observed variation (20.8% vs. 17.7%). Within 15/21 (71%) of laboratories, significant inter-pathologist variation existed. Conclusion: Substantial variation in prostate cancer grading was observed between and within Dutch pathology laboratories. Case-mix correction did not explain the variation. Better standardization of prostate cancer grading is warranted to optimize and harmonize treatment.

Grading of invasive breast carcinoma: the way forward

Histologic grading has been a simple and inexpensive method to assess tumor behavior and prognosis of invasive breast cancer grading, thereby identifying patients at risk for adverse outcomes, who may be eligible for (neo)adjuvant therapies. Histologic grading needs to be performed accurately, on properly fixed specimens, and by adequately trained dedicated pathologists that take the time to diligently follow the protocol methodology. In this paper, we review the history of histologic grading, describe the basics of grading, review prognostic value and reproducibility issues, compare performance of grading to gene expression profiles, and discuss how to move forward to improve reproducibility of grading by training, feedback and artificial intelligence algorithms, and special stains to better recognize mitoses. We conclude that histologic grading, when adequately carried out, remains to be of important prognostic value in breast cancer patients.

Variability in grading of ductal carcinoma in situ among an international group of pathologists

The prognostic value of cytonuclear grade in ductal carcinoma in situ (DCIS) is debated, partly due to high interobserver variability and the use of multiple guidelines. The aim of this study was to evaluate interobserver agreement in grading DCIS between Dutch, British, and American pathologists. Haematoxylin and eosin-stained slides of 425 women with primary DCIS were independently reviewed by nine breast pathologists based in the Netherlands, the UK, and the USA. Chance-corrected kappa (κma ) for association between pathologists was calculated based on a generalised linear mixed model using the ordinal package in R. Overall κma for grade of DCIS (low, intermediate, or high) was estimated to be 0.50 (95% confidence interval [CI] 0.44-0.56), indicating a moderate association between pathologists. When the model was adjusted for national guidelines, the association for grade did not change (κma = 0.53; 95% CI 0.48-0.57); subgroup analysis for pathologists using the UK pathology guidelines only had significantly higher association (κma = 0.58; 95% CI 0.56-0.61). To assess if concordance of grading relates to the expression of the oestrogen receptor (ER) and HER2, archived immunohistochemistry was analysed on a subgroup (n = 106). This showed that non-high grade according to the majority opinion was associated with ER positivity and HER2 negativity (100 and 89% of non-high grade cases, respectively). In conclusion, DCIS grade showed only moderate association using whole slide images scored by nine breast pathologists. As therapeutic decisions and inclusion in ongoing clinical trials are guided by DCIS grade, there is a pressing need to reduce interobserver variability in grading. ER and HER2 might be supportive to prevent the accidental and unwanted inclusion of high-grade DCIS in such trials.

The increasing importance of histologic grading in tailoring adjuvant systemic therapy in 30,843 breast cancer patients

Purpose

The large variation in histologic grading of invasive breast cancer (IBC) that has been reported likely influences tailoring adjuvant therapy. The role of grading in therapeutic decision-making in daily practice, was evaluated using the Dutch national guidelines for IBC-management.

Methods

Synoptic reports of IBC resection-specimens, obtained between 2013 and 2016, were extracted from the nationwide Dutch Pathology Registry, and linked to treatment-data from the Netherlands Cancer Registry. The relevance of grading for adjuvant chemotherapy (aCT) was quantified by identifying patients for whom grade was the determinative factor. In addition, the relation between grade and aCT-administration was evaluated by multivariate logistic regression for patients with a guideline-aCT-indication.

Results

30,843 patients were included. Applying the guideline that was valid between 2013 and 2016, grade was the determinative factor for the aCT-indication in 7744 (25.1%) patients, a percentage that even increased according to the current guideline where grade would be decisive for aCT in 10,869 (35.2%) patients. Also in current practice, the indication for adjuvant endocrine therapy (aET) would be based on grade in 9173 (29.7%) patients. Finally, as patients with lower-grade tumors receive aCT significantly less often, grade was also decisive in tailoring aCT de-escalation.

Conclusions

In the largest study published so far we illustrate the increasing importance of histologic grade in tailoring adjuvant systemic breast cancer therapy. Next to playing a key-role in aCT-indication and de-escalation, the role of grading has expanded to the indication for aET. Optimizing histologic grading by pathologists is urgently needed to diminish the risk of worse patient outcome due to non-optimal treatment.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10549-021-06098-7.