Clinical Decision Support for Ovarian Carcinoma Subtype Classification: A Pilot Observer Study With Pathology Trainees

The use of imputation in clinical decision support systems: a cardiovascular risk management pilot vignette study among clinicians

Aims

A major challenge of the use of prediction models in clinical care is missing data. Real-time imputation may alleviate this. However, to what extent clinicians accept this solution remains unknown. We aimed to assess acceptance of real-time imputation for missing patient data in a clinical decision support system (CDSS) including 10-year cardiovascular absolute risk for the individual patient.

Methods and results

We performed a vignette study extending an existing CDSS with the real-time imputation method joint modelling imputation (JMI). We included 17 clinicians to use the CDSS with three different vignettes, describing potential use cases (missing data, no risk estimate; imputed values, risk estimate based on imputed data; complete information). In each vignette, missing data were introduced to mimic a situation as could occur in clinical practice. Acceptance of end-users was assessed on three different axes: clinical realism, comfortableness, and added clinical value. Overall, the imputed predictor values were found to be clinically reasonable and according to the expectations. However, for binary variables, use of a probability scale to express uncertainty was deemed inconvenient. The perceived comfortableness with imputed risk prediction was low, and confidence intervals were deemed too wide for reliable decision-making. The clinicians acknowledged added value for using JMI in clinical practice when used for educational, research, or informative purposes.

Conclusion

Handling missing data in CDSS via JMI is useful, but more accurate imputations are needed to generate comfort in clinicians for use in routine care. Only then can CDSS create clinical value by improving decision-making.

Graduate Medical Education in Pathology: A Scoping Review

CONTEXT.—

Pathologists have produced a substantial body of literature on graduate medical education (GME). However, this body of literature is diverse and has not yet been characterized.

OBJECTIVE.—

To chart the concepts, research methods, and publication patterns of studies on GME in pathology.

DATA SOURCES.—

This was a systematic scoping review covering all literature produced since 1980 in the PubMed and Embase databases.

CONCLUSIONS.—

Research on GME in pathology is evenly dispersed across educational topics. This body of literature would benefit from research based on theory, stronger study designs, and studies that can provide evidence to support decisions on educational policies.

Selection of Representative Histologic Slides in Interobserver Reproducibility Studies: Insights from Expert Review for Ovarian Carcinoma Subtype Classification

BACKGROUND

Observer studies in pathology often utilize a limited number of representative slides per case, selected and reported in a nonstandardized manner. Reference diagnoses are commonly assumed to be generalizable to all slides of a case. We examined these issues in the context of pathologist concordance for histologic subtype classification of ovarian carcinomas (OCs).

MATERIALS AND METHODS

A cohort of 114 OCs consisting of 72 cases with a single representative slide (Group 1) and 42 cases with multiple representative slides (148 slides, 2-6 sections per case, Group 2) was independently reviewed by three experts in gynecologic pathology (case-based review). In a follow-up study, each individual slide was independently reviewed in a randomized order by the same pathologists (section-based review).

RESULTS

Average interobserver concordance varied from 100% for Group 1 to 64.3% for Group 2 (86.8% across all cases). Across Group 2, 19 cases (45.2%) had at least one slide classified as a different subtype than the subtype assigned from case-based review, demonstrating the impact of intratumoral heterogeneity. Section-based concordance across individual sections from Group 2 was comparable to case-based concordance for those cases indicating diagnostic challenges at the individual section level. Findings demonstrate the increased diagnostic complexity of heterogeneous tumors that require multiple section sampling and its impact on pathologist performance.

CONCLUSIONS

The proportion of cases with multiple representative slides in cohorts used in validation studies, such as those conducted to evaluate artificial intelligence/machine learning tools, can influence diagnostic performance, and if not accounted for, can cause disparities between research and real-world observations and between research studies. Case selection in validation studies should account for tumor heterogeneity to create balanced datasets in terms of diagnostic complexity.

Pathologist Concordance for Ovarian Carcinoma Subtype Classification and Identification of Relevant Histologic Features Using Microscope and Whole Slide Imaging: A Multisite Observer Study

CONTEXT.—

Despite several studies focusing on the validation of whole slide imaging (WSI) across organ systems or subspecialties, the use of WSI for specific primary diagnosis tasks has been underexamined.

OBJECTIVE.—

To assess pathologist performance for the histologic subtyping of individual sections of ovarian carcinomas using the light microscope and WSI.

DESIGN.—

A panel of 3 experienced gynecologic pathologists provided reference subtype diagnoses for 212 histologic sections from 109 ovarian carcinomas based on optical microscopy review. Two additional attending pathologists provided diagnoses and also identified the presence of a set of 8 histologic features important for ovarian tumor subtyping. Two experienced gynecologic pathologists and 2 fellows reviewed the corresponding WSI images for subtype classification and feature identification.

RESULTS.—

Across pathologists specialized in gynecologic pathology, concordance with the reference diagnosis for the 5 major ovarian carcinoma subtypes was significantly higher for a pathologist reading on microscope than each of 2 pathologists reading on WSI. Differences were primarily due to more frequent classification of mucinous carcinomas as endometrioid with WSI. Pathologists had generally low agreement in identifying histologic features important to ovarian tumor subtype classification, with either optical microscopy or WSI. This result suggests the need for refined histologic criteria for identifying such features. Interobserver agreement was particularly low for identifying intracytoplasmic mucin with WSI. Inconsistencies in evaluating nuclear atypia and mitoses with WSI were also observed.

CONCLUSIONS.—

Further research is needed to specify the reasons for these diagnostic challenges and to inform users and manufacturers of WSI technology.

Evaluation of the Use of Combined Artificial Intelligence and Pathologist Assessment to Review and Grade Prostate Biopsies

IMPORTANCE

Expert-level artificial intelligence (AI) algorithms for prostate biopsy grading have recently been developed. However, the potential impact of integrating such algorithms into pathologist workflows remains largely unexplored.

OBJECTIVE

To evaluate an expert-level AI-based assistive tool when used by pathologists for the grading of prostate biopsies.

DESIGN, SETTING, AND PARTICIPANTS

This diagnostic study used a fully crossed multiple-reader, multiple-case design to evaluate an AI-based assistive tool for prostate biopsy grading. Retrospective grading of prostate core needle biopsies from 2 independent medical laboratories in the US was performed between October 2019 and January 2020. A total of 20 general pathologists reviewed 240 prostate core needle biopsies from 240 patients. Each pathologist was randomized to 1 of 2 study cohorts. The 2 cohorts reviewed every case in the opposite modality (with AI assistance vs without AI assistance) to each other, with the modality switching after every 10 cases. After a minimum 4-week washout period for each batch, the pathologists reviewed the cases for a second time using the opposite modality. The pathologist-provided grade group for each biopsy was compared with the majority opinion of urologic pathology subspecialists.

EXPOSURE

An AI-based assistive tool for Gleason grading of prostate biopsies.

MAIN OUTCOMES AND MEASURES

Agreement between pathologists and subspecialists with and without the use of an AI-based assistive tool for the grading of all prostate biopsies and Gleason grade group 1 biopsies.

RESULTS

Biopsies from 240 patients (median age, 67 years; range, 39-91 years) with a median prostate-specific antigen level of 6.5 ng/mL (range, 0.6-97.0 ng/mL) were included in the analyses. Artificial intelligence-assisted review by pathologists was associated with a 5.6% increase (95% CI, 3.2%-7.9%; P < .001) in agreement with subspecialists (from 69.7% for unassisted reviews to 75.3% for assisted reviews) across all biopsies and a 6.2% increase (95% CI, 2.7%-9.8%; P = .001) in agreement with subspecialists (from 72.3% for unassisted reviews to 78.5% for assisted reviews) for grade group 1 biopsies. A secondary analysis indicated that AI assistance was also associated with improvements in tumor detection, mean review time, mean self-reported confidence, and interpathologist agreement.

CONCLUSIONS AND RELEVANCE

In this study, the use of an AI-based assistive tool for the review of prostate biopsies was associated with improvements in the quality, efficiency, and consistency of cancer detection and grading.