Inter- and Intrarater Reliability and Agreement Among Danish Head and Neck Pathologists Assessing Extranodal Extension in Lymph Node Metastases from Oropharyngeal Squamous Cell Carcinomas

International Multi-Specialty Expert Physician Preoperative Identification of Extranodal Extension n Oropharyngeal Cancer Patients using Computed Tomography: Prospective Blinded Human Inter-Observer Performance Evaluation

Importance

Extranodal extension (pENE) is a critical prognostic factor in oropharyngeal cancer (OPC) that drives therapeutic disposition. Determination of pENE from radiological imaging has been associated with high inter-observer variability. However, the impact of clinician specialty on human observer performance of imaging-detected extranodal extension (iENE) remains poorly understood.

Objective

To characterize the impact of clinician specialty on the accuracy of pre-operative iENE in human papillomavirus-positive (HPV+) OPC using computed tomography (CT) images.

Design Setting and Participants

This prospective observational human performance study analyzed pre-therapy CT images from 24 HPV+ OPC patients, with duplication of 6 scans (n=30) of which 21 were pathologically confirmed pENE. Thirty-four expert observers, including 11 radiologists, 12 surgeons, and 11 radiation oncologists, independently assessed these scans for iENE and reported human-detected radiologic criteria and observer confidence.

Main Outcomes and Measures

The primary outcomes included accuracy, sensitivity, specificity, area under the receiver operating characteristic curve (AUC), and Brier score for each physician, compared to ground-truth pENE. The significance of radiographic signs for prediction of pENE were determined through logistic regression analysis. Fleiss' kappa measured interobserver agreement, and Hanley-MacNeil AUC discrimination testing.

Results

Median accuracy across all specialties was 0.57 (95%CI 0.39 to 0.73), with no specialty showing discriminate performance greater than random estimation (median AUC 0.64, 95%CI 0.44 to 0.83). Significant differences between radiologists and surgeons in Brier scores (0.33 vs. 0.26, p < 0.01), radiation oncologists and surgeons in sensitivity (0.48 vs. 0.69, p > 0.1), and radiation oncologists and radiologists/surgeons in specificity (0.89 vs. 0.56, p > 0.1). Indistinct capsular contour and nodal necrosis were significant predictors of correct pENE status among all specialties. Interobserver agreement was weak for all the radiographic criteria, regardless of specialty (κ<0.6).

Conclusions and Relevance

Multiobserver testing shows physician discrimination of HPV+OPC pENE on pre-operative CT remains non-different than blind guessing, with high interrater variability and low diagnostic accuracy, regardless of clinician specialty. While minor differences in diagnostic performance among specialties are noted, they do not significantly affect the overall poor agreement and discrimination rates observed. The findings underscore the need for further research into automated detection systems or enhanced imaging techniques to improve the accuracy and reliability of iENE assessments in clinical practice.

Extranodal extension in head and neck squamous cell carcinoma: need for accurate pretherapeutic staging to select optimum treatment and minimize toxicity

PURPOSE OF REVIEW

In 2017, the American Joint Committee on Cancer (AJCC) introduced the inclusion of extracapsular nodal extension (ENE) into the N staging of nonviral head and neck squamous cell carcinoma (HNSCC), while retaining the traditional N classification based on the number and sizes of metastatic nodes. The extent of ENE was further defined as microscopic ENE (ENEmi) and major ENE (ENEma) based on extent of disease beyond the nodal capsule (≤ or > 2 mm). This article reviews the evidence and progress made since these changes were introduced.

RECENT FINDINGS

The 'gold standard' for evaluation ENE is histopathologic examination, the current preferred primary treatment of patients with HNSCC is by radiation-based therapy ± chemotherapy or biotherapy. The current pretreatment staging is by imaging, which needs improved reliability of radiologic rENE assessment with reporting needs to consider both sensitivity and specificity (currently computed tomography images have high-specificity but low-sensitivity). Adjuvant chemotherapy is indicated for patients with ENEma to enhance disease control, whereas for patients with ENEmi, there is a need to assess the benefit of adjuvant chemotherapy. Evidence that the presence of pENE in HPV-positive oropharyngeal carcinoma is an independent prognostic factor and should be considered for inclusion in future AJCC editions has recently emerged.

SUMMARY

There remains a paucity of data on the reliability of imaging in the staging of rENE, more so the for the accurate assessment of ENEmi. Optimistic early results from use of artificial intelligence/deep learning demonstrate progress and may pave the way for better capabilities in tumor staging, treatment outcome prediction, resulting in improved survival outcomes.