Outcome-Supervised Deep Learning on Pathologic Whole Slide Images for Survival Prediction of Immunotherapy in Patients with Non-Small Cell Lung Cancer

Although programmed death-(ligand) 1 (PD-(L)1) inhibitors are marked by durable efficacy in patients with non-small cell lung cancer (NSCLC), approximately 60% of the patients still suffer from recurrence and metastasis after PD-(L)1 inhibitor treatment. To accurately predict the response to PD-(L)1 inhibitors, we presented a deep learning model using a Vision Transformer (ViT) network based on hematoxylin and eosin (H&E)-stained specimens of patients with NSCLC. Two independent cohorts of patients with NSCLC receiving PD-(L)1 inhibitors from Shandong Cancer Hospital and Institute and Shandong Provincial Hospital were enrolled for model training and external validation, respectively. Whole slide images (WSIs) of H&E-stained histologic specimens were obtained from these patients and patched into 1024 × 1024 pixels. The patch-level model was trained based on ViT to identify the predictive patches, and patch-level probability distribution was performed. Then, we trained a patient-level survival model based on the ViT-Recursive Neural Network framework and externally validated it in the Shandong Provincial Hospital cohort. A total of 291 WSIs of H&E-stained histologic specimens from 198 patients with NSCLC in Shandong Cancer Hospital and 62 WSIs from 30 patients with NSCLC in Shandong Provincial Hospital were included in the model training and validation. The model achieved an accuracy of 88.6% in the internal validation cohort and 81% in the external validation cohort. The survival model also remained a statistically independent predictor of survival from PD-(L)1 inhibitors. In conclusion, the outcome-supervised ViT-Recursive Neural Network survival model based on pathologic WSIs could be used to predict immunotherapy efficacy in patients with NSCLC.