A 2.5D convolutional neural network for HPV prediction in advanced oropharyngeal cancer

Assessing the Reporting Quality of Machine Learning Algorithms in Head and Neck Oncology

OBJECTIVE

This study aimed to assess reporting quality of machine learning (ML) algorithms in the head and neck oncology literature using the TRIPOD-AI criteria.

DATA SOURCES

A comprehensive search was conducted using PubMed, Scopus, Embase, and Cochrane Database of Systematic Reviews, incorporating search terms related to "artificial intelligence," "machine learning," "deep learning," "neural network," and various head and neck neoplasms.

REVIEW METHODS

Two independent reviewers analyzed each published study for adherence to the 65-point TRIPOD-AI criteria. Items were classified as "Yes," "No," or "NA" for each publication. The proportion of studies satisfying each TRIPOD-AI criterion was calculated. Additionally, the evidence level for each study was evaluated independently by two reviewers using the Oxford Centre for Evidence-Based Medicine (OCEBM) Levels of Evidence. Discrepancies were reconciled through discussion until consensus was reached.

RESULTS

The study highlights the need for improvements in ML algorithm reporting in head and neck oncology. This includes more comprehensive descriptions of datasets, standardization of model performance reporting, and increased sharing of ML models, data, and code with the research community. Adoption of TRIPOD-AI is necessary for achieving standardized ML research reporting in head and neck oncology.

CONCLUSION

Current reporting of ML algorithms hinders clinical application, reproducibility, and understanding of the data used for model training. To overcome these limitations and improve patient and clinician trust, ML developers should provide open access to models, code, and source data, fostering iterative progress through community critique, thus enhancing model accuracy and mitigating biases.

LEVEL OF EVIDENCE

NA Laryngoscope, 135:687-694, 2025.

Potential value of novel multiparametric MRI radiomics for preoperative prediction of microsatellite instability and Ki-67 expression in endometrial cancer

Exploring the potential of advanced artificial intelligence technology in predicting microsatellite instability (MSI) and Ki-67 expression of endometrial cancer (EC) is highly significant. This study aimed to develop a novel hybrid radiomics approach integrating multiparametric magnetic resonance imaging (MRI), deep learning, and multichannel image analysis for predicting MSI and Ki-67 status. A retrospective study included 156 EC patients who were subsequently categorized into MSI and Ki-67 groups. The hybrid radiomics model (HMRadSum) was developed by extracting quantitative imaging features and deep learning features from multiparametric MRI using emerging attention mechanism. Tumor markers were subsequently predicted utilizing an XGBoost classifier. Model performance and interpretability were evaluated using standard classification metrics, Gradient-weighted Class Activation Mapping (Grad-CAM), and SHapley Additive exPlanations (SHAP) techniques. For the MSI prediction task, the HMRadSum model achieved area-under-curve (AUC) value of 0.945 (95% CI 0.862-1.000) and accuracy of 0.889. For the Ki-67 prediction task, the AUC and accuracy of HMRadSum model was 0.888 (95% CI 0.743-1.000) and 0.810. This hybrid radiomics model effectively extracted features associated with EC gene expression, providing potential clinical implications for personalized diagnosis, treatment, and treatment strategy optimization.

Construction of a 2.5D Deep Learning Model for Predicting Early Postoperative Recurrence of Hepatocellular Carcinoma Using Multi-View and Multi-Phase CT Images

Purpose

To construct a 2.5-dimensional (2.5D) CT radiomics-based deep learning (DL) model to predict early postoperative recurrence of hepatocellular carcinoma (HCC).

Patients and Methods

We retrospectively analyzed the data of patients who underwent HCC resection at 2 centers. The 232 patients from center 1 were randomly divided into the training (162 patients) and internal validation cohorts (70 patients); 91 patients from center 2 formed the external validation cohort. We developed a 2.5D DL model based on a central 2D image with the maximum tumor cross-section and adjacent slices. Multiple views (transverse, sagittal, and coronal) and phases (arterial, plain, and portal) were incorporated. Multi-instance learning techniques were applied to the extracted data; the resulting comprehensive feature set was modeled using Logistic Regression, RandomForest, ExtraTrees, XGBoost, and LightGBM, with 5-fold cross validation and hyperparameter optimization with Grid-search. Receiver operating characteristic curves, calibration curves, DeLong test, and decision curve analysis were used to evaluate model performance.

Results

The 2.5D DL model performed well in the training (AUC: 0.920), internal validation (AUC: 0.825), and external validation cohorts (AUC: 0.795). The 3D DL model performed well in the training cohort and poorly in the internal and external validation cohorts (AUCs: 0.751, 0.666, and 0.567, respectively), indicating overfitting. The combined model (2.5D DL+clinical) performed well in all cohorts (AUCs: 0.921, 0.835, 0.804). The Hosmer-Lemeshow test, DeLong test, and decision curve analysis confirmed the superiority of the combined model over the other signatures.

Conclusion

The combined model integrating 2.5D DL and clinical features accurately predicts early postoperative HCC recurrence.

Genome composition-based deep learning predicts oncogenic potential of HPVs

Human papillomaviruses (HPVs) account for more than 30% of cancer cases, with definite identification of the oncogenic role of viral E6 and E7 genes. However, the identification of high-risk HPV genotypes has largely relied on lagged biological exploration and clinical observation, with types unclassified and oncogenicity unknown for many HPVs. In the present study, we retrieved and cleaned HPV sequence records with high quality and analyzed their genomic compositional traits of dinucleotide (DNT) and DNT representation (DCR) to overview the distribution difference among various types of HPVs. Then, a deep learning model was built to predict the oncogenic potential of all HPVs based on E6 and E7 genes. Our results showed that the main three groups of Alpha, Beta, and Gamma HPVs were clearly separated between/among types in the DCR trait for either E6 or E7 coding sequence (CDS) and were clustered within the same group. Moreover, the DCR data of either E6 or E7 were learnable with a convolutional neural network (CNN) model. Either CNN classifier predicted accurately the oncogenicity label of high and low oncogenic HPVs. In summary, the compositional traits of HPV oncogenicity-related genes E6 and E7 were much different between the high and low oncogenic HPVs, and the compositional trait of the DCR-based deep learning classifier predicted the oncogenic phenotype accurately of HPVs. The trained predictor in this study will facilitate the identification of HPV oncogenicity, particularly for those HPVs without clear genotype or phenotype.

Explainable prediction model for the human papillomavirus status in patients with oropharyngeal squamous cell carcinoma using CNN on CT images

Several studies have emphasised how positive and negative human papillomavirus (HPV+  and HPV-, respectively) oropharyngeal squamous cell carcinoma (OPSCC) has distinct molecular profiles, tumor characteristics, and disease outcomes. Different radiomics-based prediction models have been proposed, by also using innovative techniques such as Convolutional Neural Networks (CNNs). Although some of these models reached encouraging predictive performances, there evidence explaining the role of radiomic features in achieving a specific outcome is scarce. In this paper, we propose some preliminary results related to an explainable CNN-based model to predict HPV status in OPSCC patients. We extracted the Gross Tumor Volume (GTV) of pre-treatment CT images related to 499 patients (356 HPV+ and 143 HPV-) included into the OPC-Radiomics public dataset to train an end-to-end Inception-V3 CNN architecture. We also collected a multicentric dataset consisting of 92 patients (43 HPV+ , 49 HPV-), which was employed as an independent test set. Finally, we applied Gradient-weighted Class Activation Mapping (Grad-CAM) technique to highlight the most informative areas with respect to the predicted outcome. The proposed model reached an AUC value of 73.50% on the independent test. As a result of the Grad-CAM algorithm, the most informative areas related to the correctly classified HPV+ patients were located into the intratumoral area. Conversely, the most important areas referred to the tumor edges. Finally, since the proposed model provided additional information with respect to the accuracy of the classification given by the visualization of the areas of greatest interest for predictive purposes for each case examined, it could contribute to increase confidence in using computer-based predictive models in the actual clinical practice.

Automatic detection of epilepsy from EEGs using a temporal convolutional network with a self-attention layer

BACKGROUND

Over 60% of epilepsy patients globally are children, whose early diagnosis and treatment are critical for their development and can substantially reduce the disease's burden on both families and society. Numerous algorithms for automated epilepsy detection from EEGs have been proposed. Yet, the occurrence of epileptic seizures during an EEG exam cannot always be guaranteed in clinical practice. Models that exclusively use seizure EEGs for detection risk artificially enhanced performance metrics. Therefore, there is a pressing need for a universally applicable model that can perform automatic epilepsy detection in a variety of complex real-world scenarios.

METHOD

To address this problem, we have devised a novel technique employing a temporal convolutional neural network with self-attention (TCN-SA). Our model comprises two primary components: a TCN for extracting time-variant features from EEG signals, followed by a self-attention (SA) layer that assigns importance to these features. By focusing on key features, our model achieves heightened classification accuracy for epilepsy detection.

RESULTS

The efficacy of our model was validated on a pediatric epilepsy dataset we collected and on the Bonn dataset, attaining accuracies of 95.50% on our dataset, and 97.37% (A v. E), and 93.50% (B vs E), respectively. When compared with other deep learning architectures (temporal convolutional neural network, self-attention network, and standardized convolutional neural network) using the same datasets, our TCN-SA model demonstrated superior performance in the automated detection of epilepsy.

CONCLUSION

The proven effectiveness of the TCN-SA approach substantiates its potential as a valuable tool for the automated detection of epilepsy, offering significant benefits in diverse and complex real-world clinical settings.

Self-reporting with checklists in artificial intelligence research on medical imaging: a systematic review based on citations of CLAIM

OBJECTIVE

To evaluate the usage of a well-known and widely adopted checklist, Checklist for Artificial Intelligence in Medical imaging (CLAIM), for self-reporting through a systematic analysis of its citations.

METHODS

Google Scholar, Web of Science, and Scopus were used to search for citations (date, 29 April 2023). CLAIM's use for self-reporting with proof (i.e., filled-out checklist) and other potential use cases were systematically assessed in research papers. Eligible papers were evaluated independently by two readers, with the help of automatic annotation. Item-by-item confirmation analysis on papers with checklist proof was subsequently performed.

RESULTS

A total of 391 unique citations were identified from three databases. Of the 118 papers included in this study, 12 (10%) provided a proof of self-reported CLAIM checklist. More than half (70; 59%) only mentioned some sort of adherence to CLAIM without providing any proof in the form of a checklist. Approximately one-third (36; 31%) cited the CLAIM for reasons unrelated to their reporting or methodological adherence. Overall, the claims on 57 to 93% of the items per publication were confirmed in the item-by-item analysis, with a mean and standard deviation of 81% and 10%, respectively.

CONCLUSION

Only a small proportion of the publications used CLAIM as checklist and supplied filled-out documentation; however, the self-reported checklists may contain errors and should be approached cautiously. We hope that this systematic citation analysis would motivate artificial intelligence community about the importance of proper self-reporting, and encourage researchers, journals, editors, and reviewers to take action to ensure the proper usage of checklists.

CLINICAL RELEVANCE STATEMENT

Only a small percentage of the publications used CLAIM for self-reporting with proof (i.e., filled-out checklist). However, the filled-out checklist proofs may contain errors, e.g., false claims of adherence, and should be approached cautiously. These may indicate inappropriate usage of checklists and necessitate further action by authorities.

KEY POINTS

• Of 118 eligible papers, only 12 (10%) followed the CLAIM checklist for self-reporting with proof (i.e., filled-out checklist). More than half (70; 59%) only mentioned some kind of adherence without providing any proof. • Overall, claims on 57 to 93% of the items were valid in item-by-item confirmation analysis, with a mean and standard deviation of 81% and 10%, respectively. • Even with the checklist proof, the items declared may contain errors and should be approached cautiously.

Multi-Modal Ensemble Deep Learning in Head and Neck Cancer HPV Sub-Typing

Oropharyngeal Squamous Cell Carcinoma (OPSCC) is one of the common forms of heterogeneity in head and neck cancer. Infection with human papillomavirus (HPV) has been identified as a major risk factor for OPSCC. Therefore, differentiating the HPV-positive and negative cases in OPSCC patients is an essential diagnostic factor influencing future treatment decisions. In this study, we investigated the accuracy of a deep learning-based method for image interpretation and automatically detected the HPV status of OPSCC in routinely acquired Computed Tomography (CT) and Positron Emission Tomography (PET) images. We introduce a 3D CNN-based multi-modal feature fusion architecture for HPV status prediction in primary tumor lesions. The architecture is composed of an ensemble of CNN networks and merges image features in a softmax classification layer. The pipeline separately learns the intensity, contrast variation, shape, texture heterogeneity, and metabolic assessment from CT and PET tumor volume regions and fuses those multi-modal features for final HPV status classification. The precision, recall, and AUC scores of the proposed method are computed, and the results are compared with other existing models. The experimental results demonstrate that the multi-modal ensemble model with soft voting outperformed single-modality PET/CT, with an AUC of 0.76 and F1 score of 0.746 on publicly available TCGA and MAASTRO datasets. In the MAASTRO dataset, our model achieved an AUC score of 0.74 over primary tumor volumes of interest (VOIs). In the future, more extensive cohort validation may suffice for better diagnostic accuracy and provide preliminary assessment before the biopsy.

Development and external validation of the multichannel deep learning model based on unenhanced CT for differentiating fat-poor angiomyolipoma from renal cell carcinoma: a two-center retrospective study

PURPOSE

There are undetectable levels of fat in fat-poor angiomyolipoma. Thus, it is often misdiagnosed as renal cell carcinoma. We aimed to develop and evaluate a multichannel deep learning model for differentiating fat-poor angiomyolipoma (fp-AML) from renal cell carcinoma (RCC).

METHODS

This two-center retrospective study included 320 patients from the First Affiliated Hospital of Sun Yat-Sen University (FAHSYSU) and 132 patients from the Sun Yat-Sen University Cancer Center (SYSUCC). Data from patients at FAHSYSU were divided into a development dataset (n = 267) and a hold-out dataset (n = 53). The development dataset was used to obtain the optimal combination of CT modality and input channel. The hold-out dataset and SYSUCC dataset were used for independent internal and external validation, respectively.

RESULTS

In the development phase, models trained on unenhanced CT images performed significantly better than those trained on enhanced CT images based on the fivefold cross-validation. The best patient-level performance, with an average area under the receiver operating characteristic curve (AUC) of 0.951 ± 0.026 (mean ± SD), was achieved using the "unenhanced CT and 7-channel" model, which was finally selected as the optimal model. In the independent internal and external validation, AUCs of 0.966 (95% CI 0.919-1.000) and 0.898 (95% CI 0.824-0.972), respectively, were obtained using the optimal model. In addition, the performance of this model was better on large tumors (≥ 40 mm) in both internal and external validation.

CONCLUSION

The promising results suggest that our multichannel deep learning classifier based on unenhanced whole-tumor CT images is a highly useful tool for differentiating fp-AML from RCC.

Artificial intelligence for HPV status prediction based on disease-specific images in head and neck cancer: A systematic review and meta-analysis

Accurate early detection of the human papillomavirus (HPV) status in head and neck cancer (HNC) is crucial to identify at-risk populations, stratify patients, personalized treatment options, and predict prognosis. Artificial intelligence (AI) is an emerging tool to dissect imaging features. This systematic review and meta-analysis aimed to evaluate the performance of AI to predict the HPV positivity through the HPV-associated diseased images in HNC patients. A systematic literature search was conducted in databases including Ovid-MEDLINE, Embase, and Web of Science Core Collection for studies continuously published from inception up to October 30, 2022. Search strategies included keywords such as "artificial intelligence," "head and neck cancer," "HPV," and "sensitivity & specificity." Duplicates, articles without HPV predictions, letters, scientific reports, conference abstracts, or reviews were excluded. Binary diagnostic data were then extracted to generate contingency tables and then used to calculate the pooled sensitivity (SE), specificity (SP), area under the curve (AUC), and their 95% confidence interval (CI). A random-effects model was used for meta-analysis, four subgroup analyses were further explored. Totally, 22 original studies were included in the systematic review, 15 of which were eligible to generate 33 contingency tables for meta-analysis. The pooled SE and SP for all studies were 79% (95% CI: 75-82%) and 74% (95% CI: 69-78%) respectively, with an AUC of 0.83 (95% CI: 0.79-0.86). When only selecting one contingency table with the highest accuracy from each study, our analysis revealed a pooled SE of 79% (95% CI: 75-83%), SP of 75% (95% CI: 69-79%), and an AUC of 0.84 (95% CI: 0.81-0.87). The respective heterogeneities were moderate (I2 for SE and SP were 51.70% and 51.01%) and only low (35.99% and 21.44%). This evidence-based study showed an acceptable and promising performance for AI algorithms to predict HPV status in HNC but was not comparable to the routine p16 immunohistochemistry. The exploitation and optimization of AI algorithms warrant further research. Compared with previous studies, future studies anticipate to make progress in the selection of databases, improvement of international reporting guidelines, and application of high-quality deep learning algorithms.

A comprehensive review for machine learning based human papillomavirus detection in forensic identification with multiple medical samples

Human papillomavirus (HPV) is a sexually transmitted virus. Cervical cancer is one of the highest incidences of cancer, almost all patients are accompanied by HPV infection. In addition, the occurrence of a variety of cancers is also associated with HPV infection. HPV vaccination has gained widespread popularity in recent years with the increase in public health awareness. In this context, HPV testing not only needs to be sensitive and specific but also needs to trace the source of HPV infection. Through machine learning and deep learning, information from medical examinations can be used more effectively. In this review, we discuss recent advances in HPV testing in combination with machine learning and deep learning.

Application of artificial intelligence for overall survival risk stratification in oropharyngeal carcinoma: A validation of ProgTOOL

BACKGROUND

In recent years, there has been a surge in machine learning-based models for diagnosis and prognostication of outcomes in oncology. However, there are concerns relating to the model's reproducibility and generalizability to a separate patient cohort (i.e., external validation).

OBJECTIVES

This study primarily provides a validation study for a recently introduced and publicly available machine learning (ML) web-based prognostic tool (ProgTOOL) for overall survival risk stratification of oropharyngeal squamous cell carcinoma (OPSCC). Additionally, we reviewed the published studies that have utilized ML for outcome prognostication in OPSCC to examine how many of these models were externally validated, type of external validation, characteristics of the external dataset, and diagnostic performance characteristics on the internal validation (IV) and external validation (EV) datasets were extracted and compared.

METHODS

We used a total of 163 OPSCC patients obtained from the Helsinki University Hospital to externally validate the ProgTOOL for generalizability. In addition, PubMed, OvidMedline, Scopus, and Web of Science databases were systematically searched according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.

RESULTS

The ProgTOOL produced a predictive performance of 86.5% balanced accuracy, Mathew's correlation coefficient of 0.78, Net Benefit (0.7) and Brier score (0.06) for overall survival stratification of OPSCC patients as either low-chance or high-chance. In addition, out of a total of 31 studies found to have used ML for the prognostication of outcomes in OPSCC, only seven (22.6%) reported a form of EV. Three studies (42.9%) each used either temporal EV or geographical EV while only one study (14.2%) used expert as a form of EV. Most of the studies reported a reduction in performance when externally validated.

CONCLUSION

The performance of the model in this validation study indicates that it may be generalized, therefore, bringing recommendations of the model for clinical evaluation closer to reality. However, the number of externally validated ML-based models for OPSCC is still relatively small. This significantly limits the transfer of these models for clinical evaluation and subsequently reduces the likelihood of the use of these models in daily clinical practice. As a gold standard, we recommend the use of geographical EV and validation studies to reveal biases and overfitting of these models. These recommendations are poised to facilitate the implementation of these models in clinical practice.