1
|
Moharrami M, Azimian Zavareh P, Watson E, Singhal S, Johnson AEW, Hosni A, Quinonez C, Glogauer M. Prognosing post-treatment outcomes of head and neck cancer using structured data and machine learning: A systematic review. PLoS One 2024; 19:e0307531. [PMID: 39046953 PMCID: PMC11268644 DOI: 10.1371/journal.pone.0307531] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2024] [Accepted: 07/07/2024] [Indexed: 07/27/2024] Open
Abstract
BACKGROUND This systematic review aimed to evaluate the performance of machine learning (ML) models in predicting post-treatment survival and disease progression outcomes, including recurrence and metastasis, in head and neck cancer (HNC) using clinicopathological structured data. METHODS A systematic search was conducted across the Medline, Scopus, Embase, Web of Science, and Google Scholar databases. The methodological characteristics and performance metrics of studies that developed and validated ML models were assessed. The risk of bias was evaluated using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). RESULTS Out of 5,560 unique records, 34 articles were included. For survival outcome, the ML model outperformed the Cox proportional hazards model in time-to-event analyses for HNC, with a concordance index of 0.70-0.79 vs. 0.66-0.76, and for all sub-sites including oral cavity (0.73-0.89 vs. 0.69-0.77) and larynx (0.71-0.85 vs. 0.57-0.74). In binary classification analysis, the area under the receiver operating characteristics (AUROC) of ML models ranged from 0.75-0.97, with an F1-score of 0.65-0.89 for HNC; AUROC of 0.61-0.91 and F1-score of 0.58-0.86 for the oral cavity; and AUROC of 0.76-0.97 and F1-score of 0.63-0.92 for the larynx. Disease-specific survival outcomes showed higher performance than overall survival outcomes, but the performance of ML models did not differ between three- and five-year follow-up durations. For disease progression outcomes, no time-to-event metrics were reported for ML models. For binary classification of the oral cavity, the only evaluated subsite, the AUROC ranged from 0.67 to 0.97, with F1-scores between 0.53 and 0.89. CONCLUSIONS ML models have demonstrated considerable potential in predicting post-treatment survival and disease progression, consistently outperforming traditional linear models and their derived nomograms. Future research should incorporate more comprehensive treatment features, emphasize disease progression outcomes, and establish model generalizability through external validations and the use of multicenter datasets.
Collapse
Affiliation(s)
- Mohammad Moharrami
- Faculty of Dentistry, University of Toronto, Toronto, Canada
- Department of Dental Oncology, Princess Margaret Cancer Centre, Toronto, Canada
- Topic Group Dental Diagnostics and Digital Dentistry, ITU/WHO Focus Group AI on Health, Geneva, Switzerland
| | - Parnia Azimian Zavareh
- Topic Group Dental Diagnostics and Digital Dentistry, ITU/WHO Focus Group AI on Health, Geneva, Switzerland
| | - Erin Watson
- Faculty of Dentistry, University of Toronto, Toronto, Canada
- Department of Dental Oncology, Princess Margaret Cancer Centre, Toronto, Canada
| | - Sonica Singhal
- Faculty of Dentistry, University of Toronto, Toronto, Canada
- Chronic Disease and Injury Prevention Department, Health Promotion, Public Health Ontario, Toronto, Canada
| | - Alistair E. W. Johnson
- Program in Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, Canada
| | - Ali Hosni
- Radiation Oncology, Princess Margaret Cancer Center, University of Toronto, Toronto, Canada
| | - Carlos Quinonez
- Faculty of Dentistry, University of Toronto, Toronto, Canada
- Schulich School of Medicine & Dentistry, Western University, London, Canada
| | - Michael Glogauer
- Faculty of Dentistry, University of Toronto, Toronto, Canada
- Department of Dental Oncology, Princess Margaret Cancer Centre, Toronto, Canada
- Department of Dentistry, Centre for Advanced Dental Research and Care, Mount Sinai Hospital, Toronto, Canada
| |
Collapse
|
2
|
Alabi RO, Almangush A, Elmusrati M, Leivo I, Mäkitie AA. Interpretable machine learning model for prediction of overall survival in laryngeal cancer. Acta Otolaryngol 2024:1-7. [PMID: 38279817 DOI: 10.1080/00016489.2023.2301648] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2023] [Accepted: 12/21/2023] [Indexed: 01/29/2024]
Abstract
Background: The mortality rates of laryngeal squamous cell carcinoma cancer (LSCC) have not significantly decreased in the last decades.Objectives: We primarily aimed to compare the predictive performance of DeepTables with the state-of-the-art machine learning (ML) algorithms (Voting ensemble, Stack ensemble, and XGBoost) to stratify patients with LSCC into chance of overall survival (OS). In addition, we complemented the developed model by providing interpretability using both global and local model-agnostic techniques.Methods: A total of 2792 patients in the Surveillance, Epidemiology, and End Results (SEER) database diagnosed with LSCC were reviewed. The global model-agnostic interpretability was examined using SHapley Additive exPlanations (SHAP) technique. Likewise, individual interpretation of the prediction was made using Local Interpretable Model Agnostic Explanations (LIME).Results: The state-of-the-art ML ensemble algorithms outperformed DeepTables. Specifically, the examined ensemble algorithms showed comparable weighted area under receiving curve of 76.9, 76.8, and 76.1 with an accuracy of 71.2%, 70.2%, and 71.8%, respectively. The global methods of interpretability (SHAP) demonstrated that the age of the patient at diagnosis, N-stage, T-stage, tumor grade, and marital status are among the prominent parameters.Conclusions: A ML model for OS prediction may serve as an ancillary tool for treatment planning of LSCC patients.
Collapse
Affiliation(s)
- Rasheed Omobolaji Alabi
- Research Program in Systems Oncology, University of Helsinki, Helsinki, Finland
- Department of Industrial Digitalization, School of Technology and Innovations, University of Vaasa, Vaasa, Finland
| | - Alhadi Almangush
- Research Program in Systems Oncology, University of Helsinki, Helsinki, Finland
- Department of Pathology, University of Helsinki, Helsinki, Finland
- Institute of Biomedicine, University of Turku, Pathology, Finland
| | - Mohammed Elmusrati
- Department of Industrial Digitalization, School of Technology and Innovations, University of Vaasa, Vaasa, Finland
| | - Ilmo Leivo
- Institute of Biomedicine, University of Turku, Pathology, Finland
| | - Antti A Mäkitie
- Research Program in Systems Oncology, University of Helsinki, Helsinki, Finland
- Department of Otorhinolaryngology - Head and Neck Surgery, University of Helsinki and Helsinki University Hospital, Helsinki, Finland
- Division of Ear, Nose and Throat Diseases, Department of Clinical Sciences, Intervention and Technology, Karolinska Institute and Karolinska University Hospital, Stockholm, Sweden
| |
Collapse
|
3
|
Li CX, Wang ZY, Tong QY, Li MQ, Wei W, Gong ZC. Effect of Prognostic Factors of Postoperative Radiotherapy in Oral Squamous Cell Carcinoma: A SEER-Based Study. EAR, NOSE & THROAT JOURNAL 2023:1455613231210388. [PMID: 37994533 DOI: 10.1177/01455613231210388] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2023] Open
Abstract
Objective: The treatment of oral squamous cell carcinoma (OSCC) is dominated by surgery and radiochemotherapy, but its prognosis is still unsatisfactory, with around five tenths of 5-year survival. This study aimed to assess the prognosis of OSCC patients treated with surgery with and without postoperative radiotherapy. Study Design: Retrospective study. Methods: The clinicopathological information and follow-up datasets on patients with OSCC (T1-4 and/or N+) registered from 2010 to 2015 were downloaded from the Surveillance, Epidemiology, and End Results database. Totally 7231 enrolled subjects were divided into a case group (surgery alone, n = 4167) and a control group (surgery combined with postoperative radiotherapy, n = 3064). One-to-one matching was performed by propensity score matching to make the baseline data comparable between the 2 subgroups. Multivariate Cox regression analysis was used to calculate hazard ratios (HR) of various clinicopathological features. The Kaplan-Meier method and log-rank test were used to plot the survival curves. Results: The majority of patients in case group were tumor stage I (n = 2569, 61.7%), whereas most patients in control group were stages III to IV (n = 2360, 77.1%). In the case group, the 1-, 3-, and 5-year overall survival (OS; 76%, 59.5%, 53.7%) were significantly lower than those of the control group (85.1%, 64.1%, 55.8%; P < .0001). Similarly, the 1-, 3-, and 5-year cancer-specific survival (CSS) of the case group (80.2%, 66.6%, 63.3%) were significantly lower than those of the control group (87.2%, 69.3%, 63.9%, respectively; P < .0001). Cox multivariate analysis indicated that age, differentiation, clinical stage, and tumor-node-metastasis stage affected the prognosis of OSCC patients, while postoperative radiotherapy was a protective factor (OS: HR = 0.649, P < .001; CSS: HR = 0.702, P < .001). Conclusions: Postoperative radiation was an independent protective factor, hence, the combination of surgery plus radiotherapy is more beneficial for the survival of patients with OSCC, particularly for advanced cases.
Collapse
Affiliation(s)
- Chen-Xi Li
- Department of Oral and Maxillofacial Oncology & Surgery, School / Hospital of Stomatology, the First Affiliated Hospital of Xinjiang Medical University, Urumqi, China
- Stomatological Research Institute of Xinjiang Uygur Autonomous Region, Urumqi, China
- Hubei Province Key Laboratory of Oral and Maxillofacial Development and Regeneration, School of Stomatology, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| | - Zheng-Ye Wang
- Department of Preventive Medicine, College of Public Health, Xinjiang Medical University, Urumqi, China
| | - Qiao-Ying Tong
- Department of Oral and Maxillofacial Oncology & Surgery, School / Hospital of Stomatology, the First Affiliated Hospital of Xinjiang Medical University, Urumqi, China
| | - Mu-Qiu Li
- Department of Oral and Maxillofacial Oncology & Surgery, School / Hospital of Stomatology, the First Affiliated Hospital of Xinjiang Medical University, Urumqi, China
- Stomatological Research Institute of Xinjiang Uygur Autonomous Region, Urumqi, China
| | - Wei Wei
- Department of Oral and Maxillofacial Oncology & Surgery, School / Hospital of Stomatology, the First Affiliated Hospital of Xinjiang Medical University, Urumqi, China
- Stomatological Research Institute of Xinjiang Uygur Autonomous Region, Urumqi, China
| | - Zhong-Cheng Gong
- Department of Oral and Maxillofacial Oncology & Surgery, School / Hospital of Stomatology, the First Affiliated Hospital of Xinjiang Medical University, Urumqi, China
- Stomatological Research Institute of Xinjiang Uygur Autonomous Region, Urumqi, China
| |
Collapse
|
4
|
Song HJ, Park YJ, Jeong HY, Kim BG, Kim JH, Im YG. Detection of Abnormal Changes on the Dorsal Tongue Surface Using Deep Learning. MEDICINA (KAUNAS, LITHUANIA) 2023; 59:1293. [PMID: 37512104 PMCID: PMC10385577 DOI: 10.3390/medicina59071293] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/02/2023] [Revised: 07/09/2023] [Accepted: 07/10/2023] [Indexed: 07/30/2023]
Abstract
Background and Objective: The tongue mucosa often changes due to various local and systemic diseases or conditions. This study aimed to investigate whether deep learning can help detect abnormal regions on the dorsal tongue surface in patients and healthy adults. Materials and Methods: The study collected 175 clinical photographic images of the dorsal tongue surface, which were divided into 7782 cropped images classified into normal, abnormal, and non-tongue regions and trained using the VGG16 deep learning model. The 80 photographic images of the entire dorsal tongue surface were used for the segmentation of abnormal regions using point mapping segmentation. Results: The F1-scores of the abnormal and normal classes were 0.960 (precision: 0.935, recall: 0.986) and 0.968 (precision: 0.987, recall: 0.950), respectively, in the prediction of the VGG16 model. As a result of evaluation using point mapping segmentation, the average F1-scores were 0.727 (precision: 0.717, recall: 0.737) and 0.645 (precision: 0.650, recall: 0.641), the average intersection of union was 0.695 and 0.590, and the average precision was 0.940 and 0.890, respectively, for abnormal and normal classes. Conclusions: The deep learning algorithm used in this study can accurately determine abnormal areas on the dorsal tongue surface, which can assist in diagnosing specific diseases or conditions of the tongue mucosa.
Collapse
Affiliation(s)
- Ho-Jun Song
- Department of Dental Materials, Dental Science Research Institute, School of Dentistry, Chonnam National University, Gwangju 61186, Republic of Korea
| | - Yeong-Joon Park
- Department of Dental Materials, Dental Science Research Institute, School of Dentistry, Chonnam National University, Gwangju 61186, Republic of Korea
| | - Hie-Yong Jeong
- Department of Artificial Intelligence Convergence, Chonnam National University, Gwangju 61186, Republic of Korea
| | - Byung-Gook Kim
- Department of Oral Medicine, Dental Science Research Institute, School of Dentistry, Chonnam National University, Gwangju 61186, Republic of Korea
| | - Jae-Hyung Kim
- Department of Oral Medicine, Dental Science Research Institute, School of Dentistry, Chonnam National University, Gwangju 61186, Republic of Korea
| | - Yeong-Gwan Im
- Department of Oral Medicine, Dental Science Research Institute, School of Dentistry, Chonnam National University, Gwangju 61186, Republic of Korea
| |
Collapse
|
5
|
Alabi RO, Elmusrati M, Leivo I, Almangush A, Mäkitie AA. Machine learning explainability in nasopharyngeal cancer survival using LIME and SHAP. Sci Rep 2023; 13:8984. [PMID: 37268685 DOI: 10.1038/s41598-023-35795-0] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2022] [Accepted: 05/24/2023] [Indexed: 06/04/2023] Open
Abstract
Nasopharyngeal cancer (NPC) has a unique histopathology compared with other head and neck cancers. Individual NPC patients may attain different outcomes. This study aims to build a prognostic system by combining a highly accurate machine learning model (ML) model with explainable artificial intelligence to stratify NPC patients into low and high chance of survival groups. Explainability is provided using Local Interpretable Model Agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP) techniques. A total of 1094 NPC patients were retrieved from the Surveillance, Epidemiology, and End Results (SEER) database for model training and internal validation. We combined five different ML algorithms to form a uniquely stacked algorithm. The predictive performance of the stacked algorithm was compared with a state-of-the-art algorithm-extreme gradient boosting (XGBoost) to stratify the NPC patients into chance of survival groups. We validated our model with temporal validation (n = 547) and geographic external validation (Helsinki University Hospital NPC cohort, n = 60). The developed stacked predictive ML model showed an accuracy of 85.9% while the XGBoost had 84.5% after the training and testing phases. This demonstrated that both XGBoost and the stacked model showed comparable performance. External geographic validation of XGBoost model showed a c-index of 0.74, accuracy of 76.7%, and area under curve of 0.76. The SHAP technique revealed that age of the patient at diagnosis, T-stage, ethnicity, M-stage, marital status, and grade were among the prominent input variables in decreasing order of significance for the overall survival of NPC patients. LIME showed the degree of reliability of the prediction made by the model. In addition, both techniques showed how each feature contributed to the prediction made by the model. LIME and SHAP techniques provided personalized protective and risk factors for each NPC patient and unraveled some novel non-linear relationships between input features and survival chance. The examined ML approach showed the ability to predict the chance of overall survival of NPC patients. This is important for effective treatment planning care and informed clinical decisions. To enhance outcome results, including survival in NPC, ML may aid in planning individualized therapy for this patient population.
Collapse
Affiliation(s)
- Rasheed Omobolaji Alabi
- Research Program in Systems Oncology, Faculty of Medicine, University of Helsinki, Helsinki, Finland.
- Department of Industrial Digitalization, School of Technology and Innovations, University of Vaasa, Vaasa, Finland.
| | - Mohammed Elmusrati
- Department of Industrial Digitalization, School of Technology and Innovations, University of Vaasa, Vaasa, Finland
| | - Ilmo Leivo
- Institute of Biomedicine, Pathology, University of Turku, Turku, Finland
| | - Alhadi Almangush
- Research Program in Systems Oncology, Faculty of Medicine, University of Helsinki, Helsinki, Finland
- Department of Pathology, University of Helsinki, Helsinki, Finland
- Faculty of Dentistry, Misurata University, Misurata, Libya
| | - Antti A Mäkitie
- Research Program in Systems Oncology, Faculty of Medicine, University of Helsinki, Helsinki, Finland
- Department of Otorhinolaryngology-Head and Neck Surgery, University of Helsinki and Helsinki University Hospital, Helsinki, Finland
- Division of Ear, Nose and Throat Diseases, Department of Clinical Sciences, Intervention and Technology, Karolinska Institute and Karolinska University Hospital, Stockholm, Sweden
| |
Collapse
|