Multimodal deep learning to predict prognosis in adult and pediatric brain tumors

A review of deep learning for brain tumor analysis in MRI

Recent progress in deep learning (DL) is producing a new generation of tools across numerous clinical applications. Within the analysis of brain tumors in magnetic resonance imaging, DL finds applications in tumor segmentation, quantification, and classification. It facilitates objective and reproducible measurements crucial for diagnosis, treatment planning, and disease monitoring. Furthermore, it holds the potential to pave the way for personalized medicine through the prediction of tumor type, grade, genetic mutations, and patient survival outcomes. In this review, we explore the transformative potential of DL for brain tumor care and discuss existing applications, limitations, and future directions and opportunities.

Artificial intelligence and omics in malignant gliomas

Glioblastoma multiforme (GBM) is one of the most common and aggressive type of malignant glioma with an average survival time of 12-18 mo. Despite the utilization of extensive surgical resections using cutting-edge neuroimaging, and advanced chemotherapy and radiotherapy, the prognosis remains unfavorable. The heterogeneity of GBM and the presence of the blood-brain barrier further complicate the therapeutic process. It is crucial to adopt a multifaceted approach in GBM research to understand its biology and advance toward effective treatments. In particular, omics research, which primarily includes genomics, transcriptomics, proteomics, and epigenomics, helps us understand how GBM develops, finds biomarkers, and discovers new therapeutic targets. The availability of large-scale multiomics data requires the development of computational models to infer valuable biological insights for the implementation of precision medicine. Artificial intelligence (AI) refers to a host of computational algorithms that is becoming a major tool capable of integrating large omics databases. Although the application of AI tools in GBM-omics is currently in its early stages, a thorough exploration of AI utilization to uncover different aspects of GBM (subtype classification, prognosis, and survival) would have a significant impact on both researchers and clinicians. Here, we aim to review and provide database resources of different AI-based techniques that have been used to study GBM pathogenesis using multiomics data over the past decade. We summarize different types of GBM-related omics resources that can be used to develop AI models. Furthermore, we explore various AI tools that have been developed using either individual or integrated multiomics data, highlighting their applications and limitations in the context of advancing GBM research and treatment.

Leveraging State-of-the-Art AI Algorithms in Personalized Oncology: From Transcriptomics to Treatment

BACKGROUND

Continuous breakthroughs in computational algorithms have positioned AI-based models as some of the most sophisticated technologies in the healthcare system. AI shows dynamic contributions in advancing various medical fields involving data interpretation and monitoring, imaging screening and diagnosis, and treatment response and survival prediction. Despite advances in clinical oncology, more effort must be employed to tailor therapeutic plans based on each patient's unique transcriptomic profile within the precision/personalized oncology frame. Furthermore, the standard analysis method is not compatible with the comprehensive deciphering of significant data streams, thus precluding the prediction of accurate treatment options.

METHODOLOGY

We proposed a novel approach that includes obtaining different tumour tissues and preparing RNA samples for comprehensive transcriptomic interpretation using specifically trained, programmed, and optimized AI-based models for extracting large data volumes, refining, and analyzing them. Next, the transcriptomic results will be scanned against an expansive drug library to predict the response of each target to the tested drugs. The obtained target-drug combination/s will be then validated using in vitro and in vivo experimental models. Finally, the best treatment combination option/s will be introduced to the patient. We also provided a comprehensive review discussing AI models' recent innovations and implementations to aid in molecular diagnosis and treatment planning.

RESULTS

The expected transcriptomic analysis generated by the AI-based algorithms will provide an inclusive genomic profile for each patient, containing statistical and bioinformatics analyses, identification of the dysregulated pathways, detection of the targeted genes, and recognition of molecular biomarkers. Subjecting these results to the prediction and pairing AI-based processes will result in statistical graphs presenting each target's likely response rate to various treatment options. Different in vitro and in vivo investigations will further validate the selection of the target drug/s pairs.

CONCLUSIONS

Leveraging AI models will provide more rigorous manipulation of large-scale datasets on specific cancer care paths. Such a strategy would shape treatment according to each patient's demand, thus fortifying the avenue of personalized/precision medicine. Undoubtedly, this will assist in improving the oncology domain and alleviate the burden of clinicians in the coming decade.

BioFusionNet: Deep Learning-Based Survival Risk Stratification in ER+ Breast Cancer Through Multifeature and Multimodal Data Fusion

Breast cancer is a significant health concern affecting millions of women worldwide. Accurate survival risk stratification plays a crucial role in guiding personalised treatment decisions and improving patient outcomes. Here we present BioFusionNet, a deep learning framework that fuses image-derived features with genetic and clinical data to obtain a holistic profile and achieve survival risk stratification of ER+ breast cancer patients. We employ multiple self-supervised feature extractors (DINO and MoCoV3) pretrained on histopathological patches to capture detailed image features. These features are then fused by a variational autoencoder and fed to a self-attention network generating patient-level features. A co-dual-cross-attention mechanism combines the histopathological features with genetic data, enabling the model to capture the interplay between them. Additionally, clinical data is incorporated using a feed-forward network, further enhancing predictive performance and achieving comprehensive multimodal feature integration. Furthermore, we introduce a weighted Cox loss function, specifically designed to handle imbalanced survival data, which is a common challenge. Our model achieves a mean concordance index of 0.77 and a time-dependent area under the curve of 0.84, outperforming state-of-the-art methods. It predicts risk (high versus low) with prognostic significance for overall survival in univariate analysis (HR=2.99, 95% CI: 1.88-4.78, p 0.005), and maintains independent significance in multivariate analysis incorporating standard clinicopathological variables (HR=2.91, 95% CI: 1.80-4.68, p 0.005).

Deep learning-based multimodal spatial transcriptomics analysis for cancer

The advent of deep learning (DL) and multimodal spatial transcriptomics (ST) has revolutionized cancer research, offering unprecedented insights into tumor biology. This book chapter explores the integration of DL with ST to advance cancer diagnostics, treatment planning, and precision medicine. DL, a subset of artificial intelligence, employs neural networks to model complex patterns in vast datasets, significantly enhancing diagnostic and treatment applications. In oncology, convolutional neural networks excel in image classification, segmentation, and tumor volume analysis, essential for identifying tumors and optimizing radiotherapy. The chapter also delves into multimodal data analysis, which integrates genomic, proteomic, imaging, and clinical data to offer a holistic understanding of cancer biology. Leveraging diverse data sources, researchers can uncover intricate details of tumor heterogeneity, microenvironment interactions, and treatment responses. Examples include integrating MRI data with genomic profiles for accurate glioma grading and combining proteomic and clinical data to uncover drug resistance mechanisms. DL's integration with multimodal data enables comprehensive and actionable insights for cancer diagnosis and treatment. The synergy between DL models and multimodal data analysis enhances diagnostic accuracy, personalized treatment planning, and prognostic modeling. Notable applications include ST, which maps gene expression patterns within tissue contexts, providing critical insights into tumor heterogeneity and potential therapeutic targets. In summary, the integration of DL and multimodal ST represents a paradigm shift towards more precise and personalized oncology. This chapter elucidates the methodologies and applications of these advanced technologies, highlighting their transformative potential in cancer research and clinical practice.

Application of artificial intelligence in cancer diagnosis and tumor nanomedicine

Cancer is a major health concern due to its high incidence and mortality rates. Advances in cancer research, particularly in artificial intelligence (AI) and deep learning, have shown significant progress. The swift evolution of AI in healthcare, especially in tools like computer-aided diagnosis, has the potential to revolutionize early cancer detection. This technology offers improved speed, accuracy, and sensitivity, bringing a transformative impact on cancer diagnosis, treatment, and management. This paper provides a concise overview of the application of artificial intelligence in the realms of medicine and nanomedicine, with a specific emphasis on the significance and challenges associated with cancer diagnosis. It explores the pivotal role of AI in cancer diagnosis, leveraging structured, unstructured, and multimodal fusion data. Additionally, the article delves into the applications of AI in nanomedicine sensors and nano-oncology drugs. The fundamentals of deep learning and convolutional neural networks are clarified, underscoring their relevance to AI-driven cancer diagnosis. A comparative analysis is presented, highlighting the accuracy and efficiency of traditional methods juxtaposed with AI-based approaches. The discussion not only assesses the current state of AI in cancer diagnosis but also delves into the challenges faced by AI in this context. Furthermore, the article envisions the future development direction and potential application of artificial intelligence in cancer diagnosis, offering a hopeful prospect for enhanced cancer detection and improved patient prognosis.

Applicability of artificial intelligence in neuropsychological rehabilitation of patients with brain injury

Neuropsychological rehabilitation plays a critical role in helping those recovering from brain injuries restore cognitive and functional abilities. Artificial Intelligence, with its potential, may revolutionize this field further; therefore, this article explores applications of AI for neuropsychological rehabilitation of patients suffering brain injuries. This study employs a systematic review methodology to comprehensively review existing literature regarding Artificial Intelligence use in neuropsychological rehabilitation for people with brain injuries. The systematic review follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. A systematic search of electronic databases (PubMed, Scopus, PsycINFO, etc.) showed a total of 212 potentially relevant articles. After removing duplicates and screening titles and abstracts, 186 articles were selected for assessment. Following the assessment, 55 articles met the inclusion criteria and were included in this systematic review. A thematic analysis approach is employed to analyze and synthesize the extracted data. Themes, patterns, and trends are identified across the included studies, allowing for a comprehensive understanding of the applicability of AI in neuropsychological rehabilitation for patients with brain injuries. The identified topics were: AI Applications in Diagnostics of Brain Injuries and their Neuropsychological Repercussions; AI in Personalization and Monitoring of Neuropsychological Rehabilitation for traumatic brain injury (TBI); Leveraging AI for Predicting and Optimizing Neuropsychological Rehabilitation Outcomes in TBI Patients. Based on the review, it was concluded that AI has the potential to enhance neuropsychological rehabilitation for patients with brain injuries. By leveraging AI techniques, personalized rehabilitation programs can be developed, treatment outcomes can be predicted, and interventions can be optimized.

High-throughput analysis of hazards in novel food based on the density functional theory and multimodal deep learning

The emergence of cultured meat presents the potential for personalized food additive manufacturing, offering a solution to address future food resource scarcity. Processing raw materials and products in synthetic food products poses challenges in identifying hazards, impacting the entire industrial chain during the industry's further evolution. It is crucial to examine the correlation of biological information at different levels and to reveal the temporal dynamics jointly. Proposed active prevention method includes four aspects: (i) Investigating the molecular-level mechanism underlying the binding and dissociation of hazards with proteins represents a novel approach to mitigate matrix effect. (ii) Identifying distinct fragments is a pivotal advancement toward developing a novel screening strategy for hazards throughout the food chain. (iii) Designing an artificial intelligence model-based approach to acquire multi-dimensional histology data also holds significant potential for various applications. (iv) Integrating multimodal data is a practical approach to enhance evaluation and feedback control accuracy.

Multimodal data fusion using sparse canonical correlation analysis and cooperative learning: a COVID-19 cohort study

Through technological innovations, patient cohorts can be examined from multiple views with high-dimensional, multiscale biomedical data to classify clinical phenotypes and predict outcomes. Here, we aim to present our approach for analyzing multimodal data using unsupervised and supervised sparse linear methods in a COVID-19 patient cohort. This prospective cohort study of 149 adult patients was conducted in a tertiary care academic center. First, we used sparse canonical correlation analysis (CCA) to identify and quantify relationships across different data modalities, including viral genome sequencing, imaging, clinical data, and laboratory results. Then, we used cooperative learning to predict the clinical outcome of COVID-19 patients: Intensive care unit admission. We show that serum biomarkers representing severe disease and acute phase response correlate with original and wavelet radiomics features in the LLL frequency channel (cor(Xu1, Zv1) = 0.596, p value < 0.001). Among radiomics features, histogram-based first-order features reporting the skewness, kurtosis, and uniformity have the lowest negative, whereas entropy-related features have the highest positive coefficients. Moreover, unsupervised analysis of clinical data and laboratory results gives insights into distinct clinical phenotypes. Leveraging the availability of global viral genome databases, we demonstrate that the Word2Vec natural language processing model can be used for viral genome encoding. It not only separates major SARS-CoV-2 variants but also allows the preservation of phylogenetic relationships among them. Our quadruple model using Word2Vec encoding achieves better prediction results in the supervised task. The model yields area under the curve (AUC) and accuracy values of 0.87 and 0.77, respectively. Our study illustrates that sparse CCA analysis and cooperative learning are powerful techniques for handling high-dimensional, multimodal data to investigate multivariate associations in unsupervised and supervised tasks.

Artificial intelligence in neuro-oncology: advances and challenges in brain tumor diagnosis, prognosis, and precision treatment

This review delves into the most recent advancements in applying artificial intelligence (AI) within neuro-oncology, specifically emphasizing work on gliomas, a class of brain tumors that represent a significant global health issue. AI has brought transformative innovations to brain tumor management, utilizing imaging, histopathological, and genomic tools for efficient detection, categorization, outcome prediction, and treatment planning. Assessing its influence across all facets of malignant brain tumor management- diagnosis, prognosis, and therapy- AI models outperform human evaluations in terms of accuracy and specificity. Their ability to discern molecular aspects from imaging may reduce reliance on invasive diagnostics and may accelerate the time to molecular diagnoses. The review covers AI techniques, from classical machine learning to deep learning, highlighting current applications and challenges. Promising directions for future research include multimodal data integration, generative AI, large medical language models, precise tumor delineation and characterization, and addressing racial and gender disparities. Adaptive personalized treatment strategies are also emphasized for optimizing clinical outcomes. Ethical, legal, and social implications are discussed, advocating for transparency and fairness in AI integration for neuro-oncology and providing a holistic understanding of its transformative impact on patient care.

Generation of synthetic whole-slide image tiles of tumours from RNA-sequencing data via cascaded diffusion models

Training machine-learning models with synthetically generated data can alleviate the problem of data scarcity when acquiring diverse and sufficiently large datasets is costly and challenging. Here we show that cascaded diffusion models can be used to synthesize realistic whole-slide image tiles from latent representations of RNA-sequencing data from human tumours. Alterations in gene expression affected the composition of cell types in the generated synthetic image tiles, which accurately preserved the distribution of cell types and maintained the cell fraction observed in bulk RNA-sequencing data, as we show for lung adenocarcinoma, kidney renal papillary cell carcinoma, cervical squamous cell carcinoma, colon adenocarcinoma and glioblastoma. Machine-learning models pretrained with the generated synthetic data performed better than models trained from scratch. Synthetic data may accelerate the development of machine-learning models in scarce-data settings and allow for the imputation of missing data modalities.

Multimodal artificial intelligence-based pathogenomics improves survival prediction in oral squamous cell carcinoma

In this study, we aimed to develop a novel prognostic algorithm for oral squamous cell carcinoma (OSCC) using a combination of pathogenomics and AI-based techniques. We collected comprehensive clinical, genomic, and pathology data from a cohort of OSCC patients in the TCGA dataset and used machine learning and deep learning algorithms to identify relevant features that are predictive of survival outcomes. Our analyses included 406 OSCC patients. Initial analyses involved gene expression analyses, principal component analyses, gene enrichment analyses, and feature importance analyses. These insights were foundational for subsequent model development. Furthermore, we applied five machine learning/deep learning algorithms (Random Survival Forest, Gradient Boosting Survival Analysis, Cox PH, Fast Survival SVM, and DeepSurv) for survival prediction. Our initial analyses revealed relevant gene expression variations and biological pathways, laying the groundwork for robust feature selection in model building. The results showed that the multimodal model outperformed the unimodal models across all methods, with c-index values of 0.722 for RSF, 0.633 for GBSA, 0.625 for FastSVM, 0.633 for CoxPH, and 0.515 for DeepSurv. When considering only important features, the multimodal model continued to outperform the unimodal models, with c-index values of 0.834 for RSF, 0.747 for GBSA, 0.718 for FastSVM, 0.742 for CoxPH, and 0.635 for DeepSurv. Our results demonstrate the potential of pathogenomics and AI-based techniques in improving the accuracy of prognostic prediction in OSCC, which may ultimately aid in the development of personalized treatment strategies for patients with this devastating disease.

Integrative analysis of RNA expression data unveils distinct cancer types through machine learning techniques

Cancer is a highly complex and heterogeneous disease. Traditional methods of cancer classification based on histopathology have limitations in guiding personalized prognosis and therapy. Gene expression profiling provides a powerful approach to unraveling molecular intricacies and better-stratifying cancer subtypes. In this study, we performed an integrative analysis of RNA sequencing data from five cancer types - BRCA, KIRC, COAD, LUAD, and PRAD. A machine learning workflow consisting of dataset identification, normalization, feature selection, dimensionality reduction, clustering, and classification was implemented. The k-means algorithm was applied to categorize samples into distinct clusters based solely on gene expression patterns. Five unique clusters emerged from the unsupervised machine learning based analysis, significantly correlating with the known cancer types. BRCA aligned predominantly with one cluster, while COAD spanned three clusters. KIRC was represented within two main clusters. LUAD is associated strongly with a single cluster and PRAD with another cluster. This demonstrates the ability of machine learning approaches to unravel complex signatures within transcriptomic profiles that can delineate cancer subtypes. The proposed study highlights the potential of integrative analytics to derive meaningful biological insights from high-dimensional omics datasets. Molecular subtyping through machine learning clustering enhances our understanding of the intrinsic heterogeneities and pathways dysregulated in different cancers. Overall, this study exemplifies a powerful computational framework to classify gene expressions of patients having different types of cancers and guide personalized therapeutic decisions. Finally, Wide Neural Network demonstrates a significantly higher accuracy, achieving 99.834% on the validation set and an even more impressive 99.995% on the test set.

Big data and artificial intelligence in cancer research

The field of oncology has witnessed an extraordinary surge in the application of big data and artificial intelligence (AI). AI development has made multiscale and multimodal data fusion and analysis possible. A new era of extracting information from complex big data is rapidly evolving. However, challenges related to efficient data curation, in-depth analysis, and utilization remain. We provide a comprehensive overview of the current state of the art in big data and computational analysis, highlighting key applications, challenges, and future opportunities in cancer research. By sketching the current landscape, we seek to foster a deeper understanding and facilitate the advancement of big data utilization in oncology, call for interdisciplinary collaborations, ultimately contributing to improved patient outcomes and a profound understanding of cancer.

Machine Learning Methods for Gene Selection in Uveal Melanoma

Uveal melanoma (UM) is the most common primary intraocular malignancy with a limited five-year survival for metastatic patients. Limited therapeutic treatments are currently available for metastatic disease, even if the genomics of this tumor has been deeply studied using next-generation sequencing (NGS) and functional experiments. The profound knowledge of the molecular features that characterize this tumor has not led to the development of efficacious therapies, and the survival of metastatic patients has not changed for decades. Several bioinformatics methods have been applied to mine NGS tumor data in order to unveil tumor biology and detect possible molecular targets for new therapies. Each application can be single domain based while others are more focused on data integration from multiple genomics domains (as gene expression and methylation data). Examples of single domain approaches include differentially expressed gene (DEG) analysis on gene expression data with statistical methods such as SAM (significance analysis of microarray) or gene prioritization with complex algorithms such as deep learning. Data fusion or integration methods merge multiple domains of information to define new clusters of patients or to detect relevant genes, according to multiple NGS data. In this work, we compare different strategies to detect relevant genes for metastatic disease prediction in the TCGA uveal melanoma (UVM) dataset. Detected targets are validated with multi-gene score analysis on a larger UM microarray dataset.

Digital profiling of cancer transcriptomes from histology images with grouped vision attention

Cancer is a heterogeneous disease that demands precise molecular profiling for better understanding and management. Recently, deep learning has demonstrated potentials for cost-efficient prediction of molecular alterations from histology images. While transformer-based deep learning architectures have enabled significant progress in non-medical domains, their application to histology images remains limited due to small dataset sizes coupled with the explosion of trainable parameters. Here, we develop SEQUOIA, a transformer model to predict cancer transcriptomes from whole-slide histology images. To enable the full potential of transformers, we first pre-train the model using data from 1,802 normal tissues. Then, we fine-tune and evaluate the model in 4,331 tumor samples across nine cancer types. The prediction performance is assessed at individual gene levels and pathway levels through Pearson correlation analysis and root mean square error. The generalization capacity is validated across two independent cohorts comprising 1,305 tumors. In predicting the expression levels of 25,749 genes, the highest performance is observed in cancers from breast, kidney and lung, where SEQUOIA accurately predicts the expression of 11,069, 10,086 and 8,759 genes, respectively. The accurately predicted genes are associated with the regulation of inflammatory response, cell cycles and metabolisms. While the model is trained at the tissue level, we showcase its potential in predicting spatial gene expression patterns using spatial transcriptomics datasets. Leveraging the prediction performance, we develop a digital gene expression signature that predicts the risk of recurrence in breast cancer. SEQUOIA deciphers clinically relevant gene expression patterns from histology images, opening avenues for improved cancer management and personalized therapies.

BrainNet: Precision Brain Tumor Classification with Optimized EfficientNet Architecture

Brain tumors significantly impact human health due to their complexity and the challenges in early detection and treatment. Accurate diagnosis is crucial for effective intervention, but existing methods often suffer from limitations in accuracy and efficiency. To address these challenges, this study presents a novel deep learning (DL) approach utilizing the EfficientNet family for enhanced brain tumor classification and detection. Leveraging a comprehensive dataset of 3064 T1‐weighted CE MRI images, our methodology incorporates advanced preprocessing and augmentation techniques to optimize model performance. The experiments demonstrate that EfficientNetB(07) achieved 99.14%, 98.76%, 99.07%, 99.69%, 99.07%, 98.76%, 98.76%, and 99.07% accuracy, respectively. The pinnacle of our research is the EfficientNetB3 model, which demonstrated exceptional performance with an accuracy rate of 99.69%. This performance surpasses many existing state‐of‐the‐art (SOTA) techniques, underscoring the efficacy of our approach. The precision of our high‐accuracy DL model promises to improve diagnostic reliability and speed in clinical settings, facilitating earlier and more effective treatment strategies. Our findings suggest significant potential for improving patient outcomes in brain tumor diagnosis.

Computer Vision in Digital Neuropathology

Digital pathology has revolutionized the field of neuropathology, enabling rapid and precise analysis of tissue samples. As neuropathologists and neurosurgeons increasingly embrace digital platforms, computer vision techniques and computerized quantitative analyses (i.e., pathomics) play a pivotal role in automating and enhancing diagnostic processes, as well as in unlocking new insights from neuropathological images. This chapter explores the role of computer vision in bridging the gap between traditional and digital neuropathology, providing a comprehensive overview of computer vision applications in neuropathology, covering image preprocessing, and decision support for clinical tasks, such as early detection of neurological disorders, classification of brain tumors, and quantitative assessment of tissue biomarkers. In addition, we further delve into the challenges and opportunities presented by large-scale image datasets and the integration of digital pathology into neurosurgical practice.

Multimodal Biomedical Data Fusion Using Sparse Canonical Correlation Analysis and Cooperative Learning: A Cohort Study on COVID-19

Through technological innovations, patient cohorts can be examined from multiple views with high-dimensional, multiscale biomedical data to classify clinical phenotypes and predict outcomes. Here, we aim to present our approach for analyzing multimodal data using unsupervised and supervised sparse linear methods in a COVID-19 patient cohort. This prospective cohort study of 149 adult patients was conducted in a tertiary care academic center. First, we used sparse canonical correlation analysis (CCA) to identify and quantify relationships across different data modalities, including viral genome sequencing, imaging, clinical data, and laboratory results. Then, we used cooperative learning to predict the clinical outcome of COVID-19 patients. We show that serum biomarkers representing severe disease and acute phase response correlate with original and wavelet radiomics features in the LLL frequency channel (corr(Xu1, Zv1) = 0.596, p-value < 0.001). Among radiomics features, histogram-based first-order features reporting the skewness, kurtosis, and uniformity have the lowest negative, whereas entropy-related features have the highest positive coefficients. Moreover, unsupervised analysis of clinical data and laboratory results gives insights into distinct clinical phenotypes. Leveraging the availability of global viral genome databases, we demonstrate that the Word2Vec natural language processing model can be used for viral genome encoding. It not only separates major SARS-CoV-2 variants but also allows the preservation of phylogenetic relationships among them. Our quadruple model using Word2Vec encoding achieves better prediction results in the supervised task. The model yields area under the curve (AUC) and accuracy values of 0.87 and 0.77, respectively. Our study illustrates that sparse CCA analysis and cooperative learning are powerful techniques for handling high-dimensional, multimodal data to investigate multivariate associations in unsupervised and supervised tasks.

Spatial cellular architecture predicts prognosis in glioblastoma

Intra-tumoral heterogeneity and cell-state plasticity are key drivers for the therapeutic resistance of glioblastoma. Here, we investigate the association between spatial cellular organization and glioblastoma prognosis. Leveraging single-cell RNA-seq and spatial transcriptomics data, we develop a deep learning model to predict transcriptional subtypes of glioblastoma cells from histology images. Employing this model, we phenotypically analyze 40 million tissue spots from 410 patients and identify consistent associations between tumor architecture and prognosis across two independent cohorts. Patients with poor prognosis exhibit higher proportions of tumor cells expressing a hypoxia-induced transcriptional program. Furthermore, a clustering pattern of astrocyte-like tumor cells is associated with worse prognosis, while dispersion and connection of the astrocytes with other transcriptional subtypes correlate with decreased risk. To validate these results, we develop a separate deep learning model that utilizes histology images to predict prognosis. Applying this model to spatial transcriptomics data reveal survival-associated regional gene expression programs. Overall, our study presents a scalable approach to unravel the transcriptional heterogeneity of glioblastoma and establishes a critical connection between spatial cellular architecture and clinical outcomes.

Deep Learning Techniques with Genomic Data in Cancer Prognosis: A Comprehensive Review of the 2021-2023 Literature

Deep learning has brought about a significant transformation in machine learning, leading to an array of novel methodologies and consequently broadening its influence. The application of deep learning in various sectors, especially biomedical data analysis, has initiated a period filled with noteworthy scientific developments. This trend has majorly influenced cancer prognosis, where the interpretation of genomic data for survival analysis has become a central research focus. The capacity of deep learning to decode intricate patterns embedded within high-dimensional genomic data has provoked a paradigm shift in our understanding of cancer survival. Given the swift progression in this field, there is an urgent need for a comprehensive review that focuses on the most influential studies from 2021 to 2023. This review, through its careful selection and thorough exploration of dominant trends and methodologies, strives to fulfill this need. The paper aims to enhance our existing understanding of applications of deep learning in cancer survival analysis, while also highlighting promising directions for future research. This paper undertakes aims to enrich our existing grasp of the application of deep learning in cancer survival analysis, while concurrently shedding light on promising directions for future research in this vibrant and rapidly proliferating field.