Non-Invasive Measurement Using Deep Learning Algorithm Based on Multi-Source Features Fusion to Predict PD-L1 Expression and Survival in NSCLC

Hybrid multimodal fusion for graph learning in disease prediction

Graph neural networks (GNNs) have gained significant attention in disease prediction where the latent embeddings of patients are modeled as nodes and the similarities among patients are represented through edges. The graph structure, which determines how information is aggregated and propagated, plays a crucial role in graph learning. Recent approaches typically create graphs based on patients' latent embeddings, which may not accurately reflect their real-world closeness. Our analysis reveals that raw data, such as demographic attributes and laboratory results, offers a wealth of information for assessing patient similarities and can serve as a compensatory measure for graphs constructed exclusively from latent embeddings. In this study, we first construct adaptive graphs from both latent representations and raw data respectively, and then merge these graphs via weighted summation. Given that the graphs may contain extraneous and noisy connections, we apply degree-sensitive edge pruning and kNN sparsification techniques to selectively sparsify and prune these edges. We conducted intensive experiments on two diagnostic prediction datasets, and the results demonstrate that our proposed method surpasses current state-of-the-art techniques.

Radiomics based on 18F-FDG PET/CT for prediction of pathological complete response to neoadjuvant therapy in non-small cell lung cancer

Purpose

This study aimed to establish and evaluate the value of integrated models involving 18F-FDG PET/CT-based radiomics and clinicopathological information in the prediction of pathological complete response (pCR) to neoadjuvant therapy (NAT) for non-small cell lung cancer (NSCLC).

Methods

A total of 106 eligible NSCLC patients were included in the study. After volume of interest (VOI) segmentation, 2,016 PET-based and 2,016 CT-based radiomic features were extracted. To select an optimal machine learning model, a total of 25 models were constructed based on five sets of machine learning classifiers combined with five sets of predictive feature resources, including PET-based alone radiomics, CT-based alone radiomics, PET/CT-based radiomics, clinicopathological features, and PET/CT-based radiomics integrated with clinicopathological features. Area under the curves (AUCs) of receiver operator characteristic (ROC) curves were used as the main outcome to assess the model performance.

Results

The hybrid PET/CT-derived radiomic model outperformed PET-alone and CT-alone radiomic models in the prediction of pCR to NAT. Moreover, addition of clinicopathological information further enhanced the predictive performance of PET/CT-derived radiomic model. Ultimately, the support vector machine (SVM)-based PET/CT radiomics combined clinicopathological information presented an optimal predictive efficacy with an AUC of 0.925 (95% CI 0.869-0.981) in the training cohort and an AUC of 0.863 (95% CI 0.740-0.985) in the test cohort. The developed nomogram involving radiomics and pathological type was suggested as a convenient tool to enable clinical application.

Conclusions

The 18F-FDG PET/CT-based SVM radiomics integrated with clinicopathological information was an optimal model to non-invasively predict pCR to NAC for NSCLC.

Automated PD-L1 status prediction in lung cancer with multi-modal PET/CT fusion

Programmed death-ligand 1 (PD-L1) expressions play a crucial role in guiding therapeutic interventions such as the use of tyrosine kinase inhibitors (TKIs) and immune checkpoint inhibitors (ICIs) in lung cancer. Conventional determination of PD-L1 status includes careful surgical or biopsied tumor specimens. These specimens are gathered through invasive procedures, representing a risk of difficulties and potential challenges in getting reliable and representative tissue samples. Using a single center cohort of 189 patients, our objective was to evaluate various fusion methods that used non-invasive computed tomography (CT) and18 F-FDG positron emission tomography (PET) images as inputs to various deep learning models to automatically predict PD-L1 in non-small cell lung cancer (NSCLC). We compared three different architectures (ResNet, DenseNet, and EfficientNet) and considered different input data (CT only, PET only, PET/CT early fusion, PET/CT late fusion without as well as with partially and fully shared weights to determine the best model performance. Models were assessed utilizing areas under the receiver operating characteristic curves (AUCs) considering their 95% confidence intervals (CI). The fusion of PET and CT images as input yielded better performance for PD-L1 classification. The different data fusion schemes systematically outperformed their individual counterparts when used as input of the various deep models. Furthermore, early fusion consistently outperformed late fusion, probably as a result of its capacity to capture more complicated patterns by merging PET and CT derived content at a lower level. When we looked more closely at the effects of weight sharing in late fusion architectures, we discovered that while it might boost model stability, it did not always result in better results. This suggests that although weight sharing could be beneficial when modality parameters are similar, the anatomical and metabolic information provided by CT and PET scans are too dissimilar to consistently lead to improved PD-L1 status predictions.

Integration of deep learning and habitat radiomics for predicting the response to immunotherapy in NSCLC patients

BACKGROUND

The non-invasive biomarkers for predicting immunotherapy response are urgently needed to prevent both premature cessation of treatment and ineffective extension. This study aimed to construct a non-invasive model for predicting immunotherapy response, based on the integration of deep learning and habitat radiomics in patients with advanced non-small cell lung cancer (NSCLC).

METHODS

Independent patient cohorts from three medical centers were enrolled for training (n = 164) and test (n = 82). Habitat imaging radiomics features were derived from sub-regions clustered from individual's tumor by K-means method. The deep learning features were extracted based on 3D ResNet algorithm. Pearson correlation coefficient, T test and least absolute shrinkage and selection operator regression were used to select features. Support vector machine was applied to implement deep learning and habitat radiomics, respectively. Then, a combination model was developed integrating both sources of data.

RESULTS

The combination model obtained a strong well-performance, achieving area under receiver operating characteristics curve of 0.865 (95% CI 0.772-0.931). The model significantly discerned high and low-risk patients, and exhibited a significant benefit in the clinical use.

CONCLUSION

The integration of deep-leaning and habitat radiomics contributed to predicting response to immunotherapy in patients with NSCLC. The developed integration model may be used as potential tool for individual immunotherapy management.

Diagnostic and prognostic value of serum soluble B7-H3 in nonsmall cell lung cancer

The aim of this study was to investigate the utility of serum soluble B7-H3 (sB7-H3) as a diagnostic marker for early-stage nonsmall cell lung cancer (NSCLC) and its potential for evaluating the prognosis of patients with advanced-stage NSCLC. In this study, an ELISA was employed to detect the expression levels of sB7-H3 in a cohort of patients diagnosed with NSCLC ( n  = 122) and a control group ( n  = 42) during the same observation period. Comparative analyses were conducted to ascertain the variations in sB7-H3 concentrations between the NSCLC cohort and the healthy control group, as well as across pathological types and the presence and absence of lymph node metastasis. (1) The concentration of sB7-H3 in patients diagnosed with NSCLC exhibited a statistically significant increase compared to that observed in the healthy control group ( P  < 0.05). Elevated expression levels of sB7-H3 demonstrated a significant correlation with pathological type, lymph node metastasis, tumor, node and metastasis stage and programmed cell death ligand (PD-L1) expression ( P  < 0.05). (2) The diagnostic utility of sB7-H3 for the diagnosis of NSCLC and the heightened expression of PD-L1 demonstrated high levels of sensitivity and specificity. (3) Elevated levels of sB7-H3 emerged as an independent risk factor impacting the overall survival of patients diagnosed with advanced NSCLC. The findings of this study suggest that sB7-H3 holds promise as a diagnostic tool for early-stage NSCLC. The elevated expression of sB7-H3 appears to serve as a reliable indicator for assessing the prognosis of patients diagnosed with advanced NSCLC.

Value of computed tomography radiomics combined with inflammation indices in predicting the efficacy of immunotherapy in patients with locally advanced and metastatic non-small cell lung cancer

Background

Although immunotherapy has revolutionized the treatment landscape of lung cancer and improved the prognosis of this malignancy, many patients with lung cancer still are not able to benefit from it because of many different reasons. The expression of programmed death ligand-1 (PD-L1) in tumor cells has been approved for the prediction of immunotherapy efficacy; however, its clinical application has been limited by the invasiveness of PD-L1 determination and the heterogeneity of tumor cells. As a promising technology, radiomics has made significant progress in the diagnosis and treatment of lung cancer. Thus, we constructed a noninvasive predictive model which based on radiomics to predict the immunotherapy efficacy of lung caner patients.

Methods

Data of 82 patients with stage IIIa/IVb NSCLC who received immunotherapy at the First Affiliated Hospital of Soochow University from December 2019 to January 2023 were retrospectively collected. These patients were followed up for durable clinical benefit (DCB), as defined by whether progression-free survival (PFS) reached 12 months. The least absolute shrinkage and selection operator (LASSO) algorithm was used to screen for the radiomic features in the training set, and a radiomics score (Rad-score) was calculated. The clinical baseline data were analyzed, and the peripheral blood inflammation indices were calculated. Univariate and multivariate analyses were performed to identify the applicable indices, which were combined with the Rad-score to create a comprehensive forecasting model (CFM) and nomograms. Internal validation was performed in the validation set.

Results

Up to the last follow-up time, 48 of 82 patients had a PFS of more than 12 months. The area under the receiver operating characteristic (ROC) curve (AUC) of the Rad-score was 0.858 and 0.812, respectively, in the training set and validation set. A systemic immune-inflammation index (SII) score of <500.88 after two cycles of immunotherapy was a protective factor for PFS >12 months [odds ratio (OR) 0.054; P=0.003]. The CFM had an AUC of 0.930 and 0.922, respectively, in the training and validation sets. The calibration curves and decision curve analysis (DCA) demonstrated the reliability and clinical applicability of the model, respectively.

Conclusions

The radiomics model performed well in predicting whether patients with locally advanced or metastatic NSCLC can achieve DCB after receiving immunotherapy. The CFM had good predictive performance and reliability.

Unveiling the Influence of AI Predictive Analytics on Patient Outcomes: A Comprehensive Narrative Review

This comprehensive literature review explores the transformative impact of artificial intelligence (AI) predictive analytics on healthcare, particularly in improving patient outcomes regarding disease progression, treatment response, and recovery rates. AI, encompassing capabilities such as learning, problem-solving, and decision-making, is leveraged to predict disease progression, optimize treatment plans, and enhance recovery rates through the analysis of vast datasets, including electronic health records (EHRs), imaging, and genetic data. The utilization of machine learning (ML) and deep learning (DL) techniques in predictive analytics enables personalized medicine by facilitating the early detection of conditions, precision in drug discovery, and the tailoring of treatment to individual patient profiles. Ethical considerations, including data privacy, bias, and accountability, emerge as vital in the responsible implementation of AI in healthcare. The findings underscore the potential of AI predictive analytics in revolutionizing clinical decision-making and healthcare delivery, emphasizing the necessity of ethical guidelines and continuous model validation to ensure its safe and effective use in augmenting human judgment in medical practice.

Deep learning for precise diagnosis and subtype triage of drug-resistant tuberculosis on chest computed tomography

Deep learning, transforming input data into target prediction through intricate network structures, has inspired novel exploration in automated diagnosis based on medical images. The distinct morphological characteristics of chest abnormalities between drug-resistant tuberculosis (DR-TB) and drug-sensitive tuberculosis (DS-TB) on chest computed tomography (CT) are of potential value in differential diagnosis, which is challenging in the clinic. Hence, based on 1176 chest CT volumes from the equal number of patients with tuberculosis (TB), we presented a Deep learning-based system for TB drug resistance identification and subtype classification (DeepTB), which could automatically diagnose DR-TB and classify crucial subtypes, including rifampicin-resistant tuberculosis, multidrug-resistant tuberculosis, and extensively drug-resistant tuberculosis. Moreover, chest lesions were manually annotated to endow the model with robust power to assist radiologists in image interpretation and the Circos revealed the relationship between chest abnormalities and specific types of DR-TB. Finally, DeepTB achieved an area under the curve (AUC) up to 0.930 for thoracic abnormality detection and 0.943 for DR-TB diagnosis. Notably, the system demonstrated instructive value in DR-TB subtype classification with AUCs ranging from 0.880 to 0.928. Meanwhile, class activation maps were generated to express a human-understandable visual concept. Together, showing a prominent performance, DeepTB would be impactful in clinical decision-making for DR-TB.

A whirl of radiomics-based biomarkers in cancer immunotherapy, why is large scale validation still lacking?

The search for understanding immunotherapy response has sparked interest in diverse areas of oncology, with artificial intelligence (AI) and radiomics emerging as promising tools, capable of gathering large amounts of information to identify suitable patients for treatment. The application of AI in radiology has grown, driven by the hypothesis that radiology images capture tumor phenotypes and thus could provide valuable insights into immunotherapy response likelihood. However, despite the rapid growth of studies, no algorithms in the field have reached clinical implementation, mainly due to the lack of standardized methods, hampering study comparisons and reproducibility across different datasets. In this review, we performed a comprehensive assessment of published data to identify sources of variability in radiomics study design that hinder the comparison of the different model performance and, therefore, clinical implementation. Subsequently, we conducted a use-case meta-analysis using homogenous studies to assess the overall performance of radiomics in estimating programmed death-ligand 1 (PD-L1) expression. Our findings indicate that, despite numerous attempts to predict immunotherapy response, only a limited number of studies share comparable methodologies and report sufficient data about cohorts and methods to be suitable for meta-analysis. Nevertheless, although only a few studies meet these criteria, their promising results underscore the importance of ongoing standardization and benchmarking efforts. This review highlights the importance of uniformity in study design and reporting. Such standardization is crucial to enable meaningful comparisons and demonstrate the validity of biomarkers across diverse populations, facilitating their implementation into the immunotherapy patient selection process.

Artificial Intelligence-Based Treatment Decisions: A New Era for NSCLC

Non-small cell lung cancer (NSCLC) is the leading cause of cancer-related mortality among women and men, in developed countries, despite the public health interventions including tobacco-free campaigns, screening and early detection methods, recent therapeutic advances, and ongoing intense research on novel antineoplastic modalities. Targeting oncogenic driver mutations and immune checkpoint inhibition has indeed revolutionized NSCLC treatment, yet there still remains the unmet need for robust and standardized predictive biomarkers to accurately inform clinical decisions. Artificial intelligence (AI) represents the computer-based science concerned with large datasets for complex problem-solving. Its concept has brought a paradigm shift in oncology considering its immense potential for improved diagnosis, treatment guidance, and prognosis. In this review, we present the current state of AI-driven applications on NSCLC management, with a particular focus on radiomics and pathomics, and critically discuss both the existing limitations and future directions in this field. The thoracic oncology community should not be discouraged by the likely long road of AI implementation into daily clinical practice, as its transformative impact on personalized treatment approaches is undeniable.

Weakly Supervised Deep Learning Predicts Immunotherapy Response in Solid Tumors Based on PD-L1 Expression

Programmed death-ligand 1 (PD-L1) IHC is the most commonly used biomarker for immunotherapy response. However, quantification of PD-L1 status in pathology slides is challenging. Neither manual quantification nor a computer-based mimicking of manual readouts is perfectly reproducible, and the predictive performance of both approaches regarding immunotherapy response is limited. In this study, we developed a deep learning (DL) method to predict PD-L1 status directly from raw IHC image data, without explicit intermediary steps such as cell detection or pigment quantification. We trained the weakly supervised model on PD-L1-stained slides from the non-small cell lung cancer (NSCLC)-Memorial Sloan Kettering (MSK) cohort (N = 233) and validated it on the pan-cancer-Vall d'Hebron Institute of Oncology (VHIO) cohort (N = 108). We also investigated the performance of the model to predict response to immune checkpoint inhibitors (ICI) in terms of progression-free survival. In the pan-cancer-VHIO cohort, the performance was compared with tumor proportion score (TPS) and combined positive score (CPS). The DL model showed good performance in predicting PD-L1 expression (TPS ≥ 1%) in both NSCLC-MSK and pan-cancer-VHIO cohort (AUC 0.88 ± 0.06 and 0.80 ± 0.03, respectively). The predicted PD-L1 status showed an improved association with response to ICIs [HR: 1.5 (95% confidence interval: 1-2.3), P = 0.049] compared with TPS [HR: 1.4 (0.96-2.2), P = 0.082] and CPS [HR: 1.2 (0.79-1.9), P = 0.386]. Notably, our explainability analysis showed that the model does not just look at the amount of brown pigment in the IHC slides, but also considers morphologic factors such as lymphocyte conglomerates. Overall, end-to-end weakly supervised DL shows potential for improving patient stratification for cancer immunotherapy by analyzing PD-L1 IHC, holistically integrating morphology and PD-L1 staining intensity.

SIGNIFICANCE

The weakly supervised DL model to predict PD-L1 status from raw IHC data, integrating tumor staining intensity and morphology, enables enhanced patient stratification in cancer immunotherapy compared with traditional pathologist assessment.

Refining PD-1/PD-L1 assessment for biomarker-guided immunotherapy: A review

Anti-programmed cell death ligand 1 (anti-PD-L1)  immunotherapy is an increasingly crucial in cancer treatment. To date, the Federal Drug Administration (FDA) has approved four PD-L1 immunohistochemistry (IHC) staining protocols, commercially available in the form of "kits", facilitating testing for PD-L1 expression. These kits comprise four PD-L1 antibodies on two separate IHC platforms, each utilizing distinct, non-interchangeable scoring systems. Several factors, including tumor heterogeneity and the size of the tissue specimens assessed, can lead to PD-L1 status misclassification, potentially hindering the initiation of therapy. Therefore, the development of more accurate predictive biomarkers to distinguish between responders and non-responders prior to anti-PD-1/PD-L1 therapy warrants further research. Achieving this goal necessitates refining sampling criteria, enhancing current methods of PD-L1 detection, and deepening our understanding of the impact of additional biomarkers. In this article, we review potential solutions to improve the predictive accuracy of PD-L1 assessment in order to more precisely anticipate patients' responses to anti-PD-1/PD-L1 therapy, monitor disease progression and predict clinical outcomes.

Cancer immunotherapy efficacy and machine learning

INTRODUCTION

Immunotherapy is one of the major breakthroughs in the treatment of cancer, and it has become a powerful clinical strategy, however, not all patients respond to immune checkpoint blockade and other immunotherapy strategies. Applying machine learning (ML) techniques to predict the efficacy of cancer immunotherapy is useful for clinical decision-making.

AREAS COVERED

Applying ML including deep learning (DL) in radiomics, pathomics, tumor microenvironment (TME) and immune-related genes analysis to predict immunotherapy efficacy. The studies in this review were searched from PubMed and ClinicalTrials.gov (January 2023).

EXPERT OPINION

An increasing number of studies indicate that ML has been applied to various aspects of oncology research, with the potential to provide more effective individualized immunotherapy strategies and enhance treatment decisions. With advances in ML technology, more efficient methods of predicting the efficacy of immunotherapy may become available in the future.

AI/ML advances in non-small cell lung cancer biomarker discovery

Lung cancer is the leading cause of cancer deaths among both men and women, representing approximately 25% of cancer fatalities each year. The treatment landscape for non-small cell lung cancer (NSCLC) is rapidly evolving due to the progress made in biomarker-driven targeted therapies. While advancements in targeted treatments have improved survival rates for NSCLC patients with actionable biomarkers, long-term survival remains low, with an overall 5-year relative survival rate below 20%. Artificial intelligence/machine learning (AI/ML) algorithms have shown promise in biomarker discovery, yet NSCLC-specific studies capturing the clinical challenges targeted and emerging patterns identified using AI/ML approaches are lacking. Here, we employed a text-mining approach and identified 215 studies that reported potential biomarkers of NSCLC using AI/ML algorithms. We catalogued these studies with respect to BEST (Biomarkers, EndpointS, and other Tools) biomarker sub-types and summarized emerging patterns and trends in AI/ML-driven NSCLC biomarker discovery. We anticipate that our comprehensive review will contribute to the current understanding of AI/ML advances in NSCLC biomarker research and provide an important catalogue that may facilitate clinical adoption of AI/ML-derived biomarkers.

Application of radiomics in diagnosis and treatment of lung cancer

Radiomics has become a research field that involves the process of converting standard nursing images into quantitative image data, which can be combined with other data sources and subsequently analyzed using traditional biostatistics or artificial intelligence (Al) methods. Due to the capture of biological and pathophysiological information by radiomics features, these quantitative radiomics features have been proven to provide fast and accurate non-invasive biomarkers for lung cancer risk prediction, diagnosis, prognosis, treatment response monitoring, and tumor biology. In this review, radiomics has been emphasized and discussed in lung cancer research, including advantages, challenges, and drawbacks.

Integrative radiomics and transcriptomics analyses reveal subtype characterization of non-small cell lung cancer

OBJECTIVES

To assess whether integrative radiomics and transcriptomics analyses could provide novel insights for radiomic features' molecular annotation and effective risk stratification in non-small cell lung cancer (NSCLC).

METHODS

A total of 627 NSCLC patients from three datasets were included. Radiomics features were extracted from segmented 3-dimensional tumour volumes and were z-score normalized for further analysis. In transcriptomics level, 186 pathways and 28 types of immune cells were assessed by using the Gene Set Variation Analysis (GSVA) algorithm. NSCLC patients were categorized into subgroups based on their radiomic features and pathways enrichment scores using consensus clustering. Subgroup-specific radiomics features were used to validate clustering performance and prognostic value. Kaplan-Meier survival analysis with the log-rank test and univariable and multivariable Cox analyses were conducted to explore survival differences among the subgroups.

RESULTS

Three radiotranscriptomics subtypes (RTSs) were identified based on the radiomics and pathways enrichment profiles. The three RTSs were characterized as having specific molecular hallmarks: RTS1 (proliferation subtype), RTS2 (metabolism subtype), and RTS3 (immune activation subtype). RTS3 showed increased infiltration of most immune cells. The RTS stratification strategy was validated in a validation cohort and showed significant prognostic value. Survival analysis demonstrated that the RTS strategy could stratify NSCLC patients according to prognosis (p = 0.009), and the RTS strategy remained an independent prognostic indicator after adjusting for other clinical parameters.

CONCLUSIONS

This radiotranscriptomics study provides a stratification strategy for NSCLC that could provide information for radiomics feature molecular annotation and prognostic prediction.

KEY POINTS

• Radiotranscriptomics subtypes (RTSs) could be used to stratify molecularly heterogeneous patients. • RTSs showed relationships between molecular phenotypes and radiomics features. • The RTS algorithm could be used to identify patients with poor prognosis.

A transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics

During the diagnostic process, clinicians leverage multimodal information, such as the chief complaint, medical images and laboratory test results. Deep-learning models for aiding diagnosis have yet to meet this requirement of leveraging multimodal information. Here we report a transformer-based representation-learning model as a clinical diagnostic aid that processes multimodal input in a unified manner. Rather than learning modality-specific features, the model leverages embedding layers to convert images and unstructured and structured text into visual tokens and text tokens, and uses bidirectional blocks with intramodal and intermodal attention to learn holistic representations of radiographs, the unstructured chief complaint and clinical history, and structured clinical information such as laboratory test results and patient demographic information. The unified model outperformed an image-only model and non-unified multimodal diagnosis models in the identification of pulmonary disease (by 12% and 9%, respectively) and in the prediction of adverse clinical outcomes in patients with COVID-19 (by 29% and 7%, respectively). Unified multimodal transformer-based models may help streamline the triaging of patients and facilitate the clinical decision-making process.

Advances in artificial intelligence to predict cancer immunotherapy efficacy

Tumor immunotherapy, particularly the use of immune checkpoint inhibitors, has yielded impressive clinical benefits. Therefore, it is critical to accurately screen individuals for immunotherapy sensitivity and forecast its efficacy. With the application of artificial intelligence (AI) in the medical field in recent years, an increasing number of studies have indicated that the efficacy of immunotherapy can be better anticipated with the help of AI technology to reach precision medicine. This article focuses on the current prediction models based on information from histopathological slides, imaging-omics, genomics, and proteomics, and reviews their research progress and applications. Furthermore, we also discuss the existing challenges encountered by AI in the field of immunotherapy, as well as the future directions that need to be improved, to provide a point of reference for the early implementation of AI-assisted diagnosis and treatment systems in the future.

Radiogenomic System for Non-Invasive Identification of Multiple Actionable Mutations and PD-L1 Expression in Non-Small Cell Lung Cancer Based on CT Images

PURPOSE

Personalized treatments such as targeted therapy and immunotherapy have revolutionized the predominantly therapeutic paradigm for non-small cell lung cancer (NSCLC). However, these treatment decisions require the determination of targetable genomic and molecular alterations through invasive genetic or immunohistochemistry (IHC) tests. Numerous previous studies have demonstrated that artificial intelligence can accurately predict the single-gene status of tumors based on radiologic imaging, but few studies have achieved the simultaneous evaluation of multiple genes to reflect more realistic clinical scenarios.

METHODS

We proposed a multi-label multi-task deep learning (MMDL) system for non-invasively predicting actionable NSCLC mutations and PD-L1 expression utilizing routinely acquired computed tomography (CT) images. This radiogenomic system integrated transformer-based deep learning features and radiomic features of CT volumes from 1096 NSCLC patients based on next-generation sequencing (NGS) and IHC tests.

RESULTS

For each task cohort, we randomly split the corresponding dataset into training (80%), validation (10%), and testing (10%) subsets. The area under the receiver operating characteristic curves (AUCs) of the MMDL system achieved 0.862 (95% confidence interval (CI), 0.758-0.969) for discrimination of a panel of 8 mutated genes, including EGFR, ALK, ERBB2, BRAF, MET, ROS1, RET and KRAS, 0.856 (95% CI, 0.663-0.948) for identification of a 10-molecular status panel (previous 8 genes plus TP53 and PD-L1); and 0.868 (95% CI, 0.641-0.972) for classifying EGFR / PD-L1 subtype, respectively.

CONCLUSIONS

To the best of our knowledge, this study is the first deep learning system to simultaneously analyze 10 molecular expressions, which might be utilized as an assistive tool in conjunction with or in lieu of ancillary testing to support precision treatment options.

What does radiomics do in PD-L1 blockade therapy of NSCLC patients?

With the in‐depth understanding of programmed cell death 1 ligand 1 (PD‐L1) in non‐small cell lung cancer (NSCLC), PD‐L1 has become a vital immunotherapy target and a significant biomarker. The clinical utility of detecting PD‐L1 by immunohistochemistry or next‐generation sequencing has been written into guidelines. However, the application of these methods is limited in some circumstances where the biopsy size is small or not accessible, or a dynamic monitor is needed. Radiomics can noninvasively, in real‐time, and quantitatively analyze medical images to reflect deeper information about diseases. Since radiomics was proposed in 2012, it has been widely used in disease diagnosis and differential diagnosis, tumor staging and grading, gene and protein phenotype prediction, treatment plan decision‐making, efficacy evaluation, and prognosis prediction. To explore the feasibility of the clinical application of radiomics in predicting PD‐L1 expression, immunotherapy response, and long‐term prognosis, we comprehensively reviewed and summarized recently published works in NSCLC. In conclusion, radiomics is expected to be a companion to the whole immunotherapy process.

Development and validation of an abnormality-derived deep-learning diagnostic system for major respiratory diseases

Respiratory diseases impose a tremendous global health burden on large patient populations. In this study, we aimed to develop DeepMRD^TR, a deep learning-based medical image interpretation system for the diagnosis of major respiratory diseases based on the automated identification of a wide range of radiological abnormalities through computed tomography (CT) and chest X-ray (CXR) from real-world, large-scale datasets. DeepMRD^TR comprises four networks (two CT-Nets and two CXR-Nets) that exploit contrastive learning to generate pre-training parameters that are fine-tuned on the retrospective dataset collected from a single institution. The performance of DeepMRD^TR was evaluated for abnormality identification and disease diagnosis on data from two different institutions: one was an internal testing dataset from the same institution as the training data and the second was collected from an external institution to evaluate the model generalizability and robustness to an unrelated population dataset. In such a difficult multi-class diagnosis task, our system achieved the average area under the receiver operating characteristic curve (AUC) of 0.856 (95% confidence interval (CI):0.843–0.868) and 0.841 (95%CI:0.832–0.887) for abnormality identification, and 0.900 (95%CI:0.872–0.958) and 0.866 (95%CI:0.832–0.887) for major respiratory diseases’ diagnosis on CT and CXR datasets, respectively. Furthermore, to achieve a clinically actionable diagnosis, we deployed a preliminary version of DeepMRD^TR into the clinical workflow, which was performed on par with senior experts in disease diagnosis, with an AUC of 0.890 and a Cohen’s k of 0.746–0.877 at a reasonable timescale; these findings demonstrate the potential to accelerate the medical workflow to facilitate early diagnosis as a triage tool for respiratory diseases which supports improved clinical diagnoses and decision-making.

The Application of Artificial Intelligence in the Diagnosis and Drug Resistance Prediction of Pulmonary Tuberculosis

With the increasing incidence and mortality of pulmonary tuberculosis, in addition to tough and controversial disease management, time-wasting and resource-limited conventional approaches to the diagnosis and differential diagnosis of tuberculosis are still awkward issues, especially in countries with high tuberculosis burden and backwardness. In the meantime, the climbing proportion of drug-resistant tuberculosis poses a significant hazard to public health. Thus, auxiliary diagnostic tools with higher efficiency and accuracy are urgently required. Artificial intelligence (AI), which is not new but has recently grown in popularity, provides researchers with opportunities and technical underpinnings to develop novel, precise, rapid, and automated implements for pulmonary tuberculosis care, including but not limited to tuberculosis detection. In this review, we aimed to introduce representative AI methods, focusing on deep learning and radiomics, followed by definite descriptions of the state-of-the-art AI models developed using medical images and genetic data to detect pulmonary tuberculosis, distinguish the infection from other pulmonary diseases, and identify drug resistance of tuberculosis, with the purpose of assisting physicians in deciding the appropriate therapeutic schedule in the early stage of the disease. We also enumerated the challenges in maximizing the impact of AI in this field such as generalization and clinical utility of the deep learning models.