Deep learning for the prediction of early on-treatment response in metastatic colorectal cancer from serial medical imaging

An Integrated Radiopathomics Machine Learning Model to Predict Pathological Response to Preoperative Chemotherapy in Gastric Cancer

RATIONALE AND OBJECTIVES

Accurately predicting the pathological response to chemotherapy before treatment is important for selecting the appropriate treatment groups, formulating individualized treatment plans, and improving the survival rates of patients with gastric cancer (GC).

METHODS

We retrospectively enrolled 151 patients diagnosed with GC who underwent preoperative chemotherapy and surgical resection at the Affiliated Hospital of Qingdao University between January 2015 and June 2023. Both pretreatment-enhanced computer technology images and whole slide images of pathological hematoxylin and eosin-stained sections were available for each patient. The image features were extracted and used to construct an ensemble radiopathomics machine learning model. In addition, a nomogram was developed by combining the imaging features and clinical characteristics.

RESULTS

In total, 962 radiomics and 999 pathomics signatures were extracted from 106 patients in the training cohort. A fusion radiopathomics model was constructed using 13 radiomics and 5 pathomics signatures. The fusion model showed favorable performance compared to single-omics models, with an area under the curve (AUC) of 0.789 in the validation cohort. Moreover, a combined radiopathomics nomogram (RPN) was developed based on radiopathomics features and the Borrmann type, which is a classification method for advanced GC according to tumor growth pattern and gross morphology. The RPN showed superior predictive performance in the training (AUC 0.880) and validation cohorts (AUC 0.797). The decision curve analysis showed that RPN could provide favorable clinical benefits to patients with GC.

CONCLUSIONS

RPN was able to predict the pathological response to preoperative chemotherapy with high accuracy, and therefore provides a novel tool for personalized treatment of GC.

Improving the prediction of patient survival with the aid of residual convolutional neural network (ResNet) in colorectal cancer with unresectable liver metastases treated with bevacizumab-based chemotherapy

BACKGROUND

To verify overall survival predictions made with residual convolutional neural network-determined morphological response (ResNet-MR) in patients with unresectable synchronous liver-only metastatic colorectal cancer (mCRC) treated with bevacizumab-based chemotherapy (BBC).

METHODS

A retrospective review of liver-only mCRC patients treated with BBC from December 2011 to Apr 2021 was performed. Patients who had metachronous liver metastases or received locoregional treatment before the initiation of BBC were excluded. The percentage of downstaging to curative treatment and overall survival (OS) were recorded. Two abdominal radiologists evaluated portal venous phase CT images based on the morphological criteria and divided the images into Groups 1, 2, and 3. These images were used to establish the radiologists-determined morphological response (RD-MR), which classified patients into responders and non-responders based on the morphological change 3 months after the initiation of BBC. Then, the Group 1 and 3 images classified by the radiologists were input into ResNet as the training dataset. The trained ResNet then redivided the Group 2 images into Groups 1, 2 and 3. The ResNet-MR was determined on the basis of these redivided images and the initial Group 1 and 3 images classified by the radiologists.

RESULTS

Eighty-four patients were included in this study (53 males and 31 females, with a median age of 60.0 years). The follow-up time ranged from 10 to 86 months. A total of 407 CT images were input into ResNet as the training dataset. Both RD-MR and ResNet-MR correlated with OS (p value = 0.0167 and 0.0225, respectively). Regarding discriminatory ability for mortality, ResNet-MR had higher area under curve than RD-MR at both 1 year and 2 years and showed a significant difference in discriminatory ability (p-value = 0.0321) at 2 years. RD-MR classified 28 patients (33.3%) as responders, and ResNet-MR classified an additional 16 patients (19.0%) as responders; these 16 patients had longer OS than the remaining non-responders in the RD-MR group (27.49 versus 21.20 months, p value = 0.043) and had a higher percentage of downstaging (37.5% versus 17.5%, p value = 0.1610).

CONCLUSIONS

In CRC patients with liver metastases treated with BBC, prediction of survival can be improved with the aid of ResNet, enabling optimized individualized treatment.

NACNet: A histology context-aware transformer graph convolution network for predicting treatment response to neoadjuvant chemotherapy in Triple Negative Breast Cancer

Neoadjuvant chemotherapy (NAC) response prediction for triple negative breast cancer (TNBC) patients is a challenging task clinically as it requires understanding complex histology interactions within the tumor microenvironment (TME). Digital whole slide images (WSIs) capture detailed tissue information, but their giga-pixel size necessitates computational methods based on multiple instance learning, which typically analyze small, isolated image tiles without the spatial context of the TME. To address this limitation and incorporate TME spatial histology interactions in predicting NAC response for TNBC patients, we developed a histology context-aware transformer graph convolution network (NACNet). Our deep learning method identifies the histopathological labels on individual image tiles from WSIs, constructs a spatial TME graph, and represents each node with features derived from tissue texture and social network analysis. It predicts NAC response using a transformer graph convolution network model enhanced with graph isomorphism network layers. We evaluate our method with WSIs of a cohort of TNBC patient (N=105) and compared its performance with multiple state-of-the-art machine learning and deep learning models, including both graph and non-graph approaches. Our NACNet achieves 90.0% accuracy, 96.0% sensitivity, 88.0% specificity, and an AUC of 0.82, through eight-fold cross-validation, outperforming baseline models. These comprehensive experimental results suggest that NACNet holds strong potential for stratifying TNBC patients by NAC response, thereby helping to prevent overtreatment, improve patient quality of life, reduce treatment cost, and enhance clinical outcomes, marking an important advancement toward personalized breast cancer treatment.

Advancements and Challenges in the Image-Based Diagnosis of Lung and Colon Cancer: A Comprehensive Review

Image-based diagnosis has become a crucial tool in the identification and management of various cancers, particularly lung and colon cancer. This review delves into the latest advancements and ongoing challenges in the field, with a focus on deep learning, machine learning, and image processing techniques applied to X-rays, CT scans, and histopathological images. Significant progress has been made in imaging technologies like computed tomography (CT), magnetic resonance imaging (MRI), and positron emission tomography (PET), which, when combined with machine learning and artificial intelligence (AI) methodologies, have greatly enhanced the accuracy of cancer detection and characterization. These advances have enabled early detection, more precise tumor localization, personalized treatment plans, and overall improved patient outcomes. However, despite these improvements, challenges persist. Variability in image interpretation, the lack of standardized diagnostic protocols, unequal access to advanced imaging technologies, and concerns over data privacy and security within AI-based systems remain major obstacles. Furthermore, integrating imaging data with broader clinical information is crucial to achieving a more comprehensive approach to cancer diagnosis and treatment. This review provides valuable insights into the recent developments and challenges in image-based diagnosis for lung and colon cancers, underscoring both the remarkable progress and the hurdles that still need to be overcome to optimize cancer care.

Integrating machine learning and artificial intelligence in life-course epidemiology: pathways to innovative public health solutions

The integration of machine learning (ML) and artificial intelligence (AI) techniques in life-course epidemiology offers remarkable opportunities to advance our understanding of the complex interplay between biological, social, and environmental factors that shape health trajectories across the lifespan. This perspective summarizes the current applications, discusses future potential and challenges, and provides recommendations for harnessing ML and AI technologies to develop innovative public health solutions. ML and AI have been increasingly applied in epidemiological studies, demonstrating their ability to handle large, complex datasets, identify intricate patterns and associations, integrate multiple and multimodal data types, improve predictive accuracy, and enhance causal inference methods. In life-course epidemiology, these techniques can help identify sensitive periods and critical windows for intervention, model complex interactions between risk factors, predict individual and population-level disease risk trajectories, and strengthen causal inference in observational studies. By leveraging the five principles of life-course research proposed by Elder and Shanahan-lifespan development, agency, time and place, timing, and linked lives-we discuss a framework for applying ML and AI to uncover novel insights and inform targeted interventions. However, the successful integration of these technologies faces challenges related to data quality, model interpretability, bias, privacy, and equity. To fully realize the potential of ML and AI in life-course epidemiology, fostering interdisciplinary collaborations, developing standardized guidelines, advocating for their integration in public health decision-making, prioritizing fairness, and investing in training and capacity building are essential. By responsibly harnessing the power of ML and AI, we can take significant steps towards creating healthier and more equitable futures across the life course.

Multiregional dynamic contrast-enhanced MRI-based integrated system for predicting pathological complete response of axillary lymph node to neoadjuvant chemotherapy in breast cancer: multicentre study

BACKGROUND

The accurate evaluation of axillary lymph node (ALN) response to neoadjuvant chemotherapy (NAC) in breast cancer holds great value. This study aimed to develop an artificial intelligence system utilising multiregional dynamic contrast-enhanced MRI (DCE-MRI) and clinicopathological characteristics to predict axillary pathological complete response (pCR) after NAC in breast cancer.

METHODS

This study included retrospective and prospective datasets from six medical centres in China between May 2018 and December 2023. A fully automated integrated system based on deep learning (FAIS-DL) was built to perform tumour and ALN segmentation and axillary pCR prediction sequentially. The predictive performance of FAIS-DL was assessed using the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, and specificity. RNA sequencing analysis were conducted on 45 patients to explore the biological basis of FAIS-DL.

FINDINGS

1145 patients (mean age, 50 years ±10 [SD]) were evaluated. Among these patients, 506 were in the training and validation sets (axillary pCR rate of 40.3%), 127 in the internal test set (axillary pCR rate of 37.8%), 414 in the pooled external test set (axillary pCR rate of 48.8%), and 98 in the prospective test set (axillary pCR rate of 43.9%). For predicting axillary pCR, FAIS-DL achieved AUCs of 0.95, 0.93, and 0.94 in the internal test set, pooled external test set, and prospective test set, respectively, which were also significantly higher than those of the clinical model and deep learning models based on single-regional DCE-MRI (all P < 0.05, DeLong test). In the pooled external and prospective test sets, the FAIS-DL decreased the unnecessary axillary lymph node dissection rate from 47.9% to 6.8%, and increased the benefit rate from 52.2% to 86.5%. RNA sequencing analysis revealed that high FAIS-DL scores were associated with the upregulation of immune-mediated genes and pathways.

INTERPRETATION

FAIS-DL has demonstrated satisfactory performance in predicting axillary pCR, which may guide the formulation of personalised treatment regimens for patients with breast cancer in clinical practice.

FUNDING

This study was supported by the National Natural Science Foundation of China (82371933), National Natural Science Foundation of Shandong Province of China (ZR2021MH120), Mount Taishan Scholars and Young Experts Program (tsqn202211378), Key Projects of China Medicine Education Association (2022KTM030), China Postdoctoral Science Foundation (314730), and Beijing Postdoctoral Research Foundation (2023-zz-012).

Automated Lugano Metabolic Response Assessment in 18F-Fluorodeoxyglucose-Avid Non-Hodgkin Lymphoma With Deep Learning on 18F-Fluorodeoxyglucose-Positron Emission Tomography

PURPOSE

Artificial intelligence can reduce the time used by physicians on radiological assessments. For 18F-fluorodeoxyglucose-avid lymphomas, obtaining complete metabolic response (CMR) by end of treatment is prognostic.

METHODS

Here, we present a deep learning-based algorithm for fully automated treatment response assessments according to the Lugano 2014 classification. The proposed four-stage method, trained on a multicountry clinical trial (ClinicalTrials.gov identifier: NCT01287741) and tested in three independent multicenter and multicountry test sets on different non-Hodgkin lymphoma subtypes and different lines of treatment (ClinicalTrials.gov identifiers NCT02257567, NCT02500407; 20% holdout in ClinicalTrials.gov identifier NCT01287741), outputs the detected lesions at baseline and follow-up to enable focused radiologist review.

RESULTS

The method's response assessment achieved high agreement with the adjudicated radiologic responses (eg, agreement for overall response assessment of 93%, 87%, and 85% in ClinicalTrials.gov identifiers NCT01287741, NCT02500407, and NCT02257567, respectively) similar to inter-radiologist agreement and was strongly prognostic of outcomes with a trend toward higher accuracy for death risk than adjudicated radiologic responses (hazard ratio for end of treatment by-model CMR of 0.123, 0.054, and 0.205 in ClinicalTrials.gov identifiers NCT01287741, NCT02500407, and NCT02257567, compared with, respectively, 0.226, 0.292, and 0.272 for CMR by the adjudicated responses). Furthermore, a radiologist review of the algorithm's assessments was conducted. The radiologist median review time was 1.38 minutes/assessment, and no statistically significant differences were observed in the level of agreement of the radiologist with the model's response compared with the level of agreement of the radiologist with the adjudicated responses.

CONCLUSION

These results suggest that the proposed method can be incorporated into radiologic response assessment workflows in cancer imaging for significant time savings and with performance similar to trained medical experts.

Potential application of artificial intelligence in cancer therapy

PURPOSE OF REVIEW

This review underscores the critical role and challenges associated with the widespread adoption of artificial intelligence in cancer care to enhance disease management, streamline clinical processes, optimize data retrieval of health information, and generate and synthesize evidence.

RECENT FINDINGS

Advancements in artificial intelligence models and the development of digital biomarkers and diagnostics are applicable across the cancer continuum from early detection to survivorship care. Additionally, generative artificial intelligence has promised to streamline clinical documentation and patient communications, generate structured data for clinical trial matching, automate cancer registries, and facilitate advanced clinical decision support. Widespread adoption of artificial intelligence has been slow because of concerns about data diversity and data shift, model reliability and algorithm bias, legal oversight, and high information technology and infrastructure costs.

SUMMARY

Artificial intelligence models have significant potential to transform cancer care. Efforts are underway to deploy artificial intelligence models in the cancer practice, evaluate their clinical impact, and enhance their fairness and explainability. Standardized guidelines for the ethical integration of artificial intelligence models in cancer care pathways and clinical operations are needed. Clear governance and oversight will be necessary to gain trust in artificial intelligence-assisted cancer care by clinicians, scientists, and patients.

Predicting gastric cancer response to anti-HER2 therapy or anti-HER2 combined immunotherapy based on multi-modal data

The sole use of single modality data often fails to capture the complex heterogeneity among patients, including the variability in resistance to anti-HER2 therapy and outcomes of combined treatment regimens, for the treatment of HER2-positive gastric cancer (GC). This modality deficit has not been fully considered in many studies. Furthermore, the application of artificial intelligence in predicting the treatment response, particularly in complex diseases such as GC, is still in its infancy. Therefore, this study aimed to use a comprehensive analytic approach to accurately predict treatment responses to anti-HER2 therapy or anti-HER2 combined immunotherapy in patients with HER2-positive GC. We collected multi-modal data, comprising radiology, pathology, and clinical information from a cohort of 429 patients: 310 treated with anti-HER2 therapy and 119 treated with a combination of anti-HER2 and anti-PD-1/PD-L1 inhibitors immunotherapy. We introduced a deep learning model, called the Multi-Modal model (MuMo), that integrates these data to make precise treatment response predictions. MuMo achieved an area under the curve score of 0.821 for anti-HER2 therapy and 0.914 for combined immunotherapy. Moreover, patients classified as low-risk by MuMo exhibited significantly prolonged progression-free survival and overall survival (log-rank test, P < 0.05). These findings not only highlight the significance of multi-modal data analysis in enhancing treatment evaluation and personalized medicine for HER2-positive gastric cancer, but also the potential and clinical value of our model.

Can supervised deep learning architecture outperform autoencoders in building propensity score models for matching?

PURPOSE

Propensity score matching is vital in epidemiological studies using observational data, yet its estimates relies on correct model-specification. This study assesses supervised deep learning models and unsupervised autoencoders for propensity score estimation, comparing them with traditional methods for bias and variance accuracy in treatment effect estimations.

METHODS

Utilizing a plasmode simulation based on the Right Heart Catheterization dataset, under a variety of settings, we evaluated (1) a supervised deep learning architecture and (2) an unsupervised autoencoder, alongside two traditional methods: logistic regression and a spline-based method in estimating propensity scores for matching. Performance metrics included bias, standard errors, and coverage probability. The analysis was also extended to real-world data, with estimates compared to those obtained via a double robust approach.

RESULTS

The analysis revealed that supervised deep learning models outperformed unsupervised autoencoders in variance estimation while maintaining comparable levels of bias. These results were supported by analyses of real-world data, where the supervised model's estimates closely matched those derived from conventional methods. Additionally, deep learning models performed well compared to traditional methods in settings where exposure was rare.

CONCLUSION

Supervised deep learning models hold promise in refining propensity score estimations in epidemiological research, offering nuanced confounder adjustment, especially in complex datasets. We endorse integrating supervised deep learning into epidemiological research and share reproducible codes for widespread use and methodological transparency.

Radiographic Findings Associated With Mild Hip Dysplasia in 3869 Patients Using a Deep Learning Measurement Tool

Background

Hip dysplasia is considered one of the leading etiologies contributing to hip degeneration and the eventual need for total hip arthroplasty (THA). We validated a deep learning (DL) algorithm to measure angles relevant to hip dysplasia and applied this algorithm to determine the prevalence of dysplasia in a large population based on incremental radiographic cutoffs.

Methods

Patients from the Osteoarthritis Initiative with anteroposterior pelvis radiographs and without previous THAs were included. A DL algorithm automated 3 angles associated with hip dysplasia: modified lateral center-edge angle (LCEA), Tönnis angle, and modified Sharp angle. The algorithm was validated against manual measurements, and all angles were measured in a cohort of 3869 patients (61.2 ± 9.2 years, 57.1% female). The percentile distributions and prevalence of dysplastic hips were analyzed using each angle.

Results

The algorithm had no significant difference (P > .05) in measurements (paired difference: 0.3°-0.7°) against readers and had excellent agreement for dysplasia classification (kappa = 0.78-0.88). In 140 minutes, 23,214 measurements were automated for 3869 patients. LCEA and Sharp angles were higher and the Tönnis angle was lower (P < .01) in females. The dysplastic hip prevalence varied from 2.5% to 20% utilizing the following cutoffs: 17.3°-25.5° (LCEA), 9.4°-15.6° (Tönnis), and 41.3°-45.9° (Sharp).

Conclusions

A DL algorithm was developed to measure and classify hips with mild hip dysplasia. The reported prevalence of dysplasia in a large patient cohort was dependent on both the measurement and threshold, with 12.4% of patients having dysplasia radiographic indices indicative of higher THA risk.

Hybrid deep learning models for the screening of Diabetic Macular Edema in optical coherence tomography volumes

Several studies published so far used highly selective image datasets from unclear sources to train computer vision models and that may lead to overestimated results, while those studies conducted in real-life remain scarce. To avoid image selection bias, we stacked convolutional and recurrent neural networks (CNN-RNN) to analyze complete optical coherence tomography (OCT) cubes in a row and predict diabetic macular edema (DME), in a real-world diabetic retinopathy screening program. A retrospective cohort study was carried out. Throughout 4-years, 5314 OCT cubes from 4408 subjects who attended to the diabetic retinopathy (DR) screening program were included. We arranged twenty-two (22) pre-trained CNNs in parallel with a bidirectional RNN layer stacked at the bottom, allowing the model to make a prediction for the whole OCT cube. The staff of retina experts built a ground truth of DME later used to train a set of these CNN-RNN models with different configurations. For each trained CNN-RNN model, we performed threshold tuning to find the optimal cut-off point for binary classification of DME. Finally, the best models were selected according to sensitivity, specificity, and area under the receiver operating characteristics curve (AUROC) with their 95% confidence intervals (95%CI). An ensemble of the best models was also explored. 5188 cubes were non-DME and 126 were DME. Three models achieved an AUROC of 0.94. Among these, sensitivity, and specificity (95%CI) ranged from 84.1-90.5 and 89.7-93.3, respectively, at threshold 1, from 89.7-92.1 and 80-83.1 at threshold 2, and from 80.2-81 and 93.8-97, at threshold 3. The ensemble model improved these results, and lower specificity was observed among subjects with sight-threatening DR. Analysis by age, gender, or grade of DME did not vary the performance of the models. CNN-RNN models showed high diagnostic accuracy for detecting DME in a real-world setting. This engine allowed us to detect extra-foveal DMEs commonly overlooked in other studies, and showed potential for application as the first filter of non-referable patients in an outpatient center within a population-based DR screening program, otherwise ended up in specialized care.

Longitudinal risk prediction for pediatric glioma with temporal deep learning

Pediatric glioma recurrence can cause morbidity and mortality; however, recurrence pattern and severity are heterogeneous and challenging to predict with established clinical and genomic markers. Resultingly, almost all children undergo frequent, long-term, magnetic resonance (MR) brain surveillance regardless of individual recurrence risk. Deep learning analysis of longitudinal MR may be an effective approach for improving individualized recurrence prediction in gliomas and other cancers but has thus far been infeasible with current frameworks. Here, we propose a self-supervised, deep learning approach to longitudinal medical imaging analysis, temporal learning, that models the spatiotemporal information from a patient's current and prior brain MRs to predict future recurrence. We apply temporal learning to pediatric glioma surveillance imaging for 715 patients (3,994 scans) from four distinct clinical settings. We find that longitudinal imaging analysis with temporal learning improves recurrence prediction performance by up to 41% compared to traditional approaches, with improvements in performance in both low- and high-grade glioma. We find that recurrence prediction accuracy increases incrementally with the number of historical scans available per patient. Temporal deep learning may enable point-of-care decision-support for pediatric brain tumors and be adaptable more broadly to patients with other cancers and chronic diseases undergoing surveillance imaging.

Longitudinal Image Data for Outcome Modeling

In oncology, medical imaging is crucial for diagnosis, treatment planning and therapy execution. Treatment responses can be complex and varied and are known to involve factors of treatment, patient characteristics and tumor microenvironment. Longitudinal image analysis is able to track temporal changes, aiding in disease monitoring, treatment evaluation, and outcome prediction. This allows for the enhancement of personalized medicine. However, analyzing longitudinal 2D and 3D images presents unique challenges, including image registration, reliable segmentation, dealing with variable imaging intervals, and sparse data. This review presents an overview of techniques and methodologies in longitudinal image analysis, with a primary focus on outcome modeling in radiation oncology.

Artificial Intelligence in Colorectal Cancer: From Patient Screening over Tailoring Treatment Decisions to Identification of Novel Biomarkers

BACKGROUND

Artificial intelligence (AI) is increasingly entering and transforming not only medical research but also clinical practice. In the last 10 years, new AI methods have enabled computers to perform visual tasks, reaching high performance and thereby potentially supporting and even outperforming human experts. This is in particular relevant for colorectal cancer (CRC), which is the 3rd most common cancer type in general, as along the CRC patient journey many complex visual tasks need to be performed: from endoscopy over imaging to histopathology; the screening, diagnosis, and treatment of CRC involve visual image analysis tasks.

SUMMARY

In all these clinical areas, AI models have shown promising results by supporting physicians, improving accuracy, and providing new biological insights and biomarkers. By predicting prognostic and predictive biomarkers from routine images/slides, AI models could lead to an improved patient stratification for precision oncology approaches in the near future. Moreover, it is conceivable that AI models, in particular together with innovative techniques such as single-cell or spatial profiling, could help identify novel clinically as well as biologically meaningful biomarkers that could pave the way to new therapeutic approaches.

KEY MESSAGES

Here, we give a comprehensive overview of AI in colorectal cancer, describing and discussing these developments as well as the next steps which need to be taken to incorporate AI methods more broadly into the clinical care of CRC.

A guide to artificial intelligence for cancer researchers

Artificial intelligence (AI) has been commoditized. It has evolved from a specialty resource to a readily accessible tool for cancer researchers. AI-based tools can boost research productivity in daily workflows, but can also extract hidden information from existing data, thereby enabling new scientific discoveries. Building a basic literacy in these tools is useful for every cancer researcher. Researchers with a traditional biological science focus can use AI-based tools through off-the-shelf software, whereas those who are more computationally inclined can develop their own AI-based software pipelines. In this article, we provide a practical guide for non-computational cancer researchers to understand how AI-based tools can benefit them. We convey general principles of AI for applications in image analysis, natural language processing and drug discovery. In addition, we give examples of how non-computational researchers can get started on the journey to productively use AI in their own work.

Tumor Size Is Not Everything: Advancing Radiomics as a Precision Medicine Biomarker in Oncology Drug Development and Clinical Care. A Report of a Multidisciplinary Workshop Coordinated by the RECIST Working Group

Radiomics, the science of extracting quantifiable data from routine medical images, is a powerful tool that has many potential applications in oncology. The Response Evaluation Criteria in Solid Tumors Working Group (RWG) held a workshop in May 2022, which brought together various stakeholders to discuss the potential role of radiomics in oncology drug development and clinical trials, particularly with respect to response assessment. This article summarizes the results of that workshop, reviewing radiomics for the practicing oncologist and highlighting the work that needs to be done to move forward the incorporation of radiomics into clinical trials.

Deep Learning-Based Multiparametric MRI Model for Preoperative T-Stage in Rectal Cancer

BACKGROUND

Conventional MRI staging can be challenging in the preoperative assessment of rectal cancer. Deep learning methods based on MRI have shown promise in cancer diagnosis and prognostication. However, the value of deep learning in rectal cancer T-staging is unclear.

PURPOSE

To develop a deep learning model based on preoperative multiparametric MRI for evaluation of rectal cancer and to investigate its potential to improve T-staging accuracy.

STUDY TYPE

Retrospective.

POPULATION

After cross-validation, 260 patients (123 with T-stage T1-2 and 134 with T-stage T3-4) with histopathologically confirmed rectal cancer were randomly divided to the training (N = 208) and test sets (N = 52).

FIELD STRENGTH/SEQUENCE

3.0 T/Dynamic contrast enhanced (DCE), T2-weighted imaging (T2W), and diffusion-weighted imaging (DWI).

ASSESSMENT

The deep learning (DL) model of multiparametric (DCE, T2W, and DWI) convolutional neural network were constructed for evaluating preoperative diagnosis. The pathological findings served as the reference standard for T-stage. For comparison, the single parameter DL-model, a logistic regression model composed of clinical features and subjective assessment of radiologists were used.

STATISTICAL TESTS

The receiver operating characteristic curve (ROC) was used to evaluate the models, the Fleiss' kappa for the intercorrelation coefficients, and DeLong test for compare the diagnostic performance of ROCs. P-values less than 0.05 were considered statistically significant.

RESULTS

The Area Under Curve (AUC) of the multiparametric DL-model was 0.854, which was significantly higher than the radiologist's assessment (AUC = 0.678), clinical model (AUC = 0.747), and the single parameter DL-models including T2W-model (AUC = 0.735), DWI-model (AUC = 0.759), and DCE-model (AUC = 0.789).

DATA CONCLUSION

In the evaluation of rectal cancer patients, the proposed multiparametric DL-model outperformed the radiologist's assessment, the clinical model as well as the single parameter models. The multiparametric DL-model has the potential to assist clinicians by providing more reliable and precise preoperative T staging diagnosis.

EVIDENCE LEVEL

3 TECHNICAL EFFICACY: Stage 2.

Combined clinical variable and radiomics of post-treatment total body scan for prediction of successful I-131 ablation in low-risk papillary thyroid carcinoma patients

To explore the feasibility of combined radiomics of post-treatment I-131 total body scan (TBS) and clinical parameter to predict successful ablation in low-risk papillary thyroid carcinoma (PTC) patients. Data of low-risk PTC patients who underwent total/near total thyroidectomy and I-131 ablation 30 mCi between April 2015 and July 2021 were retrospectively reviewed. The clinical factors studied included age, sex, and pre-ablative serum thyroglobulin (Tg). Radiomic features were extracted via PyRadiomics, and radiomic feature selection was performed. The predictive performance for successful ablation of the clinical parameter, radiomic, and combined models (radiomics combined with clinical parameter) was calculated using the area under the receiver operating characteristic curve (AUC). One hundred and thirty patients were included. Successful ablation was achieved in 77 patients (59.2%). The mean pre-ablative Tg in the unsuccessful group (15.50 ± 18.04 ng/ml) was statistically significantly higher than those in the successful ablation group (7.12 ± 7.15 ng/ml). The clinical parameter, radiomic, and combined models produced AUCs of 0.66, 0.77, and 0.87 in the training sets, and 0.65, 0.69, and 0.78 in the validation sets, respectively. The combined model produced a significantly higher AUC than that of the clinical parameter (p < 0.05). Radiomic analysis of the post-treatment TBS combined with pre-ablative serum Tg showed a significant improvement in the predictive performance of successful ablation in low-risk PTC patients compared to the use of clinical parameter alone.Thai Clinical Trials Registry TCTR identification number is TCTR20230816004 ( https://www.thaiclinicaltrials.org/show/TCTR20230816004 ).

Deep learning model based on multi-lesion and time series CT images for predicting the benefits from anti-HER2 targeted therapy in stage IV gastric cancer

OBJECTIVE

To develop and validate a deep learning model based on multi-lesion and time series CT images in predicting overall survival (OS) in patients with stage IV gastric cancer (GC) receiving anti-HER2 targeted therapy.

METHODS

A total of 207 patients were enrolled in this multicenter study, with 137 patients for retrospective training and internal validation, 33 patients for prospective validation, and 37 patients for external validation. All patients received anti-HER2 targeted therapy and underwent pre- and post-treatment CT scans (baseline and at least one follow-up). The proposed deep learning model evaluated the multiple lesions in time series CT images to predict risk probabilities. We further evaluated and validated the risk score of the nomogram combining a two-follow-up lesion-based deep learning model (LDLM-2F), tumor markers, and clinical information for predicting the benefits from treatment (Nomo-LDLM-2F).

RESULTS

In the internal validation and prospective cohorts, the one-year AUCs for Nomo-LDLM-2F using the time series medical images and tumor markers were 0.894 (0.728-1.000) and 0.809 (0.561-1.000), respectively. In the external validation cohort, the one-year AUC of Nomo-LDLM-2F without tumor markers was 0.771 (0.510-1.000). Patients with a low Nomo-LDLM-2F score derived survival benefits from anti-HER2 targeted therapy significantly compared to those with a high Nomo-LDLM-2F score (all p < 0.05).

CONCLUSION

The Nomo-LDLM-2F score derived from multi-lesion and time series CT images holds promise for the effective readout of OS probability in patients with HER2-positive stage IV GC receiving anti-HER2 therapy.

CRITICAL RELEVANCE STATEMENT

The deep learning model using baseline and early follow-up CT images aims to predict OS in patients with stage IV gastric cancer receiving anti-HER2 targeted therapy. This model highlights the spatiotemporal heterogeneity of stage IV GC, assisting clinicians in the early evaluation of the efficacy of anti-HER2 therapy.

KEY POINTS

• Multi-lesion and time series model revealed the spatiotemporal heterogeneity in anti-HER2 therapy. • The Nomo-LDLM-2F score was a valuable prognostic marker for anti-HER2 therapy. • CT-based deep learning model incorporating time-series tumor markers improved performance.

Treatment response prediction of neoadjuvant chemotherapy for rectal cancer by deep learning of colonoscopy images

In current clinical practice, several treatment methods, including neoadjuvant therapy, are being developed to improve overall survival or local recurrence rates for locally advanced rectal cancer. The response to neoadjuvant therapy is usually evaluated using imaging data collected before and after preoperative treatment or postsurgical pathological diagnosis. However, there is a need to accurately predict the response to preoperative treatment before treatment is administered. The present study used a deep learning network to examine colonoscopy images and construct a model to predict the response of rectal cancer to neoadjuvant chemotherapy. A total of 53 patients who underwent preoperative chemotherapy followed by radical resection for advanced rectal cancer at the Osaka University Hospital between January 2011 and August 2019 were retrospectively analyzed. A convolutional neural network model was constructed using 403 images from 43 patients as the learning set. The diagnostic accuracy of the deep learning model was evaluated using 84 images from 10 patients as the validation set. The model demonstrated a sensitivity, specificity, accuracy, positive predictive value and area under the curve of 77.6% (38/49), 62.9% (22/33), 71.4% (60/84), 74.5% (38/51) and 0.713, respectively, in predicting a poor response to neoadjuvant therapy. Overall, deep learning of colonoscopy images may contribute to an accurate prediction of the response of rectal cancer to neoadjuvant chemotherapy.

Response prediction by mutation- or methylation-specific detection of ctDNA dynamics in pretreated metastatic colorectal cancer

Background

Serial analysis of circulating tumor DNA (ctDNA) levels is a promising tool for both relapse prediction in the curative setting, as well as predicting clinical benefit from systemic treatment in metastasic colorectal cancer (mCRC). Most data in this context are derived from treatment naive patients.

Objective

To predict progressive disease (PD) as early as possible through monitoring of changes in ctDNA levels during systemic treatment in pretreated patients with mCRC.

Design

A prospective, single-center, observational study.

Methods

Patients treated beyond first-line were prospectively included between February 2020 and September 2021. Blood for ctDNA detection was taken before every treatment cycle from start of treatment until first restaging by CT-scan. ctDNA was detected by mutation- (mut-ctDNA) and methylation-specific ddPCR. Receiver Operating Characteristic (ROC)-analysis was used to describe sensitivity and specificity for prediction of PD at restaging for all time points.

Results

A total of 42 patients were included who all carried a mutation in tumor tissue. Detection rate of mut-ctDNA was 88.1% and 74.4% for meth-ctDNA. Absolute ctDNA levels before treatment were prognostic in terms of overall survival. Levels of ctDNA were significantly higher in patients with PD at restaging. Median time from start of treatment to restaging was 93 days (95% CI 88.8-96). After a median of 19 days of treatment (95% CI 16.1-20.2), a decline of either mutation- or methylation-specific ctDNA levels of ⩽58% predicted PD at restaging with a sensitivity/specificity of 92.9/85.7% and 85.7/100%, respectively. Median time to restaging was 66 days (95% CI 56.8-75.2). There was no significant increase of sensitivity/specificity at later time points of ctDNA measurements.

Conclusion

Monitoring early changes of ctDNA levels either by mut- or meth-ctDNA allows for early prediction of PD in pretreated patients with mCRC. This has the potential to complement RECIST-based treatment assessment with the aim to switch potentially insufficient treatments as early as possible, which is of particular interest in higher treatment lines.

Predicting Neoadjuvant Chemotherapy Response and High-Grade Serous Ovarian Cancer From CT Images in Ovarian Cancer with Multitask Deep Learning: A Multicenter Study

RATIONALE AND OBJECTIVES

Accurate prediction neoadjuvant chemotherapy (NACT) response in ovarian cancer (OC) is essential for personalized medicine. We aimed to develop and validate a deep learning (DL) model based on pretreatment contrast-enhanced CT (CECT) images for predicting NACT responses and classifying high-grade serous ovarian cancer (HGSOC) to identify patients who may benefit from NACT.

MATERIALS AND METHODS

This multicenter study, which contained both retrospective and prospective studies, included consecutive OC patients (n = 757) from three hospitals. Using WHO RECIST 1.1 for the reference standard, a total of 587 women with 1761 images were included in the training and validation sets, 67 women with 201 images were included in the prospective sets, and 103 women with 309 images were included in the external sets. A multitask DL model based on the multiperiod CT image was developed to predict NACT response and HGSOC.

RESULTS

Logistic regression analysis showed that peritoneal invasion, retinal invasion, and inguinal lymph node metastasis were independent predictors. The DL achieved promising segmentation performances with DICEmean= 0.83 (range: 0.78-0.87). For predicting NACT response, the DL model combined with clinical risk factors obtained area under the receiver operating characteristic curve (AUCs) of 0.87 (0.83-0.89), 0.88 (0.86-0.91), 0.86 (0.82-0.89), and 0.79 (0.75-0.82) in the training, validation, prospective, and external sets, respectively. The AUCs were 0.91 (0.87-0.94), 0.89 (0.86-0.91), 0.80 (0.76-0.84), and 0.80 (0.75-0.85) in four sets in HGSOC classification.

CONCLUSION

The multitask DL model developed using multiperiod CT images exhibited a promising performance for predicting NACT response and HGSOC with OC, which could provide valuable information for individualized treatment.

A review on deep learning applications in highly multiplexed tissue imaging data analysis

Since its introduction into the field of oncology, deep learning (DL) has impacted clinical discoveries and biomarker predictions. DL-driven discoveries and predictions in oncology are based on a variety of biological data such as genomics, proteomics, and imaging data. DL-based computational frameworks can predict genetic variant effects on gene expression, as well as protein structures based on amino acid sequences. Furthermore, DL algorithms can capture valuable mechanistic biological information from several spatial "omics" technologies, such as spatial transcriptomics and spatial proteomics. Here, we review the impact that the combination of artificial intelligence (AI) with spatial omics technologies has had on oncology, focusing on DL and its applications in biomedical image analysis, encompassing cell segmentation, cell phenotype identification, cancer prognostication, and therapy prediction. We highlight the advantages of using highly multiplexed images (spatial proteomics data) compared to single-stained, conventional histopathological ("simple") images, as the former can provide deep mechanistic insights that cannot be obtained by the latter, even with the aid of explainable AI. Furthermore, we provide the reader with the advantages/disadvantages of DL-based pipelines used in preprocessing highly multiplexed images (cell segmentation, cell type annotation). Therefore, this review also guides the reader to choose the DL-based pipeline that best fits their data. In conclusion, DL continues to be established as an essential tool in discovering novel biological mechanisms when combined with technologies such as highly multiplexed tissue imaging data. In balance with conventional medical data, its role in clinical routine will become more important, supporting diagnosis and prognosis in oncology, enhancing clinical decision-making, and improving the quality of care for patients. Since its introduction into the field of oncology, deep learning (DL) has impacted clinical discoveries and biomarker predictions. DL-driven discoveries and predictions in oncology are based on a variety of biological data such as genomics, proteomics, and imaging data. DL-based computational frameworks can predict genetic variant effects on gene expression, as well as protein structures based on amino acid sequences. Furthermore, DL algorithms can capture valuable mechanistic biological information from several spatial "omics" technologies, such as spatial transcriptomics and spatial proteomics. Here, we review the impact that the combination of artificial intelligence (AI) with spatial omics technologies has had on oncology, focusing on DL and its applications in biomedical image analysis, encompassing cell segmentation, cell phenotype identification, cancer prognostication, and therapy prediction. We highlight the advantages of using highly multiplexed images (spatial proteomics data) compared to single-stained, conventional histopathological ("simple") images, as the former can provide deep mechanistic insights that cannot be obtained by the latter, even with the aid of explainable AI. Furthermore, we provide the reader with the advantages/disadvantages of the DL-based pipelines used in preprocessing the highly multiplexed images (cell segmentation, cell type annotation). Therefore, this review also guides the reader to choose the DL-based pipeline that best fits their data. In conclusion, DL continues to be established as an essential tool in discovering novel biological mechanisms when combined with technologies such as highly multiplexed tissue imaging data. In balance with conventional medical data, its role in clinical routine will become more important, supporting diagnosis and prognosis in oncology, enhancing clinical decision-making, and improving the quality of care for patients.

Deep learning-enabled fully automated pipeline system for segmentation and classification of single-mass breast lesions using contrast-enhanced mammography: a prospective, multicentre study

Background

Breast cancer is the leading cause of cancer-related deaths in women. However, accurate diagnosis of breast cancer using medical images heavily relies on the experience of radiologists. This study aimed to develop an artificial intelligence model that diagnosed single-mass breast lesions on contrast-enhanced mammography (CEM) for assisting the diagnostic workflow.

Methods

A total of 1912 women with single-mass breast lesions on CEM images before biopsy or surgery were included from June 2017 to October 2022 at three centres in China. Samples were divided into training and validation sets, internal testing set, pooled external testing set, and prospective testing set. A fully automated pipeline system (FAPS) using RefineNet and the Xception + Pyramid pooling module (PPM) was developed to perform the segmentation and classification of breast lesions. The performances of six radiologists and adjustments in Breast Imaging Reporting and Data System (BI-RADS) category 4 under the FAPS-assisted strategy were explored in pooled external and prospective testing sets. The segmentation performance was assessed using the Dice similarity coefficient (DSC), and the classification was assessed using heatmaps, area under the receiver operating characteristic curve (AUC), sensitivity, and specificity. The radiologists' reading time was recorded for comparison with the FAPS. This trial is registered with China Clinical Trial Registration Centre (ChiCTR2200063444).

Findings

The FAPS-based segmentation task achieved DSCs of 0.888 ± 0.101, 0.820 ± 0.148 and 0.837 ± 0.132 in the internal, pooled external and prospective testing sets, respectively. For the classification task, the FAPS achieved AUCs of 0.947 (95% confidence interval [CI]: 0.916-0.978), 0.940 (95% [CI]: 0.894-0.987) and 0.891 (95% [CI]: 0.816-0.945). It outperformed radiologists in terms of classification efficiency based on single lesions (6 s vs 3 min). Moreover, the FAPS-assisted strategy improved the performance of radiologists. BI-RADS category 4 in 12.4% and 13.3% of patients was adjusted in two testing sets with the assistance of FAPS, which may play an important guiding role in the selection of clinical management strategies.

Interpretation

The FAPS based on CEM demonstrated the potential for the segmentation and classification of breast lesions, and had good generalisation ability and clinical applicability.

Funding

This study was supported by the Taishan Scholar Foundation of Shandong Province of China (tsqn202211378), National Natural Science Foundation of China (82001775), Natural Science Foundation of Shandong Province of China (ZR2021MH120), and Special Fund for Breast Disease Research of Shandong Medical Association (YXH2021ZX055).

Deep learning for pancreatic diseases based on endoscopic ultrasound: A systematic review

BACKGROUND AND AIMS

Endoscopic ultrasonography (EUS) is one of the main examinations in pancreatic diseases. A series of the studies reported the application of deep learning (DL)-assisted EUS in the diagnosis of pancreatic diseases. This systematic review is to evaluate the role of DL algorithms in assisting EUS diagnosis of pancreatic diseases.

METHODS

Literature search were conducted in PubMed and Semantic Scholar databases. Studies that developed DL models for pancreatic diseases based on EUS were eligible for inclusion. This review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines and quality assessment of the included studies was performed according to the IJMEDI checklist.

RESULTS

A total of 23 studies were enrolled into this systematic review, which could be categorized into three groups according to computer vision tasks: classification, detection and segmentation. Seventeen studies focused on the classification task, among which five studies developed simple neural network (NN) models while twelve studies constructed convolutional NN (CNN) models. Three studies were concerned the detection task and five studies were the segmentation task, all based on CNN architectures. All models presented in the studies performed well based on EUS images, videos or voice. According to the IJMEDI checklist, six studies were recognized as high-grade quality, with scores beyond 35 points.

CONCLUSIONS

DL algorithms show great potential in EUS images/videos/voice for pancreatic diseases. However, there is room for improvement such as sample sizes, multi-center cooperation, data preprocessing, model interpretability, and code sharing.

From patterns to patients: Advances in clinical machine learning for cancer diagnosis, prognosis, and treatment

Machine learning (ML) is increasingly used in clinical oncology to diagnose cancers, predict patient outcomes, and inform treatment planning. Here, we review recent applications of ML across the clinical oncology workflow. We review how these techniques are applied to medical imaging and to molecular data obtained from liquid and solid tumor biopsies for cancer diagnosis, prognosis, and treatment design. We discuss key considerations in developing ML for the distinct challenges posed by imaging and molecular data. Finally, we examine ML models approved for cancer-related patient usage by regulatory agencies and discuss approaches to improve the clinical usefulness of ML.

A comprehensive regulatory and industry review of modeling and simulation practices in oncology clinical drug development

Exposure-response (E-R) analyses are an integral component in the development of oncology products. Characterizing the relationship between drug exposure metrics and response allows the sponsor to use modeling and simulation to address both internal and external drug development questions (e.g., optimal dose, frequency of administration, dose adjustments for special populations). This white paper is the output of an industry-government collaboration among scientists with broad experience in E-R modeling as part of regulatory submissions. The goal of this white paper is to provide guidance on what the preferred methods for E-R analysis in oncology clinical drug development are and what metrics of exposure should be considered.

Intelligent oncology: The convergence of artificial intelligence and oncology

With increasingly explored ideologies and technologies for potential applications of artificial intelligence (AI) in oncology, we here describe a holistic and structured concept termed intelligent oncology. Intelligent oncology is defined as a cross-disciplinary specialty which integrates oncology, radiology, pathology, molecular biology, multi-omics and computer sciences, aiming to promote cancer prevention, screening, early diagnosis and precision treatment. The development of intelligent oncology has been facilitated by fast AI technology development such as natural language processing, machine/deep learning, computer vision, and robotic process automation. While the concept and applications of intelligent oncology is still in its infancy, and there are still many hurdles and challenges, we are optimistic that it will play a pivotal role for the future of basic, translational and clinical oncology.

Accurate tumor segmentation and treatment outcome prediction with DeepTOP

BACKGROUND

Accurate outcome prediction prior to treatment can facilitate trial design and clinical decision making to achieve better treatment outcome.

METHOD

We developed the DeepTOP tool with deep learning approach for region-of-interest segmentation and clinical outcome prediction using magnetic resonance imaging (MRI). DeepTOP was constructed with an automatic pipeline from tumor segmentation to outcome prediction. In DeepTOP, the segmentation model used U-Net with a codec structure, and the prediction model was built with a three-layer convolutional neural network. In addition, the weight distribution algorithm was developed and applied in the prediction model to optimize the performance of DeepTOP.

RESULTS

A total of 1889 MRI slices from 99 patients in the phase III multicenter randomized clinical trial (NCT01211210) on neoadjuvant treatment for rectal cancer was used to train and validate DeepTOP. We systematically optimized and validated DeepTOP with multiple devised pipelines in the clinical trial, demonstrating a better performance than other competitive algorithms in accurate tumor segmentation (Dice coefficient: 0.79; IoU: 0.75; slice-specific sensitivity: 0.98) and predicting pathological complete response to chemo/radiotherapy (accuracy: 0.789; specificity: 0.725; and sensitivity: 0.812). DeepTOP is a deep learning tool that could avoid manual labeling and feature extraction and realize automatic tumor segmentation and treatment outcome prediction by using the original MRI images.

CONCLUSION

DeepTOP is open to provide a tractable framework for the development of other segmentation and predicting tools in clinical settings. DeepTOP-based tumor assessment can provide a reference for clinical decision making and facilitate imaging marker-driven trial design.

The Applications of Artificial Intelligence in Digestive System Neoplasms: A Review

Importance

Digestive system neoplasms (DSNs) are the leading cause of cancer-related mortality with a 5-year survival rate of less than 20%. Subjective evaluation of medical images including endoscopic images, whole slide images, computed tomography images, and magnetic resonance images plays a vital role in the clinical practice of DSNs, but with limited performance and increased workload of radiologists or pathologists. The application of artificial intelligence (AI) in medical image analysis holds promise to augment the visual interpretation of medical images, which could not only automate the complicated evaluation process but also convert medical images into quantitative imaging features that associated with tumor heterogeneity.

Highlights

We briefly introduce the methodology of AI for medical image analysis and then review its clinical applications including clinical auxiliary diagnosis, assessment of treatment response, and prognosis prediction on 4 typical DSNs including esophageal cancer, gastric cancer, colorectal cancer, and hepatocellular carcinoma.

Conclusion

AI technology has great potential in supporting the clinical diagnosis and treatment decision-making of DSNs. Several technical issues should be overcome before its application into clinical practice of DSNs.

CT-based delta radiomics in predicting the prognosis of stage IV gastric cancer to immune checkpoint inhibitors

Introduction

To explore the prognostic value of CT-based delta radiomics in predicting the prognosis of patients with stage IV gastric cancer treated with immune checkpoint inhibitors (ICI).

Materials and methods

Forty-two patients with stage IV gastric cancer, who had received ICI monotherapy, were enrolled in this retrospective study. Baseline and first follow-up CT scans were analyzed. Intratumoral and peritumoral regions of interest (ROI) were contoured, enabling the extraction of 192 features from each ROI. The intraclass correlation coefficients were used to select features with high stability. The least absolute shrinkage and selection operator was used to select features with high weights for predicting patient prognosis. Kaplan-Meier analysis and log-rank test were performed to explore the association between features and progression free survival (PFS). Cox regression analyses were used to identify predictors for PFS. The C-index was used to assess the prediction performance of features.

Results

Two radiomics features of ΔVintra_ZV and postVperi_Sphericity were identified from intratumoral and peritumoral regions, respectively. The Kaplan-Meier analysis revealed significant differences in PFS between patients with low and high feature value (ΔVintra_ZV: P=0.000; postVperi_Sphericity: P=0.012), and the multivariable cox analysis demonstrated that ΔVintra_ZV was independent predictor for PFS (HR, 1.911; 95% CI: 1.163-3.142; P=0.011), with C-index of 0.705.

Conclusions

Based on CT scans at baseline and first follow-up, the delta radiomics features could efficiently predict the PFS of gastric cancer patients treated with ICI therapy.

Artificial Intelligence-Based Prediction of Recurrence after Curative Resection for Colorectal Cancer from Digital Pathological Images

BACKGROUND

To develop an artificial intelligence-based model to predict recurrence after curative resection for stage I-III colorectal cancer from digitized pathological slides.

PATIENTS AND METHODS

In this retrospective study, 471 consecutive patients who underwent curative resection for stage I-III colorectal cancer at our institution from 2004 to 2015 were enrolled, and 512 randomly selected tiles from digitally scanned images of hematoxylin and eosin-stained tumor tissue sections were used to train a convolutional neural network. Five-fold cross-validation was used to validate the model. The association between recurrence and the model's output scores were analyzed in the test cohorts.

RESULTS

The area under the receiver operating characteristic curve of the cross-validation was 0.7245 [95% confidence interval (CI) 0.6707-0.7783; P < 0.0001]. The score successfully classified patients into those with better and worse recurrence free survival (P < 0.0001). Multivariate analysis revealed that a high score was significantly associated with worse recurrence free survival [odds ratio (OR) 1.857; 95% CI 1.248-2.805; P = 0.0021], which was independent from other predictive factors: male sex (P = 0.0238), rectal cancer (P = 0.0396), preoperative abnormal carcinoembryonic antigen (CEA) level (P = 0.0216), pathological T3/T4 stage (P = 0.0162), and pathological positive lymph node metastasis (P < 0.0001).

CONCLUSIONS

The artificial intelligence-based prediction model discriminated patients with a high risk of recurrence. This approach could help decision-makers consider the benefits of adjuvant chemotherapy.

Deep learning based radiomics for gastrointestinal cancer diagnosis and treatment: A minireview

Gastrointestinal (GI) cancers are the major cause of cancer-related mortality globally. Medical imaging is an important auxiliary means for the diagnosis, assessment and prognostic prediction of GI cancers. Radiomics is an emerging and effective technology to decipher the encoded information within medical images, and traditional machine learning is the most commonly used tool. Recent advances in deep learning technology have further promoted the development of radiomics. In the field of GI cancer, although there are several surveys on radiomics, there is no specific review on the application of deep-learning-based radiomics (DLR). In this review, a search was conducted on Web of Science, PubMed, and Google Scholar with an emphasis on the application of DLR for GI cancers, including esophageal, gastric, liver, pancreatic, and colorectal cancers. Besides, the challenges and recommendations based on the findings of the review are comprehensively analyzed to advance DLR.

Imaging standardisation in metastatic colorectal cancer: A joint EORTC-ESOI-ESGAR expert consensus recommendation

BACKGROUND

Treatment monitoring in metastatic colorectal cancer (mCRC) relies on imaging to evaluate the tumour burden. Response Evaluation Criteria in Solid Tumors provide a framework on reporting and interpretation of imaging findings yet offer no guidance on a standardised imaging protocol tailored to patients with mCRC. Imaging protocol heterogeneity remains a challenge for the reproducibility of conventional imaging end-points and is an obstacle for research on novel imaging end-points.

PATIENTS AND METHODS

Acknowledging the recently highlighted potential of radiomics and artificial intelligence tools as decision support for patient care in mCRC, a multidisciplinary, international and expert panel of imaging specialists was formed to find consensus on mCRC imaging protocols using the Delphi method.

RESULTS

Under the guidance of the European Organisation for Research and Treatment of Cancer (EORTC) Imaging and Gastrointestinal Tract Cancer Groups, the European Society of Oncologic Imaging (ESOI) and the European Society of Gastrointestinal and Abdominal Radiology (ESGAR), the EORTC-ESOI-ESGAR core imaging protocol was identified.

CONCLUSION

This consensus protocol attempts to promote standardisation and to diminish variations in patient preparation, scan acquisition and scan reconstruction. We anticipate that this standardisation will increase reproducibility of radiomics and artificial intelligence studies and serve as a catalyst for future research on imaging end-points. For ongoing and future mCRC trials, we encourage principal investigators to support the dissemination of these imaging standards across recruiting centres.

Radiomic and Volumetric Measurements as Clinical Trial Endpoints—A Comprehensive Review

Simple Summary

The extraction of quantitative data from standard-of-care imaging modalities offers opportunities to improve the relevance and salience of imaging biomarkers used in drug development. This review aims to identify the challenges and opportunities for discovering new imaging-based biomarkers based on radiomic and volumetric assessment in the single-site solid tumor sites: breast cancer, rectal cancer, lung cancer and glioblastoma. Developing approaches to harmonize three essential areas: segmentation, validation and data sharing may expedite regulatory approval and adoption of novel cancer imaging biomarkers.

Abstract

Clinical trials for oncology drug development have long relied on surrogate outcome biomarkers that assess changes in tumor burden to accelerate drug registration (i.e., Response Evaluation Criteria in Solid Tumors version 1.1 (RECIST v1.1) criteria). Drug-induced reduction in tumor size represents an imperfect surrogate marker for drug activity and yet a radiologically determined objective response rate is a widely used endpoint for Phase 2 trials. With the addition of therapies targeting complex biological systems such as immune system and DNA damage repair pathways, incorporation of integrative response and outcome biomarkers may add more predictive value. We performed a review of the relevant literature in four representative tumor types (breast cancer, rectal cancer, lung cancer and glioblastoma) to assess the preparedness of volumetric and radiomics metrics as clinical trial endpoints. We identified three key areas—segmentation, validation and data sharing strategies—where concerted efforts are required to enable progress of volumetric- and radiomics-based clinical trial endpoints for wider clinical implementation.

Artificial Intelligence Predictive Models of Response to Cytotoxic Chemotherapy Alone or Combined to Targeted Therapy for Metastatic Colorectal Cancer Patients: A Systematic Review and Meta-Analysis

Tailored treatments for metastatic colorectal cancer (mCRC) have not yet completely evolved due to the variety in response to drugs. Therefore, artificial intelligence has been recently used to develop prognostic and predictive models of treatment response (either activity/efficacy or toxicity) to aid in clinical decision making. In this systematic review, we have examined the ability of learning methods to predict response to chemotherapy alone or combined with targeted therapy in mCRC patients by targeting specific narrative publications in Medline up to April 2022 to identify appropriate original scientific articles. After the literature search, 26 original articles met inclusion and exclusion criteria and were included in the study. Our results show that all investigations conducted on this field have provided generally promising results in predicting the response to therapy or toxic side-effects. By a meta-analytic approach we found that the overall weighted means of the area under the receiver operating characteristic (ROC) curve (AUC) were 0.90, 95% C.I. 0.80-0.95 and 0.83, 95% C.I. 0.74-0.89 in training and validation sets, respectively, indicating a good classification performance in discriminating response vs. non-response. The calculation of overall HR indicates that learning models have strong ability to predict improved survival. Lastly, the delta-radiomics and the 74 gene signatures were able to discriminate response vs. non-response by correctly identifying up to 99% of mCRC patients who were responders and up to 100% of patients who were non-responders. Specifically, when we evaluated the predictive models with tests reaching 80% sensitivity (SE) and 90% specificity (SP), the delta radiomics showed an SE of 99% and an SP of 94% in the training set and an SE of 85% and SP of 92 in the test set, whereas for the 74 gene signatures the SE was 97.6% and the SP 100% in the training set.

John Charnley Award: Deep Learning Prediction of Hip Joint Center on Standard Pelvis Radiographs

BACKGROUND

Accurate hip joint center (HJC) determination is critical for preoperative planning, intraoperative execution, clinical outcomes after total hip arthroplasty, and commonly used classification systems in primary and revision hip replacement. However, current methods of preoperative HJC estimation are prone to subjectivity and human error. The purpose of the study was to leverage deep learning (DL) to develop a rapid and objective HJC estimation tool on anteroposterior (AP) pelvis radiographs.

METHODS

Radiographs from 3,965 patients (7,930 hips) were included. A DL model workflow was created to detect bony landmarks and estimate HJC based on a pelvic height ratio method. The workflow was utilized to conduct a grid-search for optimal nonspecific, sex-specific, and patient-specific (using contralateral hip) pelvic height ratios on the training/validation cohort (6,344 hips). Algorithm performance was assessed on an independent testing cohort for HJC estimation comparison.

RESULTS

The algorithm estimated HJC for the testing cohort at a rate of 0.65 seconds/hip based on features in AP radiographs alone. The model predicted HJC within 5 mm of error for 80% of hips using nonspecific ratios, which increased to 83% with sex-specific and 91% with patient-specific pelvic height ratio models. Mean error decreased utilizing the patient-specific model (3.09 ± 1.69 mm, P < .001).

CONCLUSION

Using DL, we developed nonspecific, sex-specific, and patient-specific models capable of estimating native HJC on AP pelvis radiographs. This tool may provide clinical value when considering preoperative component position in patients planned to undergo THA and in reducing the subjective variability in HJC estimation.

LEVEL OF EVIDENCE

Diagnostic, level IV.

Multi-Size Deep Learning Based Preoperative Computed Tomography Signature for Prognosis Prediction of Colorectal Cancer

Background: Preoperative and postoperative evaluation of colorectal cancer (CRC) patients is crucial for subsequent treatment guidance. Our study aims to provide a timely and rapid assessment of the prognosis of CRC patients with deep learning according to non-invasive preoperative computed tomography (CT) and explore the underlying biological explanations.

Methods: A total of 808 CRC patients with preoperative CT (development cohort: n = 426, validation cohort: n = 382) were enrolled in our study. We proposed a novel end-to-end Multi-Size Convolutional Neural Network (MSCNN) to predict the risk of CRC recurrence with CT images (CT signature). The prognostic performance of CT signature was evaluated by Kaplan-Meier curve. An integrated nomogram was constructed to improve the clinical utility of CT signature by combining with other clinicopathologic factors. Further visualization and correlation analysis for CT deep features with paired gene expression profiles were performed to reveal the molecular characteristics of CRC tumors learned by MSCNN in radiographic imaging.

Results: The Kaplan-Meier analysis showed that CT signature was a significant prognostic factor for CRC disease-free survival (DFS) prediction [development cohort: hazard ratio (HR): 50.7, 95% CI: 28.4–90.6, p &lt; 0.001; validation cohort: HR: 2.04, 95% CI: 1.44–2.89, p &lt; 0.001]. Multivariable analysis confirmed the independence prognostic value of CT signature (development cohort: HR: 30.7, 95% CI: 19.8–69.3, p &lt; 0.001; validation cohort: HR: 1.83, 95% CI: 1.19–2.83, p = 0.006). Dimension reduction and visualization of CT deep features demonstrated a high correlation with the prognosis of CRC patients. Functional pathway analysis further indicated that CRC patients with high CT signature presented down-regulation of several immunology pathways. Correlation analysis found that CT deep features were mainly associated with activation of metabolic and proliferative pathways.

Conclusions: Our deep learning based preoperative CT signature can effectively predict prognosis of CRC patients. Integration analysis of multi-omic data revealed that some molecular characteristics of CRC tumor can be captured by deep learning in CT images.

Radiophysiomics: Brain Tumors Classification by Machine Learning and Physiological MRI Data

The precise initial characterization of contrast-enhancing brain tumors has significant consequences for clinical outcomes. Various novel neuroimaging methods have been developed to increase the specificity of conventional magnetic resonance imaging (cMRI) but also the increased complexity of data analysis. Artificial intelligence offers new options to manage this challenge in clinical settings. Here, we investigated whether multiclass machine learning (ML) algorithms applied to a high-dimensional panel of radiomic features from advanced MRI (advMRI) and physiological MRI (phyMRI; thus, radiophysiomics) could reliably classify contrast-enhancing brain tumors. The recently developed phyMRI technique enables the quantitative assessment of microvascular architecture, neovascularization, oxygen metabolism, and tissue hypoxia. A training cohort of 167 patients suffering from one of the five most common brain tumor entities (glioblastoma, anaplastic glioma, meningioma, primary CNS lymphoma, or brain metastasis), combined with nine common ML algorithms, was used to develop overall 135 classifiers. Multiclass classification performance was investigated using tenfold cross-validation and an independent test cohort. Adaptive boosting and random forest in combination with advMRI and phyMRI data were superior to human reading in accuracy (0.875 vs. 0.850), precision (0.862 vs. 0.798), F-score (0.774 vs. 0.740), AUROC (0.886 vs. 0.813), and classification error (5 vs. 6). The radiologists, however, showed a higher sensitivity (0.767 vs. 0.750) and specificity (0.925 vs. 0.902). We demonstrated that ML-based radiophysiomics could be helpful in the clinical routine diagnosis of contrast-enhancing brain tumors; however, a high expenditure of time and work for data preprocessing requires the inclusion of deep neural networks.