Artificial Neural Network with Composite Architectures for Prediction of Local Control in Radiotherapy

Longitudinal Image Data for Outcome Modeling

In oncology, medical imaging is crucial for diagnosis, treatment planning and therapy execution. Treatment responses can be complex and varied and are known to involve factors of treatment, patient characteristics and tumor microenvironment. Longitudinal image analysis is able to track temporal changes, aiding in disease monitoring, treatment evaluation, and outcome prediction. This allows for the enhancement of personalized medicine. However, analyzing longitudinal 2D and 3D images presents unique challenges, including image registration, reliable segmentation, dealing with variable imaging intervals, and sparse data. This review presents an overview of techniques and methodologies in longitudinal image analysis, with a primary focus on outcome modeling in radiation oncology.

Deep-Learning Model Prediction of Radiation Pneumonitis Using Pretreatment Chest Computed Tomography and Clinical Factors

Objectives: This study aimed to build a comprehensive deep-learning model for the prediction of radiation pneumonitis using chest computed tomography (CT), clinical, dosimetric, and laboratory data. Introduction: Radiation therapy is an effective tool for treating patients with lung cancer. Despite its effectiveness, the risk of radiation pneumonitis limits its application. Although several studies have demonstrated models to predict radiation pneumonitis, no reliable model has been developed yet. Herein, we developed prediction models using pretreatment chest CT and various clinical data to assess the likelihood of radiation pneumonitis in lung cancer patients. Methods: This retrospective study analyzed 3-dimensional (3D) lung volume data from chest CT scans and 27 features including dosimetric, clinical, and laboratory data from 548 patients who were treated at our institution between 2010 and 2021. We developed a neural network, named MergeNet, which processes lung 3D CT, clinical, dosimetric, and laboratory data. The MergeNet integrates a convolutional neural network with subsequent fully connected layers. A support vector machine (SVM) and light gradient boosting machine (LGBM) model were also implemented for comparison. For comparison, the convolution-only neural network was implemented as well. Three-dimensional Resnet-10 network and 4-fold cross-validation were used. Results: Classification performance was quantified by using the area under the receiver operative characteristic curve (AUC) metrics. MergeNet showed the AUC of 0.689. SVM, LGBM, and convolution-only networks showed AUCs of 0.525, 0.541, and 0.550, respectively. Application of DeLong test to pairs of receiver operating characteristic curves respectively yielded P values of .001 for the MergeNet-SVM pair and 0.001 for the MergeNet-LGBM pair. Conclusion: The MergeNet model, which incorporates chest CT, clinical, dosimetric, and laboratory data, demonstrated superior performance compared to other models. However, since its prediction performance has not yet reached an efficient level for clinical application, further research is required. Contribution: This study showed that MergeNet may be an effective means to predict radiation pneumonitis. Various predictive factors can be used together for the radiation pneumonitis prediction task via the MergeNet.

Artificial intelligence (AI) and machine learning (ML) in precision oncology: a review on enhancing discoverability through multiomics integration

Multiomics data including imaging radiomics and various types of molecular biomarkers have been increasingly investigated for better diagnosis and therapy in the era of precision oncology. Artificial intelligence (AI) including machine learning (ML) and deep learning (DL) techniques combined with the exponential growth of multiomics data may have great potential to revolutionize cancer subtyping, risk stratification, prognostication, prediction and clinical decision-making. In this article, we first present different categories of multiomics data and their roles in diagnosis and therapy. Second, AI-based data fusion methods and modeling methods as well as different validation schemes are illustrated. Third, the applications and examples of multiomics research in oncology are demonstrated. Finally, the challenges regarding the heterogeneity data set, availability of omics data, and validation of the research are discussed. The transition of multiomics research to real clinics still requires consistent efforts in standardizing omics data collection and analysis, building computational infrastructure for data sharing and storing, developing advanced methods to improve data fusion and interpretability, and ultimately, conducting large-scale prospective clinical trials to fill the gap between study findings and clinical benefits.

The role of artificial intelligence in veterinary radiation oncology

Veterinary radiation oncology regularly deploys sophisticated contouring, image registration, and treatment planning optimization software for patient care. Over the past decade, advances in computing power and the rapid development of neural networks, open-source software packages, and data science have been realized and resulted in new research and clinical applications of artificial intelligent (AI) systems in radiation oncology. These technologies differ from conventional software in their level of complexity and ability to learn from representative and local data. We provide clinical and research application examples of AI in human radiation oncology and their potential applications in veterinary medicine throughout the patient's care-path: from treatment simulation, deformable registration, auto-segmentation, automated treatment planning and plan selection, quality assurance, adaptive radiotherapy, and outcomes modeling. These technologies have the potential to offer significant time and cost savings in the veterinary setting; however, since the range of usefulness of these technologies have not been well studied nor understood, care must be taken if adopting AI technologies in clinical practice. Over the next several years, some practical and realizable applications of AI in veterinary radiation oncology include automated segmentation of normal tissues and tumor volumes, deformable registration, multi-criteria plan optimization, and adaptive radiotherapy. Keys in achieving success in adopting AI in veterinary radiation oncology include: establishing "truth-data"; data harmonization; multi-institutional data and collaborations; standardized dose reporting and taxonomy; adopting an open access philosophy, data collection and curation; open-source algorithm development; and transparent and platform-independent code development.

Integrating Multiomics Information in Deep Learning Architectures for Joint Actuarial Outcome Prediction in Non-Small Cell Lung Cancer Patients After Radiation Therapy

PURPOSE

Novel actuarial deep learning neural network (ADNN) architectures are proposed for joint prediction of radiation therapy outcomes-radiation pneumonitis (RP) and local control (LC)-in stage III non-small cell lung cancer (NSCLC) patients. Unlike normal tissue complication probability/tumor control probability models that use dosimetric information solely, our proposed models consider complex interactions among multiomics information including positron emission tomography (PET) radiomics, cytokines, and miRNAs. Additional time-to-event information is also used in the actuarial prediction.

METHODS AND MATERIALS

Three architectures were investigated: ADNN-DVH considered dosimetric information only; ADNN-com integrated multiomics information; and ADNN-com-joint combined RP2 (RP grade ≥2) and LC prediction. In these architectures, differential dose-volume histograms (DVHs) were fed into 1D convolutional neural networks (CNN) for extracting reduced representations. Variational encoders were used to learn representations of imaging and biological data. Reduced representations were fed into Surv-Nets to predict time-to-event probabilities for RP2 and LC independently and jointly by incorporating time information into designated loss functions.

RESULTS

Models were evaluated on 117 retrospective patients and were independently tested on 25 newly accrued patients prospectively. A multi-institutional RTOG0617 data set of 327 patients was used for external validation. ADNN-DVH yielded cross-validated c-indexes (95% confidence intervals) of 0.660 (0.630-0.690) for RP2 prediction and 0.727 (0.700-0.753) for LC prediction, outperforming a generalized Lyman model for RP2 (0.613 [0.583-0.643]) and a generalized log-logistic model for LC (0.569 [0.545-0.594]). The independent internal test and external validation yielded similar results. ADNN-com achieved an even better performance than ADNN-DVH on both cross-validation and independent internal test. Furthermore, ADNN-com-joint, which yielded performance similar to ADNN-com, realized joint prediction with c-indexes of 0.705 (0.676-0.734) for RP2 and 0.740 (0.714-0.765) for LC and achieved an area under a free-response receiving operator characteristic curve (AU-FROC) of 0.729 (0.697-0.773) for the joint prediction of RP2 and LC.

CONCLUSION

Novel deep learning architectures that integrate multiomics information outperformed traditional normal tissue complication probability/tumor control probability models in actuarial prediction of RP2 and LC.

Artificial intelligence in radiotherapy: a technological review

Radiation therapy (RT) is widely used to treat cancer. Technological advances in RT have occurred in the past 30 years. These advances, such as three-dimensional image guidance, intensity modulation, and robotics, created challenges and opportunities for the next breakthrough, in which artificial intelligence (AI) will possibly play important roles. AI will replace certain repetitive and labor-intensive tasks and improve the accuracy and consistency of others, particularly those with increased complexity because of technological advances. The improvement in efficiency and consistency is important to manage the increasing cancer patient burden to the society. Furthermore, AI may provide new functionalities that facilitate satisfactory RT. The functionalities include superior images for real-time intervention and adaptive and personalized RT. AI may effectively synthesize and analyze big data for such purposes. This review describes the RT workflow and identifies areas, including imaging, treatment planning, quality assurance, and outcome prediction, that benefit from AI. This review primarily focuses on deep-learning techniques, although conventional machine-learning techniques are also mentioned.

Introduction to machine and deep learning for medical physicists

Recent years have witnessed tremendous growth in the application of machine learning (ML) and deep learning (DL) techniques in medical physics. Embracing the current big data era, medical physicists equipped with these state-of-the-art tools should be able to solve pressing problems in modern radiation oncology. Here, a review of the basic aspects involved in ML/DL model building, including data processing, model training, and validation for medical physics applications is presented and discussed. Machine learning can be categorized based on the underlying task into supervised learning, unsupervised learning, or reinforcement learning; each of these categories has its own input/output dataset characteristics and aims to solve different classes of problems in medical physics ranging from automation of processes to predictive analytics. It is recognized that data size requirements may vary depending on the specific medical physics application and the nature of the algorithms applied. Data processing, which is a crucial step for model stability and precision, should be performed before training the model. Deep learning as a subset of ML is able to learn multilevel representations from raw input data, eliminating the necessity for hand crafted features in classical ML. It can be thought of as an extension of the classical linear models but with multilayer (deep) structures and nonlinear activation functions. The logic of going "deeper" is related to learning complex data structures and its realization has been aided by recent advancements in parallel computing architectures and the development of more robust optimization methods for efficient training of these algorithms. Model validation is an essential part of ML/DL model building. Without it, the model being developed cannot be easily trusted to generalize to unseen data. Whenever applying ML/DL, one should keep in mind, according to Amara's law, that humans may tend to overestimate the ability of a technology in the short term and underestimate its capability in the long term. To establish ML/DL role into standard clinical workflow, models considering balance between accuracy and interpretability should be developed. Machine learning/DL algorithms have potential in numerous radiation oncology applications, including automatizing mundane procedures, improving efficiency and safety of auto-contouring, treatment planning, quality assurance, motion management, and outcome predictions. Medical physicists have been at the frontiers of technology translation into medicine and they ought to be prepared to embrace the inevitable role of ML/DL in the practice of radiation oncology and lead its clinical implementation.