A prospective study comparing the predictions of doctors versus models for treatment outcome of lung cancer patients: a step toward individualized care and shared decision making

Federated Learning Survival Model and Potential Radiotherapy Decision Support Impact Assessment for Non-small Cell Lung Cancer Using Real-World Data

AIMS

The objective of this study was to develop a two-year overall survival model for inoperable stage I-III non-small cell lung cancer (NSCLC) patients using routine radiation oncology data over a federated (distributed) learning network and evaluate the potential of decision support for curative versus palliative radiotherapy.

METHODS

A federated infrastructure of data extraction, de-identification, standardisation, image analysis, and modelling was installed for seven clinics to obtain clinical and imaging features and survival information for patients treated in 2011-2019. A logistic regression model was trained for the 2011-2016 curative patient cohort and validated for the 2017-2019 cohort. Features were selected with univariate and model-based analysis and optimised using bootstrapping. System performance was assessed by the receiver operating characteristic (ROC) and corresponding area under curve (AUC), C-index, calibration metrics and Kaplan-Meier survival curves, with risk groups defined by model probability quartiles. Decision support was evaluated using a case-control analysis using propensity matching between treatment groups.

RESULTS

1655 patient datasets were included. The overall model AUC was 0.68. Fifty-eight percent of patients treated with palliative radiotherapy had a low-to-moderate risk prediction according to the model, with survival times not significantly different (p = 0.87 and 0.061) from patients treated with curative radiotherapy classified as high-risk by the model. When survival was simulated by risk group and model-indicated treatment, there was an estimated 11% increase in survival rate at two years (p < 0.01).

CONCLUSION

Federated learning over multiple institution data can be used to develop and validate decision support systems for lung cancer while quantifying the potential impact of their use in practice. This paves the way for personalised medicine, where decisions can be based more closely on individual patient details from routine care.

Swallowing disorders and mortality in adults with advanced cancer outside the head and neck and upper gastrointestinal tract: a systematic review

BACKGROUND

Although oncological palliative care is increasingly being offered by multidisciplinary teams, there is still a lack of data about some symptoms handled by these teams, such as dysphagia, in patients with advanced cancer outside swallow regions. This study aimed to estimate the occurrence of dysphagia in prognosis studies of adults with advanced cancer outside the head, neck, and upper gastrointestinal tract, and to determine if there is an association with mortality.

METHODS

A systematic review of studies that evaluated dysphagia and mortality was conducted (PROSPERO: CRD42021257172).

DATA SOURCES

BVS, PubMed, CINAHL, Web of Science, and Scopus. Data between 2011 and 2023 were selected.

RESULTS

Among the 608 articles screened, only 14 were included, which covered different types of cancer, primarily Lung, and Genitourinary, Skin, Hematological, and Central Nervous System as well. Dysphagia demonstrated a variable frequency, and almost half of the studies found a percentage of dysphagia above 60%, appearing most as a symptom that affects health-related quality of life and prove to be a toxicity of treatment. The association between dysphagia and mortality was only evaluated in three articles that studied advanced lung cancer, in which, after controlling for covariates, swallowing disorders were associated with worse survival, with prevalences of dysphagia and hazard ratios of 78.5% (1.12 [1.04-1.20]), 4% (1.34 [1.28-1.35]), and 3% (1.40 [1.07-1.81]), respectively.

CONCLUSIONS

The occurrence of dysphagia in advanced cancer outside the head, neck, and upper GI tract is common, and there seems to be an association with significantly decreased survival in patients with advanced lung cancer.

ABVS-Based Radiomics for Early Predicting the Efficacy of Neoadjuvant Chemotherapy in Patients with Breast Cancers

Background

Neoadjuvant chemotherapy (NAC) plays a significant role in breast cancer (BC) management; however, its efficacy varies among patients. Current evaluation methods may lead to delayed treatment alterations, and traditional imaging modalities often yield inaccurate results. Radiomics, an emerging field in medical imaging, offers potential for improved tumor characterization and personalized medicine. Nevertheless, its application in early and accurately predicting NAC response remains underinvestigated.

Objective

This study aims to develop an automated breast volume scanner (ABVS)-based radiomics model to facilitate early detection of suboptimal NAC response, ultimately promoting personalized therapeutic approaches for BC patients.

Methods

This retrospective study involved 248 BC patients receiving NAC. Standard guidelines were followed, and patients were classified as responders or non-responders based on treatment outcomes. ABVS images were obtained before and during NAC, and radiomics features were extracted using the PyRadiomics toolkit. Inter-observer consistency and hierarchical feature selection were assessed. Three machine learning classifiers, logistic regression, support vector machine, and random forest, were trained and validated using a five-fold cross-validation with three repetitions. Model performance was comprehensively evaluated based on discrimination, calibration, and clinical utility.

Results

Of the 248 BC patients, 157 (63.3%) were responders, and 91 (36.7%) were non-responders. Radiomics feature selection revealed 7 pre-NAC and 6 post-NAC ABVS features, with higher weights for post-NAC features (min >0.05) than pre-NAC (max <0.03). The three post-NAC classifiers demonstrated AUCs of approximately 0.9, indicating excellent discrimination. DCA curves revealed a substantial net benefit when the threshold probability exceeded 40%. Conversely, the three pre-NAC classifiers had AUCs between 0.7 and 0.8, suggesting moderate discrimination and limited clinical utility based on their DCA curves.

Conclusion

The ABVS-based radiomics model effectively predicted suboptimal NAC responses in BC patients, with early post-NAC classifiers outperforming pre-NAC classifiers in discrimination and clinical utility. It could enhance personalized treatment and improve patient outcomes in BC management.

A CT-based radiomics nomogram for distinguishing between malignant and benign Bosniak IIF masses: a two-centre study

AIM

To establish and assess a computed tomography (CT)-based radiomics nomogram for identifying malignant and benign Bosniak IIF masses.

MATERIALS AND METHODS

In total, 150 patients with Bosniak IIF masses were separated into a training set (n=106) and a test set (n=44) in a ratio of 7:3. A radiomics signature was calculated based on extracted features from the three phases of CT images. A clinical model was constructed based on clinical characteristics and CT features, and a nomogram incorporating the radiomics signature and independent clinical variables was established. The calibration ability, discrimination accuracy, and clinical value of the nomogram model were assessed.

RESULTS

Twelve features derived from CT images were applied to establish the radiomics signature. The performance levels of three machine-learning models were improved by adding the synthetic minority oversampling technique algorithm. The optimised machine learning model was a combination of the minimum redundancy maximum relevance-least absolute shrinkage and selection operator feature screening method + logistic regression classifier + synthetic minority oversampling technique algorithm, which demonstrated excellent identification ability on the test set (area under the curve [AUC], 0.970; 95% confidence interval [CI], 0.940-1.000). The nomogram model displayed outstanding discrimination ability on the test set (AUC, 0.972; 95% CI, 0.942-1.000).

CONCLUSIONS

The CT-based radiomics nomogram was useful for discriminating between malignant and benign Bosniak IIF masses, which improved the precision of preoperative diagnosis.

deepPERFECT: Novel Deep Learning CT Synthesis Method for Expeditious Pancreatic Cancer Radiotherapy

Major sources of delay in the standard of care RT workflow are the need for multiple appointments and separate image acquisition. In this work, we addressed the question of how we can expedite the workflow by synthesizing planning CT from diagnostic CT. This idea is based on the theory that diagnostic CT can be used for RT planning, but in practice, due to the differences in patient setup and acquisition techniques, separate planning CT is required. We developed a generative deep learning model, deepPERFECT, that is trained to capture these differences and generate deformation vector fields to transform diagnostic CT into preliminary planning CT. We performed detailed analysis both from an image quality and a dosimetric point of view, and showed that deepPERFECT enabled the preliminary RT planning to be used for preliminary and early plan dosimetric assessment and evaluation.

A new prognostic model for brain metastases of specific primary tumors with stereotactic radiotherapy

PURPOSE

Stereotactic radiotherapy (SRT) was widely used in brain metastases (BM), especially in oligometastases. It is imperative to develop a new prognostic score to predict the overall survival (OS) of brain metastases based on prognostic factors for specific primary tumors.

MATERIAL AND METHOD

One hundred and ninety-seven patients were involved in the training cohort to develop a new prognostic score to predict the overall survival (OS) of brain metastases for specific primary tumors. Independent prognostic factors were confirmed using a Cox regression model. The score was developed based on clinical prognostic factors of OS with Cox proportional hazards model. The result was validated in another cohort with 56 participants to evaluate the performance of the score.

RESULTS

One hundred and ninety-seven patients with 329 brain metastases received SRT. For NSCLC, the significant prognostic factors were extracranial metastases, target therapy and number of brain metastases. For gastrointestinal cancer, the significant prognostic factors were target therapy and number of brain metastases.

CONCLUSION

The prognostic factors scores were varied by the histologic types which can be used to efficiently stratify for selected patients with brain-metastasis.

Incorporation of a machine learning pathological diagnosis algorithm into the thyroid ultrasound imaging data improves the diagnosis risk of malignant thyroid nodules

Objective

This study aimed at establishing a new model to predict malignant thyroid nodules using machine learning algorithms.

Methods

A retrospective study was performed on 274 patients with thyroid nodules who underwent fine-needle aspiration (FNA) cytology or surgery from October 2018 to 2020 in Xianyang Central Hospital. The least absolute shrinkage and selection operator (lasso) regression analysis and logistic analysis were applied to screen and identified variables. Six machine learning algorithms, including Decision Tree (DT), Extreme Gradient Boosting (XGBoost), Gradient Boosting Machine (GBM), Naive Bayes Classifier (NBC), Random Forest (RF), and Logistic Regression (LR), were employed and compared in constructing the predictive model, coupled with preoperative clinical characteristics and ultrasound features. Internal validation was performed by using 10-fold cross-validation. The performance of the model was measured by the area under the receiver operating characteristic curve (AUC), accuracy, precision, recall, F1 score, Shapley additive explanations (SHAP) plot, feature importance, and correlation of features. The best cutoff value for risk stratification was identified by probability density function (PDF) and clinical utility curve (CUC).

Results

The malignant rate of thyroid nodules in the study cohort was 53.2%. The predictive models are constructed by age, margin, shape, echogenic foci, echogenicity, and lymph nodes. The XGBoost model was significantly superior to any one of the machine learning models, with an AUC value of 0.829. According to the PDF and CUC, we recommended that 51% probability be used as a threshold for determining the risk stratification of malignant nodules, where about 85.6% of patients with malignant nodules could be detected. Meanwhile, approximately 89.8% of unnecessary biopsy procedures would be saved. Finally, an online web risk calculator has been built to estimate the personal likelihood of malignant thyroid nodules based on the best-performing ML-ed model of XGBoost.

Conclusions

Combining clinical characteristics and features of ultrasound images, ML algorithms can achieve reliable prediction of malignant thyroid nodules. The online web risk calculator based on the XGBoost model can easily identify in real-time the probability of malignant thyroid nodules, which can assist clinicians to formulate individualized management strategies for patients.

Gross tumour volume radiomics for prognostication of recurrence & death following radical radiotherapy for NSCLC

Recurrence occurs in up to 36% of patients treated with curative-intent radiotherapy for NSCLC. Identifying patients at higher risk of recurrence for more intensive surveillance may facilitate the earlier introduction of the next line of treatment. We aimed to use radiotherapy planning CT scans to develop radiomic classification models that predict overall survival (OS), recurrence-free survival (RFS) and recurrence two years post-treatment for risk-stratification. A retrospective multi-centre study of >900 patients receiving curative-intent radiotherapy for stage I-III NSCLC was undertaken. Models using radiomic and/or clinical features were developed, compared with 10-fold cross-validation and an external test set, and benchmarked against TNM-stage. Respective validation and test set AUCs (with 95% confidence intervals) for the radiomic-only models were: (1) OS: 0.712 (0.592–0.832) and 0.685 (0.585–0.784), (2) RFS: 0.825 (0.733–0.916) and 0.750 (0.665–0.835), (3) Recurrence: 0.678 (0.554–0.801) and 0.673 (0.577–0.77). For the combined models: (1) OS: 0.702 (0.583–0.822) and 0.683 (0.586–0.78), (2) RFS: 0.805 (0.707–0.903) and 0·755 (0.672–0.838), (3) Recurrence: 0·637 (0.51–0.·765) and 0·738 (0.649–0.826). Kaplan-Meier analyses demonstrate OS and RFS difference of >300 and >400 days respectively between low and high-risk groups. We have developed validated and externally tested radiomic-based prediction models. Such models could be integrated into the routine radiotherapy workflow, thus informing a personalised surveillance strategy at the point of treatment. Our work lays the foundations for future prospective clinical trials for quantitative personalised risk-stratification for surveillance following curative-intent radiotherapy for NSCLC.

A deep learning-based diagnostic pattern for ultrasound breast imaging: can it reduce unnecessary biopsy?

Background

Early studies have demonstrated the potential of deep learning in bringing revolutionary changes in medical analysis. However, it is unknown which deep learning based diagnostic pattern is more effective for differentiating malignant and benign breast lesions (BLs) and can assist radiologists to reduce unnecessary biopsies.

Methods

A total of 506 malignant BLs and 557 benign BLs were enrolled in this study after excluding incomplete ultrasound images. 396 malignant BLs and 447 benign BLs were included in the training cohort while 110 malignant and 110 benign BLs were included in the validation cohort. All BLs in the training and validation cohort were biopsy-proven. The most common convolutional neural networks (VGG-16 and VGG-19) were applied to identify malignant and benign BLs using grey-scale ultrasound images. Two radiologists determined the malignant (suggestion for biopsy) and benign (suggestion for follow-up) BLs with a 2-step reading session. The first step was based on conventional ultrasound (US) images alone to make a biopsy or follow-up decision. The second step was to take deep learning results into account for the decision adjustment. If a deep learning result of a first-classified benign BL was above the cut-off value, then it was re-classified as malignant.

Results

In terms of area under the curve (AUC), the VGG-19 model yielded the best diagnostic performance in both training [0.939, 95% confidence interval (CI): 0.924-0.954] and testing dataset (0.959, 95% CI: 0.937-0.982). With the aid of deep learning models, the AUC of radiologists improved from 0.805 (95% CI: 0.744-0.865) to 0.827 (95% CI: 0.771-0.875, VGG-16) and 0.914 (95% CI: 0.871-0.957, VGG-19). The unnecessary biopsies decreased from 10.0% (11/110) to 8.2% (9/110) (assisted by VGG-16) and 0.9% (1/110) (assisted by VGG-19).

Conclusions

The application of deep learning patterns in breast US may improve the diagnostic performance of radiologists by offering a second opinion. And thus, the assist of deep learning algorithm can considerably reduce the unnecessary biopsy rate in the clinical management of breast lesions.

Infrastructure platform for privacy-preserving distributed machine learning development of computer-assisted theragnostics in cancer

INTRODUCTION

Emerging evidence suggests that data-driven support tools have found their way into clinical decision-making in a number of areas, including cancer care. Improving them and widening their scope of availability in various differing clinical scenarios, including for prognostic models derived from retrospective data, requires co-ordinated data sharing between clinical centres, secondary analyses of large multi-institutional clinical trial data, or distributed (federated) learning infrastructures. A systematic approach to utilizing routinely collected data across cancer care clinics remains a significant challenge due to privacy, administrative and political barriers.

METHODS

An information technology infrastructure and web service software was developed and implemented which uses machine learning to construct clinical decision support systems in a privacy-preserving manner across datasets geographically distributed in different hospitals. The infrastructure was deployed in a network of Australian hospitals. A harmonized, international ontology-linked, set of lung cancer databases were built with the routine clinical and imaging data at each centre. The infrastructure was demonstrated with the development of logistic regression models to predict major cardiovascular events following radiation therapy.

RESULTS

The infrastructure implemented forms the basis of the Australian computer-assisted theragnostics (AusCAT) network for radiation oncology data extraction, reporting and distributed learning. Four radiation oncology departments (across seven hospitals) in New South Wales (NSW) participated in this demonstration study. Infrastructure was deployed at each centre and used to develop a model predicting for cardiovascular admission within a year of receiving curative radiotherapy for non-small cell lung cancer. A total of 10417 lung cancer patients were identified with 802 being eligible for the model. Twenty features were chosen for analysis from the clinical record and linked registries. After selection, 8 features were included and a logistic regression model achieved an area under the receiver operating characteristic (AUROC) curve of 0.70 and C-index of 0.65 on out-of-sample data.

CONCLUSION

The infrastructure developed was demonstrated to be usable in practice between clinical centres to harmonize routinely collected oncology data and develop models with federated learning. It provides a promising approach to enable further research studies in radiation oncology using real world clinical data.

Open-source distributed learning validation for a larynx cancer survival model following radiotherapy

INTRODUCTION

Prediction models are useful to design personalised treatment. However, safe and effective implementation relies on external validation. Retrospective data are available in many institutions, but sharing between institutions can be challenging due to patient data sensitivity and governance or legal barriers. This study validates a larynx cancer survival model performed using distributed learning without any sensitive data leaving the institution.

METHODS

Open-source distributed learning software based on a stratified Cox proportional hazard model was developed and used to validate the Egelmeer et al. MAASTRO survival model across two hospitals in two countries. The validation optimised a single scaling parameter multiplied by the original predicted prognostic index. All analyses and figures were based on the distributed system, ensuring no information leakage from the individual centres. All applied software is provided as freeware to facilitate distributed learning in other institutions.

RESULTS

1745 patients received radiotherapy for larynx cancer in the two centres from Jan 2005 to Dec 2018. Limiting to a maximum of one missing value in the parameters of the survival model reduced the cohort to 1095 patients. The Harrell C-index was 0.74 (CI95%, 0.71-0.76) and 0.70 (0.66-0.75) for the two centres. However, the model needed a scaling update. In addition, it was found that survival predictions of patients undergoing hypofractionation were less precise.

CONCLUSION

Open-source distributed learning software was able to validate, and suggest a minor update to the original survival model without central access to patient sensitive information. Even without the update, the original MAASTRO survival model of Egelmeer et al. performed reasonably well, providing similar results in this validation as in its original validation.

Clinician perspectives on clinical decision support systems in lung cancer: Implications for shared decision-making

Background

Lung cancer treatment decisions are typically made among clinical experts in a multidisciplinary tumour board (MTB) based on clinical data and guidelines. The rise of artificial intelligence and cultural shifts towards patient autonomy are changing the nature of clinical decision‐making towards personalized treatments. This can be supported by clinical decision support systems (CDSSs) that generate personalized treatment information as a basis for shared decision‐making (SDM). Little is known about lung cancer patients' treatment decisions and the potential for SDM supported by CDSSs. The aim of this study is to understand to what extent SDM is done in current practice and what clinicians need to improve it.

Objective

To explore (1) the extent to which patient preferences are taken into consideration in non‐small‐cell lung cancer (NSCLC) treatment decisions; (2) clinician perspectives on using CDSSs to support SDM.

Design

Mixed methods study consisting of a retrospective cohort study on patient deviation from MTB advice and reasons for deviation, qualitative interviews with lung cancer specialists and observations of MTB discussions and patient consultations.

Setting and Participants

NSCLC patients (N = 257) treated at a single radiotherapy clinic and nine lung cancer specialists from six Dutch clinics.

Results

We found a 10.9% (n = 28) deviation rate from MTB advice; 50% (n = 14) were due to patient preference, of which 85.7% (n = 12) chose a less intensive treatment than MTB advice. Current MTB recommendations are based on clinician experience, guidelines and patients' performance status. Most specialists (n = 7) were receptive towards CDSSs but cited barriers, such as lack of trust, lack of validation studies and time. CDSSs were considered valuable during MTB discussions rather than in consultations.

Conclusion

Lung cancer decisions are heavily influenced by clinical guidelines and experience, yet many patients prefer less intensive treatments. CDSSs can support SDM by presenting the harms and benefits of different treatment options rather than giving single treatment advice. External validation of CDSSs should be prioritized.

Patient or Public Contribution

This study did not involve patients or the public explicitly; however, the study design was informed by prior interviews with volunteers of a cancer patient advocacy group. The study objectives and data collection were supported by Dutch health care insurer CZ for a project titled ‘My Best Treatment’ that improves patient‐centeredness and the lung cancer patient pathway in the Netherlands.

Predicting 2-year survival in stage I-III non-small cell lung cancer: the development and validation of a scoring system from an Australian cohort

Background

There are limited data on survival prediction models in contemporary inoperable non-small cell lung cancer (NSCLC) patients. The objective of this study was to develop and validate a survival prediction model in a cohort of inoperable stage I-III NSCLC patients treated with radiotherapy.

Methods

Data from inoperable stage I-III NSCLC patients diagnosed from 1/1/2016 to 31/12/2017 were collected from three radiation oncology clinics. Patient, tumour and treatment-related variables were selected for model inclusion using univariate and multivariate analysis. Cox proportional hazards regression was used to develop a 2-year overall survival prediction model, the South West Sydney Model (SWSM) in one clinic (n = 117) and validated in the other clinics (n = 144). Model performance, assessed internally and on one independent dataset, was expressed as Harrell’s concordance index (c-index).

Results

The SWSM contained five variables: Eastern Cooperative Oncology Group performance status, diffusing capacity of the lung for carbon monoxide, histological diagnosis, tumour lobe and equivalent dose in 2 Gy fractions. The SWSM yielded a c-index of 0.70 on internal validation and 0.72 on external validation. Survival probability could be stratified into three groups using a risk score derived from the model.

Conclusions

A 2-year survival model with good discrimination was developed. The model included tumour lobe as a novel variable and has the potential to guide treatment decisions. Further validation is needed in a larger patient cohort.

Bayesian network structure for predicting local tumor recurrence in rectal cancer patients treated with neoadjuvant chemoradiation followed by surgery

Background and Purpose

Tumor recurrence, a characteristic of malignant tumors, is the biggest concern for rectal cancer survivors. The epidemiology of the disease calls for a pressing need to improve healthcare quality and patient outcomes. Prediction models such as Bayesian networks, which can probabilistically reason under uncertainty, could assist caregivers with patient management. However, some concerns are associated with the standard approaches to developing these structures in medicine. Therefore, this study aims to compare Bayesian network structures that stem from these two techniques.

Patients and Methods

A retrospective analysis was performed on 6754 locally advanced rectal cancer (LARC) patients enrolled in 14 international clinical trials. Local tumor recurrence at 2, 3, and 5-years was defined as the endpoints of interest. Five rectal cancer treating physicians from three countries elicited the expert structure. The algorithmic structure was inferred from the data with the hill-climbing algorithm. Structural performance was assessed with calibration plots and area under the curve values.

Results

The area under the curve for the expert structure on the training and validation data was above 0.9 and 0.8, respectively, for all the time points. However, the algorithmic structure had superior predictive performance over the expert structure for all time points of interest.

Conclusion

We have developed and internally validated a Bayesian networks structure from experts' opinions, which can predict the risk of a LARC patient developing a tumor recurrence at 2, 3, and 5 years. Our result shows that the algorithmic-based structures are more performant and less interpretable than expert-based structures.

Deep Multi-Objective Learning from Low-Dose CT for Automatic Lung-RADS Report Generation

Radiology report generation through chest radiography interpretation is a time-consuming task that involves the interpretation of images by expert radiologists. It is common for fatigue-induced diagnostic error to occur, and especially difficult in areas of the world where radiologists are not available or lack diagnostic expertise. In this research, we proposed a multi-objective deep learning model called CT2Rep (Computed Tomography to Report) for generating lung radiology reports by extracting semantic features from lung CT scans. A total of 458 CT scans were used in this research, from which 107 radiomics features and 6 slices of segmentation related nodule features were extracted for the input of our model. The CT2Rep can simultaneously predict position, margin, and texture, which are three important indicators of lung cancer, and achieves remarkable performance with an F1-score of 87.29%. We conducted a satisfaction survey for estimating the practicality of CT2Rep, and the results show that 95% of the reports received satisfactory ratings. The results demonstrate the great potential in this model for the production of robust and reliable quantitative lung diagnosis reports. Medical personnel can obtain important indicators simply by providing the lung CT scan to the system, which can bring about the widespread application of the proposed framework.

A comparison of machine learning methods for predicting recurrence and death after curative-intent radiotherapy for non-small cell lung cancer: Development and validation of multivariable clinical prediction models

BACKGROUND

Surveillance is universally recommended for non-small cell lung cancer (NSCLC) patients treated with curative-intent radiotherapy. High-quality evidence to inform optimal surveillance strategies is lacking. Machine learning demonstrates promise in accurate outcome prediction for a variety of health conditions. The purpose of this study was to utilise readily available patient, tumour, and treatment data to develop, validate and externally test machine learning models for predicting recurrence, recurrence-free survival (RFS) and overall survival (OS) at 2 years from treatment.

METHODS

A retrospective, multicentre study of patients receiving curative-intent radiotherapy for NSCLC was undertaken. A total of 657 patients from 5 hospitals were eligible for inclusion. Data pre-processing derived 34 features for predictive modelling. Combinations of 8 feature reduction methods and 10 machine learning classification algorithms were compared, producing risk-stratification models for predicting recurrence, RFS and OS. Models were compared with 10-fold cross validation and an external test set and benchmarked against TNM-stage and performance status. Youden Index was derived from validation set ROC curves to distinguish high and low risk groups and Kaplan-Meier analyses performed.

FINDINGS

Median follow-up time was 852 days. Parameters were well matched across training-validation and external test sets: Mean age was 73 and 71 respectively, and recurrence, RFS and OS rates at 2 years were 43% vs 34%, 54% vs 47% and 54% vs 47% respectively. The respective validation and test set AUCs were as follows: 1) RFS: 0·682 (0·575-0·788) and 0·681 (0·597-0·766), 2) Recurrence: 0·687 (0·582-0·793) and 0·722 (0·635-0·81), and 3) OS: 0·759 (0·663-0·855) and 0·717 (0·634-0·8). Our models were superior to TNM stage and performance status in predicting recurrence and OS.

INTERPRETATION

This robust and ready to use machine learning method, validated and externally tested, sets the stage for future clinical trials entailing quantitative personalised risk-stratification and surveillance following curative-intent radiotherapy for NSCLC.

FUNDING

A full list of funding bodies that contributed to this study can be found in the Acknowledgements section.

Evaluation of Prognostic and Predictive Models in the Oncology Clinic

Predictive and prognostic models hold great potential to support clinical decision making in oncology and could ultimately facilitate a paradigm shift to a more personalised form of treatment. While a large number of models relevant to the field of oncology have been developed, few have been translated into clinical use and assessment of clinical utility is not currently considered a routine part of model development. In this narrative review of the clinical evaluation of prediction models in oncology, we propose a high-level process diagram for the life cycle of a clinical model, encompassing model commissioning, clinical implementation and ongoing quality assurance, which aims to bridge the gap between model development and clinical implementation.

Improving Early Identification of Significant Weight Loss Using Clinical Decision Support System in Lung Cancer Radiation Therapy

PURPOSE

Early identification of patients who may be at high risk of significant weight loss (SWL) is important for timely clinical intervention in lung cancer radiotherapy (RT). A clinical decision support system (CDSS) for SWL prediction was implemented within the routine clinical workflow and assessed on a prospective cohort of patients.

MATERIALS AND METHODS

CDSS incorporated a machine learning prediction model on the basis of radiomics and dosiomics image features and was connected to a web-based dashboard for streamlined patient enrollment, feature extraction, SWL prediction, and physicians' evaluation processes. Patients with lung cancer (N = 37) treated with definitive RT without prior RT were prospectively enrolled in the study. Radiomics and dosiomics features were extracted from CT and 3D dose volume, and SWL probability (≥ 0.5 considered as SWL) was predicted. Two physicians predicted whether the patient would have SWL before and after reviewing the CDSS prediction. The physician's prediction performance without and with CDSS and prediction changes before and after using CDSS were compared.

RESULTS

CDSS showed significantly better prediction accuracy than physicians (0.73 v 0.54) with higher specificity (0.81 v 0.50) but with lower sensitivity (0.55 v 0.64). Physicians changed their original prediction after reviewing CDSS prediction for four cases (three correctly and one incorrectly), for all of which CDSS prediction was correct. Physicians' prediction was improved with CDSS in accuracy (0.54-0.59), sensitivity (0.64-0.73), specificity (0.50-0.54), positive predictive value (0.35-0.40), and negative predictive value (0.76-0.82).

CONCLUSION

Machine learning-based CDSS showed the potential to improve SWL prediction in lung cancer RT. More investigation on a larger patient cohort is needed to properly interpret CDSS prediction performance and its benefit in clinical decision making.

Machine learning applications in radiation oncology

Machine learning technology has a growing impact on radiation oncology with an increasing presence in research and industry. The prevalence of diverse data including 3D imaging and the 3D radiation dose delivery presents potential for future automation and scope for treatment improvements for cancer patients. Harnessing this potential requires standardization of tools and data, and focused collaboration between fields of expertise. The rapid advancement of radiation oncology treatment technologies presents opportunities for machine learning integration with investments targeted towards data quality, data extraction, software, and engagement with clinical expertise. In this review, we provide an overview of machine learning concepts before reviewing advances in applying machine learning to radiation oncology and integrating these techniques into the radiation oncology workflows. Several key areas are outlined in the radiation oncology workflow where machine learning has been applied and where it can have a significant impact in terms of efficiency, consistency in treatment and overall treatment outcomes. This review highlights that machine learning has key early applications in radiation oncology due to the repetitive nature of many tasks that also currently have human review. Standardized data management of routinely collected imaging and radiation dose data are also highlighted as enabling engagement in research utilizing machine learning and the ability integrate these technologies into clinical workflow to benefit patients. Physicists need to be part of the conversation to facilitate this technical integration.

Ethical evaluation of artificial intelligence applications in radiotherapy using the Four Topics Approach

Artificial Intelligence is the capability of a machine to imitate intelligent human behavior. An important impact can be expected from Artificial Intelligence throughout the workflow of radiotherapy (such as automated organ segmentation, treatment planning, prediction of outcome and quality assurance). However, ethical concerns regarding the binding agreement between the patient and the physician have followed the introduction of artificial intelligence. Through the recording of personal and social moral values in addition to the usual demographics and the implementation of these as distinctive inputs to matching algorithms, ethical concerns such as consistency, applicability and relevance can be solved. In the meantime, physicians' awareness of the ethical dimension in their decision-making should be challenged, so that they prioritize treating their patients and not diseases, remain vigilant to preserve patient safety, avoid unintended harm and establish institutional policies on these issues.

A Comparative Analysis of Two Machine Learning-Based Diagnostic Patterns with Thyroid Imaging Reporting and Data System for Thyroid Nodules: Diagnostic Performance and Unnecessary Biopsy Rate

Background: The risk stratification system of the American College of Radiology Thyroid Imaging Reporting and Data System (ACR TI-RADS) for thyroid nodules is affected by low diagnostic specificity. Machine learning (ML) methods can optimize the diagnostic performance in medical image analysis. However, it is unknown which ML-based diagnostic pattern is more effective in improving diagnostic performance for thyroid nodules and reducing nodule biopsies. Therefore, we compared ML-assisted visual approaches and radiomics approaches with ACR TI-RADS in diagnostic performance and unnecessary fine-needle aspiration biopsy (FNAB) rate for thyroid nodules. Methods: This retrospective study evaluated a data set of ultrasound (US) and shear wave elastography (SWE) images in patients with biopsy-proven thyroid nodules (≥1 cm) from the Shanghai Tenth People's Hospital (743 nodules in 720 patients from September 2017 to January 2019) and an independent test data set from the Ma'anshan People's Hospital (106 nodules in 102 patients from February 2019 to April 2019). Six US features and five SWE parameters from the radiologists' interpretation were used for building the ML-assisted visual approaches. The radiomics features extracted from the US and SWE images were used with ML methods for developing the radiomics approaches. The diagnostic performance for differentiating thyroid nodules and the unnecessary FNAB rate of the ML-assisted visual approaches and the radiomics approaches were compared with ACR TI-RADS. Results: The ML-assisted US visual approach had the best diagnostic performance than the US radiomics approach and ACR TI-RADS (area under the curve [AUC]: 0.900 vs. 0.789 vs. 0.689 for the validation data set, 0.917 vs. 0.770 vs. 0.681 for the test data set). After adding SWE, the ML-assisted visual approach had a better diagnostic performance than US alone (AUC: 0.951 vs. 0.900 for the validation data set, 0.953 vs. 0.917 for the test data set). When applying the ML-assisted US+SWE visual approach, the unnecessary FNAB rate decreased from 30.0% to 4.5% in the validation data set and from 37.7% to 4.7% in the test data set in comparison to ACR TI-RADS. Conclusions: The ML-assisted dual modalities visual approach can assist radiologists to diagnose thyroid nodules more effectively and considerably reduce the unnecessary FNAB rate in the clinical management of thyroid nodules.

Enhanced computed tomography radiomics-based machine-learning methods for predicting the Fuhrman grades of renal clear cell carcinoma

OBJECTIVE

To develop and test an optimal machine learning model based on the enhanced computed tomography (CT) to preoperatively predict pathological grade of clear cell renal cell carcinoma (ccRCC).

METHODS

A retrospective analysis of 53 pathologically confirmed cases of ccRCC was performed and 25 consecutive ccRCC cases were selected as a prospective testing set. All patients underwent routine preoperative abdominal CT plain and enhanced scans. Renal tumor lesions were segmented on arterial phase images and 396 radiomics features were extracted. In the training set, seven discrimination classifiers for high- and low-grade ccRCCs were constructed based on seven different machine learning models, respectively, and their performance and stability for predicting ccRCC grades were evaluated through receiver operating characteristic (ROC) analysis and cross-validation. Prediction accuracy and area under ROC curve were used as evaluation indices. Finally, the diagnostic efficacy of the optimal model was verified in the testing set.

RESULTS

The accuracies and AUC values achieved by support vector machine with radial basis function kernel (svmRadial), random forest and naïve Bayesian models were 0.860±0.158 and 0.919±0.118, 0.840±0.160 and 0.915±0.138, 0.839±0.147 and 0.921±0.133, respectively, which showed high predictive performance, whereas K-nearest neighborhood model yielded lower accuracy of 0.720±0.188 and lower AUC value of 0.810±0.150. Additionally, svmRadial had smallest relative standard deviation (RSD, 0.13 for AUC, 0.17 for accuracy), which indicates higher stability.

CONCLUSION

svmRadial performs best in predicting pathological grades of ccRCC using radiomics features computed from the preoperative CT images, and thus may have high clinical potential in guiding preoperative decision.

Machine Learning-Based Radiomics Nomogram Using Magnetic Resonance Images for Prediction of Neoadjuvant Chemotherapy Efficacy in Breast Cancer Patients

Purpose: The construction and validation of a radiomics nomogram based on machine learning using magnetic resonance image (MRI) for predicting the efficacy of neoadjuvant chemotherapy (NACT) in patients with breast cancer (BCa).

Methods: This retrospective investigation consisted of 158 patients who were diagnosed with BCa and underwent MRI before NACT, of which 33 patients experienced pathological complete response (pCR) by the postoperative pathological examination. The patients with BCa were divided into the training set (n = 110) and test set (n = 48) randomly. The features were selected by the maximum relevance minimum redundancy (mRMR) and absolute shrinkage and selection operator (LASSO) algorithm in the training set. In return, the radiomics signature was established using machine learning. The predictive score of each patient was calculated using the radiomics signature formula. Finally, the predictive scores and clinical factors were used to perform the multivariate logistic regression and construct the nomogram. Receiver operating characteristics (ROC) analyses were used to assess and validate the diagnostic accuracy of the nomogram in the test set. Lastly, the usefulness of the nomogram was confirmed via decision curve analysis (DCA).

Results: The radiomics signature was well-discriminated in the training set [AUC 0.835, specificity 71.32%, and sensitivity 82.61%], and test set (AUC 0.834, specificity 73.21%, and sensitivity 80%). Containing the radiomics signature and hormone status, the radiomics nomogram showed good calibration and discrimination in the training set [AUC 0.888, specificity 79.31%, and sensitivity 86.96%] and test set (AUC 0.879, specificity 82.19%, and sensitivity 83.57%). The decision curve indicated the clinical usefulness of our nomogram.

Conclusion: Our radiomics nomogram showed good discrimination in patients with BCa who experience pCR after NACT. The model may aid physicians in predicting how specific patients may respond to BCa treatments in the future.

Predicting Lung Cancer Patients' Survival Time via Logistic Regression-based Models in a Quantitative Radiomic Framework

BACKGROUND

Selection of the best treatment modalities for lung cancer depends on many factors, like survival time, which are usually determined by imaging.

OBJECTIVES

To predict the survival time of lung cancer patients using the advantages of both radiomics and logistic regression-based classification models.

MATERIAL AND METHODS

Fifty-nine patients with primary lung adenocarcinoma were included in this retrospective study and pre-treatment contrast-enhanced CT images were acquired. The patients lived more than 2 years were classified as the 'Alive' class and otherwise as the 'Dead' class. In our proposed quantitative radiomic framework, we first extracted the associated regions of each lung lesion from pre-treatment CT images for each patient via grow cut segmentation algorithm. Then, 40 radiomic features were extracted from the segmented lung lesions. In order to enhance the generalizability of the classification models, the mutual information-based feature selection method was applied to each feature vector. We investigated the performance of six logistic regression-based classification models.

RESULTS

It was observed that the mutual information feature selection method can help the classifier to achieve better predictive results. In our study, the Logistic regression (LR) and Dual Coordinate Descent method for Logistic Regression (DCD-LR) models achieved the best results indicating that these classification models have strong potential for classifying the more important class (i.e., the 'Alive' class).

CONCLUSION

The proposed quantitative radiomic framework yielded promising results, which can guide physicians to make better and more precise decisions and increase the chance of treatment success.

Utility of a Clinical Decision Support System in Weight Loss Prediction After Head and Neck Cancer Radiotherapy

PURPOSE

To evaluate the utility of a clinical decision support system (CDSS) using a weight loss prediction model.

METHODS

A prediction model for significant weight loss (loss of greater than or equal to 7.5% of body mass at 3-month post radiotherapy) was created with clinical, dosimetric, and radiomics predictors from 63 patients in an independent training data set (accuracy, 0.78; area under the curve [AUC], 0.81) using least absolute shrinkage and selection operator logistic regression. Four physicians with varying experience levels were then recruited to evaluate 100 patients in an independent validation data set of head and neck cancer twice (ie, a pre-post design): first without and then with the aid of a CDSS derived from the prediction model. At both evaluations, physicians were asked to predict the development (yes/no) and probability of significant weight loss for each patient on the basis of patient characteristics, including pretreatment dysphagia and weight loss and information from the treatment plan. At the second evaluation, physicians were also provided with the prediction model's results for weight loss probability. Physicians' predictions were compared with actual weight loss, and accuracy and AUC were investigated between the two evaluations.

RESULTS

The mean accuracy of the physicians' ability to identify patients who will experience significant weight loss (yes/no) increased from 0.58 (range, 0.47 to 0.63) to 0.63 (range, 0.58 to 0.72) with the CDSS ( P = .06). The AUC of weight loss probability predicted by physicians significantly increased from 0.56 (range, 0.46 to 0.64) to 0.69 (range, 0.63 to 0.73) with the aid of the CDSS ( P < .05). Specifically, more improvement was observed among less-experienced physicians ( P < .01).

CONCLUSION

Our preliminary results demonstrate that physicians' decisions may be improved by a weight loss CDSS model, especially among less-experienced physicians. Additional study with a larger cohort of patients and more participating physicians is thus warranted for understanding the usefulness of CDSSs.

Preoperative MRI-Based Radiomic Machine-Learning Nomogram May Accurately Distinguish Between Benign and Malignant Soft-Tissue Lesions: A Two-Center Study

BACKGROUND

Preoperative differentiation between malignant and benign soft-tissue masses is important for treatment decisions.

PURPOSE/HYPOTHESIS

To construct/validate a radiomics-based machine method for differentiation between malignant and benign soft-tissue masses.

STUDY TYPE

Retrospective.

POPULATION

In all, 206 cases.

FIELD STRENGTH/SEQUENCE

The T₁ sequence was acquired with the following range of parameters: relaxation time / echo time (TR/TE), 352-550/2.75-19 msec. The T₂ sequence was acquired with the following parameters: TR/TE, 700-6370/40-120 msec. The data were divided into a 3.0T training cohort, a 1.5T MR validation cohort, and a 3.0T external validationcohort.

ASSESSMENT

Twelve machine-learning methods were trained to establish classification models to predict the likelihood of malignancy of each lesion. The data of 206 cases were separated into a training set (n = 69) and two validation sets (n = 64, 73, respectively).

STATISTICAL TESTS

1) Demographic characteristics: a one-way analysis of variance (ANOVA) test was performed for continuous variables as appropriate. The χ² test or Fisher's exact test was performed for comparing categorical variables as appropriate. 2) The performance of four feature selection methods (least absolute shrinkage and selection operator [LASSO], Boruta, Recursive feature elimination [RFE, and minimum redundancy maximum relevance [mRMR]) and three classifiers (support vector machine [SVM], generalized linear models [GLM], and random forest [RF]) were compared for selecting the likelihood of malignancy of each lesion. The performance of the radiomics model was assessed using area under the receiver-operating characteristic curve (AUC) and accuracy (ACC) values.

RESULTS

The LASSO feature method + RF classifier achieved the highest AUC of 0.86 and 0.82 in the two validation cohorts. The nomogram achieved AUCs of 0.96 and 0.88, respectively, in the two validation sets, which was higher than that of the radiomic algorithm in the two validation sets and clinical model of the validation 1 set (0.92, 0.88 respectively). The accuracy, sensitivity, and specificity of the radiomics nomogram were 90.5%, 100%, and 80.6%, respectively, for validation set 1; and 80.8%, 75.8%, and 85.0% for validation set 2.

DATA CONCLUSION

A machine-learning nomogram based on radiomics was accurate for distinguishing between malignant and benign soft-tissue masses.

EVIDENCE LEVEL

3 TECHNICAL EFFICACY: Stage 2 J. Magn. Reson. Imaging 2020;52:873-882.

Artificial Intelligence: reshaping the practice of radiological sciences in the 21st century

Advances in computing hardware and software platforms have led to the recent resurgence in artificial intelligence (AI) touching almost every aspect of our daily lives by its capability for automating complex tasks or providing superior predictive analytics. AI applications are currently spanning many diverse fields from economics to entertainment, to manufacturing, as well as medicine. Since modern AI's inception decades ago, practitioners in radiological sciences have been pioneering its development and implementation in medicine, particularly in areas related to diagnostic imaging and therapy. In this anniversary article, we embark on a journey to reflect on the learned lessons from past AI's chequered history. We further summarize the current status of AI in radiological sciences, highlighting, with examples, its impressive achievements and effect on re-shaping the practice of medical imaging and radiotherapy in the areas of computer-aided detection, diagnosis, prognosis, and decision support. Moving beyond the commercial hype of AI into reality, we discuss the current challenges to overcome, for AI to achieve its promised hope of providing better precision healthcare for each patient while reducing cost burden on their families and the society at large.

Designing for Physician Trust: Toward a Machine Learning Decision Aid for Radiation Toxicity Risk

The application of machine learning (ML) technologies in health care is expected to improve care delivery and patient outcomes. However, there are no best practices for designing these technologies for use in clinical settings. To explore user needs and design requirements for a user interface of a ML risk prediction tool in development, we consulted with subject matter experts and physicians. We explored physician expectations of using a ML tool in clinical practice and their preferences on designs. Our process revealed physician perspectives on trusting a ML tool and opportunities to design for these considerations, while navigating ambiguity in the tool’s outputs.

Artificial Intelligence in Radiation Oncology

The integration of artificial intelligence in the radiation oncologist's workflow has multiple applications and significant potential. From the initial patient encounter, artificial intelligence may aid in pretreatment disease outcome and toxicity prediction. It may subsequently aid in treatment planning, and enhanced dose optimization. Artificial intelligence may also optimize the quality assurance process and support a higher level of safety, quality, and efficiency of care. This article describes components of the radiation consultation, planning, and treatment process and how the thoughtful integration of artificial intelligence may improve shared decision making, planning efficiency, planning quality, patient safety, and patient outcomes.

Mining Prognosis Index of Brain Metastases Using Artificial Intelligence

This study is to identify the optimum prognosis index for brain metastases by machine learning. Seven hundred cancer patients with brain metastases were enrolled and divided into 446 training and 254 testing cohorts. Seven features and seven prediction methods were selected to evaluate the performance of cancer prognosis for each patient. We used mutual information and rough set with particle swarm optimization (MIRSPSO) methods to predict patient’s prognosis with the highest accuracy at area under the curve (AUC) = 0.978 ± 0.06. The improvement by MIRSPSO in terms of AUC was at 1.72%, 1.29%, and 1.83% higher than that of the traditional statistical method, sequential feature selection (SFS), mutual information with particle swarm optimization(MIPSO), and mutual information with sequential feature selection (MISFS), respectively. Furthermore, the clinical performance of the best prognosis was superior to conventional statistic method in accuracy, sensitivity, and specificity. In conclusion, identifying optimal machine-learning methods for the prediction of overall survival in brain metastases is essential for clinical applications. The accuracy rate by machine-learning is far higher than that of conventional statistic methods.

Machine Learning-Assisted System for Thyroid Nodule Diagnosis

Background: Ultrasound (US) examination is helpful in the differential diagnosis of thyroid nodules (malignant vs. benign), but its accuracy relies heavily on examiner experience. Therefore, the aim of this study was to develop a less subjective diagnostic model aided by machine learning. Methods: A total of 2064 thyroid nodules (2032 patients, 695 male; M_age = 45.25 ± 13.49 years) met all of the following inclusion criteria: (i) hemi- or total thyroidectomy, (ii) maximum nodule diameter 2.5 cm, (iii) examination by conventional US and real-time elastography within one month before surgery, and (iv) no previous thyroid surgery or percutaneous thermotherapy. Models were developed using 60% of randomly selected samples based on nine commonly used algorithms, and validated using the remaining 40% of cases. All models function with a validation data set that has a pretest probability of malignancy of 10%. The models were refined with machine learning that consisted of 1000 repetitions of derivatization and validation, and compared to diagnosis by an experienced radiologist. Sensitivity, specificity, accuracy, and area under the curve (AUC) were calculated. Results: A random forest algorithm led to the best diagnostic model, which performed better than radiologist diagnosis based on conventional US only (AUC = 0.924 [confidence interval (CI) 0.895-0.953] vs. 0.834 [CI 0.815-0.853]) and based on both conventional US and real-time elastography (AUC = 0.938 [CI 0.914-0.961] vs. 0.843 [CI 0.829-0.857]). Conclusions: Machine-learning algorithms based on US examinations, particularly the random forest classifier, may diagnose malignant thyroid nodules better than radiologists.

Mortality prediction following non-traumatic amputation of the lower extremity

BACKGROUND

Patients who undergo lower extremity amputation secondary to the complications of diabetes or peripheral artery disease have poor long-term survival. Providing patients and surgeons with individual-patient, rather than population, survival estimates provides them with important information to make individualized treatment decisions.

METHODS

Patients with peripheral artery disease and/or diabetes undergoing their first unilateral transmetatarsal, transtibial or transfemoral amputation were identified in the Veterans Affairs Surgical Quality Improvement Program (VASQIP) database. Stepdown logistic regression was used to develop a 1-year mortality risk prediction model from a list of 33 candidate predictors using data from three of five Department of Veterans Affairs national geographical regions. External geographical validation was performed using data from the remaining two regions. Calibration and discrimination were assessed in the development and validation samples.

RESULTS

The development sample included 5028 patients and the validation sample 2140. The final mortality prediction model (AMPREDICT-Mortality) included amputation level, age, BMI, race, functional status, congestive heart failure, dialysis, blood urea nitrogen level, and white blood cell and platelet counts. The model fit in the validation sample was good. The area under the receiver operating characteristic (ROC) curve for the validation sample was 0·76 and Cox calibration regression indicated excellent calibration (slope 0·96, 95 per cent c.i. 0·85 to 1·06; intercept 0·02, 95 per cent c.i. -0·12 to 0·17). Given the external validation characteristics, the development and validation samples were combined, giving a total sample of 7168.

CONCLUSION

The AMPREDICT-Mortality prediction model is a validated parsimonious model that can be used to inform the 1-year mortality risk following non-traumatic lower extremity amputation of patients with peripheral artery disease or diabetes.

Comparison between linear and nonlinear machine-learning algorithms for the classification of thyroid nodules

BACKGROUND

A key challenge in thyroid carcinoma is preoperatively diagnosing malignant thyroid nodules. The purpose of this study was to compare the classification performance of linear and nonlinear machine-learning algorithms for the evaluation of thyroid nodules using pathological reports as reference standard.

METHODS

Ethical approval was obtained for this retrospective analysis, and the informed consent requirement was waived. A total of 1179 thyroid nodules (training cohort, n = 700; validation cohort, n = 479) were confirmed by pathological reports or fine-needle aspiration (FNA) biopsy. The following ultrasonography (US) featu res were measured for each nodule: size (maximum diameter), margins, shape, aspect ratio, capsule, hypoechoic halo, composition, echogenicity, calcification pattern, vascularity, and cervical lymph node status. We analyzed five nonlinear and three linear machine-learning algorithms. The diagnostic performance of each algorithm was compared by using the area under the curve (AUC) of the receiver operating characteristic curve. We repeated this process 1000 times to obtain the mean AUC and 95% confidence interval (CI).

RESULTS

Overall, nonlinear machine-learning algorithms demonstrated similar AUCs compared with linear algorithms. The Random Forest and Kernel Support Vector Machines algorithms achieved slightly greater AUCs in the validation cohort (0.954, 95% CI: 0.939-0.969; 0.954 95%CI: 0.939-0.969, respectively) than other algorithms.

CONCLUSIONS

Overall, nonlinear machine-learning algorithms share similar performance compared with linear algorithms for the evaluation the malignancy risk of thyroid nodules.

The METABANK score: A clinical tool to predict survival after stereotactic radiotherapy for oligometastatic disease

BACKGROUND AND PURPOSE

Stereotactic radiotherapy (SRT, SBRT) is widely used in oligometastatic cancer, but the heterogeneity of the population complicates estimation of the prognosis. We investigated the role of different clinical and inflammatory parameters.

MATERIALS AND METHODS

We included all patients treated with SRT for 1-5 oligometastases between 2003 and 2017 in our center. Patients were randomized between a model training set (2/3) and a separate validation set (1/3). A Cox regression model was built, validated and risk points were attributed to the resulting parameters.

RESULTS

403 patients received SRT for 760 metastases. Treated sites were mainly lung, liver, nodal areas, and brain. Most common primaries were colorectal and lung cancer. Median follow-up for living patients reached 42 months and median overall survival (MS) was 26.6 months (95% CI 23.8-29.3). Five independent adverse factors were discriminated: male sex, synchronous timing of oligometastases, brain metastasis, non-adenocarcinoma histology, KPS <80. A risk score is formed by summation of the points of each factor (M:4, T:2, B:7, N:7, K:8). Four risk groups were defined: (1) 0-2 points: MS 41.2 months (95% CI 30.2-52.3); (2) 3-8 points: 29.3 months (24.6-34.0); (3) 9-13 points: 17.4 months (10.1-24.7), and (4) 14-28 points: 7.9 months (5.5-10.3).

CONCLUSION

We propose a prognostic score applicable in a variety of primary tumors and disease locations, including presence of brain metastases. The nomogram and risk groups can be used to stratify patients in new trials and to support individualized care for oligometastatic patients. An online calculator will become available at predictcancer.org.

Automated data extraction and ensemble methods for predictive modeling of breast cancer outcomes after radiation therapy

PURPOSE

The purpose of this study was to compare the effectiveness of ensemble methods (e.g., random forests) and single-model methods (e.g., logistic regression and decision trees) in predictive modeling of post-RT treatment failure and adverse events (AEs) for breast cancer patients using automatically extracted EMR data.

METHODS

Data from 1967 consecutive breast radiotherapy (RT) courses at one institution between 2008 and 2015 were automatically extracted from EMRs and oncology information systems using extraction software. Over 230 variables were extracted spanning the following variable segments: patient demographics, medical/surgical history, tumor characteristics, RT treatment history, and AEs tracked using CTCAEv4.0. Treatment failure was extracted algorithmically by searching posttreatment encounters for evidence of local, nodal, or distant failure. Individual models were trained using decision trees, logistic regression, random forests, and boosted decision trees to predict treatment failures and AEs. Models were fit on 75% of the data and evaluated for probability calibration and area under the ROC curve (AUC) on the remaining test set. The impact of each variable segment was assessed by retraining without the segment and measuring change in AUC (ΔAUC).

RESULTS

All AUC values were statistically significant (P < 0.05). Ensemble methods outperformed single-model methods across all outcomes. The best ensemble method outperformed decision trees and logistic regression by an average AUC of 0.053 and 0.034, respectively. Model probabilities were well calibrated as evidenced by calibration curves. Excluding the patient medical history variable segment led to the largest AUC reduction in all models (Average ΔAUC = -0.025), followed by RT treatment history (-0.021) and tumor information (-0.015).

CONCLUSION

In this largest such study in breast cancer performed to date, automatically extracted EMR data provided a basis for reliable outcome predictions across multiple statistical methods. Ensemble methods provided substantial advantages over single-model methods. Patient medical history contributed the most to prediction quality.

Deep learning for lung cancer prognostication: A retrospective multi-cohort radiomics study

BACKGROUND

Non-small-cell lung cancer (NSCLC) patients often demonstrate varying clinical courses and outcomes, even within the same tumor stage. This study explores deep learning applications in medical imaging allowing for the automated quantification of radiographic characteristics and potentially improving patient stratification.

METHODS AND FINDINGS

We performed an integrative analysis on 7 independent datasets across 5 institutions totaling 1,194 NSCLC patients (age median = 68.3 years [range 32.5-93.3], survival median = 1.7 years [range 0.0-11.7]). Using external validation in computed tomography (CT) data, we identified prognostic signatures using a 3D convolutional neural network (CNN) for patients treated with radiotherapy (n = 771, age median = 68.0 years [range 32.5-93.3], survival median = 1.3 years [range 0.0-11.7]). We then employed a transfer learning approach to achieve the same for surgery patients (n = 391, age median = 69.1 years [range 37.2-88.0], survival median = 3.1 years [range 0.0-8.8]). We found that the CNN predictions were significantly associated with 2-year overall survival from the start of respective treatment for radiotherapy (area under the receiver operating characteristic curve [AUC] = 0.70 [95% CI 0.63-0.78], p < 0.001) and surgery (AUC = 0.71 [95% CI 0.60-0.82], p < 0.001) patients. The CNN was also able to significantly stratify patients into low and high mortality risk groups in both the radiotherapy (p < 0.001) and surgery (p = 0.03) datasets. Additionally, the CNN was found to significantly outperform random forest models built on clinical parameters-including age, sex, and tumor node metastasis stage-as well as demonstrate high robustness against test-retest (intraclass correlation coefficient = 0.91) and inter-reader (Spearman's rank-order correlation = 0.88) variations. To gain a better understanding of the characteristics captured by the CNN, we identified regions with the most contribution towards predictions and highlighted the importance of tumor-surrounding tissue in patient stratification. We also present preliminary findings on the biological basis of the captured phenotypes as being linked to cell cycle and transcriptional processes. Limitations include the retrospective nature of this study as well as the opaque black box nature of deep learning networks.

CONCLUSIONS

Our results provide evidence that deep learning networks may be used for mortality risk stratification based on standard-of-care CT images from NSCLC patients. This evidence motivates future research into better deciphering the clinical and biological basis of deep learning networks as well as validation in prospective data.

Machine learning and modeling: Data, validation, communication challenges

With the era of big data, the utilization of machine learning algorithms in radiation oncology is rapidly growing with applications including: treatment response modeling, treatment planning, contouring, organ segmentation, image-guidance, motion tracking, quality assurance, and more. Despite this interest, practical clinical implementation of machine learning as part of the day-to-day clinical operations is still lagging. The aim of this white paper is to further promote progress in this new field of machine learning in radiation oncology by highlighting its untapped advantages and potentials for clinical advancement, while also presenting current challenges and open questions for future research. The targeted audience of this paper includes newcomers as well as practitioners in the field of medical physics/radiation oncology. The paper also provides general recommendations to avoid common pitfalls when applying these powerful data analytic tools to medical physics and radiation oncology problems and suggests some guidelines for transparent and informative reporting of machine learning results.

Artificial intelligence in radiation oncology: A specialty-wide disruptive transformation?

Artificial intelligence (AI) is emerging as a technology with the power to transform established industries, and with applications from automated manufacturing to advertising and facial recognition to fully autonomous transportation. Advances in each of these domains have led some to call AI the "fourth" industrial revolution [1]. In healthcare, AI is emerging as both a productive and disruptive force across many disciplines. This is perhaps most evident in Diagnostic Radiology and Pathology, specialties largely built around the processing and complex interpretation of medical images, where the role of AI is increasingly seen as both a boon and a threat. In Radiation Oncology as well, AI seems poised to reshape the specialty in significant ways, though the impact of AI has been relatively limited at present, and may rightly seem more distant to many, given the predominantly interpersonal and complex interventional nature of the specialty. In this overview, we will explore the current state and anticipated future impact of AI on Radiation Oncology, in detail, focusing on key topics from multiple stakeholder perspectives, as well as the role our specialty may play in helping to shape the future of AI within the larger spectrum of medicine.

Functional promoter rs189037 variant of ATM is associated with decrease in lung diffusing capacity after irradiation for non-small-cell lung cancer

Objective

Single-nucleotide polymorphisms (SNPs) in the ataxia telangiectasia-mutated gene ATM have been linked with pneumonitis after radiotherapy for lung cancer but have not been evaluated in terms of pulmonary function impairment. Here we investigated potential associations between SNPs in ATM and changes in diffusing capacity of the lung for carbon monoxide (DLCO) in patients with non-small-cell lung cancer (NSCLC) after radiotherapy.

Methods

From November 1998 through June 2009, 448 consecutive patients with inoperable primary NSCLC underwent definitive (≥60 Gy) radiotherapy, with or without chemotherapy. After excluding patients with a history of thoracic surgery, radiation, or lung cancer; without DNA samples available for analysis; or without pulmonary function testing within the 12 months before and the 12 months after radiotherapy, 100 patients were identified who are the subjects of this study. We genotyped two SNPs of ATM previously found to be associated with radiation-induced pneumonitis (rs189037 and rs228590) and evaluated potential correlations between these SNPs and impairment (decreases) in DLCO by using logistic regression analysis.

Results

Univariate and multivariate analyses showed that the AA genotype of ATM rs189037 was associated with decreased DLCO after definitive radiotherapy than the GG/AG genotypes [univariate coefficient, -0.122; 95% confidence interval (CI), -0.236 to -0.008; P = 0.037; and multivariate coefficient, -0.102; 95% CI, -0.198 to -0.005; P = 0.038]. No such correlations were found for rs228590 (univariate coefficient, -0.096; 95% CI, -0.208 to 0.017; P = 0.096).

Conclusions

The AA genotype of ATM rs189037 was associated with higher risk of lung injury than were the GG/AG genotypes in patients with NSCLC treated with radiotherapy. This finding should be validated prospectively with other patient populations.

18F-fluorodeoxyglucose positron-emission tomography (FDG-PET)-Radiomics of metastatic lymph nodes and primary tumor in non-small cell lung cancer (NSCLC) - A prospective externally validated study

BACKGROUND

Lymph node stage prior to treatment is strongly related to disease progression and poor prognosis in non-small cell lung cancer (NSCLC). However, few studies have investigated metabolic imaging features derived from pre-radiotherapy 18F-fluorodeoxyglucose (FDG) positron-emission tomography (PET) of metastatic hilar/mediastinal lymph nodes (LNs). We hypothesized that these would provide complementary prognostic information to FDG-PET descriptors to only the primary tumor (tumor).

METHODS

Two independent cohorts of 262 and 50 node-positive NSCLC patients were used for model development and validation. Image features (i.e. Radiomics) including shape and size, first order statistics, texture, and intensity-volume histograms (IVH) (http://www.radiomics.io/) were evaluated by univariable Cox regression on the development cohort. Prognostic modeling was conducted with a 10-fold cross-validated least absolute shrinkage and selection operator (LASSO), automatically selecting amongst FDG-PET-Radiomics descriptors from (1) tumor, (2) LNs or (3) both structures. Performance was assessed with the concordance-index. Development data are publicly available at www.cancerdata.org and Dryad (doi:10.5061/dryad.752153b).

RESULTS

Common SUV descriptors (maximum, peak, and mean) were significantly related to overall survival when extracted from LNs, as were LN volume and tumor load (summed tumor and LNs' volumes), though this was not true for either SUV metrics or tumor's volume. Feature selection exclusively from imaging information based on FDG-PET-Radiomics, exhibited performances of (1) 0.53 -external 0.54, when derived from the tumor, (2) 0.62 -external 0.56 from LNs, and (3) 0.62 -external 0.59 from both structures, including at least one feature from each sub-category, except IVH.

CONCLUSION

Combining imaging information based on FDG-PET-Radiomics features from tumors and LNs is desirable to achieve a higher prognostic discriminative power for NSCLC.

Radiomic features analysis in computed tomography images of lung nodule classification

Purpose

Radiomics, which extract large amount of quantification image features from diagnostic medical images had been widely used for prognostication, treatment response prediction and cancer detection. The treatment options for lung nodules depend on their diagnosis, benign or malignant. Conventionally, lung nodule diagnosis is based on invasive biopsy. Recently, radiomics features, a non-invasive method based on clinical images, have shown high potential in lesion classification, treatment outcome prediction.

Methods

Lung nodule classification using radiomics based on Computed Tomography (CT) image data was investigated and a 4-feature signature was introduced for lung nodule classification. Retrospectively, 72 patients with 75 pulmonary nodules were collected. Radiomics feature extraction was performed on non-enhanced CT images with contours which were delineated by an experienced radiation oncologist.

Result

Among the 750 image features in each case, 76 features were found to have significant differences between benign and malignant lesions. A radiomics signature was composed of the best 4 features which included Laws_LSL_min, Laws_SLL_energy, Laws_SSL_skewness and Laws_EEL_uniformity. The accuracy using the signature in benign or malignant classification was 84% with the sensitivity of 92.85% and the specificity of 72.73%.

Conclusion

The classification signature based on radiomics features demonstrated very good accuracy and high potential in clinical application.