Federated learning enables big data for rare cancer boundary detection

Although machine learning (ML) has shown promise across disciplines, out-of-sample generalizability is concerning. This is currently addressed by sharing multi-site data, but such centralization is challenging/infeasible to scale due to various limitations. Federated ML (FL) provides an alternative paradigm for accurate and generalizable ML, by only sharing numerical model updates. Here we present the largest FL study to-date, involving data from 71 sites across 6 continents, to generate an automatic tumor boundary detector for the rare disease of glioblastoma, reporting the largest such dataset in the literature (n = 6, 314). We demonstrate a 33% delineation improvement for the surgically targetable tumor, and 23% for the complete tumor extent, over a publicly trained model. We anticipate our study to: 1) enable more healthcare studies informed by large diverse data, ensuring meaningful results for rare diseases and underrepresented populations, 2) facilitate further analyses for glioblastoma by releasing our consensus model, and 3) demonstrate the FL effectiveness at such scale and task-complexity as a paradigm shift for multi-site collaborations, alleviating the need for data-sharing.

Imaging Biomarkers of Glioblastoma Treatment Response: A Systematic Review and Meta-Analysis of Recent Machine Learning Studies

OBJECTIVE

Monitoring biomarkers using machine learning (ML) may determine glioblastoma treatment response. We systematically reviewed quality and performance accuracy of recently published studies.

METHODS

Following Preferred Reporting Items for Systematic Reviews and Meta-Analysis: Diagnostic Test Accuracy, we extracted articles from MEDLINE, EMBASE and Cochrane Register between 09/2018-01/2021. Included study participants were adults with glioblastoma having undergone standard treatment (maximal resection, radiotherapy with concomitant and adjuvant temozolomide), and follow-up imaging to determine treatment response status (specifically, distinguishing progression/recurrence from progression/recurrence mimics, the target condition). Using Quality Assessment of Diagnostic Accuracy Studies Two/Checklist for Artificial Intelligence in Medical Imaging, we assessed bias risk and applicability concerns. We determined test set performance accuracy (sensitivity, specificity, precision, F1-score, balanced accuracy). We used a bivariate random-effect model to determine pooled sensitivity, specificity, area-under the receiver operator characteristic curve (ROC-AUC). Pooled measures of balanced accuracy, positive/negative likelihood ratios (PLR/NLR) and diagnostic odds ratio (DOR) were calculated. PROSPERO registered (CRD42021261965).

RESULTS

Eighteen studies were included (1335/384 patients for training/testing respectively). Small patient numbers, high bias risk, applicability concerns (particularly confounding in reference standard and patient selection) and low level of evidence, allow limited conclusions from studies. Ten studies (10/18, 56%) included in meta-analysis gave 0.769 (0.649-0.858) sensitivity [pooled (95% CI)]; 0.648 (0.749-0.532) specificity; 0.706 (0.623-0.779) balanced accuracy; 2.220 (1.560-3.140) PLR; 0.366 (0.213-0.572) NLR; 6.670 (2.800-13.500) DOR; 0.765 ROC-AUC.

CONCLUSION

ML models using MRI features to distinguish between progression and mimics appear to demonstrate good diagnostic performance. However, study quality and design require improvement.

Glioblastoma and radiotherapy: A multicenter AI study for Survival Predictions from MRI (GRASP study)

BACKGROUND

The aim was to predict survival of glioblastoma at 8 months after radiotherapy (a period allowing for completing a typical course of adjuvant temozolomide), by applying deep learning to the first brain MRI after radiotherapy completion.

METHODS

Retrospective and prospective data were collected from 206 consecutive glioblastoma, isocitrate dehydrogenase -wildtype patients diagnosed between March 2014 and February 2022 across 11 UK centers. Models were trained on 158 retrospective patients from 3 centers. Holdout test sets were retrospective (n = 19; internal validation), and prospective (n = 29; external validation from 8 distinct centers). Neural network branches for T2-weighted and contrast-enhanced T1-weighted inputs were concatenated to predict survival. A nonimaging branch (demographics/MGMT/treatment data) was also combined with the imaging model. We investigated the influence of individual MR sequences; nonimaging features; and weighted dense blocks pretrained for abnormality detection.

RESULTS

The imaging model outperformed the nonimaging model in all test sets (area under the receiver-operating characteristic curve, AUC P = .038) and performed similarly to a combined imaging/nonimaging model (P > .05). Imaging, nonimaging, and combined models applied to amalgamated test sets gave AUCs of 0.93, 0.79, and 0.91. Initializing the imaging model with pretrained weights from 10 000s of brain MRIs improved performance considerably (amalgamated test sets without pretraining 0.64; P = .003).

CONCLUSIONS

A deep learning model using MRI images after radiotherapy reliably and accurately determined survival of glioblastoma. The model serves as a prognostic biomarker identifying patients who will not survive beyond a typical course of adjuvant temozolomide, thereby stratifying patients into those who might require early second-line or clinical trial treatment.

Overcoming challenges of translating deep-learning models for glioblastoma: the ZGBM consortium

OBJECTIVE

To report imaging protocol and scheduling variance in routine care of glioblastoma patients in order to demonstrate challenges of integrating deep-learning models in glioblastoma care pathways. Additionally, to understand the most common imaging studies and image contrasts to inform the development of potentially robust deep-learning models.

METHODS

MR imaging data were analysed from a random sample of five patients from the prospective cohort across five participating sites of the ZGBM consortium. Reported clinical and treatment data alongside DICOM header information were analysed to understand treatment pathway imaging schedules.

RESULTS

All sites perform all structural imaging at every stage in the pathway except for the presurgical study, where in some sites only contrast-enhanced T1-weighted imaging is performed. Diffusion MRI is the most common non-structural imaging type, performed at every site.

CONCLUSION

The imaging protocol and scheduling varies across the UK, making it challenging to develop machine-learning models that could perform robustly at other centres. Structural imaging is performed most consistently across all centres.

ADVANCES IN KNOWLEDGE

Successful translation of deep-learning models will likely be based on structural post-treatment imaging unless there is significant effort made to standardise non-structural or peri-operative imaging protocols and schedules.

Efficacy of attention bias modification via smartphones in a large population sample

Negative affective biases are a key feature of anxiety and depression that uphold and promote negative mood. Bias modification aims to reduce these biases using computerized training, but shows mixed success and has not been tested at scale. The aim was to determine whether bias modification delivered via smartphones can improve mood in a large sample. In total, 153 385 self-referring participants were randomly assigned to modification or sham bias training on a dot-probe or visual-search task. The primary outcome of interest was balance of mood, assessed on the Positive and Negative Affect Schedule. In total, 22 933 participants who provided at least two mood ratings were included in analyses. There was a large amount of participant attrition. In the remaining smaller sample, results supported the prediction that visual-search modification would result in improved mood (95%CI [0.10, 0.82]; p = 0.01, d = 0.05, N = 2588 after two ratings; 95%CI [1.75,6.54]; p = 0.001, d = 0.32, N = 118 after six ratings), which was not seen for the sham version (N = 4818 after two ratings; N = 138 after six ratings). Dot-probe modification was not associated with mood improvements (p = 0.52). Visual-search, but not dot-probe, bias modification slightly but significantly improved mood. Although this effect size is very small and subject to large participant drop-off, it might be worth considering an adjunct to current treatments.

Change in infection control practices and awareness of hospital medical staff in the aftermath of SARS

Severe Acute Respiratory Syndrome (SARS) epidemic illustrated the crucial role of infection surveillance and control measures in the combat of any highly transmissible disease. We conducted an interview survey of 121 medical staff 145 doctors, 46 staff nurses and 30 medical assistants) in a state hospital in Malaysia three months after the end of SARS epidemic (from October to December 2003). Staff was grouped according to those directly involved in the care of suspected SARS patients [S+ group n=41] and those who were not [S- group; n=80]. On hand washing following sneezing, coughing and touching patients, the proportions of medical staff that reported an increase after the SARS crisis were 22.3%, 16.5% and 45.5% respectively. On wearing masks, gloves, and aprons when meeting potentially infectious patients, the proportions that reported an increase were 39.7%, 47.1% and 32.2% respectively. Significantly more staff in S+ than S- group reported these increases. Sixty percent of staff was aware of changes in hospital infection control policies after SARS; 93.4% was aware of notifying procedures, and 81.8% was aware of whom to notify in the hospital. Regarding infection isolation ward, Infectious Control Nurse and Infection Control Committee Chairman in the hospital, the proportions of staff that could correctly name them were 39.7%, 38.3% and 15.7% respectively. Significantly more in S+ than S- group could do so. However, more than half the staff claimed ignorance on the knowledge of infection isolation ward (56.2%), Infection Control Nurse (57.9%) and Chairman (65.3%). Our findings demonstrated that SARS crisis had some positive impact on the infection control practices and awareness of medical staff especially on those with direct SARS involvement. Implications for future control of infectious diseases are obvious.

OS08.6.A Glioblastoma treatment response machine learning monitoring biomarkers: a systematic review and meta-analysis

BACKGROUND

The aim of the systematic review was to assess recently published studies on diagnostic test accuracy of glioblastoma treatment response monitoring biomarkers in adults, developed through machine learning (ML).

MATERIAL AND METHODS

PRISMA methodology was followed. Articles published 09/2018-01/2021 (since previous reviews) were searched for using MEDLINE, EMBASE, and the Cochrane Register by two reviewers independently. Included study participants were adult patients with high grade glioma who had undergone standard treatment (maximal resection, radiotherapy with concomitant and adjuvant temozolomide) and subsequently underwent follow-up imaging to determine treatment response status (specifically, distinguishing progression/recurrence from progression/recurrence mimics - the target condition). Risk of bias and applicability was assessed with QUADAS 2. A third reviewer arbitrated any discrepancy. Contingency tables were created for hold-out test sets and recall, specificity, precision, F1-score, balanced accuracy calculated. A meta-analysis was performed using a bivariate model for recall, false positive rate and area-under the receiver operator characteristic curve (AUC).

RESULTS

Eighteen studies were included with 1335 patients in training sets and 384 in test sets. To determine whether there was progression or a mimic, the reference standard combination of follow-up imaging and histopathology at re-operation was applied in 67% (13/18) of studies. The small numbers of patient included in studies, the high risk of bias and concerns of applicability in the study designs (particularly in relation to the reference standard and patient selection due to confounding), and the low level of evidence, suggest that limited conclusions can be drawn from the data. Ten studies (10/18, 56%) had internal or external hold-out test set data that could be included in a meta-analysis of monitoring biomarker studies. The pooled sensitivity was 0.77 (0.65–0.86). The pooled false positive rate (1-specificity) was 0.35 (0.25–0.47). The summary point estimate for the AUC was 0.77.

CONCLUSION

There is likely good diagnostic performance of machine learning models that use MRI features to distinguish between progression and mimics. The diagnostic performance of ML using implicit features did not appear to be superior to ML using explicit features. There are a range of ML-based solutions poised to become treatment response monitoring biomarkers for glioblastoma. To achieve this, the development and validation of ML models require large, well-annotated datasets where the potential for confounding in the study design has been carefully considered. Therefore, multidisciplinary efforts and multicentre collaborations are necessary.

Efficacy of attention bias modification via smartphones in a large population sample

Negative affective biases are a key feature of anxiety and depression that uphold and promote negative mood. Bias modification aims to reduce these biases using computerized training, but shows mixed success and has not been tested at scale. The aim was to determine whether bias modification delivered via smartphones can improve mood in a large sample. In total, 153 385 self-referring participants were randomly assigned to modification or sham bias training on a dot-probe or visual-search task. The primary outcome of interest was balance of mood, assessed on the Positive and Negative Affect Schedule. In total, 22 933 participants who provided at least two mood ratings were included in analyses. There was a large amount of participant attrition. In the remaining smaller sample, results supported the prediction that visual-search modification would result in improved mood (95%CI [0.10, 0.82]; p = 0.01, d = 0.05, N = 2588 after two ratings; 95%CI [1.75,6.54]; p = 0.001, d = 0.32, N = 118 after six ratings), which was not seen for the sham version (N = 4818 after two ratings; N = 138 after six ratings). Dot-probe modification was not associated with mood improvements (p = 0.52). Visual-search, but not dot-probe, bias modification slightly but significantly improved mood. Although this effect size is very small and subject to large participant drop-off, it might be worth considering an adjunct to current treatments.