Comparison of volumetric and 2D-based response methods in the PNOC-001 pediatric low-grade glioma clinical trial

Background

Although response in pediatric low-grade glioma (pLGG) includes volumetric assessment, more simplified 2D-based methods are often used in clinical trials. The study's purpose was to compare volumetric to 2D methods.

Methods

An expert neuroradiologist performed solid and whole tumor (including cyst and edema) volumetric measurements on MR images using a PACS-based manual segmentation tool in 43 pLGG participants (213 total follow-up images) from the Pacific Pediatric Neuro-Oncology Consortium (PNOC-001) trial. Classification based on changes in volumetric and 2D measurements of solid tumor were compared to neuroradiologist visual response assessment using the Brain Tumor Reporting and Data System (BT-RADS) criteria for a subset of 65 images using receiver operating characteristic (ROC) analysis. Longitudinal modeling of solid tumor volume was used to predict BT-RADS classification in 54 of the 65 images.

Results

There was a significant difference in ROC area under the curve between 3D solid tumor volume and 2D area (0.96 vs 0.78, P = .005) and between 3D solid and 3D whole volume (0.96 vs 0.84, P = .006) when classifying BT-RADS progressive disease (PD). Thresholds of 15-25% increase in 3D solid tumor volume had an 80% sensitivity in classifying BT-RADS PD included in their 95% confidence intervals. The longitudinal model of solid volume response had a sensitivity of 82% and a positive predictive value of 67% for detecting BT-RADS PD.

Conclusions

Volumetric analysis of solid tumor was significantly better than 2D measurements in classifying tumor progression as determined by BT-RADS criteria and will enable more comprehensive clinical management.

Application of novel PACS-based informatics platform to identify imaging based predictors of CDKN2A allelic status in glioblastomas

Gliomas with CDKN2A mutations are known to have worse prognosis but imaging features of these gliomas are unknown. Our goal is to identify CDKN2A specific qualitative imaging biomarkers in glioblastomas using a new informatics workflow that enables rapid analysis of qualitative imaging features with Visually AcceSAble Rembrandtr Images (VASARI) for large datasets in PACS. Sixty nine patients undergoing GBM resection with CDKN2A status determined by whole-exome sequencing were included. GBMs on magnetic resonance images were automatically 3D segmented using deep learning algorithms incorporated within PACS. VASARI features were assessed using FHIR forms integrated within PACS. GBMs without CDKN2A alterations were significantly larger (64 vs. 30%, p = 0.007) compared to tumors with homozygous deletion (HOMDEL) and heterozygous loss (HETLOSS). Lesions larger than 8 cm were four times more likely to have no CDKN2A alteration (OR: 4.3; 95% CI 1.5-12.1; p < 0.001). We developed a novel integrated PACS informatics platform for the assessment of GBM molecular subtypes and show that tumors with HOMDEL are more likely to have radiographic evidence of pial invasion and less likely to have deep white matter invasion or subependymal invasion. These imaging features may allow noninvasive identification of CDKN2A allele status.

Systematic Literature Review of Machine Learning Algorithms Using Pretherapy Radiologic Imaging for Glioma Molecular Subtype Prediction

BACKGROUND

The molecular profile of gliomas is a prognostic indicator for survival, driving clinical decision-making for treatment. Pathology-based molecular diagnosis is challenging because of the invasiveness of the procedure, exclusion from neoadjuvant therapy options, and the heterogeneous nature of the tumor.

PURPOSE

We performed a systematic review of algorithms that predict molecular subtypes of gliomas from MR Imaging.

DATA SOURCES

Data sources were Ovid Embase, Ovid MEDLINE, Cochrane Central Register of Controlled Trials, Web of Science.

STUDY SELECTION

Per the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, 12,318 abstracts were screened and 1323 underwent full-text review, with 85 articles meeting the inclusion criteria.

DATA ANALYSIS

We compared prediction results from different machine learning approaches for predicting molecular subtypes of gliomas. Bias analysis was conducted for each study, following the Prediction model Risk Of Bias Assessment Tool (PROBAST) guidelines.

DATA SYNTHESIS

Isocitrate dehydrogenase mutation status was reported with an area under the curve and accuracy of 0.88 and 85% in internal validation and 0.86 and 87% in limited external validation data sets, respectively. For the prediction of O6-methylguanine-DNA methyltransferase promoter methylation, the area under the curve and accuracy in internal validation data sets were 0.79 and 77%, and in limited external validation, 0.89 and 83%, respectively. PROBAST scoring demonstrated high bias in all articles.

LIMITATIONS

The low number of external validation and studies with incomplete data resulted in unequal data analysis. Comparing the best prediction pipelines of each study may introduce bias.

CONCLUSIONS

While the high area under the curve and accuracy for the prediction of molecular subtypes of gliomas are reported in internal and external validation data sets, limited use of external validation and the increased risk of bias in all articles may present obstacles for clinical translation of these techniques.

NIMG-07. APPLYING A GLIOMA-TRAINED DEEP LEARNING AUTO-SEGMENTATION TOOL ON BM PRE- AND POST-RADIOSURGERY

PURPOSE

Stereotactic radiosurgery (SRS) has become the mainstay to treat BM. Follow-up MRI provides important information on lesion treatment response and guides future therapy planning. Volumetric measurements of BM have shown promise over traditional uni- and two-dimensional measurements in more accurate and repeatable assessment. However, routine clinical use has yet to be achieved because the workflow is laborious. In previous work, we developed a PACS-integrated deep learning algorithm for automatic high- and low-grade glioma 3D segmentation. In this work, we applied this U-Net to segment BM on pre- and post-Gamma Knife (GK) MRI and evaluated the performance.

METHODS

10 pre- and post-GK studies were autosegmented in five randomly selected patients (melanoma n= 3, breast n= 2). The glioma trained algorithm segmented the “Whole Tumor” (tumor core+peritumoral edema on T2w-FLAIR) and “Tumor Core” (CE tumor core+necrosis on SPGR). The AI generated segmentation was then revised as needed by a board-certified neuroradiologist and the dice-similarity-coefficient (DSC) between the revised and automatic volumetric segmentations were calculated.

RESULTS

Four patients had multicentric (2-4 BM) lesions. The mean± SD DSC for Whole Tumor and Tumor Core were 0.92±0.06 and 0.46±0.30 for pretreatment, 0.84±0.09 and 0.41±0.25 for posttreatment BM, respectively. The tool detected lesions with a sensitivity of 45% (5/11) for pretreatment and 50% (3/6) for posttreatment lesions. Three pretreatment and all posttreatment lesions that were not detected by the autosegmentation tool showed a very faint hyperintense peritumoral edema in T2w-FLAIR.

CONCLUSION

Volumetric segmentation of edema on FLAIR using the glioma-trained segmentation algorithm on pre- and post-GK BM did not require major adjustment of segmentation if it detects the lesion. On the other hand, with low sensitivity of lesion detection and low DSC for enhancing component, dedicated training of the algorithm on annotated BM data will be needed.

Machine Learning Models for Classifying High- and Low-Grade Gliomas: A Systematic Review and Quality of Reporting Analysis

Objectives

To systematically review, assess the reporting quality of, and discuss improvement opportunities for studies describing machine learning (ML) models for glioma grade prediction.

Methods

This study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses of Diagnostic Test Accuracy (PRISMA-DTA) statement. A systematic search was performed in September 2020, and repeated in January 2021, on four databases: Embase, Medline, CENTRAL, and Web of Science Core Collection. Publications were screened in Covidence, and reporting quality was measured against the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) Statement. Descriptive statistics were calculated using GraphPad Prism 9.

Results

The search identified 11,727 candidate articles with 1,135 articles undergoing full text review and 85 included in analysis. 67 (79%) articles were published between 2018-2021. The mean prediction accuracy of the best performing model in each study was 0.89 ± 0.09. The most common algorithm for conventional machine learning studies was Support Vector Machine (mean accuracy: 0.90 ± 0.07) and for deep learning studies was Convolutional Neural Network (mean accuracy: 0.91 ± 0.10). Only one study used both a large training dataset (n>200) and external validation (accuracy: 0.72) for their model. The mean adherence rate to TRIPOD was 44.5% ± 11.1%, with poor reporting adherence for model performance (0%), abstracts (0%), and titles (0%).

Conclusions

The application of ML to glioma grade prediction has grown substantially, with ML model studies reporting high predictive accuracies but lacking essential metrics and characteristics for assessing model performance. Several domains, including generalizability and reproducibility, warrant further attention to enable translation into clinical practice.

Systematic Review Registration

PROSPERO, identifier CRD42020209938.

Identifying clinically applicable machine learning algorithms for glioma segmentation: recent advances and discoveries

Background

While there are innumerable machine learning (ML) research algorithms used for segmentation of gliomas, there is yet to be a US FDA cleared product. The aim of this study is to explore the systemic limitations of research algorithms that have prevented translation from concept to product by a review of the current research literature.

Methods

We performed a systematic literature review on 4 databases. Of 11 727 articles, 58 articles met the inclusion criteria and were used for data extraction and screening using TRIPOD.

Results

We found that while many articles were published on ML-based glioma segmentation and report high accuracy results, there were substantial limitations in the methods and results portions of the papers that result in difficulty reproducing the methods and translation into clinical practice.

Conclusions

In addition, we identified that more than a third of the articles used the same publicly available BRaTS and TCIA datasets and are responsible for the majority of patient data on which ML algorithms were trained, which leads to limited generalizability and potential for overfitting and bias.

Trends in Development of Novel Machine Learning Methods for the Identification of Gliomas in Datasets That Include Non-Glioma Images: A Systematic Review

Purpose

Machine learning has been applied to the diagnostic imaging of gliomas to augment classification, prognostication, segmentation, and treatment planning. A systematic literature review was performed to identify how machine learning has been applied to identify gliomas in datasets which include non-glioma images thereby simulating normal clinical practice.

Materials and Methods

Four databases were searched by a medical librarian and confirmed by a second librarian for all articles published prior to February 1, 2021: Ovid Embase, Ovid MEDLINE, Cochrane trials (CENTRAL), and Web of Science-Core Collection. The search strategy included both keywords and controlled vocabulary combining the terms for: artificial intelligence, machine learning, deep learning, radiomics, magnetic resonance imaging, glioma, as well as related terms. The review was conducted in stepwise fashion with abstract screening, full text screening, and data extraction. Quality of reporting was assessed using TRIPOD criteria.

Results

A total of 11,727 candidate articles were identified, of which 12 articles were included in the final analysis. Studies investigated the differentiation of normal from abnormal images in datasets which include gliomas (7 articles) and the differentiation of glioma images from non-glioma or normal images (5 articles). Single institution datasets were most common (5 articles) followed by BRATS (3 articles). The median sample size was 280 patients. Algorithm testing strategies consisted of five-fold cross validation (5 articles), and the use of exclusive sets of images within the same dataset for training and for testing (7 articles). Neural networks were the most common type of algorithm (10 articles). The accuracy of algorithms ranged from 0.75 to 1.00 (median 0.96, 10 articles). Quality of reporting assessment utilizing TRIPOD criteria yielded a mean individual TRIPOD ratio of 0.50 (standard deviation 0.14, range 0.37 to 0.85).

Conclusion

Systematic review investigating the identification of gliomas in datasets which include non-glioma images demonstrated multiple limitations hindering the application of these algorithms to clinical practice. These included limited datasets, a lack of generalizable algorithm training and testing strategies, and poor quality of reporting. The development of more robust and heterogeneous datasets is needed for algorithm development. Future studies would benefit from using external datasets for algorithm testing as well as placing increased attention on quality of reporting standards.

Systematic Review Registration

www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42020209938, International Prospective Register of Systematic Reviews (PROSPERO 2020 CRD42020209938).

NIMG-46. SYSTEMATIC LITERATURE REVIEW OF ARTIFICIAL INTELLIGENCE ALGORITHMS USING PRE-THERAPY MR IMAGING FOR GLIOMA MOLECULAR SUBTYPE CLASSIFICATION

PURPOSE

Identifying molecular subtypes in gliomas has prognostic and therapeutic value, traditionally after invasive neurosurgical tumor resection or biopsy. Recent advances using artificial intelligence (AI) show promise in using pre-therapy imaging for predicting molecular subtype. We performed a systematic review of recent literature on AI methods used to predict molecular subtypes of gliomas.

METHODS

Literature review conforming to PRSIMA guidelines was performed for publications prior to February 2021 using 4 databases: Ovid Embase, Ovid MEDLINE, Cochrane trials (CENTRAL), and Web of Science core-collection. Keywords included: artificial intelligence, machine learning, deep learning, radiomics, magnetic resonance imaging, glioma, and glioblastoma. Non-machine learning and non-human studies were excluded. Screening was performed using Covidence software. Bias analysis was done using TRIPOD guidelines.

RESULTS

11,727 abstracts were retrieved. After applying initial screening exclusion criteria, 1,135 full text reviews were performed, with 82 papers remaining for data extraction. 57% used retrospective single center hospital data, 31.6% used TCIA and BRATS, and 11.4% analyzed multicenter hospital data. An average of 146 patients (range 34-462 patients) were included. Algorithms predicting IDH status comprised 51.8% of studies, MGMT 18.1%, and 1p19q 6.0%. Machine learning methods were used in 71.4%, deep learning in 27.4%, and 1.2% directly compared both methods. The most common algorithm for machine learning were support vector machine (43.3%), and for deep learning convolutional neural network (68.4%). Mean prediction accuracy was 76.6%.

CONCLUSION

Machine learning is the predominant method for image-based prediction of glioma molecular subtypes. Major limitations include limited datasets (60.2% with under 150 patients) and thus limited generalizability of findings. We recommend using larger annotated datasets for AI network training and testing in order to create more robust AI algorithms, which will provide better prediction accuracy to real world clinical datasets and provide tools that can be translated to clinical practice.

NIMG-71. IDENTIFYING CLINICALLY APPLICABLE MACHINE LEARNING ALGORITHMS FOR GLIOMA SEGMENTATION USING A SYSTEMATIC LITERATURE REVIEW

PURPOSE

Nowadays Machine learning (ML) algorithms are often used for segmentation of gliomas, but which algorithms provide the most accurate method for implementation into clinical practice has not fully been identified. We performed a systematic review of the literature to characterize the methods used for glioma segmentation and their accuracy.

METHODS

In accordance to PRISMA, a literature review was performed on four databases, Ovid Embase, Ovid MEDLINE, Cochrane trials (CENTRAL) and Web of science core-collection first in October 2020 and in February 2021. Keywords and controlled vocabulary included artificial intelligence, machine learning, deep learning, radiomics, magnetic resonance imaging, glioma, and glioblastoma. Publications were screened in Covidence and the bias analysis was done in agreement with TRIPOD.

RESULTS

Sixty-six articles were used for data extraction. BRATS and TCIA datasets were used in 36.6% of all studies, with average number of patients being 141 (range: 1 to 622). ML methods represented 45.3% of studies, with deep learning used in 54.7%; Dice score for the tumor core ranged from 0.72 to 0.95. The most common algorithm used in the machine learning papers was support vector machines (SVM) and for deep learning papers, it was Convolutional Neural Networks (CNN). Preliminary TRIPOD analysis yielded an average score from 12 (range: 7-16) with the majority of papers demonstrating deficiencies in description of the ML algorithm, funding role, data acquisition and measures of model performance.

CONCLUSION

In the last years, many articles were published on segmentation of gliomas using machine learning, thus establishing this method for tumor segmentation with high accuracy. However, the major limitations for clinically applicable use of ML in glioma segmentation include more than one-third of publications use the same datasets, thus limiting generalizability, increase the likelihood of overfitting, show and lack of ML network description and standardization in accuracy reporting.

NIMG-38. MEASURING ADHERENCE TO TRIPOD OF ARTIFICIAL INTELLIGENCE PAPERS IN THE GLIOMA SEGMENTATION

PURPOSE

Generalizability, reproducibility and objectivity are critical elements that need to be considered when translating machine learning models into clinical practice. While a large body of literature has been published on machine learning methods for segmentation of brain tumors, a systematic evaluation of paper quality and reproducibility has not been done. We investigated the use of “Transparent Reporting of studies on prediction models for Individual Prognosis Or Diagnosis” (TRIPOD) items, among papers published in this relatively new and growing field.

METHODS

According to PRISMA a literature review was performed on four databases, Ovid Embase, Ovid MEDLINE, Cochrane trials (CENTRAL) and Web of science core-collection first in October 2020 and a second time in February 2021. Keywords and controlled vocabulary included artificial intelligence, machine learning, deep learning, radiomics, magnetic resonance imaging, glioma, and glioblastoma. The publications were assessed in order to the TRIPOD items.

RESULTS

37 publications from our database search were screened in TRIPOD and yielded an average score of 12.08 with the maximum score being 16 and the minimum score 7. The best scoring item was interpretation (item 19) where all papers scored a point. The lowest scoring items were the title, the abstract, risk groups and the model performance (items number 1, 2, 11 and 16), where no paper scored a point. Less than 1% of the papers discussed the problem of missing data (item 9) and the funding of research (item 22).

CONCLUSION

TRIPOD analysis showed that a majority of the papers do not score high on critical elements that allow reproducibility, translation, and objectivity of research. An average score of 12.08 (40%) indicates that the publications usually achieve a relatively low score. The categories that were consistently poorly described include the ML network description, measuring model performance, title details and inclusion of information into the abstract.

Intracortical microinjections may cause spreading depression and suppress absence seizures

Intracerebral microinjection is a commonly used technique for local delivery of biologically active agents. However, it is known that mechanical injury of the cortex can induce spreading depression (SD), a wave of transient cellular depolarization. We examined the effects of intracortical microinjections of a new selective I(h) channel antagonist ORG 34167 and of different control treatments (saline and sham microinjections) on spontaneously occurring spike-wave discharges (SWDs) in WAG/Rij rats, a valid genetic model of absence epilepsy. Electroencephalographic (EEG) recording in awake rats has shown that both the drug and control microinjections are followed by long-term (for more than an hour) suppression of SWDs. dc-EEG recording in WAG/Rij rats has revealed that sham microinjections induce SD in 65% (31/48) cases. Number of SWDs decreased substantially for at least 90 min after the sham injections which induced cortical SD but remained unchanged if SD was not triggered by microinjection. These findings suggest that SD induced by intracortical microinjection may contribute to long-term suppression of non-convulsive epileptic activity after this experimental procedure.