Robustness and performance of radiomic features in diagnosing cystic renal masses

PURPOSE

We study the inter-reader variability in manual delineation of cystic renal masses (CRMs) presented in computerized tomography (CT) images and its effect on the classification performance of a machine learning algorithm in distinguishing benign from potentially malignant CRMs. In addition, we assessed whether the inclusion of higher-order robust radiomic features improves the classification performance over the use of first-order features.

METHODS

230 CRMs were independently delineated by two radiologists. Through a combination of random fluctuations, dilation, and erosion operations over the original region of interests (ROIs), we generated four additional sets of synthetic ROIs to capture the inter-reader variability realistically, as confirmed by dice coefficient measurements and visual assessment. We then identified the robust features based on the intra-class coefficient (ICC > 0.85) across these datasets. We applied a tenfold stratified cross-validation (CV) to train and test the performance of the random forest model for the classification of CRMs into benign and potentially malignant.

RESULTS

The mean area under the curve (AUC), sensitivity, specificity, positive predictive value, and negative predictive value were 0.87, 0.82, 0.90, 0.85, and 0.93, respectively. With the usage of first-order features alone, the corresponding values were nearly identical.

CONCLUSION

AUC ranged for the robust and uncorrelated features from 0.83 ± 0.09 to 0.93 ± 0.04 and for the first-order features from 0.84 ± 0.09 to 0.91 ± 0.04. Our study indicates that the first-order features alone are sufficient for the classification of CRMs, and that inclusion of higher-order features does not necessarily improve performance.

Standardized Classification of Lumbar Spine Degeneration on Magnetic Resonance Imaging Reduces Intra- and Inter-subspecialty Variability

BACKGROUND AND PURPOSE

To determine the efficacy of standardized definitions of degenerative change in reducing variability in interpretation of lumbar spine magnetic resonance imaging within and between groups of subspecialty-trained neuroradiologists (NR) and musculoskeletal radiologists (MSK).

MATERIALS AND METHODS

Six radiologists, three from both NR and MSK groups were trained on a standardized classification system of degenerative change. After an 11-month washout period, they independently re-interpreted fifty exams at the L4-L5 and L5-S1 levels. Responses were converted to a six-point ordinal scale for the assessment of neural foraminal stenosis and spinal canal stenosis (SCS), three-point scale for lateral recess stenosis, and four-point scale for facet osteoarthritis (FO). Intra-subspecialty and inter-subspecialty analysis was performed using the weighted Cohen's kappa with a binary matrix of all reader pairs.

RESULTS

Inter-subspecialty agreement improved from k=0.527 (moderate) to k=0.602 (substantial) for neural foraminal stenosis, from k=0.540 (moderate) to k=0.652 (substantial) for SCS, from k=0.0818 (slight) to k=0.337 (fair) for lateral recess stenosis, and from k=0.176 (slight) to k=0.495 (moderate) for FO. The NR group demonstrated improved intra-subspecialty agreement for the assessment of SCS, from k=0.368 (fair) to k=0.638 (substantial). The MSK group demonstrated improved intra-subspecialty agreement for the assessment of FO, from k=0.134 (slight) to k=0.413 (moderate). Intra-subspecialty agreement was similar for other parameters before and after training.

CONCLUSIONS

As result of the standardized definitions training, the NR and MSK groups each improved in one of the four parameters, while inter-subspecialty variability improved in all four parameters. These definitions may be useful in clinical practice across radiology subspecialties.

Simplified Universal Grading of Lumbar Spine MRI Degenerative Findings: Inter-Reader Agreement of Non-Radiologist Spine Experts

OBJECTIVE

1) To describe a simplified multidisciplinary grading system for the most clinically relevant lumbar spine degenerative changes. 2) To measure the inter-reader variability among non-radiologist spine experts in their use of the classification system for interpretation of a consecutive series of lumbar spine magnetic resonance imaging (MRI) examinations.

METHODS

ATS multidisciplinary and collaborative standardized grading of spinal stenosis, foraminal stenosis, lateral recess stenosis, and facet arthropathy was developed. Our institution's picture archiving and communication system was searched for 50 consecutive patients who underwent non-contrast MRI of the lumbar spine for chronic back pain, radiculopathy, or symptoms of spinal stenosis. Three fellowship-trained spine subspecialists from neurosurgery, orthopedic surgery, and physiatry interpreted the 50 exams using the classification at the L4-L5 and L5-S1 levels. Inter-reader agreement was assessed with Cohen's kappa coefficient.

RESULTS

For spinal stenosis, the readers demonstrated substantial agreement (κ = 0.702). For foraminal stenosis and facet arthropathy, the three readers demonstrated moderate agreement (κ = 0.544, and 0.557, respectively). For lateral recess stenosis, there was fair agreement (κ = 0.323).

CONCLUSIONS

A simplified universal grading system of lumbar spine MRI degenerative findings is newly described. Use of this multidisciplinary grading system in the assessment of clinically relevant degenerative changes revealed moderate to substantial agreement among non-radiologist spine physicians. This standardized grading system could serve as a foundation for interdisciplinary communication.

A Comparative Retrospective Study of Immunotherapy RANO Versus Standard RANO Criteria in Glioblastoma Patients Receiving Immune Checkpoint Inhibitor Therapy

Objectives

Real-time assessment of treatment response in glioblastoma (GBM) patients on immune checkpoint blockade (ICB) remains challenging because inflammatory effects of therapy may mimic progressive disease, and the temporal evolution of these inflammatory findings is poorly understood. We compare GBM patient response during ICB as assessed with the Immunotherapy Response Assessment in Neuro-Oncology (iRANO) and the standard Response Assessment in Neuro-Oncology (RANO) radiological criteria.

Methods

49 GBM patients (seven newly diagnosed and 42 recurrent) treated with ICBs at a single institution were identified. Tumor burden was quantified on serial MR scans according to RANO criteria during ICB. Radiographic response assessment by iRANO and RANO were compared.

Results

82% (40/49) of patients received anti–PD-1, 16% (8/49) received anti-PD-L1, and 2% (1/49) received anti-PD-1 and anti-CTLA4 treatment. Change in tumor burden and best overall response ranged from −100 to +557% (median: +48%). 12% (6/49) of patients were classified as concordant non-progressors by both RANO and iRANO (best response: one CR, one PR, and four SD). Another12% (6/49) had discordant assessments: 15% (6/41) of RANO grade progressive disease (PD) patients had iRANO grade of progressive disease unconfirmed (PDU). The final classification of these discordant patients was pseudoprogression (PsP) in three of six, PD in two of six, and PDU in one of six who went off study before the iRANO assessment of PDU. iRANO delayed diagnosis of PD by 42 and 93 days in the two PD patients. 76% (37/49) patients were classified as concordant PD by both RANO and iRANO. 12% (6/49) of all patients were classified as PsP, starting at a median of 12 weeks (range, 4–30 weeks) after ICB initiation.

Conclusions

Standard RANO and iRANO have high concordance for assessing PD in patients within 6 months of ICB initiation. iRANO was beneficial in 6% (3/49) cases later proven to be PsP, but delayed confirmation of PD by <3 months in 4% (2/49). PsP occurred in 12% of patients, starting at up to 7 months after initiation of ICB. Further study to define the utility of modified RANO compared with iRANO in ICB GBM patients is needed.

Frequency and Evolution of New Postoperative Enhancement on 3 Tesla Intraoperative and Early Postoperative Magnetic Resonance Imaging

BACKGROUND

Intraoperative magnetic resonance imaging (IO-MRI) provides real-time assessment of extent of resection of brain tumor. Development of new enhancement during IO-MRI can confound interpretation of residual enhancing tumor, although the incidence of this finding is unknown.

OBJECTIVE

To determine the frequency of new enhancement during brain tumor resection on intraoperative 3 Tesla (3T) MRI. To optimize the postoperative imaging window after brain tumor resection using 1.5 and 3T MRI.

METHODS

We retrospectively evaluated 64 IO-MRI performed for patients with enhancing brain lesions referred for biopsy or resection as well as a subset with an early postoperative MRI (EP-MRI) within 72 h of surgery (N = 42), and a subset with a late postoperative MRI (LP-MRI) performed between 120 h and 8 wk postsurgery (N = 34). Three radiologists assessed for new enhancement on IO-MRI, and change in enhancement on available EP-MRI and LP-MRI. Consensus was determined by majority response. Inter-rater agreement was assessed using percentage agreement.

RESULTS

A total of 10 out of 64 (16%) of the IO-MRI demonstrated new enhancement. Seven of 10 patients with available EP-MRI demonstrated decreased/resolved enhancement. One out of 42 (2%) of the EP-MRI demonstrated new enhancement, which decreased on LP-MRI. Agreement was 74% for the assessment of new enhancement on IO-MRI and 81% for the assessment of new enhancement on the EP-MRI.

CONCLUSION

New enhancement occurs in intraoperative 3T MRI in 16% of patients after brain tumor resection, which decreases or resolves on subsequent MRI within 72 h of surgery. Our findings indicate the opportunity for further study to optimize the postoperative imaging window.

Population-Scale CT-based Body Composition Analysis of a Large Outpatient Population Using Deep Learning to Derive Age-, Sex-, and Race-specific Reference Curves

Background Although CT-based body composition (BC) metrics may inform disease risk and outcomes, obtaining these metrics has been too resource intensive for large-scale use. Thus, population-wide distributions of BC remain uncertain. Purpose To demonstrate the validity of fully automated, deep learning BC analysis from abdominal CT examinations, to define demographically adjusted BC reference curves, and to illustrate the advantage of use of these curves compared with standard methods, along with their biologic significance in predicting survival. Materials and Methods After external validation and equivalency testing with manual segmentation, a fully automated deep learning BC analysis pipeline was applied to a cross-sectional population cohort that included any outpatient without a cardiovascular disease or cancer who underwent abdominal CT examination at one of three hospitals in 2012. Demographically adjusted population reference curves were generated for each BC area. The z scores derived from these curves were compared with sex-specific thresholds for sarcopenia by using χ² tests and used to predict 2-year survival in multivariable Cox proportional hazards models that included weight and body mass index (BMI). Results External validation showed excellent correlation (R = 0.99) and equivalency (P < .001) of the fully automated deep learning BC analysis method with manual segmentation. With use of the fully automated BC data from 12 128 outpatients (mean age, 52 years; 6936 [57%] women), age-, race-, and sex-normalized BC reference curves were generated. All BC areas varied significantly with these variables (P < .001 except for subcutaneous fat area vs age [P = .003]). Sex-specific thresholds for sarcopenia demonstrated that age and race bias were not present if z scores derived from the reference curves were used (P < .001). Skeletal muscle area z scores were significantly predictive of 2-year survival (P = .04) in combined models that included BMI. Conclusion Fully automated body composition (BC) metrics vary significantly by age, race, and sex. The z scores derived from reference curves for BC parameters better capture the demographic distribution of BC compared with standard methods and can help predict survival. © RSNA, 2020 Online supplemental material is available for this article. See also the editorial by Summers in this issue.

Beyond business as usual: Radiology residency educational response to the COVID-2019 pandemic

The COVID-19 pandemic has disrupted standard hospital operations and diagnostic radiology resident education at academic medical centers across the country. Deferment of elective surgeries and procedures coupled with a shift of resources toward increased inpatient clinical needs for the care of COVID-19 patients has resulted in substantially decreased imaging examinations at many institutions. Additionally, both infection control and risk mitigation measures have resulted in minimal on-site staffing of both trainees and staff radiologists at many institutions. As a result, residents have been placed in nonstandard learning environments, including working from home, engaging in a virtual curriculum, and participating in training sessions in preparation for potential reassignment to other patient care settings. Typically, for residents to gain the necessary knowledge, skills, and experience to practice independently upon graduation, radiology training programs must provide an optimal balance between resident education and clinical obligations. We describe our experience adapting to the challenges in educational interruptions and clinical work reassignments of 41 interventional and diagnostic radiology residents at a large academic center. We highlight opportunities for collaboration and teamwork in creatively adjusting and planning for the short and long-term impact of the pandemic on resident education. This experience shows how the residency educational paradigm was shifted during a pandemic and can serve as a template to address future disruptions.

•

The COVID-19 pandemic disrupted normal operations and trainee education in our large academic medical center.

•

Multitiered crisis staffing model and weekly schedule created to accommodate clinical demands and public health needs.

•

Robust virtual curriculum included synchronous and asynchronous sessions for subspecialty-specific clinical education.

•

Prepared for resident reassignment to other patient care settings such as medicine inpatient units.

Quantifying Impact of Disruption to Radiology Education During the COVID-19 Pandemic and Implications for Future Training

Purpose

The aim of this study was to assess the impact on radiology resident education due to the COVID-19 pandemic in order to inform future educational planning.

Methods

During a 10-week study period from March 16 to May 22, 2020, changes to educational block-weeks (BW) of first through fourth year residents (R1-4) were documented as disrupted in the setting of the COVID-19 pandemic. The first 5 weeks and the second 5 weeks were evaluated separately for temporal differences. Overall and mean disrupted BW per resident were documented. Wilcoxon rank-sum tests were used to assess pairwise differences between classes with Bonferroni-adjusted P-values, as well as differences in the early versus later phase of the pandemic.

Results

Of 373 BW, 56.6% were assigned to virtual curriculum, 39.4% radiology clinical duties, 2.9% illness, and 1.1% reassignment. Scheduling intervention affected 6.2 ± 2.3 (range 1-10) mean BW per resident over the 10-week study period. The R3 class experienced the largest disruption, greater than the R2 classes, and statistically significantly more than the R1 and R4 classes (both P < 0.05). The second half of the pandemic caused statistically significantly more schedule disruptions than the first half (P = 0.009).

Discussion

The impact of COVID-19 pandemic varied by residency class year, with the largest disruption of the R3 class and the least disruption of the R4 class. To optimize future educational opportunities, shifting to a competency-based education paradigm may help to achieve proficiency without extending the length of the training program.

The role of computed tomography angiography as initial imaging tool for acute hemorrhage in the head and neck

OBJECTIVES

Acute hemorrhage in the head and neck (AHNH) is life-threatening due to asphyxiation and hemorrhagic shock. When conservative measures fail, some patients benefit from endovascular therapy (EVT). While CTA is routinely used to localize bleeding and plan EVT in gastrointestinal hemorrhage, the diagnostic value of CTA in AHNH and role of CTA in treatment-planning are uncertain.

METHODS

We retrospectively reviewed neck CTAs from June 2015 to October 2018 indicated for AHNH. When performed, digital subtraction angiography (DSA) findings and EVT were documented. Extravasation or pseudoaneurysm on DSA was considered positive for bleed localization.

RESULTS

Thirty CTA exams were performed for AHNH in 18 patients (mean age = 56.6, male% = 55.6%). Eleven out of 30 exams (36.7%) had immediate DSA follow-up within 24 h. Etiologies of hemorrhage included malignancy 11/18 (61.1%) and coagulopathy (4/18, 22.2%) among others. CTA reports identified definite or possible source of bleeding in 7/30 (23.3%) exams. Seven out of 7 (100%) patients with definite or possible source of bleeding on CTA underwent DSA and 4/23 (17.4%) patients underwent DSA despite negative CTA. With DSA as the gold standard, CTA had a sensitivity of 70% and a specificity of 100%.

CONCLUSIONS

CTA has high specificity and reasonable sensitivity for detecting arterial source of bleeding in patients presenting with AHNH. Patients with negative CTA may avoid catheter angiography in most cases; however, false-negative CTA should not preclude angiography in high-risk patients.

Deep Transfer Learning and Radiomics Feature Prediction of Survival of Patients with High-Grade Gliomas

BACKGROUND AND PURPOSE

Patient survival in high-grade glioma remains poor, despite the recent developments in cancer treatment. As new chemo-, targeted molecular, and immune therapies emerge and show promising results in clinical trials, image-based methods for early prediction of treatment response are needed. Deep learning models that incorporate radiomics features promise to extract information from brain MR imaging that correlates with response and prognosis. We report initial production of a combined deep learning and radiomics model to predict overall survival in a clinically heterogeneous cohort of patients with high-grade gliomas.

MATERIALS AND METHODS

Fifty patients with high-grade gliomas from our hospital and 128 patients with high-grade glioma from The Cancer Genome Atlas were included. For each patient, we calculated 348 hand-crafted radiomics features and 8192 deep features generated by a pretrained convolutional neural network. We then applied feature selection and Elastic Net-Cox modeling to differentiate patients into long- and short-term survivors.

RESULTS

In the 50 patients with high-grade gliomas from our institution, the combined feature analysis framework classified the patients into long- and short-term survivor groups with a log-rank test P value < .001. In the 128 patients from The Cancer Genome Atlas, the framework classified patients into long- and short-term survivors with a log-rank test P value of .014. For the mixed cohort of 50 patients from our institution and 58 patients from The Cancer Genome Atlas, it yielded a log-rank test P value of .035.

CONCLUSIONS

A deep learning model combining deep and radiomics features can dichotomize patients with high-grade gliomas into long- and short-term survivors.

Intra- and Intersubspecialty Variability in Lumbar Spine MRI Interpretation: A Multireader Study Comparing Musculoskeletal Radiologists and Neuroradiologists

BACKGROUND AND PURPOSE

The purpose of this study is to assess the differences in degenerative spine MRI reporting between subspecialty-trained attending neuroradiologists and musculoskeletal radiologists (MSK) at a single institution, academic medical center.

MATERIALS AND METHODS

Fifty consecutive outpatient noncontrast lumbar spine examinations were selected from the Picture Archiving and Communication System. Three MSK and 3 neuroradiologists (NR) independently reviewed and interpreted the exams at the L4-L5 and L5-S1 levels in the same manner as in clinical practice. The assessment of neural foraminal stenosis (NFS) and spinal canal stenosis (SCS) was converted to a 5-point ordinal scale. The assessment of lateral recess stenosis (LRS) and facet osteoarthritis (FO) was recorded as present/absent. Intersubspecialty and intrasubspecialty analysis was performed using Cohen's kappa coefficient with a binary matrix of all reader pairs.

RESULTS

There was moderate intersubspecialty agreement (k = 0.527) for NFS and SCS (k = 0.540). Intersubspecialty agreement was slight for LRS (k = 0.0818) and FO (k = 0.176). The MSK group demonstrated greater intrasubspecialty agreement in assessment of NFS and SCS compared to the NR group, with nonoverlapping confidence intervals. The NR group demonstrated greater nominal intrasubspecialty agreement in the assessment of both LRS and FO, although with nonoverlapping confidence intervals.

CONCLUSION

There is moderate intersubspecialty agreement between MSK radiologists and neuroradiologists in reporting the severity of NFS and SCS, although MSK radiologists demonstrated greater intrasubspecialty agreement. There is slight intersubspecialty agreement for LRS and FO. The demonstration of differences in inter-reader agreement is a crucial first step to attempt to ameliorate these variabilities.

CT-guided transforaminal epidural steroid injections: do needle position and degree of foraminal stenosis affect the pattern of epidural flow?

OBJECTIVE

To determine the effect of needle position and foraminal stenosis on contrast flow directionality during CT-guided transforaminal epidural steroid injections (TFESI).

MATERIALS AND METHODS

One hundred five consecutive CT-guided injections were performed in 68 patients (mean age, 65.5 years) from January 1 to December 31 2017, all with preceding MRI. Two readers independently reviewed CT images to assess needle position and to determine direction of contrast flow, which was defined as central or peripheral. The MRIs were independently reviewed by the readers to determine the degree of foraminal stenosis. Inter-reader agreement for both was evaluated with the kappa statistic. Analyses were performed to determine effect of needle position, degree of foraminal stenosis, and volume of contrast injected with directionality of contrast flow, and association between contrast flow directionality with immediate post-procedural pain scores.

RESULTS

Central direction of contrast flow was demonstrated in 41/78 (52.6%) of cases with posterolateral needle position, and 20/27 (74.1%) with central or anteromedial needle position (p = 0.07). There was no difference in direction of contrast flow with high-grade versus absence of high-grade neuroforaminal narrowing, or with volume of contrast injected. There was no difference in immediate post-procedure pain scores regardless of contrast flow directionality.

CONCLUSIONS

Needle position is not significantly associated with contrast flow directionality during CT-guided TFESI, although there was a trend towards relatively decreased central flow with posterolateral positioning. Degree of foraminal stenosis and volume of injected contrast did not affect contrast flow directionality. There was no difference in immediate post-procedural pain scores with either direction of contrast flow.

Diffusional Kurtosis along the Corticospinal Tract in Adult Normal Pressure Hydrocephalus

BACKGROUND AND PURPOSE

Normal Pressure Hydrocephalus is a reversible form of dementia characterized by enlarged ventricles, which can deform and cause disruptions to adjacent white matter fibers. The purpose of this work was to examine how diffusion and kurtosis parameters vary along the corticospinal tract and determine where along this path microstructure is compromised in patients diagnosed with normal pressure hydrocephalus. We hypothesized that disruption of the corticospinal tract from ventricular enlargement can be measured using diffusion MR imaging and this will be quantified in periventricular regions.

MATERIALS AND METHODS

We developed a method to analyze diffusion parameters at discrete points along neural tracts. We then used diffusion MR imaging data from patients with Alzheimer disease and healthy controls to compare whether diffusion along the corticospinal tract differs from that of patients with normal pressure hydrocephalus.

RESULTS

We found that diffusion parameters can differentiate patients with normal pressure hydrocephalus from those with Alzheimer disease and healthy controls: Axial diffusion, axial kurtosis, and the axonal water fraction were found to differ significantly across groups (P < .05) in an area located close to the superior internal capsule and corona radiata but below the cortex.

CONCLUSIONS

A lower axonal water fraction indicates a lower axonal density in the corticospinal tract, which may indicate permanent damage. Lower axial kurtosis may imply that axons are being more aligned due to compression.

Diagnosis of Normal-Pressure Hydrocephalus: Use of Traditional Measures in the Era of Volumetric MR Imaging

Purpose To assess the diagnostic performance of the callosal angle (CA) and Evans index (EI) measures and to determine their role versus automated volumetric methods in clinical radiology. Materials and Methods Magnetic resonance (MR) examinations performed before surgery (within 1-5 months of the MR examination) in 36 shunt-responsive patients with normal-pressure hydrocephalus (NPH; mean age, 75 years; age range, 58-87 years; 26 men, 10 women) and MR examinations of age- and sex-matched patients with Alzheimer disease (n = 34) and healthy control volunteers (n = 36) were studied. Three blinded observers independently measured EI and CA for each patient. Volumetric segmentation of global gray matter, white matter, ventricles, and hippocampi was performed by using software. These measures were tested by using multivariable logistic regression models to determine which combination of metrics is most accurate in diagnosis. Results The model that used CA and EI demonstrated 89.6%-93.4% accuracy and average area under the curve of 0.96 in differentiating patients with NPH from patients without NPH (ie, Alzheimer disease and healthy control). The regression model that used volumetric predictors of gray matter and white matter was 94.3% accurate. Conclusion CA and EI may serve as a screening tool to help the radiologist differentiate patients with NPH from patients without NPH, which would allow for designation of patients for further volumetric assessment. ^© RSNA, 2017.