An approach to comparing accuracies of two FLAIR MR sequences in the detection of multiple sclerosis lesions in the brain in the absence of gold standard

Economic impact of selective use of contrast for routine follow-up MRI of patients with multiple sclerosis

BACKGROUND AND PURPOSE

Imaging and autopsy studies show intracranial gadolinium deposition in patients who have undergone serial contrast-enhanced MRIs. This observation has raised concerns when using contrast administration in patients who receive frequent MRIs. To address this, we implemented a contrast-conditional protocol wherein gadolinium is administered only for multiple sclerosis (MS) patients with imaging evidence of new disease activity on precontrast imaging. In this study, we explore the economic impact of our new MRI protocol.

METHODS

We compared scanner time and Medicare reimbursement using our contrast-conditional methodology versus that of prior protocols where all patients received gadolinium.

RESULTS

For 422 patients over 4 months, the contrast-conditional protocol amounted to 60% decrease in contrast injection and savings of approximately 20% of MRI scanner time. If the extra scanner time was used for performing MS follow-up MRIs in additional patients, the contrast-conditional protocol would amount to net revenue loss of $21,707 (∼3.7%).

CONCLUSIONS

Implementation of a new protocol to limit contrast in MS follow-up MRIs led to a minimal decrease in revenue when controlled for scanner time utilized and is outweighed by other benefits, including substantial decreased gadolinium administration, increased patient comfort, and increased availability of scanner time, which depending on type of studies performed could result in additional financial benefit.

Whole volume brain extraction for multi-centre, multi-disease FLAIR MRI datasets

Automatic segmentation of the brain from magnetic resonance images (MRI) is a fundamental step in many neuroimaging processing frameworks. There are mature technologies for this task for T1- and T2-weighted MRI; however, a widely-accepted brain extraction method for Fluid-Attenuated Inversion Recovery (FLAIR) MRI has yet to be established. FLAIR MRI are becoming increasingly important for the analysis of neurodegenerative diseases and tools developed for this sequence would have clinical value. To maximize translation opportunities and for large scale research studies, algorithms for brain extraction in FLAIR MRI should generalize to multi-centre (MC) data. To this end, this work proposes a fully automated, whole volume brain extraction methodology for MC FLAIR MRI datasets. The framework is built using a novel standardization framework which reduces acquisition artifacts, standardizes the intensities of tissues and normalizes the spatial coordinates of brain tissue across MC datasets. Using the standardized datasets, an intuitive set of features based on intensity, spatial location and gradients are extracted and classified using a random forest (RF) classifier to segment the brain tissue class. A series of experiments were conducted to optimize classifier parameters, and to determine segmentation accuracy for standardized and unstandardized (original) data, as a function of scanner vendor, feature type and disease type. The models are trained, tested and validated on 156 image volumes (∼8000 image slices) from two multi-centre, multi-disease datasets, acquired with varying imaging parameters from 30 centres and three scanner vendors. The image datasets, denoted as CAIN and ADNI for vascular and dementia disease, respectively, represent a diverse collection of MC data to test the generalization capabilities of the proposed design. Results demonstrate the importance of standardization for segmentation of MC data, as models trained on standardized data yielded a drastic improvement in brain extraction accuracy compared to the original, unstandardized data (CAIN: DSC = 91% and ADNI: DSC = 86% vs. CAIN: 78% and ADNI: 65%). It was also found that models created from one scanner vendor based on unstandardized data yielded poor segmentation results in data acquired from other scanner vendors, which was improved through standardization. These results demonstrate that to create consistency in segmentations from multi-institutional datasets it is paramount that MC variability be mitigated to improve stability and to ensure generalization of machine learning algorithms for MRI.

An Initiative to Reduce Unnecessary Gadolinium-Based Contrast in Multiple Sclerosis Patients

OBJECTIVE

Patients with multiple sclerosis (MS) routinely undergo serial contrast-enhanced MRIs. Given concerns regarding tissue deposition of gadolinium-based contrast agents (GBCAs) and evidence that enhancement of lesions is only seen in patients with new disease activity on noncontrast imaging, we set out to implement a prospective quality improvement project whereby intravenous contrast would be reserved only for patients with evidence of new disease activity on noncontrast images.

METHODS

To prospectively implement such a protocol, we leveraged our in-house computer-assisted detection (CAD) software and 3-D laboratory radiology technologists to perform real-time preliminary assessments of the CAD-processed T2 fluid attenuated inversion recovery (FLAIR) noncontrast images as a basis for deciding whether to inject contrast. Before implementation, we held multidisciplinary meetings with neurology, neuroradiology, and MR technologists and distributed surveys to objectively assess opinions and obstacles to clinical implementation. We evaluated reduction in GBCA utilization and technologist performance relative to final neuroradiologist interpretations.

RESULTS

During a 2-month trial period, 153 patients were imaged under the new protocol. Technologists using the CAD software were able to identify patients with new or enlarging lesions on FLAIR images with 95% accuracy and 97% negative predictive value relative to final neuroradiologist interpretations, which allowed us to avoid the use of contrast and additional imaging sequences in 87% of patients.

DISCUSSION

A multidisciplinary effort to implement a quality improvement project to limit contrast in MS patients receiving follow-up MRIs allowed for improved safety and cost by targeting patients that would benefit from the use of intravenous contrast in real-time.

Multi-reader multi-case studies using the area under the receiver operator characteristic curve as a measure of diagnostic accuracy: systematic review with a focus on quality of data reporting

INTRODUCTION

We examined the design, analysis and reporting in multi-reader multi-case (MRMC) research studies using the area under the receiver-operating curve (ROC AUC) as a measure of diagnostic performance.

METHODS

We performed a systematic literature review from 2005 to 2013 inclusive to identify a minimum 50 studies. Articles of diagnostic test accuracy in humans were identified via their citation of key methodological articles dealing with MRMC ROC AUC. Two researchers in consensus then extracted information from primary articles relating to study characteristics and design, methods for reporting study outcomes, model fitting, model assumptions, presentation of results, and interpretation of findings. Results were summarized and presented with a descriptive analysis.

RESULTS

Sixty-four full papers were retrieved from 475 identified citations and ultimately 49 articles describing 51 studies were reviewed and extracted. Radiological imaging was the index test in all. Most studies focused on lesion detection vs. characterization and used less than 10 readers. Only 6 (12%) studies trained readers in advance to use the confidence scale used to build the ROC curve. Overall, description of confidence scores, the ROC curve and its analysis was often incomplete. For example, 21 (41%) studies presented no ROC curve and only 3 (6%) described the distribution of confidence scores. Of 30 studies presenting curves, only 4 (13%) presented the data points underlying the curve, thereby allowing assessment of extrapolation. The mean change in AUC was 0.05 (-0.05 to 0.28). Non-significant change in AUC was attributed to underpowering rather than the diagnostic test failing to improve diagnostic accuracy.

CONCLUSIONS

Data reporting in MRMC studies using ROC AUC as an outcome measure is frequently incomplete, hampering understanding of methods and the reliability of results and study conclusions. Authors using this analysis should be encouraged to provide a full description of their methods and results.

Sickle cell anemia: intracranial stenosis and silent cerebral infarcts in children with low risk of stroke

PURPOSE

Children with sickle cell anemia (SCA), who have mean blood flow velocities <170 cm/s in the terminal internal carotid (tICA) or middle cerebral (MCA) arteries on transcranial Doppler ultrasonography (TCD), are considered to be at low risk of stroke. The prevalence of intracranial stenosis, which raises the risk of stroke, is not known in these children. Here, we estimated the prevalence of stenosis and explored its association with silent cerebral infarcts determined based on Magnetic Resonance (MR) scans.

PATIENTS/METHODS

We studied prospectively a cohort of 67 children with SCA without prior clinically overt stroke or TIA (median age 8.8 years; range limits 2.3-13.1 years; 33 females) and with TCD mean velocity <170 cm/s. They underwent MR imaging of the brain and MR angiography of intracranial arteries.

RESULTS

In 7 children (10.5%, 95% CI: 4.9-20.3%) we found 10 stenoses, including 4 with isolated left tICA stenosis and 3 with multiple stenoses. We found silent infarcts in 26 children (37.7%, 95% CI: 27.2-49.5%). The median number of infarcts in an affected child was 2 (range limits: 1-9), median volume of infarcts was 171 mm(3) (range limits: 7-1060 mm(3)), and median infarct volume in relation to total brain volume was 0.020% (range limits: 0.001-0.101%). The number and volume of infarcts were significantly higher in children with arterial stenosis (both p=0.023).

CONCLUSIONS

The prevalence of intracranial arterial stenosis in children with SCA classified as at low risk of stroke by TCD mean velocity <170 cm/s is high. Children with stenosis are at higher risk of brain parenchymal injury as they have more silent cerebral infarcts.