Intra- and inter-observer reliability of MRI examination of intervertebral disc abnormalities in patients with cervical myelopathy

Time for change? Radiologists highly concordant assessing change in stenoses on follow-up cervical spine MRI

OBJECTIVE

There is considerable variability among radiologists when grading spinal canal and foraminal stenosis on MRI. However, to date, studies have not evaluated radiologists' agreement when assessing interval change in cervical spine stenoses. The purpose of this study was to evaluate radiologists' concordance for change in cervical spine stenoses on follow-up MRIs, a major indication for these exams.

METHODS

Initial and follow-up cervical MRIs were retrospectively reviewed by three blinded radiologists. Spinal canal and foramina from C1 through T1 were rated for interval change and concordance between the blinded raters was calculated. The original MRI reports were also reviewed for specific language assessing interval change on the follow-up exams.

RESULTS

40 cervical MRI exams and 40 corresponding MRI follow-ups were assessed. Agreement for interval change in spinal canal and foraminal stenosis was near perfect amongst all readers (kappa values of 0.78-0.94). 97% of the original MRI reports used the standard severity scale. 68% of follow-up MRI reports specifically assessed for change.

DISCUSSION

Blinded radiologists had high agreement when assessing for change in spinal canal and foraminal stenosis on follow-up cervical spine MRIs. Because of inter-rater variability in stenosis grading, reports that do not emphasize change assessment, may imply change that is not truly present. For clarity and consistency in reporting of cervical spine stenoses, change assessment should be emphasized and added to structured reporting templates.

Novel MRI signs of ruptured disc in the cervical spine with intraoperative comparisons

OBJECTIVES

To evaluate the diagnostic performance of two novel MRI signs for extruded disc (ED) and uncontained ED (ruptured disc, RD) in the cervical spine using intraoperative findings as reference.

METHODS

This retrospective study included patients who underwent cervical spine MRI and surgery for disc pathology with intraoperative confirmation of RD from September 1, 2016, to January 31, 2021. Two radiologists determined whether the disc was extruded or ruptured with and without the aid of two novel MRI signs suggesting RD (sign 1: blurred margin of the disc; sign 2: mushroom-shaped disc). The diagnostic performance was analyzed using surgical findings as reference. Intra- and interobserver agreements were measured for each MRI sign.

RESULTS

A total of 91 patients totaling 131 discs were enrolled (mean age: 56.02 ± 12.93; range: 26-88; 62 men and 29 women), of whom 62 were surgically confirmed with RD. When the diagnosis was based exclusively on existing ED definitions, ED was diagnosed with 62.9-79.0% sensitivity and 80.2% accuracy, whereas RD was diagnosed with 35.5-45.2% sensitivity and 67.9-71.0% accuracy. However, when the two novel MRI signs were used as an aid in the diagnosis, ED was diagnosed with 95.2-96.8% sensitivity and 84.0-88.5% accuracy, while RD was diagnosed with 95.2-96.8% sensitivity and 84.0-89.3% accuracy. Intra- and interobserver agreement was substantial (k = 00.77-0.86, 0.69-0.79, respectively, p < 0.001).

CONCLUSIONS

The detection of two novel MRI signs on preoperative MRI can lead to a more accurate RD diagnosis.

KEY POINTS

• The diagnostic sensitivity of MRI for cervical ruptured disc is very low (about 35-45 %) using the standardized definition of lumbar disc nomenclature. • Two novel MRI signs can lead to a more accurate diagnosis of the surgically confirmed ruptured disc in the cervical spine. • These two novel MRI signs showed substantial intra-and interobserver reliabilities.

Should Somatosensory and Motor Evoked Potential Monitoring Be Used Routinely in All Posterior Cervical Operations for Degenerative Conditions of the Cervical Spine?

OBJECTIVE

Intraoperative neuromonitoring (IONM) is useful during spinal cord operations, but whether IONM is necessary for posterior cervical surgeries for degenerative conditions is unknown. We evaluated the utility of somatosensory evoked potential (SSEP) and motor evoked potential (MEP) monitoring as a tool for predicting new postoperative neurologic deficits during posterior decompression and fusion for degenerative cervical spine conditions.

METHODS

We retrospectively reviewed posterior cervical operations performed at our institute over a 4-year period. Patients with postoperative neurologic deficits were identified, and a detailed analysis performed to ascertain whether SSEP or MEP monitoring accurately predicted the onset of new postoperative deficits.

RESULTS

Overall, 498 patients were included in the analysis (median age 66 years; range: 22-93 years). SSEP monitoring was performed in all patients, and both SSEP and MEP monitoring were performed in 121 patients (24%). Twenty-one patients (4.2%) had new postoperative neurologic deficits. SSEP had significantly higher specificity (90%) but lower sensitivity (33%) than MEP (74% specificity [P = 0.008], 50% sensitivity [P = 0.01]) for detecting neurologic compromise intraoperatively. For SSEP, the positive predictive value (PPV) and negative predictive value (NPV) in detecting intraoperative changes that translated to new postoperative neurological deficits were 12% and 97%, respectively, whereas for MEP, the PPV and NPV were 6% (P = 0.009) and 98% (P = 0.20), respectively.

CONCLUSIONS

IONM during posterior cervical operations for degenerative conditions of the spine is not reliable at predicting new postoperative neurologic deficits in patients treated for degenerative conditions, but may provide peace of mind to the surgeon intraoperatively when no abnormalities are detected.

Interobserver and intraobserver variability in magnetic resonance imaging evaluation of patients with suspected disc herniation

OBJECTIVE

Magnetic resonance imaging (MRI) is usually the modality of choice to assess sciatica and intervertebral disc herniation. Despite remarkable progression in diagnostic imaging and surgical techniques, definite diagnosis based on imaging interpretation is still a great challenge. The aim of this study was to determine interobserver and intraobserver variability in reporting lumbar MRI between two neuroradiologists based on the new 2014 version of disc nomenclature.

PATIENTS AND METHODS

The study population was composed of 134 irresponsive to conservative therapy patients with clinical presentations of disc herniation and lumbar radiculopathy. MRI was taken from all the participants using a 1.5 T MRI system. Two neuroradiologists evaluated the images, separately and one of them did it twice and interpreted the scans in sagittal and axial planes. Disc bulge, disc herniation and nerve root compression were evaluated at each level. Interobserver and interaobserver agreements between two neuroradiologists, and one neuroradiologist in two times of reporting were calculated for the evaluation of bulging and herniated discs and nerve root compression by applying the Kappa statistics.

RESULTS

Bulging disc, herniated disc, the type of disc, location of the discs, and nerve root compression diagnosis were significantly in excellent agreement (kappa>0.7, p-value<0.001) through intraobserver assessments, while interobserver assessments presented statistically significant with a fair agreement (kappa:0.4-0.7 and p-value<0.05).

CONCLUSION

Remarkable intraobserver agreement was found between diagnoses of disc-related pathologies of the lumbar spine while interobserver assessments revealed only fair concordance.

Diagnostic Quality of 3D T2-SPACE Compared with T2-FSE in the Evaluation of Cervical Spine MRI Anatomy

BACKGROUND AND PURPOSE

Spinal anatomy has been variably investigated using 3D MRI. We aimed to compare the diagnostic quality of T2 sampling perfection with application-optimized contrasts by using flip angle evolution (SPACE) with T2-FSE sequences for visualization of cervical spine anatomy. We predicted that T2-SPACE will be equivalent or superior to T2-FSE for visibility of anatomic structures.

MATERIALS AND METHODS

Adult patients undergoing cervical spine MR imaging with both T2-SPACE and T2-FSE sequences for radiculopathy or myelopathy between September 2014 and February 2015 were included. Two blinded subspecialty-trained radiologists independently assessed the visibility of 12 anatomic structures by using a 5-point scale and assessed CSF pulsation artifact by using a 4-point scale. Sagittal images and 6 axial levels from C2-T1 on T2-FSE were reviewed; 2 weeks later and after randomization, T2-SPACE was evaluated. Diagnostic quality for each structure and CSF pulsation artifact visibility on both sequences were compared by using a paired t test. Interobserver agreement was calculated (κ).

RESULTS

Forty-five patients were included (mean age, 57 years; 40% male). The average visibility scores for intervertebral disc signal, neural foramina, ligamentum flavum, ventral rootlets, and dorsal rootlets were higher for T2-SPACE compared with T2-FSE for both reviewers (P < .001). Average scores for remaining structures were either not statistically different or the superiority of one sequence was discordant between reviewers. T2-SPACE showed less degree of CSF flow artifact (P < .001). Interobserver variability ranged between -0.02-0.20 for T2-SPACE and -0.02-0.30 for T2-FSE (slight to fair agreement).

CONCLUSIONS

T2-SPACE may be equivalent or superior to T2-FSE for the evaluation of cervical spine anatomic structures, and T2-SPACE shows a lower degree of CSF pulsation artifact.

Comparison of agreement of cervical spine degenerative pathology findings in magnetic resonance imaging studies

BACKGROUND CONTEXT

Magnetic resonance imaging (MRI) is often used in the evaluation of degenerative conditions of the cervical spine. However, the agreement of interpreting and reporting varying degenerative findings on cervical MRI has not been well assessed.

PURPOSE

This study aimed to compare the inter-rater and intra-rater agreement of MRI findings between common degenerative findings of the cervical spine.

STUDY DESIGN

A retrospective diagnostic study was used as study design.

PATIENT SAMPLE

The sample consisted of 48 patients who underwent routine cervical spine MRI at our institution between January 2011 and June 2012.

OUTCOME MEASURES

Reviewers evaluated each MRI study at each vertebral level for disc hydration, disc space height, central stenosis, foraminal stenosis, end plate changes, spondylolisthesis, and cord signal change.

METHODS

A panel of two orthopedic spine surgeons and four musculoskeletal radiologists independently reviewed 48 sets of T2-weighted axial and sagittal MRI sequences for a series of preselected criteria, and their findings were compared with those of the other panelists to determine inter-rater agreement. Each panelist also re-reviewed the first 10 studies to determine intra-rater agreement. Absolute inter-rater and intra-rater agreements were then calculated and compared for different findings. A modified analysis ignored disagreements between the least severe grades of findings to determine the inter-rater and intra-rater agreements of the most clinically important severity grades.

RESULTS

Absolute inter-rater agreement ranged from 54.6% to 95.0%. Disc hydration (54.6%), central stenosis (72.7%), and foraminal stenosis (73.1%) demonstrated the lowest inter-rater agreement, whereas spondylolisthesis (95.0%) and cord signal change (92.9%) demonstrated the highest agreement. The modified analysis found better inter-rater agreement, ranging from 80.9% to 95.0%. Absolute intra-rater agreement ranged from 74.2% to 94.7%. The modified analysis again found better agreement, ranging from 85.0% to 94.7%. As would be expected, overall intra-rater agreement (81.6%, 95% CI 78.9%-84.3%) was higher than inter-rater agreement (75.7%, 95% CI 74.4%-77.0%). The clinical specialty of the reviewer had no significant impact on inter- or intra-rater agreement.

CONCLUSIONS

MRI findings play an important role in the management of patients with cervical spine conditions. For this reason, consistent descriptions of these findings are essential and physicians should be aware of the relative reliability of these findings. This systematic study developed standardized grading criteria and nomenclature for common clinically significant MRI findings in the cervical spine. Even in this optimized research setting, we found significant ranges in agreement across these MRI findings. In the clinical setting, inter- and intra-rater agreements may be lower, and the range of agreements between findings may be greater. Physicians should be aware of inconsistencies inherent in the interpretation of cervical MRI findings and should be aware that some findings demonstrate lower agreement than others.

ACR Appropriateness Criteria Myelopathy

Patients presenting with myelopathic symptoms may have a number of causative intradural and extradural etiologies, including disc degenerative diseases, spinal masses, infectious or inflammatory processes, vascular compromise, and vertebral fracture. Patients may present acutely or insidiously and may progress toward long-term paralysis if not treated promptly and effectively. Noncontrast CT is the most appropriate first examination in acute trauma cases to diagnose vertebral fracture as the cause of acute myelopathy. In most nontraumatic cases, MRI is the modality of choice to evaluate the location, severity, and causative etiology of spinal cord myelopathy, and predicts which patients may benefit from surgery. Myelopathy from spinal stenosis and spinal osteoarthritis is best confirmed without MRI intravenous contrast. Many other myelopathic conditions are more easily visualized after contrast administration. Imaging performed should be limited to the appropriate spinal levels, based on history, physical examination, and clinical judgment. The ACR Appropriateness Criteria are evidence-based guidelines for specific clinical conditions that are reviewed every three years by a multidisciplinary expert panel. The guideline development and review include an extensive analysis of current medical literature from peer-reviewed journals, and the application of a well-established consensus methodology (modified Delphi) to rate the appropriateness of imaging and treatment procedures by the panel. In those instances in which evidence is lacking or not definitive, expert opinion may be used to recommend imaging or treatment.

Canadian Triage and Acuity Scale: testing the mental health categories

PURPOSE

The study tested the inter-rater reliability and accuracy of triage nurses' assignment of urgency ratings for mental health patient scenarios based on the 2008 Canadian Triage and Acuity Scale (CTAS) guidelines, using a standardized triage tool. The influence of triage experience, educational preparation, and comfort level with mental health presentations on the accuracy of urgency ratings was also explored.

METHODS

Study participants assigned urgency ratings to 20 mental health patient scenarios in randomized order using the CTAS. The scenarios were developed using actual triage notes and were reviewed by an expert panel of emergency and mental health clinicians for face and content validity.

RESULTS

The overall Fleiss' kappa, the measure of inter-rater reliability for this sample of triage nurses (n=18), was 0.312, representing only fair albeit statistically significant (P<0.0001) agreement. Kendall's coefficient of concordance for the sample was calculated to be 0.680 (P<0.0001), which signifies moderate agreement. Although the sample reported high levels of education, comfort with mental health presentations, and experience, accuracy in urgency ratings measured by the percentage of correct responses ranged from 0.05% to 94% (mean: 54%). Greater accuracy in urgency ratings was recorded for triage nurses who used second-order modifiers and avoided the use of override.

CONCLUSION

Specific focus on the use of second-order modifiers in orientation and ongoing education of triage nurses may improve the reliability and validity of the CTAS when used to assign urgency ratings to mental health presentations.

Interrater and intrarater agreements of magnetic resonance imaging findings in the lumbar spine: significant variability across degenerative conditions

BACKGROUND CONTEXT

Magnetic resonance imaging (MRI) is frequently used in the evaluation of degenerative conditions in the lumbar spine. The relative interrater and intrarater agreements of MRI findings across different pathologic conditions are underexplored, as most studies are focused on specific findings.

PURPOSE

The purpose of this study was to characterize the interrater and intrarater agreements of MRI findings used to assess the degenerative lumbar spine.

STUDY DESIGN

A retrospective diagnostic study at a large academic medical center was undertaken with a panel of orthopedic surgeons and musculoskeletal radiologists to assess lumbar MRIs using standardized criteria.

PATIENT SAMPLE

Seventy-five subjects who underwent routine lumbar spine MRI at our institution were included.

OUTCOME MEASURES

Each MRI study was assessed for 10 lumbar degenerative findings using standardized criteria. Lumbar vertebral levels were assessed independently, where applicable, for a total of 52 data points collected per study.

METHODS

T2-weighted axial and sagittal MRI sequences were presented in random order to the four reviewers (two orthopedic spine surgeons and two musculoskeletal radiologists) independently to determine interrater agreement. The first 10 studies were reevaluated at the end to determine intrarater agreement. Images were assessed using standardized and pilot-tested criteria to assess disc degeneration, stenosis, and other degenerative changes. Interrater and intrarater absolute percent agreements were calculated. To highlight the most clinically important MRI disagreements, a modified agreement analysis was also performed (in which disagreements between the lowest two severity grades for applicable conditions were ignored). Fleiss kappa coefficients for interrater agreement were determined.

RESULTS

The overall absolute and modified interrater agreements were 76.9% and 93.5%, respectively. The absolute and modified intrarater agreements were 81.3% and 92.7%, respectively. Average Fleiss kappa coefficient was 0.431, suggesting moderate overall agreement. However, when stratified by condition, absolute interrater agreement ranged from 65.1% to 92.0%. Disc hydration, disc space height, and bone marrow changes exhibited the lowest absolute interrater agreements. The absolute intrarater agreement had a narrower range, from 74.5% to 91.5%. Fleiss kappa coefficients ranged from fair-to-substantial agreement (0.282-0.618).

CONCLUSIONS

Even in a study using standardized evaluation criteria, there was significant variability in the interrater and intrarater agreements of MRI in assessing different degenerative conditions of the lumbar spine. Clinicians should be aware of the condition-specific diagnostic limitations of MRI interpretation.

Extension MRI is clinically useful in cervical myelopathy

INTRODUCTION

Cervical spine MRI with the neck in extension has been well described over the last 10 years, but its clinical value remains unknown.

METHODS

We performed extension imaging in 60 patients in whom the initial neutral study showed borderline cord compression. Images were assessed using a previously validated grading system for cord compression. Multiple linear and area measurements were also obtained. Images were scored blindly and randomly. Inter- and intra-rater variability were determined in a subset of 20 cases. Independent clinical assessment utilised the Ranwat criteria.

RESULTS

For most parameters inter/intra-observer variance of kappa/ICC > 0.6 was highly satisfactory. Standard MR was poor at discriminating between patients with and without myelopathy (ROC analysis, area under the curve (AUC), 0.52). This was considerably improved with extension imaging (AUC, 0.60), or by using the change in compression score between neutral and extension studies. Most measurements were not helpful; however, the ratio of cord area/CSF area at the level of maximum compression on extended images was the best discriminator (AUC, 0.71), as well as the presence of T2 change in cord substance (AUC, 0.68).

CONCLUSION

This is the first study to demonstrate added clinical value utilising extension MRI. In this cohort of difficult patients, when there was no T2 signal change in the cord, the presence of clinical myelopathy could only be predicted by utilising the data from extension imaging.

Reliability and diagnostic accuracy of clinical special tests for myelopathy in patients seen for cervical dysfunction

STUDY DESIGN

Case control study.

BACKGROUND

Myelopathy is a clinical diagnosis based largely on initial examination findings during a clinical screen, followed by imaging verification of cord injury or compression. At present, few studies have examined the reliability and diagnostic accuracy of clinical examination measures.

OBJECTIVES

To determine the reliability and diagnostic accuracy of neurological tests associated with the diagnosis of myelopathy.

METHODS AND MEASURES

Reliability and diagnostic accuracy of 7 frequently used tests and measures and subjective findings associated with myelopathy were examined on consecutive patients with cervical pain. Interrater reliability and diagnostic accuracy values, including posttest probability, based on a pretest probability of 40%, were calculated for each test and for combinations of tests and measures.

RESULTS

Four of the 7 diagnostic tests were found to have a substantial interrater reliability. None of the single or clusters of tests yielded low negative likelihood ratios. Of the individual tests, the Babinski sign demonstrated the highest positive likelihood ratio (LR+, 4.0; 95% CI: 1.1-16.6) and posttest probability (73%) for diagnosis, but yielded only a moderate negative likelihood ratio (LR-, 0.7; 95% CI: 0.6-0.9). Combinations of tests did not yield improved accuracy values over single test results.

CONCLUSION

This study demonstrated that 4 of 7 tests used to screen for myelopathy offered substantial levels of interrater agreement when used on individuals with cervical dysfunction. None of the tests when performed individually or in combinations are effective for screening; however, the Babinski sign did alter posttest probability more significantly than combinations of test findings.

LEVEL OF EVIDENCE

Diagnosis, Level 2b.

The reproducibility of quantitative measurements in lumbar magnetic resonance imaging of children from the general population

STUDY DESIGN

Quantitative lumbar magnetic resonance imaging (MRI) measurements in children were taken twice and analyzed for intra- and intertester reproducibility.

OBJECTIVE

To evaluate the reproducibility of a variety of lumbar quantitative measurements taken from MRIs of children from the general population.

SUMMARY OF BACKGROUND DATA

The reproducibility of lumbar quantitative measurements has never been tested on MRIs of children from the general population. This is a prerequisite for a reliable anatomic characterization of the immature spine in normal and pathologic conditions.

METHODS

Lumbar quantitative measurements from 40 T2-weighted lumbar (L1-S1) MRIs of children from the general population were taken twice by 1 tester for intratester reproducibility and compared with the same measurements (L4-S1) of the same MRIs taken by a second tester for intertester reproducibility. The following parameters were measured using the iQ-VIEW system (IMAGE Information Systems Ltd., version 1.2.2, Plauen, Germany): Linear measurements--zygoappophyseal facets and interfacet widths, and vertebral body (VB), pedicle and intervertebral discs heights, widths, and lengths. Angular measurements--zygoappophyseal tranverse superior facet angles, sagittal VB, and disc wedging, lumbar lordosis, and sacral inclination. Statistical analysis included the concordance correlation coefficient (CCC), and Bland and Altman's limits of agreement (LOA).

RESULTS

A total of 6160 measurements were analyzed. Good to excellent intratester reproducibility (0.75 < or =CCC < or =0.99) was indicated for all linear measurements (except for pedicle heights) (LOA: -4.76; 5.74 mm) and for angular measurements related to the facets orientation, lumbar lordosis, and sacral inclination (LOA: 11.22 degrees ; 12.34 degrees). VB and disc sagittal wedging manifested poor intratester reproducibility (CCC < 0.75). Good to excellent intertester reproducibility was indicated for 70% of all linear measurements (LOA: -5.83; 4.59 mm) and for all angular measurements in the lumbar spine (L4-S1) (LOA: -9.96 degrees ; 14.62 degrees) except for VB and disc sagittal wedging at L5 (CCC < 0.75).

CONCLUSION

Quantitative lumbar MRI measurements in children from the general population were found to be reproducible indicating a good visualization of immature vertebral anatomic margins on MRIs and an accurate definition of the measurement protocol.