Verification bias: an under-recognized source of error in assessing the efficacy of MRI of the meniscii

Shoulder MRI-based radiomics for diagnosis and severity staging assessment of surgically treated supraspinatus tendon tears

OBJECTIVE

To develop and validate MRI-based radiomics models capable of evaluating supraspinatus tendon tears within the shoulder joints by using arthroscopy as the reference standard.

METHODS

A total of 432 patients (332 in the training set and 100 in the external validation set) with intact supraspinatus tendon (n = 202) and supraspinatus tendon tear (n = 230, 130 full-thickness tears and 100 partial-thickness tears) were enrolled. Radiomics features were extracted from fat-saturated T2-weighted coronal images. Two radiomics signature models for detecting supraspinatus tendon abnormalities (tear or not), and stage lesion severity (full- or partial-thickness tear) and radiomics scores (Rad-score), were constructed and calculated using multivariate logistic regression analysis. The diagnostic performance of the two models was validated using ROC curves on the training and validation datasets.

RESULTS

For the radiomics model of no tears or tears, thirteen features from MR images were used to build the radiomics signature with an AUC value of 0.98 in the training set, 0.97 in the internal validation set, and 0.98 in the external validation set. For the radiomics model of full- or partial-thickness tears, thirteen features from MR images were used to build the radiomics signature with an AUC value of 0.79 in the training set, 0.69 in the internal validation set, and 0.77 in the external validation set.

CONCLUSION

The proposed radiomics models in this study can accurately rule out supraspinatus tendon tears and are capable of assessing the severity staging of tears with moderate accuracy based on shoulder MR images.

KEY POINTS

• The radiomics model of no tears or tears achieved a high overall accuracy of 93.6%, sensitivity of 91.6%, and specificity of 95.2% for supraspinatus tendon tears. • The radiomics model of full- or partial-thickness tears displayed moderate performance with an accuracy of 76.4%, a sensitivity of 79.2%, and a specificity of 74.3% for supraspinatus tendon tears severity staging.

Fully Automated Diagnosis of Anterior Cruciate Ligament Tears on Knee MR Images by Using Deep Learning

Purpose

To investigate the feasibility of using a deep learning-based approach to detect an anterior cruciate ligament (ACL) tear within the knee joint at MRI by using arthroscopy as the reference standard.

Materials and Methods

A fully automated deep learning-based diagnosis system was developed by using two deep convolutional neural networks (CNNs) to isolate the ACL on MR images followed by a classification CNN to detect structural abnormalities within the isolated ligament. With institutional review board approval, sagittal proton density-weighted and fat-suppressed T2-weighted fast spin-echo MR images of the knee in 175 subjects with a full-thickness ACL tear (98 male subjects and 77 female subjects; average age, 27.5 years) and 175 subjects with an intact ACL (100 male subjects and 75 female subjects; average age, 39.4 years) were retrospectively analyzed by using the deep learning approach. Sensitivity and specificity of the ACL tear detection system and five clinical radiologists for detecting an ACL tear were determined by using arthroscopic results as the reference standard. Receiver operating characteristic (ROC) analysis and two-sided exact binomial tests were used to further assess diagnostic performance.

Results

The sensitivity and specificity of the ACL tear detection system at the optimal threshold were 0.96 and 0.96, respectively. In comparison, the sensitivity of the clinical radiologists ranged between 0.96 and 0.98, while the specificity ranged between 0.90 and 0.98. There was no statistically significant difference in diagnostic performance between the ACL tear detection system and clinical radiologists at P < .05. The area under the ROC curve for the ACL tear detection system was 0.98, indicating high overall diagnostic accuracy.

Conclusion

There was no significant difference between the diagnostic performance of the ACL tear detection system and clinical radiologists for determining the presence or absence of an ACL tear at MRI.© RSNA, 2019Supplemental material is available for this article.

Imaging of the post-operative shoulder: does injection of iodinated contrast in addition to MR contrast during arthrography improve diagnostic accuracy and patient throughput?

OBJECTIVE

Post-operative shoulder patients are often difficult to image due to scar tissue, metallic artifact, and residual irregularity of anatomic structures. The purpose of this study was to assess MR versus MR arthrography versus CT arthrography in the post-operative shoulder. Also, we assessed whether injecting CT contrast in addition to MR contrast during arthrography improved patient care.

METHODS AND MATERIALS

One hundred consecutive post-operative conventional shoulder MR and MR arthrography exams performed on the same patients were reviewed retrospectively by two musculoskeletal radiologists. A combination of gadolinium and CT contrast was injected at arthrography so CT imaging could be performed post arthrography if metallic artifact precluded MR imaging. Twenty-two of these patients also had CT exams performed post-arthrography due to metallic artifact on MR exam. Exams were assessed for labral tears and supraspinatus tendon tears. All patients went on to arthroscopy.

RESULTS

Of these 100 patients, 35 had SLAP (superior labral anterior to posterior) tears, 22 had posterior labral tears, 24 had anterior labral tears, and 46 had full-thickness supraspinatus tendon tears on conventional MR exam. On MR arthrography, 48 patients had SLAP tears, 26 had posterior labral tears, 27 had anterior labral tears, and 54 had full-thickness supraspinatus tendon tears. MR arthrogram detected 12 SLAP tears, three posterior labral tears, three anterior labral tears, and nine supraspinatus tendon tears not detected on conventional MR exam. Twenty-two patients had additional imaging performed with CT arthrography due to metallic artifacts precluding MR assessment of shoulder pathology. There were four SLAP tears, six posterior labral tears, five anterior labral tears, and five supraspinatus tendon tear seen on CT arthrography not seen on MR exam.

CONCLUSIONS

MR arthrography is more accurate than conventional MR in assessment of post-operative shoulder pathology. CT arthrography can detect additional pathology when there is metallic artifact in post-operative patients. It is beneficial to inject a combination of gadolinium and CT contrast at arthrography so CT imaging can be performed post-arthrography if metallic artifact precludes imaging shoulder pathology by MR.

Focusing on Inattention: The Diagnostic Accuracy of Brief Measures of Inattention for Detecting Delirium

BACKGROUND

Delirium is frequently missed in most clinical settings. Brief delirium assessments are needed.

OBJECTIVE

To determine the diagnostic accuracy of reciting the months of year backwards (MOTYB) from December to July (MOTYB-6) and December to January (MOTYB-12) for delirium as diagnosed by a psychiatrist and to explore the diagnostic accuracies of the following other brief attention tasks: (1) spell the word "LUNCH" backwards, (2) recite the days of the week backwards, (3) 10-letter vigilance "A" task, and (4) 5 picture recognition task.

DESIGN

Preplanned secondary analysis of a prospective observational study.

SETTING

Emergency department located within an academic, tertiary care hospital.

PARTICIPANTS

234 acutely ill patients who were =65 years old.

MEASUREMENTS

The inattention tasks were administered by a physician. The reference standard for delirium was a comprehensive psychiatrist assessment using Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision criteria. Sensitivities and specificities were calculated.

RESULTS

Making any error on the MOTYB-6 task had a sensitivity of 80.0% (95% confidence interval [CI], 60.9%-91.1%) and specificity of 57.1% (95% CI, 50.4%- 63.7%). Making any error on the MOTYB-12 task had a sensitivity of 84.0% (95% CI, 65.4%-93.6%) and specificity of 51.9% (95% CI, 45.2%-58.5%). The best combination of sensitivity and specificity was reciting the days of the week backwards task; if the patient made any error, this was 84.0% (95% CI, 65.4%-93.6%) sensitive and 81.9% (95% CI, 76.1%-86.5%) specific.

CONCLUSIONS

MOTYB-6 and MOTYB-12 had very good sensitivities but had modest specificities for delirium, limiting their use as a standalone assessment. Reciting the days of the week backwards appeared to have the best combination of sensitivity and specificity for delirium.

Accuracy of Two-Dimensional Transvaginal Sonography and Office Hysteroscopy for Detection of Uterine Abnormalities in Patients with Repeated Implantation Failures or Recurrent Pregnancy Loss

Background

We sought to compare diagnostic values of two-dimensional transvaginal sonography (2D TVS) and office hysteroscopy (OH) for evaluation of endometrial pathologies in cases with repeated implantation failure (RIF) or recurrent pregnancy loss (RPL).

MATERIALS AND METHODS

This prospective study was performed at Royan Institute from December 2013 to January 2015. TVS was performed before hysteroscopy as part of the routine diagnostic work-up in 789 patients with RIF or RPL. Uterine biopsy was performed in cases with abnormal diagnosis in TVS and/or hysteroscopy. We compared the diagnostic accuracy values of TVS in detection of uterine abnormalities with OH by receiver operating characteristic (ROC) curve analysis.

RESULTS

TVS examination detected 545 (69%) normal cases and 244 (31%) pathologic cases, which included 84 (10.6%) endometrial polyps, 15 (1.6%) uterine fibroids, 10 (1.3%) Asherman's syndrome, 9 (1.1%) endometrial hypertrophy, and 126 (15.9%) septate and arcuate uterus. TVS and OH concurred in 163 pathologic cases, although TVS did not detect some pathology cases (n=120). OH had 94% sensitivity, 95% specificity, 62% positive predictive value (PPV), and 99% negative predictive value (NPV) for detection of endometrial polyps. In the diagnosis of myoma, sensitivity, specificity, PPV, and NPV were 100%. TVS had a sensitivity of 50% and specificity of 98% for the diagnosis of myoma. For polyps, TVS had a sensitivity of 54% and specificity of 80%. Area under the ROC curve (AUROC) was 70.69% for the accuracy of TVS compared to OH.

CONCLUSION

TVS had high specificity and low sensitivity for detection of uterine pathologies in patients with RIF or RPL compared with OH. OH should be considered as a workup method prior to treatment in patients with normal TVS findings.

[Systematic errors in clinical studies : A comprehensive survey]

Systematic errors and related phenomena represent an intrinsic challenge to the quality of clinical research. As a consequence even otherwise methodologically demanding studies may produce results that systematically differ from the true values. Systematic errors relating to investigative medicine are divided into six groups according to their affiliation with the consecutive chronological sections of the study. Bias can occur in preliminary literature research in the field, specifying the study design and selecting the study sample, measuring exposure and outcome, analyzing the data, interpreting the analyses and publishing the results. The most important systematic errors that concern diagnostic and interventional studies are created by access to the data of previous tests, calculated study design, preselection of the participants, comparison with non-contemporaneous controls, antedating the time of diagnosis and overdiagnosis of slowly progressive forms of diseases examined. Checking the measured values often leads to a mosaic of several biases with one being more or less dominant. Even by exercising due care in the preparation and performance of the study, the majority of distortions cannot be eliminated but only diminished. It is essential to consider each detected bias as a potential full or partial argument in support of an observed correlation. The control of systematic errors and related phenomena is both a significant element of the discussion of the study report and a key element for assessment of its scientific value.

[Systematic errors in clinical studies : A comprehensive survey]

Systematic errors and related phenomena represent an intrinsic challenge to the quality of clinical research. As a consequence even otherwise methodologically demanding studies may produce results that systematically differ from the true values. Systematic errors relating to investigative medicine are divided into six groups according to their affiliation with the consecutive chronological sections of the study. Bias can occur in preliminary literature research in the field, specifying the study design and selecting the study sample, measuring exposure and outcome, analyzing the data, interpreting the analyses and publishing the results. The most important systematic errors that concern diagnostic and interventional studies are created by access to the data of previous tests, calculated study design, preselection of the participants, comparison with non-contemporaneous controls, antedating the time of diagnosis and overdiagnosis of slowly progressive forms of diseases examined. Checking the measured values often leads to a mosaic of several biases with one being more or less dominant. Even by exercising due care in the preparation and performance of the study, the majority of distortions cannot be eliminated but only diminished. It is essential to consider each detected bias as a potential full or partial argument in support of an observed correlation. The control of systematic errors and related phenomena is both a significant element of the discussion of the study report and a key element for assessment of its scientific value.

A systematic review and meta-analysis of the diagnostic accuracy of MRI for suspected ACL and meniscal tears of the knee

PURPOSE

To determine the diagnostic accuracy of magnetic resonance imaging (MRI) and ultrasound (US) in the diagnosis of anterior cruciate ligament (ACL), medial meniscus and lateral meniscus tears in people with suspected ACL and/or meniscal tears.

METHODS

MEDLINE, Web of Science and the Cochrane library were searched from inception to March 2014. All prospective studies of the diagnostic accuracy of MRI or US against arthroscopy as the reference standard were included in the systematic review. Studies with a retrospective design and those with evidence of verification bias were excluded. Methodological quality of included studies was assessed using the QUADAS-2 tool. A meta-analysis of studies evaluating MRI to calculate the pooled sensitivity and specificity for each target condition was performed using a bivariate model with random effects. Sub-group and sensitivity analysis were used to examine the effect of methodological and other study variables.

RESULTS

There were 14 studies included in the meta-analysis of the accuracy of MRI for ACL tears, 19 studies included for medial meniscal tears and 19 studies for lateral meniscal tears. The summary estimates of sensitivity and specificity of MRI were 87 % (95 % CI 77-94 %) and 93 % (95 % CI 91-96 %), respectively, for ACL tears; 89 % (95 % CI 83-94 %) and 88 % (95 % CI 82-93 %), respectively, for medial meniscal tears; and 78 % (95 % CI 66-87 %) and 95 % (95 % CI 91-97 %), respectively, for lateral meniscal tears. Magnetic field strength had no significant effect on accuracy. Most studies had a high or unclear risk of bias. There were an insufficient number of studies that evaluated US to perform a meta-analysis.

CONCLUSION

This study provides a systematic review and meta-analysis of diagnostic accuracy studies of MRI and applies strict exclusion criteria in relation to the risk of verification bias. The risk of bias in most studies is high or unclear in relation to the reference standard. Concerns regarding the applicability of patient selection are also present in most studies.

LEVEL OF EVIDENCE

III.

Society of Skeletal Radiology 2011 Annual Meeting

Peer-reviewed abstracts presented at the 2011 Society of Skeletal Radiology (SSR) Annual Meeting were reviewed again following oral presentation. Topics representing new concepts, new or novel imaging techniques, and instructive case series involving the musculoskeletal system, felt to be of potential interest to investigators and practicing clinicians, have been highlighted in this compilation and analysis. ePoster highlights have also been submitted.