Development of Image Overlay and Knowledge Transfer Module Technologies Aimed at Enhancing Feasibility and External Validation of Magnetic Resonance Imaging-based Scoring Systems

Deep-learning based quantification model for hip bone marrow edema and synovitis in patients with spondyloarthritis based on magnetic resonance images

Objectives: Hip inflammation is one of the most common complications in patients with spondyloarthritis (SpA). Herein, we employed use of a deep learning-based magnetic resonance imaging (MRI) evaluation model to identify irregular and multiple inflammatory lesions of the hip. Methods: All of the SpA patients were enrolled at the Xijing Hospital. The erythrocyte sediment rate (ESR), C-reactive protein (CRP), hip function Harris score, and disease activity were evaluated by clinicians. Manual MRI annotations including bone marrow edema (BME) and effusion/synovitis, and a hip MRI scoring system (HIMRISS) assessment was performed by experienced musculoskeletal radiologists. The segmentation accuracies of four deep learning models, including U-Net, UNet++, Attention-Unet, and HRNet, were compared using five-fold cross-validation. The clinical agreement of U-Net was evaluated with clinical symptoms and HIMRISS results. Results: A total of 1945 MRI slices of STIR/T2WI sequences were obtained from 195 SpA patients with hip involvement. After the five-fold cross-validation, U-Net achieved an average segmentation accuracy of 88.48% for the femoral head and 69.36% for inflammatory lesions, which are higher than those obtained by the other three models. The UNet-score, which was calculated based on the same MRI slices as HIMRISS, was significantly correlated with the HIMRISS scores and disease activity indexes (p values <0.05). Conclusion: This deep-learning based automatic MRI evaluation model could achieve similar quantification performance as an expert radiologist, and it has the potential to improve the accuracy and efficiency of clinical diagnosis for SpA patients with hip involvement.

Hip Inflammation MRI Scoring System (HIMRISS) to predict response to hyaluronic acid injection in hip osteoarthritis

OBJECTIVE

To assess predictors of response, according to hip MRI inflammatory scoring system (HIMRISS), in a sample of patients with hip osteoarthritis (OA) treated by hyaluronic acid (HA) injection.

METHOD

Sixty patients with hip OA were included. Clinical outcomes were assessed at baseline and three months after HA injection by WOMAC. On hip MRI performed before HA injection, bone marrow lesion (BML) and synovitis were assessed by HIMRISS by four readers. The inter-reader reliability of HIMRISS was for HIMRISS total, acetabular BML, femoral BML and synovitis-effusion respectively 0.86, 0.64, 0.83 and 0.78. Associations between MRI features and clinical data were assessed. Logistic regression (univariate and multivariate) was used to explore associations between MRI features and response to HA injection, according to WOMAC50 response at three months.

RESULTS

In total, 45.5% of patients met WOMAC50 response. Five adverse events were reported. At baseline, WOMAC function correlated significantly to HIMRISS synovitis-effusion (r=0.27, P=0.03). In univariate analysis, BML femoral according to binary assessment (P=0.025), HIMRISS BML femoral (P=0.0038), HIMRISS BML acetabular (P=0.042), HIMRISS total (P=0.0092) were associated negatively with WOMAC50 response. In multivariate analysis, adjusted for age and BMI, HIMRISS femoral BML (P=0.02) and HIMRISS total (P=0.016) were negatively associated with response. At a HIMRISS threshold of<15, 82% of patients were responders, with specificity SP=0.97, sensitivity SN=0.39, and positive and negative predictive values of 0.91 and 0.64, respectively.

CONCLUSION

HIMRISS is reliable for total scores and sub-domains. It permits identification of responders to HA injection in hip OA patients.

Validation of a Knowledge Transfer Tool According to the OMERACT Filter: Does Web-based Real-time Iterative Calibration Enhance the Evaluation of Bone Marrow Lesions in Hip Osteoarthritis?

OBJECTIVE

To assess reliability and feasibility of using a Web-based interface and interactive online calibration tool for magnetic resonance imaging (MRI) scoring of bone marrow lesions (BML) in osteoarthritis (OA), applied to the Hip MR Inflammation Scoring System (HIMRISS).

METHODS

Seven readers new to HIMRISS (3 radiologists, 4 rheumatologists) scored coronal short-tau inversion recovery MRI from a hip OA observational study obtained pre- and 8-week poststeroid injection (n = 40 × 2 scans × 2 hips = 160 hips). By crossover design, Group B (4 readers) scored 20 patients (40 hips) using conventional spreadsheet-based methods and then another 20 using a Web-based interface and an online real-time iterative calibration (RETIC) training module. Group A (3 readers) reversed the order, scoring the first 20 subjects by the new method and the final 20 conventionally. Outcomes included ICC and reader survey.

RESULTS

Interobserver reliability for BML status was high by both spreadsheet and Web-based methods (0.84-0.90), regardless of the order in which scoring was performed. Reliability of change scores was moderate and improved with training. Improvement was greater in readers who began with the spreadsheet method and then used the Web-based method than in those who began with the Web-based method, especially at the acetabulum. Readers found Web-based/RETIC scoring more user-friendly and nearly 50% faster than traditional spreadsheet methods.

CONCLUSION

HIMRISS offers reliable BML scoring in OA, whether by conventional spreadsheet-based scoring or by a Web-based interface with interactive feedback. The new method allowed faster readings, provided a consistent training environment that helped inexperienced readers achieve reliability equivalent to that of conventional methods, and was preferred by the readers.

Validation of a Knowledge Transfer Tool for the Knee Inflammation MRI Scoring System for Bone Marrow Lesions According to the OMERACT Filter: Data from the Osteoarthritis Initiative

Objective.

To assess feasibility and reliability of scoring bone marrow lesions (BML) on knee magnetic resonance imaging (MRI) in osteoarthritis using the Outcome Measures in Rheumatology Knee Inflammation MRI Scoring System (KIMRISS), with a Web-based interface and online training with real-time iterative calibration.

Methods.

Six readers new to the KIMRISS (3 radiologists, 3 rheumatologists) scored sagittal T2-weighted fat-saturated MRI in 20 subjects randomly selected from the Osteoarthritis Initiative data, at baseline and 1-year followup. In the KIMRISS, the reader moves a transparent overlay grid within a Web-based interface to fit bones, then clicks or touches each region containing BML per slice, to score 1 if BML is present. Regional and total scores are automatically calculated. Outcomes include the interreader intraclass correlation coefficients (ICC) and the smallest detectable change (SDC).

Results.

Scoring took 3–12 min per scan and all readers rated the process as moderately to very user friendly. Despite a low BML burden (average score 2.8% of maximum possible) and small changes, interobserver reliability was moderate to high for BML status and change in the femur and tibia (ICC 0.78–0.88). Four readers also scored the patella reliably, whereas 2 readers were outliers, likely because of image artifacts. SDC of 1.5–5.6 represented 0.7% of the maximum possible score.

Conclusion.

We confirmed feasibility of knee BML scoring by new readers using interactive training and a Web-based touch-sensitive overlay system, finding high reliability and sensitivity to change. Further work will include adjustments to training materials regarding patellar scoring, and study in therapeutic trial datasets with higher burden of BML and larger changes.