Knee menisci segmentation using convolutional neural networks: data from the Osteoarthritis Initiative

A Scientometric Worldview of Artificial Intelligence in Musculoskeletal Diseases Since the 21st Century

Purpose

Over the past 24 years, significant advancements have been made in applying artificial intelligence (AI) to musculoskeletal (MSK) diseases. However, there is a lack of analytical and descriptive investigations on the trajectory, essential research directions, current research scenario, pivotal focuses, and future perspectives. This research aims to provide a thorough update on the progress in AI for MSK diseases over the last 24 years.

Methods

Data from the Web of Science database, covering January 1, 2000, to March 1, 2024, was analyzed. Using advanced analytical tools, we conducted comprehensive scientometric and visual analyses.

Results

The findings highlight the predominant influence of the USA, which accounts for 28.53% of the total publications and plays a key role in shaping research in this field. Notable productivity was seen at institutions such as the University of California, San Francisco, Harvard Medical School, and Seoul National University. Valentina Pedoia is identified as the most prolific contributor. Scientific Reports had the highest number of publications in this area. The five most significant diseases are joint diseases, bone fractures, bone tumors, cartilage diseases, and spondylitis.

Conclusion

This comprehensive scientometric assessment benefits both experienced researchers and newcomers, providing quick access to essential information and fostering the development of innovative concepts in this field.

Radiomics analysis using magnetic resonance imaging of bone marrow edema for diagnosing knee osteoarthritis

This study aimed to develop and validate a bone marrow edema model using a magnetic resonance imaging-based radiomics nomogram for the diagnosis of osteoarthritis. Clinical and magnetic resonance imaging (MRI) data of 302 patients with and without osteoarthritis were retrospectively collected from April 2022 to October 2023 at Longhua Hospital affiliated with the Shanghai University of Traditional Chinese Medicine. The participants were randomly divided into two groups (a training group, n = 211 and a testing group, n = 91). We used logistic regression to analyze clinical characteristics and established a clinical model. Radiomics signatures were developed by extracting radiomic features from the bone marrow edema area using MRI. A nomogram was developed based on the rad-score and clinical characteristics. The diagnostic performance of the three models was compared using the receiver operating characteristic curve and Delong's test. The accuracy and clinical application value of the nomogram were evaluated using calibration curve and decision curve analysis. Clinical characteristics such as age, radiographic grading, Western Ontario and McMaster Universities Arthritis Index score, and radiological features were significantly correlated with the diagnosis of osteoarthritis. The Rad score was constructed from 11 radiological features. A clinical model was developed to diagnose osteoarthritis (training group: area under the curve [AUC], 0.819; testing group: AUC, 0.815). Radiomics models were used to effectively diagnose osteoarthritis (training group,: AUC, 0.901; testing group: AUC, 0.841). The nomogram model composed of Rad score and clinical characteristics had better diagnostic performance than a simple clinical model (training group: AUC, 0.906; testing group: AUC, 0.845; p < 0.01). Based on DCA, the nomogram model can provide better diagnostic performance in most cases. In conclusion, the MRI-bone marrow edema-based radiomics-clinical nomogram model showed good performance in diagnosing early osteoarthritis.

Horizontal Cleavage Meniscus Tear: "The Quad Tendon Augmentation Technique"

The management of medial meniscus horizontal cleavage tears can be challenging. Currently, several treatment options, including nonoperative and surgical options, have been proposed in the literature. Different repair techniques aiming to promote the healing process have been reported and have shown good outcomes. However, recurrent parameniscal cysts and decreased meniscal volume have also been reported. In this Technical Note, a novel surgical technique to repair a horizontal cleavage tear of the posterior horn of the medial meniscus is reported in young patients. The technique uses a strip of autologous quadriceps tendon to fill the void between the upper and lower meniscal leaflets followed by an all-inside compression suture. Both of these technical features aim to overcome the limitations of current repair techniques.

Integrating Radiomics and Neural Networks for Knee Osteoarthritis Incidence Prediction

OBJECTIVE

Accurately predicting knee osteoarthritis (KOA) is essential for early detection and personalized treatment. We aimed to develop and test a magnetic resonance imaging (MRI)-based joint space (JS) radiomic model (RM) to predict radiographic KOA incidence through neural networks by integrating meniscus and femorotibial cartilage radiomic features.

METHODS

In the Osteoarthritis Initiative cohort, participants with knees without radiographic KOA at baseline but at high risk for radiographic KOA were included. Patients' knees developed radiographic KOA, whereas control knees did not over four years. We randomly split the participants into development and test cohorts (8:2) and extracted features from baseline three-dimensional double-echo steady-state sequence MRI. Model performance was evaluated using an area under the receiver operating characteristic curve (AUC), sensitivity, and specificity in both cohorts. Nine resident surgeons performed the reader experiment without/with the JS-RM aid.

RESULTS

Our study included 549 knees in the development cohort (275 knees of patients with KOA vs 274 knees of controls) and 137 knees in the test cohort (68 knees of patients with KOA vs 69 knees of controls). In the test cohort, JS-RM had a favorable accuracy for predicting the radiographic KOA incidence with an AUC of 0.931 (95% confidence interval [CI] 0.876-0.963), a sensitivity of 84.4% (95% CI 83.9%-84.9%), and a specificity of 85.6% (95% CI 85.2%-86.0%). The mean specificity and sensitivity of resident surgeons through MRI reading in predicting radiographic KOA incidence were increased from 0.474 (95% CI 0.333-0.614) and 0.586 (95% CI 0.429-0.743) without the assistance of JS-RM to 0.874 (95% CI 0.847-0.901) and 0.812 (95% CI 0.742-0.881) with JS-RM assistance, respectively (P < 0.001).

CONCLUSION

JS-RM integrating the features of the meniscus and cartilage showed improved predictive values in radiographic KOA incidence.

Adaptive Fusion of Deep Learning With Statistical Anatomical Knowledge for Robust Patella Segmentation From CT Images

Kneeosteoarthritis (KOA), as a leading joint disease, can be decided by examining the shapes of patella to spot potential abnormal variations. To assist doctors in the diagnosis of KOA, a robust automatic patella segmentation method is highly demanded in clinical practice. Deep learning methods, especially convolutional neural networks (CNNs) have been widely applied to medical image segmentation in recent years. Nevertheless, poor image quality and limited data still impose challenges to segmentation via CNNs. On the other hand, statistical shape models (SSMs) can generate shape priors which give anatomically reliable segmentation to varying instances. Thus, in this work, we propose an adaptive fusion framework, explicitly combining deep neural networks and anatomical knowledge from SSM for robust patella segmentation. Our adaptive fusion framework will accordingly adjust the weight of segmentation candidates in fusion based on their segmentation performance. We also propose a voxel-wise refinement strategy to make the segmentation of CNNs more anatomically correct. Extensive experiments and thorough assessment have been conducted on various mainstream CNN backbones for patella segmentation in low-data regimes, which demonstrate that our framework can be flexibly attached to a CNN model, significantly improving its performance when labeled training data are limited and input image data are of poor quality.

Integration of Swin UNETR and statistical shape modeling for a semi-automated segmentation of the knee and biomechanical modeling of articular cartilage

Simulation studies, such as finite element (FE) modeling, provide insight into knee joint mechanics without patient involvement. Generic FE models mimic the biomechanical behavior of the tissue, but overlook variations in geometry, loading, and material properties of a population. Conversely, subject-specific models include these factors, resulting in enhanced predictive precision, but are laborious and time intensive. The present study aimed to enhance subject-specific knee joint FE modeling by incorporating a semi-automated segmentation algorithm using a 3D Swin UNETR for an initial segmentation of the femur and tibia, followed by a statistical shape model (SSM) adjustment to improve surface roughness and continuity. For comparison, a manual FE model was developed through manual segmentation (i.e., the de-facto standard approach). Both FE models were subjected to gait loading and the predicted mechanical response was compared. The semi-automated segmentation achieved a Dice similarity coefficient (DSC) of over 98% for both the femur and tibia. Hausdorff distance (mm) between the semi-automated and manual segmentation was 1.4 mm. The mechanical results (max principal stress and strain, fluid pressure, fibril strain, and contact area) showed no significant differences between the manual and semi-automated FE models, indicating the effectiveness of the proposed semi-automated segmentation in creating accurate knee joint FE models. We have made our semi-automated models publicly accessible to support and facilitate biomechanical modeling and medical image segmentation efforts ( https://data.mendeley.com/datasets/k5hdc9cz7w/1 ).

A Comprehensive Review on MRI-based Knee Joint Segmentation and Analysis Techniques

Using magnetic resonance imaging (MRI) in osteoarthritis pathogenesis research has proven extremely beneficial. However, it is always challenging for both clinicians and researchers to detect morphological changes in knee joints from magnetic resonance (MR) imaging since the surrounding tissues produce identical signals in MR studies, making it difficult to distinguish between them. Segmenting the knee bone, articular cartilage and menisci from the MR images allows one to examine the complete volume of the bone, articular cartilage, and menisci. It can also be used to assess certain characteristics quantitatively. However, segmentation is a laborious and time-consuming operation that requires sufficient training to complete correctly. With the advancement of MRI technology and computational methods, researchers have developed several algorithms to automate the task of individual knee bone, articular cartilage and meniscus segmentation during the last two decades. This systematic review aims to present available fully and semi-automatic segmentation methods for knee bone, cartilage, and meniscus published in different scientific articles. This review provides a vivid description of the scientific advancements to clinicians and researchers in this field of image analysis and segmentation, which helps the development of novel automated methods for clinical applications. The review also contains the recently developed fully automated deep learning-based methods for segmentation, which not only provides better results compared to the conventional techniques but also open a new field of research in Medical Imaging.

The influence of meniscal pathology in the incidence of knee osteoarthritis: a review

IMPORTANCE

Knee osteoarthritis (OA) is a common cause of pain and disability in older persons, affecting approximately 14 million individuals in the USA. Meniscal damage is also common in this age group with a prevalence of 35% in a middle-aged and older community sample and 82% in persons with evidence of radiographic knee osteoarthritis. This paper systematically reviews evidence on the association of meniscal pathology and incident radiographic knee OA.

OBSERVATIONS

We included 15 articles, published between 2013 and 2021, assessing the relationship between meniscal pathology and OA incidence (Fig. 1). The menisci are crucial load-bearing structures, and the resulting increase in biomechanical stress due to meniscal damage increases the risk for OA development. While some discrepancies are present in the literature, a clinically meaningful association has been generally established between the presence of a meniscal tear or meniscal extrusion and subsequent development of incident OA. Of note, larger radial tears as well as complex and more severe tears exhibit the strongest association with the development of incident OA. The relationship between other features of meniscal morphology-such as meniscal volume and meniscal coverage-and incident OA is less clearly documented.

CONCLUSIONS AND RELEVANCE

The early detection of meniscal pathology can be used to trigger preventative and therapeutic strategies designed to avert or delay knee OA in this at-risk population.

Quantitative bone marrow lesion, meniscus, and synovitis measurement: current status

Imaging plays a pivotal role in osteoarthritis research, particularly in epidemiological and clinical trials of knee osteoarthritis (KOA), with the ultimate goal being the development of an effective drug treatment for future prevention or cessation of disease. Imaging assessment methods can be semi-quantitative, quantitative, or a combination, with quantitative methods usually relying on software to assist. The software generally attempts image segmentation (outlining of relevant structures). New techniques using artificial intelligence (AI) or deep learning (DL) are currently a frequent topic of research. This review article provides an overview of the literature to date, focusing primarily on the current status of quantitative software-based assessment techniques of KOA using magnetic resonance (MR) imaging. We will concentrate on the imaging evaluation of three specific structural imaging biomarkers: bone marrow lesions (BMLs), meniscus, and synovitis consisting of effusion synovitis (ES) and Hoffa's synovitis (HS). A brief clinical and imaging background review of osteoarthritis evaluation, particularly relating to these three structural markers, is provided as well as a general summary of the software methods. A summary of the literature with respect to each KOA assessment method will be presented overall as well as with respect to each specific biomarker individually. Novel techniques, as well as future goals and directions using quantitative imaging assessment, will be discussed.

Could a three-dimensional contralateral meniscus segmentation for allograft or scaffold sizing be possible? A prospective study

PURPOSE

Meniscal allografts and biodegradable meniscal implants are attractive surgical options for painful subtotal or total meniscectomies. In order to get the best results, these should be as similar as possible to the original meniscus in terms of shape, structure, and volume. Three-dimensional meniscus sizing could be an approach to improve the accuracy of meniscus matching. Therefore, the aims of this study were to perform a comparative morphological and volumetric analysis of the healthy meniscus based on manual tri-planar segmentation and to demonstrate that the menisci from the contralateral knee could be used as a reference in the sizing of a meniscal graft or a scaffold.

METHODS

Three-dimensional meniscal models were created based on 120 MRIs in 60 healthy subjects (bilateral knees). The differences between the pairs of menisci concerning the widths, thicknesses, lateromedial distances, anteroposterior distances, angles of coverage, and meniscal volumes were evaluated. T-Student tests were used to compare the quantitative numerical variables of the different groups. Pearson's linear regression was used to determine if correlations existed between demographic variables (age, gender, height, weight) and anatomical parameters. Statistical significance was set at p < 0.05.

RESULTS

Comparing the 120 pairs of menisci of each subject, there was no statistically significant difference for all parameters studied for both the medial and lateral meniscus. When the measurements were stratified by gender, statistically significant differences were observed for all parameters except meniscal coverage angles. We observed that anteroposterior and lateromedial distances were positively correlated with height and body mass index both at the level of the medial meniscus (r = 0.68; r = 0.66; r = 0.65; and r = 0.63) and lateral (r = 0.68; r = 0.69; r = 0.61; and r = 0.60).

CONCLUSION

Our study demonstrated that the intra-individual 3D shapes of the left and right menisci are very similar. Therefore, the contralateral side could be used as a template for the 3D sizing of meniscal allografts or meniscal implants.

Magnetic resonance imaging assessments for knee segmentation and their use in combination with machine/deep learning as predictors of early osteoarthritis diagnosis and prognosis

Knee osteoarthritis (OA) is a prevalent and disabling disease that can develop over decades. This disease is heterogeneous and involves structural changes in the whole joint, encompassing multiple tissue types. Detecting OA before the onset of irreversible changes is crucial for early management, and this could be achieved by allowing knee tissue visualization and quantifying their changes over time. Although some imaging modalities are available for knee structure assessment, magnetic resonance imaging (MRI) is preferred. This narrative review looks at existing literature, first on MRI-developed approaches for evaluating knee articular tissues, and second on prediction using machine/deep-learning-based methodologies and MRI as input or outcome for early OA diagnosis and prognosis. A substantial number of MRI methodologies have been developed to assess several knee tissues in a semi-quantitative and quantitative fashion using manual, semi-automated and fully automated systems. This dynamic field has grown substantially since the advent of machine/deep learning. Another active area is predictive modelling using machine/deep-learning methodologies enabling robust early OA diagnosis/prognosis. Moreover, incorporating MRI markers as input/outcome in such predictive models is important for a more accurate OA structural diagnosis/prognosis. The main limitation of their usage is the ability to move them in rheumatology practice. In conclusion, MRI knee tissue determination and quantification provide early indicators for individuals at high risk of developing this disease or for patient prognosis. Such assessment of knee tissues, combined with the development of models/tools from machine/deep learning using, in addition to other parameters, MRI markers for early diagnosis/prognosis, will maximize opportunities for individualized risk assessment for use in clinical practice permitting precision medicine. Future efforts should be made to integrate such prediction models into open access, allowing early disease management to prevent or delay the OA outcome.

Personalized statistical modeling of soft tissue structures in the knee

Background and Objective: As in vivo measurements of knee joint contact forces remain challenging, computational musculoskeletal modeling has been popularized as an encouraging solution for non-invasive estimation of joint mechanical loading. Computational musculoskeletal modeling typically relies on laborious manual segmentation as it requires reliable osseous and soft tissue geometry. To improve on feasibility and accuracy of patient-specific geometry predictions, a generic computational approach that can easily be scaled, morphed and fitted to patient-specific knee joint anatomy is presented.

Methods: A personalized prediction algorithm was established to derive soft tissue geometry of the knee, originating solely from skeletal anatomy. Based on a MRI dataset (n = 53), manual identification of soft-tissue anatomy and landmarks served as input for our model by use of geometric morphometrics. Topographic distance maps were generated for cartilage thickness predictions. Meniscal modeling relied on wrapping a triangular geometry with varying height and width from the anterior to the posterior root. Elastic mesh wrapping was applied for ligamentous and patellar tendon path modeling. Leave-one-out validation experiments were conducted for accuracy assessment.

Results: The Root Mean Square Error (RMSE) for the cartilage layers of the medial tibial plateau, the lateral tibial plateau, the femur and the patella equaled respectively 0.32 mm (range 0.14–0.48), 0.35 mm (range 0.16–0.53), 0.39 mm (range 0.15–0.80) and 0.75 mm (range 0.16–1.11). Similarly, the RMSE equaled respectively 1.16 mm (range 0.99–1.59), 0.91 mm (0.75–1.33), 2.93 mm (range 1.85–4.66) and 2.04 mm (1.88–3.29), calculated over the course of the anterior cruciate ligament, posterior cruciate ligament, the medial and the lateral meniscus.

Conclusion: A methodological workflow is presented for patient-specific, morphological knee joint modeling that avoids laborious segmentation. By allowing to accurately predict personalized geometry this method has the potential for generating large (virtual) sample sizes applicable for biomechanical research and improving personalized, computer-assisted medicine.

Deep Learning-Based Medical Images Segmentation of Musculoskeletal Anatomical Structures: A Survey of Bottlenecks and Strategies

By leveraging the recent development of artificial intelligence algorithms, several medical sectors have benefited from using automatic segmentation tools from bioimaging to segment anatomical structures. Segmentation of the musculoskeletal system is key for studying alterations in anatomical tissue and supporting medical interventions. The clinical use of such tools requires an understanding of the proper method for interpreting data and evaluating their performance. The current systematic review aims to present the common bottlenecks for musculoskeletal structures analysis (e.g., small sample size, data inhomogeneity) and the related strategies utilized by different authors. A search was performed using the PUBMED database with the following keywords: deep learning, musculoskeletal system, segmentation. A total of 140 articles published up until February 2022 were obtained and analyzed according to the PRISMA framework in terms of anatomical structures, bioimaging techniques, pre/post-processing operations, training/validation/testing subset creation, network architecture, loss functions, performance indicators and so on. Several common trends emerged from this survey; however, the different methods need to be compared and discussed based on each specific case study (anatomical region, medical imaging acquisition setting, study population, etc.). These findings can be used to guide clinicians (as end users) to better understand the potential benefits and limitations of these tools.

Automatic segmentation of human knee anatomy by a convolutional neural network applying a 3D MRI protocol

BACKGROUND

To study deep learning segmentation of knee anatomy with 13 anatomical classes by using a magnetic resonance (MR) protocol of four three-dimensional (3D) pulse sequences, and evaluate possible clinical usefulness.

METHODS

The sample selection involved 40 healthy right knee volumes from adult participants. Further, a recently injured single left knee with previous known ACL reconstruction was included as a test subject. The MR protocol consisted of the following 3D pulse sequences: T1 TSE, PD TSE, PD FS TSE, and Angio GE. The DenseVNet neural network was considered for these experiments. Five input combinations of sequences (i) T1, (ii) T1 and FS, (iii) PD and FS, (iv) T1, PD, and FS and (v) T1, PD, FS and Angio were trained using the deep learning algorithm. The Dice similarity coefficient (DSC), Jaccard index and Hausdorff were used to compare the performance of the networks.

RESULTS

Combining all sequences collectively performed significantly better than other alternatives. The following DSCs (±standard deviation) were obtained for the test dataset: Bone medulla 0.997 (±0.002), PCL 0.973 (±0.015), ACL 0.964 (±0.022), muscle 0.998 (±0.001), cartilage 0.966 (±0.018), bone cortex 0.980 (±0.010), arteries 0.943 (±0.038), collateral ligaments 0.919 (± 0.069), tendons 0.982 (±0.005), meniscus 0.955 (±0.032), adipose tissue 0.998 (±0.001), veins 0.980 (±0.010) and nerves 0.921 (±0.071). The deep learning network correctly identified the anterior cruciate ligament (ACL) tear of the left knee, thus indicating a future aid to orthopaedics.

CONCLUSIONS

The convolutional neural network proves highly capable of correctly labeling all anatomical structures of the knee joint when applied to 3D MR sequences. We have demonstrated that this deep learning model is capable of automatized segmentation that may give 3D models and discover pathology. Both useful for a preoperative evaluation.

A review of advances in image-guided orthopedic surgery

Orthopedic surgery remains technically demanding due to the complex anatomical structures and cumbersome surgical procedures. The introduction of image-guided orthopedic surgery (IGOS) has significantly decreased the surgical risk and improved the operation results. This review focuses on the application of recent advances in artificial intelligence (AI), deep learning (DL), augmented reality (AR) and robotics in image-guided spine surgery, joint arthroplasty, fracture reduction and bone tumor resection. For the pre-operative stage, key technologies of AI and DL based medical image segmentation, 3D visualization and surgical planning procedures are systematically reviewed. For the intra-operative stage, the development of novel image registration, surgical tool calibration and real-time navigation are reviewed. Furthermore, the combination of the surgical navigation system with AR and robotic technology is also discussed. Finally, the current issues and prospects of the IGOS system are discussed, with the goal of establishing a reference and providing guidance for surgeons, engineers, and researchers involved in the research and development of this area.

Application of artificial intelligence to imaging interpretations in the musculoskeletal area: Where are we? Where are we going?

The interest of researchers, clinicians and radiologists, in artificial intelligence (AI) continues to grow. Deep learning is a subset of machine learning, in which the computer algorithm itself can determine the optimal imaging features to answer a clinical question. Convolutional neural networks are the most common architecture for performing deep learning on medical images. The various musculoskeletal applications of deep learning are the detection of abnormalities on X-rays or cross-sectional images (CT, MRI), for example the detection of fractures, meniscal tears, anterior cruciate ligament tears, degenerative lesions of the spine, bone metastases, classification of e.g., dural sac stenosis, degeneration of intervertebral discs, assessment of skeletal age, and segmentation, for example of cartilage. Software developments are already impacting the daily practice of orthopedic imaging by automatically detecting fractures on radiographs. Improving image acquisition protocols, improving the quality of low-dose CT images, reducing acquisition times in MRI, or improving MR image resolution is possible through deep learning. Deep learning offers an automated way to offload time-consuming manual processes and improve practitioner performance. This article reviews the current state of AI in musculoskeletal imaging.

Robust automatic hexahedral cartilage meshing framework enables population-based computational studies of the knee

Osteoarthritis of the knee is increasingly prevalent as our population ages, representing an increasing financial burden, and severely impacting quality of life. The invasiveness of in vivo procedures and the high cost of cadaveric studies has left computational tools uniquely suited to study knee biomechanics. Developments in deep learning have great potential for efficiently generating large-scale datasets to enable researchers to perform population-sized investigations, but the time and effort associated with producing robust hexahedral meshes has been a limiting factor in expanding finite element studies to encompass a population. Here we developed a fully automated pipeline capable of taking magnetic resonance knee images and producing a working finite element simulation. We trained an encoder-decoder convolutional neural network to perform semantic image segmentation on the Imorphics dataset provided through the Osteoarthritis Initiative. The Imorphics dataset contained 176 image sequences with varying levels of cartilage degradation. Starting from an open-source swept-extrusion meshing algorithm, we further developed this algorithm until it could produce high quality meshes for every sequence and we applied a template-mapping procedure to automatically place soft-tissue attachment points. The meshing algorithm produced simulation-ready meshes for all 176 sequences, regardless of the use of provided (manually reconstructed) or predicted (automatically generated) segmentation labels. The average time to mesh all bones and cartilage tissues was less than 2 min per knee on an AMD Ryzen 5600X processor, using a parallel pool of three workers for bone meshing, followed by a pool of four workers meshing the four cartilage tissues. Of the 176 sequences with provided segmentation labels, 86% of the resulting meshes completed a simulated flexion-extension activity. We used a reserved testing dataset of 28 sequences unseen during network training to produce simulations derived from predicted labels. We compared tibiofemoral contact mechanics between manual and automated reconstructions for the 24 pairs of successful finite element simulations from this set, resulting in mean root-mean-squared differences under 20% of their respective min-max norms. In combination with further advancements in deep learning, this framework represents a feasible pipeline to produce population sized finite element studies of the natural knee from subject-specific models.

Automated meniscus segmentation and tear detection of knee MRI with a 3D mask-RCNN

Background

The diagnostic results of magnetic resonance imaging (MRI) are essential references for arthroscopy as an invasive procedure. A deviation between medical imaging diagnosis and arthroscopy results may cause irreversible damage to patients and lead to excessive medical treatment. To improve the accurate diagnosis of meniscus injury, it is urgent to develop auxiliary diagnosis algorithms to improve the accuracy of radiological diagnosis.

Purpose

This study aims to present a fully automatic 3D deep convolutional neural network (DCNN) for meniscus segmentation and detects arthroscopically proven meniscus tears.

Materials and methods

Our institution retrospectively included 533 patients with 546 knees who underwent knee magnetic resonance imaging (MRI) and knee arthroscopy. Sagittal proton density-weighted (PDW) images in MRI of 382 knees were regarded as a training set to train our 3D-Mask RCNN. The remaining data from 164 knees were used to validate the trained network as a test set. The masks were hand-drawn by an experienced radiologist, and the reference standard is arthroscopic surgical reports. The performance statistics included Dice accuracy, sensitivity, specificity, FROC, receiver operating characteristic (ROC) curve analysis, and bootstrap test statistics. The segmentation performance was compared with a 3D-Unet, and the detection performance was compared with radiological evaluation by two experienced musculoskeletal radiologists without knowledge of the arthroscopic surgical diagnosis.

Results

Our model produced strong Dice coefficients for sagittal PDW of 0.924, 0.95 sensitivity with 0.823 FPs/knee. 3D-Unet produced a Dice coefficient for sagittal PDW of 0.891, 0.95 sensitivity with 1.355 FPs/knee. The difference in the areas under 3D-Mask-RCNN FROC and 3D-Unet FROC was statistically significant (p = 0.0011) by bootstrap test. Our model detection performance achieved an area under the curve (AUC) value, accuracy, and sensitivity of 0.907, 0.924, 0.941, and 0.785, respectively. Based on the radiological evaluations, the AUC value, accuracy, sensitivity, and specificity were 0.834, 0.835, 0.889, and 0.754, respectively. The difference in the areas between 3D-Mask-RCNN ROC and radiological evaluation ROC was statistically significant (p = 0.0009) by bootstrap test. 3D Mask RCNN significantly outperformed the 3D-Unet and radiological evaluation demonstrated by these results.

Conclusions

3D-Mask RCNN has demonstrated efficacy and precision for meniscus segmentation and tear detection in knee MRI, which can assist radiologists in improving the accuracy and efficiency of diagnosis. It can also provide effective diagnostic indicators for orthopedic surgeons before arthroscopic surgery and further promote precise treatment.

Automated Knee MR Images Segmentation of Anterior Cruciate Ligament Tears

The anterior cruciate ligament (ACL) is one of the main stabilizer parts of the knee. ACL injury leads to causes of osteoarthritis risk. ACL rupture is common in the young athletic population. Accurate segmentation at an early stage can improve the analysis and classification of anterior cruciate ligaments tears. This study automatically segmented the anterior cruciate ligament (ACL) tears from magnetic resonance imaging through deep learning. The knee mask was generated on the original Magnetic Resonance (MR) images to apply a semantic segmentation technique with convolutional neural network architecture U-Net. The proposed segmentation method was measured by accuracy, intersection over union (IoU), dice similarity coefficient (DSC), precision, recall and F1-score of 98.4%, 99.0%, 99.4%, 99.6%, 99.6% and 99.6% on 11451 training images, whereas on the validation images of 3817 was, respectively, 97.7%, 93.8%,96.8%, 96.5%, 97.3% and 96.9%. We also provide dice loss of training and test datasets that have remained 0.005 and 0.031, respectively. The experimental results show that the ACL segmentation on JPEG MRI images with U-Nets achieves accuracy that outperforms the human segmentation. The strategy has promising potential applications in medical image analytics for the segmentation of knee ACL tears for MR images.

An Expert-Supervised Registration Method for Multiparameter Description of the Knee Joint Using Serial Imaging

As knee osteoarthritis is a disease of the entire joint, our pathophysiological understanding could be improved by the characterization of the relationships among the knee components. Diverse quantitative parameters can be characterized using magnetic resonance imaging (MRI) and computed tomography (CT). However, a lack of methods for the coordinated measurement of multiple parameters hinders global analyses. This study aimed to design an expert-supervised registration method to facilitate multiparameter description using complementary image sets obtained by serial imaging. The method is based on three-dimensional tissue models positioned in the image sets of interest using manually placed attraction points. Two datasets, with 10 knees CT-scanned twice and 10 knees imaged by CT and MRI were used to assess the method when registering the distal femur and proximal tibia. The median interoperator registration errors, quantified using the mean absolute distance and Dice index, were ≤0.45 mm and ≥0.96 unit, respectively. These values differed by less than 0.1 mm and 0.005 units compared to the errors obtained with gold standard methods. In conclusion, an expert-supervised registration method was introduced. Its capacity to register the distal femur and proximal tibia supports further developments for multiparameter description of healthy and osteoarthritic knee joints, among other applications.

Tractography of Porcine Meniscus Microstructure Using High-Resolution Diffusion Magnetic Resonance Imaging

To noninvasively evaluate the three-dimensional collagen fiber architecture of porcine meniscus using diffusion MRI, meniscal specimens were scanned using a 3D diffusion-weighted spin-echo pulse sequence at 7.0 T. The collagen fiber alignment was revealed in each voxel and the complex 3D collagen network was visualized for the entire meniscus using tractography. The proposed automatic segmentation methods divided the whole meniscus to different zones (Red-Red, Red-White, and White-White) and different parts (anterior, body, and posterior). The diffusion tensor imaging (DTI) metrics were quantified based on the segmentation results. The heatmap was generated to investigate the connections among different regions of meniscus. Strong zonal-dependent diffusion properties were demonstrated by DTI metrics. The fractional anisotropy (FA) value increased from 0.13 (White-White zone) to 0.26 (Red-Red zone) and the radial diffusivity (RD) value changed from 1.0 × 10^-3 mm²/s (White-White zone) to 0.7 × 10^-3 mm²/s (Red-Red zone). Coexistence of both radial and circumferential collagen fibers in the meniscus was evident by diffusion tractography. Weak connections were found between White-White zone and Red-Red zone in each part of the meniscus. The anterior part and posterior part were less connected, while the body part showed high connections to both anterior part and posterior part. The tractography based on diffusion MRI may provide a complementary method to study the integrity of meniscus and nondestructively visualize the 3D collagen fiber architecture.

Cross-Cohort Automatic Knee MRI Segmentation With Multi-Planar U-Nets

Background

Segmentation of medical image volumes is a time‐consuming manual task. Automatic tools are often tailored toward specific patient cohorts, and it is unclear how they behave in other clinical settings.

Purpose

To evaluate the performance of the open‐source Multi‐Planar U‐Net (MPUnet), the validated Knee Imaging Quantification (KIQ) framework, and a state‐of‐the‐art two‐dimensional (2D) U‐Net architecture on three clinical cohorts without extensive adaptation of the algorithms.

Study Type

Retrospective cohort study.

Subjects

A total of 253 subjects (146 females, 107 males, ages 57 ± 12 years) from three knee osteoarthritis (OA) studies (Center for Clinical and Basic Research [CCBR], Osteoarthritis Initiative [OAI], and Prevention of OA in Overweight Females [PROOF]) with varying demographics and OA severity (64/37/24/53/2 scans of Kellgren and Lawrence [KL] grades 0–4).

Field Strength/Sequence

0.18 T, 1.0 T/1.5 T, and 3 T sagittal three‐dimensional fast‐spin echo T1w and dual‐echo steady‐state sequences.

Assessment

All models were fit without tuning to knee magnetic resonance imaging (MRI) scans with manual segmentations from three clinical cohorts. All models were evaluated across KL grades.

Statistical Tests

Segmentation performance differences as measured by Dice coefficients were tested with paired, two‐sided Wilcoxon signed‐rank statistics with significance threshold α = 0.05.

Results

The MPUnet performed superior or equal to KIQ and 2D U‐Net on all compartments across three cohorts. Mean Dice overlap was significantly higher for MPUnet compared to KIQ and U‐Net on CCBR (0.83±0.04 vs. 0.81±0.06 and 0.82±0.05), significantly higher than KIQ and U‐Net OAI (0.86±0.03 vs. 0.84±0.04 and 0.85±0.03), and not significantly different from KIQ while significantly higher than 2D U‐Net on PROOF (0.78±0.07 vs. 0.77±0.07, P=0.10, and 0.73±0.07). The MPUnet performed significantly better on N=22 KL grade 3 CCBR scans with 0.78±0.06 vs. 0.75±0.08 for KIQ and 0.76±0.06 for 2D U‐Net.

Data Conclusion

The MPUnet matched or exceeded the performance of state‐of‐the‐art knee MRI segmentation models across cohorts of variable sequences and patient demographics. The MPUnet required no manual tuning making it both accurate and easy‐to‐use.

Level of Evidence

3

Technical Efficacy

Stage 2

A generalized deep learning framework for automatic rheumatoid arthritis severity grading

Knee rheumatoid arthritis (RA) is the highly prevalent, chronic, progressive condition in the world. To diagnose this disease in the early stage in detail analysis with magnetic resonance (MR) image is possible. The imaging modality feature allows unbiased assessment of joint space narrowing (JSN), cartilage volume, and other vital features. This provides a fine-grained RA severity evaluation of the knee, contrasted to the benchmark, and generally used Kellgren Lawrence (KL) assessment. In this research, an intelligent system is developed to predict KL grade from the knee dataset. Our approach is based on hybrid deep learning of 50 layers (ResNet50) with skip connections. The proposed approach also uses Adam optimizer to provide learning linearity in the training stage. Our approach yields KL grade and JSN for femoral and tibial tissue with lateral and medial compartments. Furthermore, the approach also yields area under curve (AUC) of 0.98, accuracy 96.85%, mean absolute error (MAE) 0.015, precision 98.31%, and other commonly used parameters for the existence of radiographic RA progression which is improved than the existing state-of-the-art.

MRI Knee Domain Translation for Unsupervised Segmentation By CycleGAN (data from Osteoarthritis initiative (OAI))

Accurate quantification of bone and cartilage features is the key to efficient management of knee osteoarthritis (OA). Bone and cartilage tissues can be accurately segmented from magnetic resonance imaging (MRI) data using supervised Deep Learning (DL) methods. DL training is commonly conducted using large datasets with expert-labeled annotations. DL models perform better if distributions of testing data (target domains) are close to those of training data (source domains). However, in practice, data distributions of images from different MRI scanners and sequences are different and DL models need to re-trained on each dataset separately. We propose a domain adaptation (DA) framework using the CycleGAN model for MRI translation that would aid in unsupervised MRI data segmentation. We have validated our pipeline on five scans from the Osteoarthritis Initiative (OAI) dataset. Using this pipeline, we translated TSE Fat Suppressed MRI sequences to pseudo-DESS images. An improved MaskRCNN (IMaskRCNN) instance segmentation network trained on DESS was used to segment cartilage and femoral head regions in TSE Fat Suppressed sequences. Segmentations of the I-MaskRCNN correlated well with approximated manual segmentation obtained from nearest DESS slices (DICE = 0.76) without the need for retraining. We anticipate this technique will aid in automatic unsupervised assessment of knee MRI using commonly acquired MRI sequences and save experts' time that would otherwise be required for manual segmentation.Clinical relevance- This technique paves the way to automatically convert one MRI sequence to its equivalent as if acquired by a different protocol or different magnet, facilitating robust, hardware-independent automated analysis. For example, routine clinically acquired knee MRI could be converted to high-resolution high-contrast images suitable for automated detection of cartilage defects.

A Multi-Task Deep Learning Method for Detection of Meniscal Tears in MRI Data from the Osteoarthritis Initiative Database

We present a novel and computationally efficient method for the detection of meniscal tears in Magnetic Resonance Imaging (MRI) data. Our method is based on a Convolutional Neural Network (CNN) that operates on complete 3D MRI scans. Our approach detects the presence of meniscal tears in three anatomical sub-regions (anterior horn, body, posterior horn) for both the Medial Meniscus (MM) and the Lateral Meniscus (LM) individually. For optimal performance of our method, we investigate how to preprocess the MRI data and how to train the CNN such that only relevant information within a Region of Interest (RoI) of the data volume is taken into account for meniscal tear detection. We propose meniscal tear detection combined with a bounding box regressor in a multi-task deep learning framework to let the CNN implicitly consider the corresponding RoIs of the menisci. We evaluate the accuracy of our CNN-based meniscal tear detection approach on 2,399 Double Echo Steady-State (DESS) MRI scans from the Osteoarthritis Initiative database. In addition, to show that our method is capable of generalizing to other MRI sequences, we also adapt our model to Intermediate-Weighted Turbo Spin-Echo (IW TSE) MRI scans. To judge the quality of our approaches, Receiver Operating Characteristic (ROC) curves and Area Under the Curve (AUC) values are evaluated for both MRI sequences. For the detection of tears in DESS MRI, our method reaches AUC values of 0.94, 0.93, 0.93 (anterior horn, body, posterior horn) in MM and 0.96, 0.94, 0.91 in LM. For the detection of tears in IW TSE MRI data, our method yields AUC values of 0.84, 0.88, 0.86 in MM and 0.95, 0.91, 0.90 in LM. In conclusion, the presented method achieves high accuracy for detecting meniscal tears in both DESS and IW TSE MRI data. Furthermore, our method can be easily trained and applied to other MRI sequences.

Automated tibiofemoral joint segmentation based on deeply supervised 2D-3D ensemble U-Net: Data from the Osteoarthritis Initiative

Improving longevity is one of the greatest achievements in humanity. Because of this, the population is growing older, and the ubiquity of knee osteoarthritis (OA) is on the rise. Nonetheless, the understanding and ability to investigate potential precursors of knee OA have been impeded by time-consuming and laborious manual delineation processes which are prone to poor reproducibility. A method for automatic segmentation of the tibiofemoral joint using magnetic resonance imaging (MRI) is presented in this work. The proposed method utilizes a deeply supervised 2D-3D ensemble U-Net, which consists of foreground class oversampling, deep supervision loss branches, and Gaussian weighted softmax score aggregation. It was designed, optimized, and tested on 507 3D double echo steady-state (DESS) MR volumes using a two-fold cross-validation approach. A state-of-the-art segmentation accuracy measured as Dice similarity coefficient (DSC) for the femur bone (98.6 ± 0.27%), tibia bone (98.8 ± 0.31%), femoral cartilage (90.3 ± 2.89%), and tibial cartilage (86.7 ± 4.07%) is achieved. Notably, the proposed method yields sub-voxel accuracy for an average symmetric surface distance (ASD) less than 0.36 mm. The model performance is not affected by the severity of radiographic osteoarthritis (rOA) grades or the presence of pathophysiological changes. The proposed method offers an accurate segmentation with high time efficiency (~62 s) per 3D volume, which is well suited for efficient processing and analysis of the large prospective cohorts of the Osteoarthritis Initiative (OAI).

Emergence of Deep Learning in Knee Osteoarthritis Diagnosis

Osteoarthritis (OA), especially knee OA, is the most common form of arthritis, causing significant disability in patients worldwide. Manual diagnosis, segmentation, and annotations of knee joints remain as the popular method to diagnose OA in clinical practices, although they are tedious and greatly subject to user variation. Therefore, to overcome the limitations of the commonly used method as above, numerous deep learning approaches, especially the convolutional neural network (CNN), have been developed to improve the clinical workflow efficiency. Medical imaging processes, especially those that produce 3-dimensional (3D) images such as MRI, possess ability to reveal hidden structures in a volumetric view. Acknowledging that changes in a knee joint is a 3D complexity, 3D CNN has been employed to analyse the joint problem for a more accurate diagnosis in the recent years. In this review, we provide a broad overview on the current 2D and 3D CNN approaches in the OA research field. We reviewed 74 studies related to classification and segmentation of knee osteoarthritis from the Web of Science database and discussed the various state-of-the-art deep learning approaches proposed. We highlighted the potential and possibility of 3D CNN in the knee osteoarthritis field. We concluded by discussing the possible challenges faced as well as the potential advancements in adopting 3D CNNs in this field.

Towards novel osteoarthritis biomarkers: Multi-criteria evaluation of 46,996 segmented knee MRI data from the Osteoarthritis Initiative

Convolutional neural networks (CNNs) are the state-of-the-art for automated assessment of knee osteoarthritis (KOA) from medical image data. However, these methods lack interpretability, mainly focus on image texture, and cannot completely grasp the analyzed anatomies' shapes. In this study we assess the informative value of quantitative features derived from segmentations in order to assess their potential as an alternative or extension to CNN-based approaches regarding multiple aspects of KOA. Six anatomical structures around the knee (femoral and tibial bones, femoral and tibial cartilages, and both menisci) are segmented in 46,996 MRI scans. Based on these segmentations, quantitative features are computed, i.e., measurements such as cartilage volume, meniscal extrusion and tibial coverage, as well as geometric features based on a statistical shape encoding of the anatomies. The feature quality is assessed by investigating their association to the Kellgren-Lawrence grade (KLG), joint space narrowing (JSN), incident KOA, and total knee replacement (TKR). Using gold standard labels from the Osteoarthritis Initiative database the balanced accuracy (BA), the area under the Receiver Operating Characteristic curve (AUC), and weighted kappa statistics are evaluated. Features based on shape encodings of femur, tibia, and menisci plus the performed measurements showed most potential as KOA biomarkers. Differentiation between non-arthritic and severely arthritic knees yielded BAs of up to 99%, 84% were achieved for diagnosis of early KOA. Weighted kappa values of 0.73, 0.72, and 0.78 were achieved for classification of the grade of medial JSN, lateral JSN, and KLG, respectively. The AUC was 0.61 and 0.76 for prediction of incident KOA and TKR within one year, respectively. Quantitative features from automated segmentations provide novel biomarkers for KLG and JSN classification and show potential for incident KOA and TKR prediction. The validity of these features should be further evaluated, especially as extensions of CNN-based approaches. To foster such developments we make all segmentations publicly available together with this publication.

Are changes in meniscus volume and extrusion associated to knee osteoarthritis development? A structural equation model

OBJECTIVE

To explore the interplay between (changes in) medial meniscus volume, meniscus extrusion and radiographic knee osteoarthritis (OA) development over 30 months follow-up (FU).

METHODS

Data from the PRevention of knee Osteoarthritis in Overweight Females study were used. This cohort included 407 middle-aged women with a body mass index ≥27 kg/m², who were free of knee OA at baseline. Demographics were collected by questionnaires at baseline. All menisci at both baseline and FU were automatically segmented from MRI scans to obtain the meniscus volume and the change over time (delta volume). Baseline and FU meniscus body extrusion was quantitatively measured on mid-coronal proton density MR images. A structural equation model was created to assess the interplay between both medial meniscus volume and central extrusion at baseline, delta volume, delta extrusion, and incident radiographic knee OA at FU.

RESULTS

The structural equation modeling yielded a fair to good fit of the data. The direct effects of both medial meniscus volume and extrusion at baseline on incident OA were statistically significant (Estimate = 0.124, p = 0.029, and Estimate = 0.194, p < 0.001, respectively). Additional indirect effects on incident radiographic OA through delta meniscus volume or delta meniscus extrusion were not statistically significant.

CONCLUSION

Baseline medial meniscus volume and extrusion were associated to incidence of radiographic knee OA at FU in middle-aged overweight and obese women, while their changes were not involved in these effects. To prevent knee OA, interventions might need to target the onset of meniscal pathologies rather than their progression.

Automatic Meniscus Segmentation Using Adversarial Learning-Based Segmentation Network with Object-Aware Map in Knee MR Images

Meniscus segmentation from knee MR images is an essential step when analyzing the length, width, height, cross-sectional area, surface area for meniscus allograft transplantation using a 3D reconstruction model based on the patient's normal meniscus. In this paper, we propose a two-stage DCNN that combines a 2D U-Net-based meniscus localization network with a conditional generative adversarial network-based segmentation network using an object-aware map. First, the 2D U-Net segments knee MR images into six classes including bone and cartilage with whole MR images at a resolution of 512 × 512 to localize the medial and lateral meniscus. Second, adversarial learning with a generator based on the 2D U-Net and a discriminator based on the 2D DCNN using an object-aware map segments the meniscus into localized regions-of-interest with a resolution of 64 × 64. The average Dice similarity coefficient of the meniscus was 85.18% at the medial meniscus and 84.33% at the lateral meniscus; these values were 10.79%p and 1.14%p, and 7.78%p and 1.12%p higher than the segmentation method without adversarial learning and without the use of an object-aware map with the Dice similarity coefficient at the medial meniscus and lateral meniscus, respectively. The proposed automatic meniscus localization through multi-class can prevent the class imbalance problem by focusing on local regions. The proposed adversarial learning using an object-aware map can prevent under-segmentation by repeatedly judging and improving the segmentation results, and over-segmentation by considering information only from the meniscus regions. Our method can be used to identify and analyze the shape of the meniscus for allograft transplantation using a 3D reconstruction model of the patient's unruptured meniscus.

Subchondral bone length in knee osteoarthritis: A deep learning derived imaging measure and its association with radiographic and clinical outcomes

OBJECTIVE

Develop a bone shape measure that reflects the extent of cartilage loss and bone flattening in knee osteoarthritis (OA) and test it against estimates of disease severity.

METHODS

A fast region-based convolutional neural network was trained to crop the knee joints in sagittal dual-echo steady state MRI sequences obtained from the Osteoarthritis Initiative (OAI). Publicly available annotations of the cartilage and menisci were used as references to annotate the tibia and the femur in 61 knees. Another deep neural network (U-Net) was developed to learn these annotations. Model predictions were compared with radiologist-driven annotations on an independent test set (27 knees). The U-Net was applied to automatically extract the knee joint structures on the larger OAI dataset (9,434 knees). We defined subchondral bone length (SBL), a novel shape measure characterizing the extent of overlying cartilage and bone flattening, and examined its relationship with radiographic joint space narrowing (JSN), concurrent WOMAC pain and disability as well as subsequent partial or total knee replacement (KR). Odds ratios for each outcome were estimated using relative changes in SBL on the OAI dataset into quartiles.

RESULT

Mean SBL values for knees with JSN were consistently different from knees without JSN. Greater changes of SBL from baseline were associated with greater pain and disability. For knees with medial or lateral JSN, the odds ratios between lowest and highest quartiles corresponding to SBL changes for future KR were 5.68 (95% CI:[3.90,8.27]) and 7.19 (95% CI:[3.71,13.95]), respectively.

CONCLUSION

SBL quantified OA status based on JSN severity. It has promise as an imaging marker in predicting clinical and structural OA outcomes.

Biomechanical Study of a Tricompartmental Unloader Brace for Patellofemoral or Multicompartment Knee Osteoarthritis

Objective: Off-loader knee braces have traditionally focused on redistributing loads away from either the medial or lateral tibiofemoral (TF) compartments. In this article, we study the potential of a novel "tricompartment unloader" (TCU) knee brace intended to simultaneously unload both the patellofemoral (PF) and TF joints during knee flexion. Three different models of the TCU brace are evaluated for their potential to unload the knee joint. Methods: A sagittal plane model of the knee was used to compute PF and TF contact forces, patellar and quadriceps tendon forces, and forces in the anterior and posterior cruciate ligaments during a deep knee bend (DKB) test using motion analysis data from eight participants. Forces were computed for the observed (no brace) and simulated braced conditions. A sensitivity and validity analysis was conducted to determine the valid output range for the model, and Statistical Parameter Mapping was used to quantify the effectual region of the different TCU brace models. Results: PF and TF joint force calculations were valid between ~0 and 100 degrees of flexion. All three simulated brace models significantly (p < 0.001) reduced predicted knee joint loads (by 30-50%) across all structures, at knee flexion angles >~30 degrees during DKB. Conclusions: The TCU brace is predicted to reduce PF and TF knee joint contact loads during weight-bearing activity requiring knee flexion angles between 30 and 100 degrees; this effect may be clinically beneficial for pain reduction or rehabilitation from common knee injuries or joint disorders. Future work is needed to assess the range of possible clinical and prophylactic benefits of the TCU brace.

Deep learning for early detection of pathological changes in X-ray bone microstructures: case of osteoarthritis

Texture features are designed to quantitatively evaluate patterns of spatial distribution of image pixels for purposes of image analysis and interpretation. Unexplained variations in the texture patterns often lead to misinterpretation and undesirable consequences in medical image analysis. In this paper we explore the ability of machine learning (ML) methods to design a radiology test of Osteoarthritis (OA) at early stage when the number of patients’ cases is small. In our experiments we use high-resolution X-ray images of knees in patients which were identified with Kellgren–Lawrence scores progressing from 1. The existing ML methods have provided a limited diagnostic accuracy, whilst the proposed Group Method of Data Handling strategy of Deep Learning has significantly extended the diagnostic test. The comparative experiments demonstrate that the proposed framework using the Zernike-based texture features has significantly improved the diagnostic accuracy on average by 11%. This allows us to conclude that the designed model for early diagnostic of OA will provide more accurate radiology tests, although new study is required when a large number of patients’ cases will be available.

Relationship between medial meniscus extrusion and cartilage measurements in the knee by fully automatic three-dimensional MRI analysis

BACKGROUND

We developed a fully automatic three-dimensional knee MRI analysis software that can quantify meniscus extrusion and cartilage measurements, including the projected cartilage area ratio (PCAR), which represents the ratio of the subject's actual cartilage area to their ideal cartilage area. We also collected 3D MRI knee data from 561 volunteers (aged 30-79 years) from the "Kanagawa Knee Study." Our purposes were to verify the accuracy of the software for automatic cartilage and meniscus segmentation using knee MRI and to examine the relationship between medial meniscus extrusion measurements and cartilage measurements from Kanagawa Knee Study data.

METHODS

We constructed a neural network for the software by randomly choosing 10 healthy volunteers and 103 patients with knee pain. We validated the algorithm by randomly selecting 108 of these 113 subjects for training, and determined Dice similarity coefficients from five other subjects. We constructed a neural network using all data (113 subjects) for training. Cartilage thickness, cartilage volume, and PCAR in the medial femoral, lateral femoral, medial tibial, and lateral tibial regions were quantified by using the trained software on Kanagawa Knee Study data and their relationship with subject height was investigated. We also quantified the medial meniscus coverage ratio (MMCR), defined as the ratio of the overlapping area between the medial meniscus area and the medial tibial cartilage area to the medial tibial cartilage area. Finally, we examined the relationship between MMCR and PCAR at middle central medial tibial (mcMT) subregion located in the center of nine subregions in the medial tibial cartilage.

RESULTS

Dice similarity coefficients for cartilage and meniscus were both approximately 0.9. The femoral and tibial cartilage thickness and volume at each region correlated with height, but PCAR did not correlate with height in most settings. PCAR at the mcMT was significantly correlated with MMCR.

CONCLUSIONS

Our software showed high segmentation accuracy for the knee cartilage and meniscus. PCAR was more useful than cartilage thickness or volume since it was less affected by height. Relations ips were observed between the medial tibial cartilage measurements and the medial meniscus extrusion measurements in our cross-sectional study.

TRIAL REGISTRATION

UMIN, UMIN000032826 ; 1 September 2018.

Medial meniscus posterior root repair restores the intra-articular volume of the medial meniscus by decreasing posteromedial extrusion at knee flexion

PURPOSE

Transtibial repair of a medial meniscus posterior root tear (MMPRT) can improve clinical outcomes, although meniscal extrusion remains. However, few studies have investigated the volume of meniscal extrusion. This study aimed to evaluate the effect of transtibial repair in reducing the volume using three-dimensional (3D) magnetic resonance imaging, at 10° and 90° knee flexion.

METHODS

Twenty patients with MMPRTs and 16 volunteers with normal knees participated. The 3D models of meniscus were constructed using SYNAPSE VINCENT^®. The meniscal extrusion and its volume were measured at 10° and 90° knee flexion. Differences between the pre- and postoperative examinations were assessed using the Wilcoxon signed-rank test. The postoperative parameters were compared to those in patients with normal knees.

RESULTS

There were no significant pre- and postoperative differences in any parameter at 10° knee flexion. At 90° knee flexion, the posterior extrusion and its meniscal volume were decreased significantly after transtibial repair (p < 0.05), even though these parameters were larger than in the normal knees. On the other hand, intra-articular meniscal volume calculated by the extrusion volume was increased to the level of the normal knee.

CONCLUSIONS

This study demonstrated that transtibial repairs improved the intra-articular/intra-tibial surface volume of the medial meniscus by reducing the posteromedial extrusion during knee flexion. This 3D analysis is clinically relevant in evaluating that, while transtibial root repair has a limited ability to reduce meniscal extrusion, it can restore the functional volume of the medial meniscus which contributes to the shock absorber postoperatively.

LEVEL OF EVIDENCE

IV.

Retrospective in silico evaluation of optimized preoperative planning for temporal bone surgery

Purpose

Robot-assisted surgery at the temporal bone utilizing a flexible drilling unit would allow safer access to clinical targets such as the cochlea or the internal auditory canal by navigating along nonlinear trajectories. One key sub-step for clinical realization of such a procedure is automated preoperative surgical planning that incorporates both segmentation of risk structures and optimized trajectory planning.

Methods

We automatically segment risk structures using 3D U-Nets with probabilistic active shape models. For nonlinear trajectory planning, we adapt bidirectional rapidly exploring random trees on Bézier Splines followed by sequential convex optimization. Functional evaluation, assessing segmentation quality based on the subsequent trajectory planning step, shows the suitability of our novel segmentation approach for this two-step preoperative pipeline.

Results

Based on 24 data sets of the temporal bone, we perform a functional evaluation of preoperative surgical planning. Our experiments show that the automated segmentation provides safe and coherent surface models that can be used in collision detection during motion planning. The source code of the algorithms will be made publicly available.

Conclusion

Optimized trajectory planning based on shape regularized segmentation leads to safe access canals for temporal bone surgery. Functional evaluation shows the promising results for both 3D U-Net and Bézier Spline trajectories.

Automatic Segmentation of Meniscus in Multispectral MRI Using Regions with Convolutional Neural Network (R-CNN)

The meniscus has a significant function in human anatomy, and Magnetic Resonance Imaging (MRI) has an essential role in meniscus examination. Due to a variety of MRI data, it is excessively difficult to segment the meniscus with image processing methods. An MRI data sequence contains multiple images, and the region features we are looking for may vary from each image in the sequence. Therefore, feature extraction becomes more difficult, and hence, explicitly programming for segmentation becomes more difficult. Convolutional Neural Network (CNN) extracts features directly from images and thus eliminates the need for manual feature extraction. Regions with Convolutional Neural Network (R-CNN) allow us to use CNN features in object detection problems by combining CNN features with Region Proposals. In this study, we designed and trained an R-CNN for detecting meniscus region in MRI data sequence. We used transfer learning for training R-CNN with a small amount of meniscus data. After detection of the meniscus region by R-CNN, we segmented meniscus by morphological image analysis using two different MRI sequences. Automatic detection of the meniscus region with R-CNN made the meniscus segmentation process easier, and the use of different contrast features of two different image sequences allowed us to differentiate the meniscus from its surroundings.

Layer-specific analysis of femorotibial cartilage t2 relaxation time based on registration of segmented double echo steady state (dess) to multi-echo-spin-echo (mese) images

OBJECTIVE

To develop and validate a 3D registration approach by which double echo steady state (DESS) MR images with cartilage thickness segmentations are used to extract the cartilage transverse relaxation time (T2) from multi-echo-spin-echo (MESE) MR images, without direct segmentations for MESE.

MATERIALS AND METHODS

Manual DESS segmentations of 89 healthy reference knees (healthy) and 60 knees with early radiographic osteoarthritis (early ROA) from the Osteoarthritis Initiative were registered to corresponding MESE images that had independent direct T2 segmentations. For validation purposes, (a) regression analysis of deep and superficial cartilage T2 was performed and (b) between-group differences between healthy vs. early ROA knees were compared for registered vs. direct MESE analysis.

RESULTS

Moderate to high correlations were observed for the deep (r = 0.80) and the superficial T2 (r = 0.81), with statistically significant between-group differences (ROA vs. healthy) of + 1.4 ms (p = 0.002) vs. + 1.3 ms (p < 0.001) for registered vs. direct T2 segmentation in the deep, and + 1.3 ms (p = 0.002) vs. + 2.3 ms (p < 0.001) in the superficial layer.

DISCUSSION

This registration approach enables extracting cartilage T2 from MESE scans using DESS (cartilage thickness) segmentations, avoiding the need for direct MESE T2 segmentations.

Osteoarthritis year in review 2019: imaging

OBJECTIVE

To provide a narrative review of original articles on osteoarthritis (OA) imaging published between April 1, 2018 and March 30, 2019.

METHODS

All original research articles on OA imaging published in English between April 1, 2018 and March 30, 2019 were identified using a PubMed database search. The search terms of "Osteoarthritis" or "OA" were combined with the search terms "Radiography", "X-Rays", "Magnetic Resonance Imaging", "MRI", "Ultrasound", "US", "Computed Tomography", "Dual Energy X-Ray Absorptiometry", "DXA", "DEXA", "CT", "Nuclear Medicine", "Scintigraphy", "Single-Photon Emission Computed Tomography", "SPECT", "Positron Emission Tomography", "PET", "PET-CT", or "PET-MRI". Articles were reviewed to determine relevance based upon the following criteria: 1) study involved human subjects with OA or risk factors for OA and 2) study involved imaging to evaluate OA disease status or OA treatment response. Relevant articles were ranked according to scientific merit, with the best publications selected for inclusion in the narrative report.

RESULTS

The PubMed search revealed a total of 1257 articles, of which 256 (20.4%) were considered relevant to OA imaging. Two-hundred twenty-six (87.1%) articles involved the knee joint, while 195 (76.2%) articles involved the use of magnetic resonance imaging (MRI). The proportion of published studies involving the use of MRI was higher than previous years. An increasing number of articles were also published on imaging of subjects with joint injury and on deep learning application in OA imaging.

CONCLUSION

MRI and other imaging modalities continue to play an important role in research studies designed to better understand the pathogenesis, progression, and treatment of OA.

Knee menisci segmentation and relaxometry of 3D ultrashort echo time cones MR imaging using attention U-Net with transfer learning

PURPOSE

To develop a deep learning-based method for knee menisci segmentation in 3D ultrashort echo time (UTE) cones MR imaging, and to automatically determine MR relaxation times, namely the T1, T1ρ , and T 2 ∗ parameters, which can be used to assess knee osteoarthritis (OA).

METHODS

Whole knee joint imaging was performed using 3D UTE cones sequences to collect data from 61 human subjects. Regions of interest (ROIs) were outlined by 2 experienced radiologists based on subtracted T1ρ -weighted MR images. Transfer learning was applied to develop 2D attention U-Net convolutional neural networks for the menisci segmentation based on each radiologist's ROIs separately. Dice scores were calculated to assess segmentation performance. Next, the T1, T1ρ , T 2 ∗ relaxations, and ROI areas were determined for the manual and automatic segmentations, then compared.

RESULTS

The models developed using ROIs provided by 2 radiologists achieved high Dice scores of 0.860 and 0.833, while the radiologists' manual segmentations achieved a Dice score of 0.820. Linear correlation coefficients for the T1, T1ρ , and T 2 ∗ relaxations calculated using the automatic and manual segmentations ranged between 0.90 and 0.97, and there were no associated differences between the estimated average meniscal relaxation parameters. The deep learning models achieved segmentation performance equivalent to the inter-observer variability of 2 radiologists.

CONCLUSION

The proposed deep learning-based approach can be used to efficiently generate automatic segmentations and determine meniscal relaxations times. The method has the potential to help radiologists with the assessment of meniscal diseases, such as OA.

Assessment of knee pain from MR imaging using a convolutional Siamese network

OBJECTIVES

It remains difficult to characterize the source of pain in knee joints either using radiographs or magnetic resonance imaging (MRI). We sought to determine if advanced machine learning methods such as deep neural networks could distinguish knees with pain from those without it and identify the structural features that are associated with knee pain.

METHODS

We constructed a convolutional Siamese network to associate MRI scans obtained on subjects from the Osteoarthritis Initiative (OAI) with frequent unilateral knee pain comparing the knee with frequent pain to the contralateral knee without pain. The Siamese network architecture enabled pairwise learning of information from two-dimensional (2D) sagittal intermediate-weighted turbo spin echo slices obtained from similar locations on both knees. Class activation mapping (CAM) was utilized to create saliency maps, which highlighted the regions most associated with knee pain. The MRI scans and the CAMs of each subject were reviewed by an expert radiologist to identify the presence of abnormalities within the model-predicted regions of high association.

RESULTS

Using 10-fold cross-validation, our model achieved an area under curve (AUC) value of 0.808. When individuals whose knee WOMAC pain scores were not discordant were excluded, model performance increased to 0.853. The radiologist review revealed that about 86% of the cases that were predicted correctly had effusion-synovitis within the regions that were most associated with pain.

CONCLUSIONS

This study demonstrates a proof of principle that deep learning can be applied to assess knee pain from MRI scans.

KEY POINTS

• Our article is the first to leverage a deep learning framework to associate MR images of the knee with knee pain. • We developed a convolutional Siamese network that had the ability to fuse information from multiple two-dimensional (2D) MRI slices from the knee with pain and the contralateral knee of the same individual without pain to predict unilateral knee pain. • Our model achieved an area under curve (AUC) value of 0.808. When individuals who had WOMAC pain scores that were not discordant for knees (pain discordance < 3) were excluded, model performance increased to 0.853.

Current applications and future directions of deep learning in musculoskeletal radiology

Deep learning with convolutional neural networks (CNN) is a rapidly advancing subset of artificial intelligence that is ideally suited to solving image-based problems. There are an increasing number of musculoskeletal applications of deep learning, which can be conceptually divided into the categories of lesion detection, classification, segmentation, and non-interpretive tasks. Numerous examples of deep learning achieving expert-level performance in specific tasks in all four categories have been demonstrated in the past few years, although comprehensive interpretation of imaging examinations has not yet been achieved. It is important for the practicing musculoskeletal radiologist to understand the current scope of deep learning as it relates to musculoskeletal radiology. Interest in deep learning from researchers, radiology leadership, and industry continues to increase, and it is likely that these developments will impact the daily practice of musculoskeletal radiology in the near future.