Automated segmentation of knee bone and cartilage combining statistical shape knowledge and convolutional neural networks: Data from the Osteoarthritis Initiative

Robust automatic hexahedral cartilage meshing framework enables population-based computational studies of the knee

Osteoarthritis of the knee is increasingly prevalent as our population ages, representing an increasing financial burden, and severely impacting quality of life. The invasiveness of in vivo procedures and the high cost of cadaveric studies has left computational tools uniquely suited to study knee biomechanics. Developments in deep learning have great potential for efficiently generating large-scale datasets to enable researchers to perform population-sized investigations, but the time and effort associated with producing robust hexahedral meshes has been a limiting factor in expanding finite element studies to encompass a population. Here we developed a fully automated pipeline capable of taking magnetic resonance knee images and producing a working finite element simulation. We trained an encoder-decoder convolutional neural network to perform semantic image segmentation on the Imorphics dataset provided through the Osteoarthritis Initiative. The Imorphics dataset contained 176 image sequences with varying levels of cartilage degradation. Starting from an open-source swept-extrusion meshing algorithm, we further developed this algorithm until it could produce high quality meshes for every sequence and we applied a template-mapping procedure to automatically place soft-tissue attachment points. The meshing algorithm produced simulation-ready meshes for all 176 sequences, regardless of the use of provided (manually reconstructed) or predicted (automatically generated) segmentation labels. The average time to mesh all bones and cartilage tissues was less than 2 min per knee on an AMD Ryzen 5600X processor, using a parallel pool of three workers for bone meshing, followed by a pool of four workers meshing the four cartilage tissues. Of the 176 sequences with provided segmentation labels, 86% of the resulting meshes completed a simulated flexion-extension activity. We used a reserved testing dataset of 28 sequences unseen during network training to produce simulations derived from predicted labels. We compared tibiofemoral contact mechanics between manual and automated reconstructions for the 24 pairs of successful finite element simulations from this set, resulting in mean root-mean-squared differences under 20% of their respective min-max norms. In combination with further advancements in deep learning, this framework represents a feasible pipeline to produce population sized finite element studies of the natural knee from subject-specific models.

A novel multi-atlas segmentation approach under the semi-supervised learning framework: Application to knee cartilage segmentation

BACKGROUND AND OBJECTIVE

Multi-atlas based segmentation techniques, which rely on an atlas library comprised of training images labeled by an expert, have proven their effectiveness in multiple automatic segmentation applications. However, the usage of exhaustive patch libraries combined with the voxel-wise labeling incur a large computational cost in terms of memory requirements and execution times.

METHODS

To confront this shortcoming, we propose a novel two-stage multi-atlas approach designed under the Semi-Supervised Learning (SSL) framework. The main properties of our method are as follows: First, instead of the voxel-wise labeling approach, the labeling of target voxels is accomplished here by exploiting the spectral content of globally sampled datasets from the target image, along with their spatially correspondent data collected from the atlases. Following SSL, voxels classification is boosted by incorporating unlabeled data from the target image, in addition to the labeled ones from atlas library. Our scheme integrates constructively fruitful concepts, including sparse reconstructions of voxels from linear neighborhoods, HOG feature descriptors of patches/regions, and label propagation via sparse graph constructions. Segmentation of the target image is carried out in two stages: stage-1 focuses on the sampling and labeling of global data, while stage-2 undertakes the above tasks for the out-of-sample data. Finally, we propose different graph-based methods for the labeling of global data, while these methods are extended to deal with the out-of-sample voxels.

RESULTS

A thorough experimental investigation is conducted on 76 subjects provided by the publicly accessible Osteoarthritis Initiative (OAI) repository. Comparative results and statistical analysis demonstrate that the suggested methodology exhibits superior segmentation performance compared to the existing patch-based methods, across all evaluation metrics (DSC:88.89%, Precision: 89.86%, Recall: 88.12%), while at the same time it requires a considerably reduced computational load (>70% reduction on average execution time with respect to other patch-based). In addition, our approach is favorably compared against other non patch-based and deep learning methods in terms of performance accuracy (on the 3-class problem). A final experimentation on a 5-class setting of the problems demonstrates that our approach is capable of achieving performance comparable to existing state-of-the-art knee cartilage segmentation methods (DSC:88.22% and DSC:85.84% for femoral and tibial cartilage respectively).

Cartilage Topography Assessment With Local-Area Cartilage Segmentation for Knee Magnetic Resonance Imaging

OBJECTIVE

Local-area cartilage segmentation (LACS) software was developed to segment medial femur (MF) cartilage on magnetic resonance imaging (MRI). Our objectives were 1) to extend LACS to the lateral femur (LF), medial tibia (MT), and lateral tibia (LT), 2) to compare LACS to an established manual segmentation method, and 3) to visualize cartilage responsiveness over each cartilage plate.

METHODS

Osteoarthritis Initiative participants with symptomatic knee osteoarthritis (OA) were selected, including knees selected at random (n = 40) and knees identified with loss of cartilage based on manual segmentation (Chondrometrics GmbH), an enriched sample of 126 knees. LACS was used to segment cartilage in the MF, LF, MT, and LT on sagittal 3D double-echo steady-state MRI scans at baseline and at 2-year follow-up. We compared LACS and Chondrometrics average thickness measures by estimating the correlation in each cartilage plate and estimating the standardized response mean (SRM) for 2-year cartilage change. We illustrated cartilage loss topographically with SRM heatmaps.

RESULTS

The estimated correlation between LACS and Chondrometrics measures was r = 0.91 (95% confidence interval [95% CI] 0.86, 0.94) for LF, r = 0.93 (95% CI 0.89, 0.95) for MF, r = 0.97 (95% CI 0.96, 0.98) for LT, and r = 0.87 (95% CI 0.81, 0.91) for MT. Estimated SRMs for LACS and Chondrometrics measures were similar in the random sample, and SRM heatmaps identified subregions of LACS-measured cartilage loss.

CONCLUSION

LACS cartilage thickness measurement in the MF and LF and tibia correlated well with established manual segmentation-based measurement, with similar responsiveness to change, among knees with symptomatic knee OA. LACS measurement of cartilage plate topography enables spatiotemporal analysis of cartilage loss in future knee OA studies.

Deep convolutional feature details for better knee disorder diagnoses in magnetic resonance images

Convolutional neural networks (CNNs) applied to magnetic resonance imaging (MRI) have demonstrated their ability in the automatic diagnosis of knee injuries. Despite the promising results, the currently available solutions do not take into account the particular anatomy of knee disorders. Existing works have shown that injuries are localized in small-sized knee regions near the center of MRI scans. Based on such insights, we propose MRPyrNet, a CNN architecture capable of extracting more relevant features from these regions. Our solution is composed of a Feature Pyramid Network with Pyramidal Detail Pooling, and can be plugged into any existing CNN-based diagnostic pipeline. The first module aims to enhance the CNN intermediate features to better detect the small-sized appearance of disorders, while the second one captures such kind of evidence by maintaining its detailed information. An extensive evaluation campaign is conducted to understand in-depth the potential of the proposed solution. The experimental results achieved demonstrate that the application of MRPyrNet to baseline methodologies improves their diagnostic capability, especially in the case of anterior cruciate ligament tear and meniscal tear because of MRPyrNet's ability in exploiting the relevant appearance features of such disorders. Code is available at https://github.com/matteo-dunnhofer/MRPyrNet.

KCB-Net: A 3D knee cartilage and bone segmentation network via sparse annotation

Knee cartilage and bone segmentation is critical for physicians to analyze and diagnose articular damage and knee osteoarthritis (OA). Deep learning (DL) methods for medical image segmentation have largely outperformed traditional methods, but they often need large amounts of annotated data for model training, which is very costly and time-consuming for medical experts, especially on 3D images. In this paper, we report a new knee cartilage and bone segmentation framework, KCB-Net, for 3D MR images based on sparse annotation. KCB-Net selects a small subset of slices from 3D images for annotation, and seeks to bridge the performance gap between sparse annotation and full annotation. Specifically, it first identifies a subset of the most effective and representative slices with an unsupervised scheme; it then trains an ensemble model using the annotated slices; next, it self-trains the model using 3D images containing pseudo-labels generated by the ensemble method and improved by a bi-directional hierarchical earth mover's distance (bi-HEMD) algorithm; finally, it fine-tunes the segmentation results using the primal-dual Internal Point Method (IPM). Experiments on four 3D MR knee joint datasets (the SKI10 dataset, OAI ZIB dataset, Iowa dataset, and iMorphics dataset) show that our new framework outperforms state-of-the-art methods on full annotation, and yields high quality results for small annotation ratios even as low as 10%.

On the Sufficient Condition for Solving the Gap-Filling Problem Using Deep Convolutional Neural Networks

Deep convolutional neural networks (DCNNs) are routinely used for image segmentation of biomedical data sets to obtain quantitative measurements of cellular structures like tissues. These cellular structures often contain gaps in their boundaries, leading to poor segmentation performance when using DCNNs like the U-Net. The gaps can usually be corrected by post-hoc computer vision (CV) steps, which are specific to the data set and require a disproportionate amount of work. As DCNNs are Universal Function Approximators, it is conceivable that the corrections should be obsolete by selecting the appropriate architecture for the DCNN. In this article, we present a novel theoretical framework for the gap-filling problem in DCNNs that allows the selection of architecture to circumvent the CV steps. Combining information-theoretic measures of the data set with a fundamental property of DCNNs, the size of their receptive field, allows us to formulate statements about the solvability of the gap-filling problem independent of the specifics of model training. In particular, we obtain mathematical proof showing that the maximum proficiency of filling a gap by a DCNN is achieved if its receptive field is larger than the gap length. We then demonstrate the consequence of this result using numerical experiments on a synthetic and real data set and compare the gap-filling ability of the ubiquitous U-Net architecture with variable depths. Our code is available at https://github.com/ai-biology/dcnn-gap-filling.

Dynamic pressure analysis of novel interpositional knee spacer implants in 3D-printed human knee models

Alternative treatment methods for knee osteoarthritis (OA) are in demand, to delay the young (< 50 Years) patient's need for osteotomy or knee replacement. Novel interpositional knee spacers shape based on statistical shape model (SSM) approach and made of polyurethane (PU) were developed to present a minimally invasive method to treat medial OA in the knee. The implant should be supposed to reduce peak strains and pain, restore the stability of the knee, correct the malalignment of a varus knee and improve joint function and gait. Firstly, the spacers were tested in artificial knee models. It is assumed that by application of a spacer, a significant reduction in stress values and a significant increase in the contact area in the medial compartment of the knee will be registered. Biomechanical analysis of the effect of novel interpositional knee spacer implants on pressure distribution in 3D-printed knee model replicas: the primary purpose was the medial joint contact stress-related biomechanics. A secondary purpose was a better understanding of medial/lateral redistribution of joint loading. Six 3D printed knee models were reproduced from cadaveric leg computed tomography. Each of four spacer implants was tested in each knee geometry under realistic arthrokinematic dynamic loading conditions, to examine the pressure distribution in the knee joint. All spacers showed reduced mean stress values by 84-88% and peak stress values by 524-704% in the medial knee joint compartment compared to the non-spacer test condition. The contact area was enlarged by 462-627% as a result of the inserted spacers. Concerning the appreciable contact stress reduction and enlargement of the contact area in the medial knee joint compartment, the premises are in place for testing the implants directly on human knee cadavers to gain further insights into a possible tool for treating medial knee osteoarthritis.

Anatomically Accurate, High-Resolution Modeling of the Human Index Finger Using In Vivo Magnetic Resonance Imaging

Anatomically accurate models of a human finger can be useful in simulating various disorders. In order to have potential clinical value, such models need to include a large number of tissue types, identified by an experienced professional, and should be versatile enough to be readily tailored to specific pathologies. Magnetic resonance images were acquired at ultrahigh magnetic field (7 T) with a radio-frequency coil specially designed for finger imaging. Segmentation was carried out under the supervision of an experienced radiologist to accurately capture various tissue types (TTs). The final segmented model of the human index finger had a spatial resolution of 0.2 mm and included 6,809,600 voxels. In total, 15 TTs were identified: subcutis, Pacinian corpuscle, nerve, vein, artery, tendon, collateral ligament, volar plate, pulley A4, bone, cartilage, synovial cavity, joint capsule, epidermis and dermis. The model was applied to the conditions of arthritic joint, ruptured tendon and variations in the geometry of a finger. High-resolution magnetic resonance images along with careful segmentation proved useful in the construction of an anatomically accurate model of the human index finger. An example illustrating the utility of the model in biomedical applications is shown. As the model includes a number of tissue types, it may present a solid foundation for future simulations of various musculoskeletal disease processes in human joints.

Automatic Bone Segmentation from MRI for Real-Time Knee Tracking in Fluoroscopic Imaging

Recent progress in real-time tracking of knee bone structures from fluoroscopic imaging using CT templates has opened the door to studying knee kinematics to improve our understanding of patellofemoral syndrome. The problem with CT imaging is that it exposes patients to extra ionising radiation, which adds to fluoroscopic imaging. This can be solved by segmenting bone templates from MRI instead of CT by using a deep neural network architecture called 2.5D U-Net. To train the network, we used the SKI10 database from the MICCAI challenge; it contains 100 knee MRIs with their corresponding annotated femur and tibia bones as the ground truth. Since patella tracking is essential in our application, the SKI10 database was augmented with a new label named UofA Patella. Using 70 MRIs from the database, a 2.5D U-Net was trained successfully after 75 epochs with an excellent final Dice score of 98%, which compared favourably with the best state-of-the-art algorithms. A test set of 30 MRIs were segmented using the trained 2.5D U-Net and then converted into 3D mesh templates by using a marching cube algorithm. The resulting 3D mesh templates were compared to the 3D mesh model extracted from the corresponding labelled data from the augmented SKI10. Even though the final Dice score (98%) compared well with the state-of-the-art algorithms, we initially found that the Euclidean distance between the segmented MRI and SKI10 meshes was over 6 mm in many regions, which is unacceptable for our application. By optimising many of the hyper-parameters of the 2.5D U-Net, we were able to find that, by changing the threshold used in the last layer of the network, one can significantly improve the average accuracy to 0.2 mm with a variance of 0.065 mm for most of the MRI mesh templates. These results illustrate that the Dice score is not always a good predictor of the geometric accuracy of segmentation and that fine-tuning hyper-parameters is critical for improving geometric accuracy.

CAN3D: Fast 3D medical image segmentation via compact context aggregation

Direct automatic segmentation of objects in 3D medical imaging, such as magnetic resonance (MR) imaging, is challenging as it often involves accurately identifying multiple individual structures with complex geometries within a large volume under investigation. Most deep learning approaches address these challenges by enhancing their learning capability through a substantial increase in trainable parameters within their models. An increased model complexity will incur high computational costs and large memory requirements unsuitable for real-time implementation on standard clinical workstations, as clinical imaging systems typically have low-end computer hardware with limited memory and CPU resources only. This paper presents a compact convolutional neural network (CAN3D) designed specifically for clinical workstations and allows the segmentation of large 3D Magnetic Resonance (MR) images in real-time. The proposed CAN3D has a shallow memory footprint to reduce the number of model parameters and computer memory required for state-of-the-art performance and maintain data integrity by directly processing large full-size 3D image input volumes with no patches required. The proposed architecture significantly reduces computational costs, especially for inference using the CPU. We also develop a novel loss function with extra shape constraints to improve segmentation accuracy for imbalanced classes in 3D MR images. Compared to state-of-the-art approaches (U-Net3D, improved U-Net3D and V-Net), CAN3D reduced the number of parameters up to two orders of magnitude and achieved much faster inference, up to 5 times when predicting with a standard commercial CPU (instead of GPU). For the open-access OAI-ZIB knee MR dataset, in comparison with manual segmentation, CAN3D achieved Dice coefficient values of (mean = 0.87 ± 0.02 and 0.85 ± 0.04) with mean surface distance errors (mean = 0.36 ± 0.32 mm and 0.29 ± 0.10 mm) for imbalanced classes such as (femoral and tibial) cartilage volumes respectively when training volume-wise under only 12G video memory. Similarly, CAN3D demonstrated high accuracy and efficiency on a pelvis 3D MR imaging dataset for prostate cancer consisting of 211 examinations with expert manual semantic labels (bladder, body, bone, rectum, prostate) now released publicly for scientific use as part of this work.

Deep collaborative network with alpha matte for precise knee tissue segmentation from MRI

BACKGROUND AND OBJECTIVE

Precise segmentation of knee tissues from magnetic resonance imaging (MRI) is critical in quantitative imaging and diagnosis. Convolutional neural networks (CNNs), being state of the art, often challenged by the lack of image-specific adaptation, such as low tissue contrasts and structural inhomogeneities, thereby leading to incomplete segmentation results.

METHODS

This paper presents a deep learning-based automatic segmentation framework for precise knee tissue segmentation. A novel deep collaborative method is proposed, which consists of an encoder-decoder-based segmentation network in combination with a low rank tensor-reconstructed segmentation network. Low rank reconstruction in MRI tensor sub-blocks is introduced to exploit the morphological variations in knee tissues. To model the tissue boundary regions and effectively utilize the superimposed regions, trimap generation is proposed for defining high, medium and low confidence regions from the multipath CNNs. The secondary path with low rank reconstructed input mitigates the conditions in which the primary segmentation network can potentially fail and overlook the boundary regions. The outcome of the segmentation is solved as an alpha matting problem by blending the trimap with the source input.

RESULTS

Experiments on Osteoarthritis Initiative (OAI) datasets with all the 6 musculoskeletal tissues provide an overall segmentation dice score of 0.8925, where Femoral and Tibial part of cartilage achieving an average dice exceeding 0.9. The volumetric metrics also indicate the superior performances in all tissue compartments.

CONCLUSIONS

Experiments on Osteoarthritis Initiative (OAI) datasets and a self-prepared scan validate the effectiveness of the proposed method. Inclusion of extra prediction scale allowed the model to distinguish and segment the tissue boundary accurately. We specifically demonstrate the application of the proposed method in a cartilage segmentation-based thickness map for diagnosis purposes.

Imaging in Osteoarthritis

Osteoarthritis (OA) is the most frequent form of arthritis with major implications on both individual and public health care levels. The field of joint imaging, and particularly magnetic resonance imaging (MRI), has evolved rapidly due to the application of technical advances to the field of clinical research. This narrative review will provide an introduction to the different aspects of OA imaging aimed at an audience of scientists, clinicians, students, industry employees, and others who are interested in OA but who do not necessarily focus on OA. The current role of radiography and recent advances in measuring joint space width will be discussed. The status of cartilage morphology assessment and evaluation of cartilage biochemical composition will be presented. Advances in quantitative three-dimensional morphologic cartilage assessment and semi-quantitative whole-organ assessment of OA will be reviewed. Although MRI has evolved as the most important imaging method used in OA research, other modalities such as ultrasound, computed tomography, and metabolic imaging play a complementary role and will also be discussed.

Deep Learning to Detect Triangular Fibrocartilage Complex Injury in Wrist MRI: Retrospective Study with Internal and External Validation

Objective: To use deep learning to predict the probability of triangular fibrocartilage complex (TFCC) injury in patients’ MRI scans. Methods: We retrospectively studied medical records over 11 years and 2 months (1 January 2009–29 February 2019), collecting 332 contrast-enhanced hand MRI scans showing TFCC injury (143 scans) or not (189 scans) from a general hospital. We employed two convolutional neural networks with the MRNet (Algorithm 1) and ResNet50 (Algorithm 2) framework for deep learning. Explainable artificial intelligence was used for heatmap analysis. We tested deep learning using an external dataset containing the MRI scans of 12 patients with TFCC injuries and 38 healthy subjects. Results: In the internal dataset, Algorithm 1 had an AUC of 0.809 (95% confidence interval—CI: 0.670–0.947) for TFCC injury detection as well as an accuracy, sensitivity, and specificity of 75.6% (95% CI: 0.613–0.858), 66.7% (95% CI: 0.438–0.837), and 81.5% (95% CI: 0.633–0.918), respectively, and an F1 score of 0.686. Algorithm 2 had an AUC of 0.871 (95% CI: 0.747–0.995) for TFCC injury detection and an accuracy, sensitivity, and specificity of 90.7% (95% CI: 0.787–0.962), 88.2% (95% CI: 0.664–0.966), and 92.3% (95% CI: 0.763–0.978), respectively, and an F1 score of 0.882. The accuracy, sensitivity, and specificity for radiologist 1 were 88.9, 94.4 and 85.2%, respectively, and for radiologist 2, they were 71.1, 100 and 51.9%, respectively. Conclusions: A modified MRNet framework enables the detection of TFCC injury and guides accurate diagnosis.

Knee Bone and Cartilage Segmentation Based on a 3D Deep Neural Network Using Adversarial Loss for Prior Shape Constraint

Fast and accurate segmentation of knee bone and cartilage on MRI images is becoming increasingly important in the orthopaedic area, as the segmentation is an essential prerequisite step to a patient-specific diagnosis, optimising implant design and preoperative and intraoperative planning. However, manual segmentation is time-intensive and subjected to inter- and intra-observer variations. Hence, in this study, a three-dimensional (3D) deep neural network using adversarial loss was proposed to automatically segment the knee bone in a resampled image volume in order to enlarge the contextual information and incorporate prior shape constraints. A restoration network was proposed to further improve the bone segmentation accuracy by restoring the bone segmentation back to the original resolution. A conventional U-Net-like network was used to segment the cartilage. The ultimate results were the combination of the bone and cartilage outcomes through post-processing. The quality of the proposed method was thoroughly assessed using various measures for the dataset from the Grand Challenge Segmentation of Knee Images 2010 (SKI10), together with a comparison with a baseline network U-Net. A fine-tuned U-Net-like network can achieve state-of-the-art results without any post-processing operations. This method achieved a total score higher than 76 in terms of the SKI10 validation dataset. This method showed to be robust to extract bone and cartilage masks from the MRI dataset, even for the pathological case.

Automated knee cartilage segmentation for heterogeneous clinical MRI using generative adversarial networks with transfer learning

BACKGROUND

This study aimed to build a deep learning model to automatically segment heterogeneous clinical MRI scans by optimizing a pre-trained model built from a homogeneous research dataset with transfer learning.

METHODS

Conditional generative adversarial networks pretrained on the Osteoarthritis Initiative MR images was transferred to 30 sets of heterogenous MR images collected from clinical routines. Two trained radiologists manually segmented the 30 sets of clinical MR images for model training, validation and test. The model performance was compared to models trained from scratch with different datasets, as well as two radiologists. A 5-fold cross validation was performed.

RESULTS

The transfer learning model obtained an overall averaged Dice coefficient of 0.819, an averaged 95 percentile Hausdorff distance of 1.463 mm, and an averaged average symmetric surface distance of 0.350 mm on the 5 random holdout test sets. A 5-fold cross validation had a mean Dice coefficient of 0.801, mean 95 percentile Hausdorff distance of 1.746 mm, and mean average symmetric surface distance of 0.364 mm. It outperformed other models and performed similarly as the radiologists.

CONCLUSIONS

A transfer learning model was able to automatically segment knee cartilage, with performance comparable to human, using heterogeneous clinical MR images with a small training data size. In addition, the model proved robust when tested through cross validation and on images from a different vendor. We found it feasible to perform fully automated cartilage segmentation of clinical knee MR images, which would facilitate the clinical application of quantitative MRI techniques and other prediction models for improved patient treatment planning.

Deep learning-based segmentation of knee MRI for fully automatic subregional morphological assessment of cartilage tissues: Data from the Osteoarthritis Initiative

Morphological changes in knee cartilage subregions are valuable imaging-based biomarkers for understanding progression of osteoarthritis, and they are typically detected from magnetic resonance imaging (MRI). So far, accurate segmentation of cartilage has been done manually. Deep learning approaches show high promise in automating the task; however, they lack clinically relevant evaluation. We introduce a fully automatic method for segmentation and subregional assessment of articular cartilage, and evaluate its predictive power in context of radiographic osteoarthritis progression. Two data sets of 3D double-echo steady-state (DESS) MRI derived from the Osteoarthritis Initiative were used: first, n = 88; second, n = 600, 0-/12-/24-month visits. Our method performed deep learning-based segmentation of knee cartilage tissues, their subregional division via multi-atlas registration, and extraction of subregional volume and thickness. The segmentation model was developed and assessed on the first data set. Subsequently, on the second data set, the morphological measurements from our and the prior methods were analyzed in correlation and agreement, and, eventually, by their discriminative power of radiographic osteoarthritis progression over 12 and 24 months, retrospectively. The segmentation model showed very high correlation (r > 0.934) and agreement (mean difference < 116 mm³ ) in volumetric measurements with the reference segmentations. Comparison of our and manual segmentation methods yielded r = 0.845-0.973 and mean differences = 262-501 mm³ for weight-bearing cartilage volume, and r = 0.770-0.962 and mean differences = 0.513-1.138 mm for subregional cartilage thickness. With regard to osteoarthritis progression, our method found most of the significant associations identified using the manual segmentation method, for both 12- and 24-month subregional cartilage changes. The method may be effectively applied in osteoarthritis progression studies to extract cartilage-related imaging biomarkers.

Artificial intelligence reshapes current understanding and management of osteoarthritis: A narrative review

Current practice of osteoarthritis has its insufficiencies which researchers are tackling with artificial intelligence (AI). This article discusses three kinds of AI models, namely diagnostic models, prediction models and morphological models. Diagnostic models enhance efficiency in diagnosis by providing an automated algorithm in knee images processing. Prediction models utilize behavioral and radiological data to assess the risk of osteoarthritis before symptom onset and needs to perform surgery. Morphological models detect biomechanical changes to facilitate understanding of pathophysiology and provide personalized intervention. Through reviewing present evidence, we demonstrate that AI could assist doctors in diagnosis, predict osteoarthritis and guide future research.

Medical image analysis on left atrial LGE MRI for atrial fibrillation studies: A review

Late gadolinium enhancement magnetic resonance imaging (LGE MRI) is commonly used to visualize and quantify left atrial (LA) scars. The position and extent of LA scars provide important information on the pathophysiology and progression of atrial fibrillation (AF). Hence, LA LGE MRI computing and analysis are essential for computer-assisted diagnosis and treatment stratification of AF patients. Since manual delineations can be time-consuming and subject to intra- and inter-expert variability, automating this computing is highly desired, which nevertheless is still challenging and under-researched. This paper aims to provide a systematic review on computing methods for LA cavity, wall, scar, and ablation gap segmentation and quantification from LGE MRI, and the related literature for AF studies. Specifically, we first summarize AF-related imaging techniques, particularly LGE MRI. Then, we review the methodologies of the four computing tasks in detail and summarize the validation strategies applied in each task as well as state-of-the-art results on public datasets. Finally, the possible future developments are outlined, with a brief survey on the potential clinical applications of the aforementioned methods. The review indicates that the research into this topic is still in the early stages. Although several methods have been proposed, especially for the LA cavity segmentation, there is still a large scope for further algorithmic developments due to performance issues related to the high variability of enhancement appearance and differences in image acquisition.

A Comprehensive Survey on Bone Segmentation Techniques in Knee Osteoarthritis Research: From Conventional Methods to Deep Learning

Knee osteoarthritis (KOA) is a degenerative joint disease, which significantly affects middle-aged and elderly people. The majority of KOA is primarily based on hyaline cartilage change, according to medical images. However, technical bottlenecks such as noise, artifacts, and modality pose enormous challenges for an objective and efficient early diagnosis. Therefore, the correct prediction of arthritis is an essential step for effective diagnosis and the prevention of acute arthritis, where early diagnosis and treatment can assist to reduce the progression of KOA. However, predicting the development of KOA is a difficult and urgent problem that, if addressed, could accelerate the development of disease-modifying drugs, in turn helping to avoid millions of total joint replacement procedures each year. In knee joint research and clinical practice there are segmentation approaches that play a significant role in KOA diagnosis and categorization. In this paper, we seek to give an in-depth understanding of a wide range of the most recent methodologies for knee articular bone segmentation; segmentation methods allow the estimation of articular cartilage loss rate, which is utilized in clinical practice for assessing the disease progression and morphological change, ranging from traditional techniques to deep learning (DL)-based techniques. Moreover, the purpose of this work is to give researchers a general review of the currently available methodologies in the area. Therefore, it will help researchers who want to conduct research in the field of KOA, as well as highlight deficiencies and potential considerations in application in clinical practice. Finally, we highlight the diagnostic value of deep learning for future computer-aided diagnostic applications to complete this review.

Automated Knee MR Images Segmentation of Anterior Cruciate Ligament Tears

The anterior cruciate ligament (ACL) is one of the main stabilizer parts of the knee. ACL injury leads to causes of osteoarthritis risk. ACL rupture is common in the young athletic population. Accurate segmentation at an early stage can improve the analysis and classification of anterior cruciate ligaments tears. This study automatically segmented the anterior cruciate ligament (ACL) tears from magnetic resonance imaging through deep learning. The knee mask was generated on the original Magnetic Resonance (MR) images to apply a semantic segmentation technique with convolutional neural network architecture U-Net. The proposed segmentation method was measured by accuracy, intersection over union (IoU), dice similarity coefficient (DSC), precision, recall and F1-score of 98.4%, 99.0%, 99.4%, 99.6%, 99.6% and 99.6% on 11451 training images, whereas on the validation images of 3817 was, respectively, 97.7%, 93.8%,96.8%, 96.5%, 97.3% and 96.9%. We also provide dice loss of training and test datasets that have remained 0.005 and 0.031, respectively. The experimental results show that the ACL segmentation on JPEG MRI images with U-Nets achieves accuracy that outperforms the human segmentation. The strategy has promising potential applications in medical image analytics for the segmentation of knee ACL tears for MR images.

Studying osteoarthritis with artificial intelligence applied to magnetic resonance imaging

The 3D nature and soft-tissue contrast of MRI makes it an invaluable tool for osteoarthritis research, by facilitating the elucidation of disease pathogenesis and progression. The recent increasing employment of MRI has certainly been stimulated by major advances that are due to considerable investment in research, particularly related to artificial intelligence (AI). These AI-related advances are revolutionizing the use of MRI in clinical research by augmenting activities ranging from image acquisition to post-processing. Automation is key to reducing the long acquisition times of MRI, conducting large-scale longitudinal studies and quantitatively defining morphometric and other important clinical features of both soft and hard tissues in various anatomical joints. Deep learning methods have been used recently for multiple applications in the musculoskeletal field to improve understanding of osteoarthritis. Compared with labour-intensive human efforts, AI-based methods have advantages and potential in all stages of imaging, as well as post-processing steps, including aiding diagnosis and prognosis. However, AI-based methods also have limitations, including the arguably limited interpretability of AI models. Given that the AI community is highly invested in uncovering uncertainties associated with model predictions and improving their interpretability, we envision future clinical translation and progressive increase in the use of AI algorithms to support clinicians in optimizing patient care.

Fully Automatic Knee Bone Detection and Segmentation on Three-Dimensional MRI

In the medical sector, three-dimensional (3D) images are commonly used like computed tomography (CT) and magnetic resonance imaging (MRI). The 3D MRI is a non-invasive method of studying the soft-tissue structures in a knee joint for osteoarthritis studies. It can greatly improve the accuracy of segmenting structures such as cartilage, bone marrow lesion, and meniscus by identifying the bone structure first. U-net is a convolutional neural network that was originally designed to segment the biological images with limited training data. The input of the original U-net is a single 2D image and the output is a binary 2D image. In this study, we modified the U-net model to identify the knee bone structures using 3D MRI, which is a sequence of 2D slices. A fully automatic model has been proposed to detect and segment knee bones. The proposed model was trained, tested, and validated using 99 knee MRI cases where each case consists of 160 2D slices for a single knee scan. To evaluate the model's performance, the similarity, dice coefficient (DICE), and area error metrics were calculated. Separate models were trained using different knee bone components including tibia, femur, patella, as well as a combined model for segmenting all the knee bones. Using the whole MRI sequence (160 slices), the method was able to detect the beginning and ending bone slices first, and then segment the bone structures for all the slices in between. On the testing set, the detection model accomplished 98.79% accuracy and the segmentation model achieved DICE 96.94% and similarity 93.98%. The proposed method outperforms several state-of-the-art methods, i.e., it outperforms U-net by 3.68%, SegNet by 14.45%, and FCN-8 by 2.34%, in terms of DICE score using the same dataset.

Entropy and distance maps-guided segmentation of articular cartilage: data from the Osteoarthritis Initiative

PURPOSE

Accurate segmentation of articular cartilage from MR images is crucial for quantitative investigation of pathoanatomical conditions such as osteoarthritis (OA). Recently, deep learning-based methods have made significant progress in hard tissue segmentation. However, it remains a challenge to develop accurate methods for automatic segmentation of articular cartilage.

METHODS

We propose a two-stage method for automatic segmentation of articular cartilage. At the first stage, nnU-Net is employed to get segmentation of both hard tissues and articular cartilage. Based on the initial segmentation, we compute distance maps as well as entropy maps, which encode the uncertainty information about the initial cartilage segmentation. At the second stage, both distance maps and entropy maps are concatenated to the original image. We then crop a sub-volume around the cartilage region based on the initial segmentation, which is used as the input to another nnU-Net for segmentation refinement.

RESULTS

We designed and conducted comprehensive experiments on segmenting three different types of articular cartilage from two datasets, i.e., an in-house dataset consisting of 25 hip MR images and a publicly available dataset from Osteoarthritis Initiative (OAI). Our method achieved an average Dice similarity coefficient (DSC) of [Formula: see text] for the combined hip cartilage, [Formula: see text] for the femoral cartilage and [Formula: see text] for the tibial cartilage, respectively.

CONCLUSION

In summary, we developed a new approach for automatic segmentation of articular cartilage from MR images. Comprehensive experiments conducted on segmenting articular cartilage of the knee and hip joints demonstrated the efficacy of the present approach. Our method achieved equivalent or better results than the state-of-the-art methods.

Cross-Cohort Automatic Knee MRI Segmentation With Multi-Planar U-Nets

Background

Segmentation of medical image volumes is a time‐consuming manual task. Automatic tools are often tailored toward specific patient cohorts, and it is unclear how they behave in other clinical settings.

Purpose

To evaluate the performance of the open‐source Multi‐Planar U‐Net (MPUnet), the validated Knee Imaging Quantification (KIQ) framework, and a state‐of‐the‐art two‐dimensional (2D) U‐Net architecture on three clinical cohorts without extensive adaptation of the algorithms.

Study Type

Retrospective cohort study.

Subjects

A total of 253 subjects (146 females, 107 males, ages 57 ± 12 years) from three knee osteoarthritis (OA) studies (Center for Clinical and Basic Research [CCBR], Osteoarthritis Initiative [OAI], and Prevention of OA in Overweight Females [PROOF]) with varying demographics and OA severity (64/37/24/53/2 scans of Kellgren and Lawrence [KL] grades 0–4).

Field Strength/Sequence

0.18 T, 1.0 T/1.5 T, and 3 T sagittal three‐dimensional fast‐spin echo T1w and dual‐echo steady‐state sequences.

Assessment

All models were fit without tuning to knee magnetic resonance imaging (MRI) scans with manual segmentations from three clinical cohorts. All models were evaluated across KL grades.

Statistical Tests

Segmentation performance differences as measured by Dice coefficients were tested with paired, two‐sided Wilcoxon signed‐rank statistics with significance threshold α = 0.05.

Results

The MPUnet performed superior or equal to KIQ and 2D U‐Net on all compartments across three cohorts. Mean Dice overlap was significantly higher for MPUnet compared to KIQ and U‐Net on CCBR (0.83±0.04 vs. 0.81±0.06 and 0.82±0.05), significantly higher than KIQ and U‐Net OAI (0.86±0.03 vs. 0.84±0.04 and 0.85±0.03), and not significantly different from KIQ while significantly higher than 2D U‐Net on PROOF (0.78±0.07 vs. 0.77±0.07, P=0.10, and 0.73±0.07). The MPUnet performed significantly better on N=22 KL grade 3 CCBR scans with 0.78±0.06 vs. 0.75±0.08 for KIQ and 0.76±0.06 for 2D U‐Net.

Data Conclusion

The MPUnet matched or exceeded the performance of state‐of‐the‐art knee MRI segmentation models across cohorts of variable sequences and patient demographics. The MPUnet required no manual tuning making it both accurate and easy‐to‐use.

Level of Evidence

3

Technical Efficacy

Stage 2

A generalized deep learning framework for automatic rheumatoid arthritis severity grading

Knee rheumatoid arthritis (RA) is the highly prevalent, chronic, progressive condition in the world. To diagnose this disease in the early stage in detail analysis with magnetic resonance (MR) image is possible. The imaging modality feature allows unbiased assessment of joint space narrowing (JSN), cartilage volume, and other vital features. This provides a fine-grained RA severity evaluation of the knee, contrasted to the benchmark, and generally used Kellgren Lawrence (KL) assessment. In this research, an intelligent system is developed to predict KL grade from the knee dataset. Our approach is based on hybrid deep learning of 50 layers (ResNet50) with skip connections. The proposed approach also uses Adam optimizer to provide learning linearity in the training stage. Our approach yields KL grade and JSN for femoral and tibial tissue with lateral and medial compartments. Furthermore, the approach also yields area under curve (AUC) of 0.98, accuracy 96.85%, mean absolute error (MAE) 0.015, precision 98.31%, and other commonly used parameters for the existence of radiographic RA progression which is improved than the existing state-of-the-art.

Deep Learning for Automatic Image Segmentation in Stomatology and Its Clinical Application

Deep learning has become an active research topic in the field of medical image analysis. In particular, for the automatic segmentation of stomatological images, great advances have been made in segmentation performance. In this paper, we systematically reviewed the recent literature on segmentation methods for stomatological images based on deep learning, and their clinical applications. We categorized them into different tasks and analyze their advantages and disadvantages. The main categories that we explored were the data sources, backbone network, and task formulation. We categorized data sources into panoramic radiography, dental X-rays, cone-beam computed tomography, multi-slice spiral computed tomography, and methods based on intraoral scan images. For the backbone network, we distinguished methods based on convolutional neural networks from those based on transformers. We divided task formulations into semantic segmentation tasks and instance segmentation tasks. Toward the end of the paper, we discussed the challenges and provide several directions for further research on the automatic segmentation of stomatological images.

MRI Knee Domain Translation for Unsupervised Segmentation By CycleGAN (data from Osteoarthritis initiative (OAI))

Accurate quantification of bone and cartilage features is the key to efficient management of knee osteoarthritis (OA). Bone and cartilage tissues can be accurately segmented from magnetic resonance imaging (MRI) data using supervised Deep Learning (DL) methods. DL training is commonly conducted using large datasets with expert-labeled annotations. DL models perform better if distributions of testing data (target domains) are close to those of training data (source domains). However, in practice, data distributions of images from different MRI scanners and sequences are different and DL models need to re-trained on each dataset separately. We propose a domain adaptation (DA) framework using the CycleGAN model for MRI translation that would aid in unsupervised MRI data segmentation. We have validated our pipeline on five scans from the Osteoarthritis Initiative (OAI) dataset. Using this pipeline, we translated TSE Fat Suppressed MRI sequences to pseudo-DESS images. An improved MaskRCNN (IMaskRCNN) instance segmentation network trained on DESS was used to segment cartilage and femoral head regions in TSE Fat Suppressed sequences. Segmentations of the I-MaskRCNN correlated well with approximated manual segmentation obtained from nearest DESS slices (DICE = 0.76) without the need for retraining. We anticipate this technique will aid in automatic unsupervised assessment of knee MRI using commonly acquired MRI sequences and save experts' time that would otherwise be required for manual segmentation.Clinical relevance- This technique paves the way to automatically convert one MRI sequence to its equivalent as if acquired by a different protocol or different magnet, facilitating robust, hardware-independent automated analysis. For example, routine clinically acquired knee MRI could be converted to high-resolution high-contrast images suitable for automated detection of cartilage defects.

A Multi-Task Deep Learning Method for Detection of Meniscal Tears in MRI Data from the Osteoarthritis Initiative Database

We present a novel and computationally efficient method for the detection of meniscal tears in Magnetic Resonance Imaging (MRI) data. Our method is based on a Convolutional Neural Network (CNN) that operates on complete 3D MRI scans. Our approach detects the presence of meniscal tears in three anatomical sub-regions (anterior horn, body, posterior horn) for both the Medial Meniscus (MM) and the Lateral Meniscus (LM) individually. For optimal performance of our method, we investigate how to preprocess the MRI data and how to train the CNN such that only relevant information within a Region of Interest (RoI) of the data volume is taken into account for meniscal tear detection. We propose meniscal tear detection combined with a bounding box regressor in a multi-task deep learning framework to let the CNN implicitly consider the corresponding RoIs of the menisci. We evaluate the accuracy of our CNN-based meniscal tear detection approach on 2,399 Double Echo Steady-State (DESS) MRI scans from the Osteoarthritis Initiative database. In addition, to show that our method is capable of generalizing to other MRI sequences, we also adapt our model to Intermediate-Weighted Turbo Spin-Echo (IW TSE) MRI scans. To judge the quality of our approaches, Receiver Operating Characteristic (ROC) curves and Area Under the Curve (AUC) values are evaluated for both MRI sequences. For the detection of tears in DESS MRI, our method reaches AUC values of 0.94, 0.93, 0.93 (anterior horn, body, posterior horn) in MM and 0.96, 0.94, 0.91 in LM. For the detection of tears in IW TSE MRI data, our method yields AUC values of 0.84, 0.88, 0.86 in MM and 0.95, 0.91, 0.90 in LM. In conclusion, the presented method achieves high accuracy for detecting meniscal tears in both DESS and IW TSE MRI data. Furthermore, our method can be easily trained and applied to other MRI sequences.

Automated segmentation of biventricular contours in tissue phase mapping using deep learning

Tissue phase mapping (TPM) is an MRI technique for quantification of regional biventricular myocardial velocities. Despite its potential, clinical use is limited due to the requisite labor-intensive manual segmentation of cardiac contours for all time frames. The purpose of this study was to develop a deep learning (DL) network for automated segmentation of TPM images, without significant loss in segmentation and myocardial velocity quantification accuracy compared with manual segmentation. We implemented a multi-channel 3D (three dimensional; 2D + time) dense U-Net that trained on magnitude and phase images and combined cross-entropy, Dice, and Hausdorff distance loss terms to improve the segmentation accuracy and suppress unnatural boundaries. The dense U-Net was trained and tested with 150 multi-slice, multi-phase TPM scans (114 scans for training, 36 for testing) from 99 heart transplant patients (44 females, 1-4 scans/patient), where the magnitude and velocity-encoded (V_x , V_y , V_z ) images were used as input and the corresponding manual segmentation masks were used as reference. The accuracy of DL segmentation was evaluated using quantitative metrics (Dice scores, Hausdorff distance) and linear regression and Bland-Altman analyses on the resulting peak radial and longitudinal velocities (V_r and V_z ). The mean segmentation time was about 2 h per patient for manual and 1.9 ± 0.3 s for DL. Our network produced good accuracy (median Dice = 0.85 for left ventricle (LV), 0.64 for right ventricle (RV), Hausdorff distance = 3.17 pixels) compared with manual segmentation. Peak V_r and V_z measured from manual and DL segmentations were strongly correlated (R ≥ 0.88) and in good agreement with manual analysis (mean difference and limits of agreement for V_z and V_r were -0.05 ± 0.98 cm/s and -0.06 ± 1.18 cm/s for LV, and -0.21 ± 2.33 cm/s and 0.46 ± 4.00 cm/s for RV, respectively). The proposed multi-channel 3D dense U-Net was capable of reducing the segmentation time by 3,600-fold, without significant loss in accuracy in tissue velocity measurements.

Artificial intelligence-based automatic assessment of lower limb torsion on MRI

Abnormal torsion of the lower limbs may adversely affect joint health. This study developed and validated a deep learning-based method for automatic measurement of femoral and tibial torsion on MRI. Axial T2-weighted sequences acquired of the hips, knees, and ankles of 93 patients (mean age, 13 ± 5 years; 52 males) were included and allocated to training (n = 60), validation (n = 9), and test sets (n = 24). A U-net convolutional neural network was trained to segment both femur and tibia, identify osseous anatomic landmarks, define pertinent reference lines, and quantify femoral and tibial torsion. Manual measurements by two radiologists provided the reference standard. Inter-reader comparisons were performed using repeated-measures ANOVA, Pearson’s r, and the intraclass correlation coefficient (ICC). Mean Sørensen-Dice coefficients for segmentation accuracy ranged between 0.89 and 0.93 and erroneous segmentations were scarce. Ranges of torsion as measured by both readers and the algorithm on the same axial image were 15.8°–18.0° (femur) and 33.9°–35.2° (tibia). Correlation coefficients (ranges, .968 ≤ r ≤ .984 [femur]; .867 ≤ r ≤ .904 [tibia]) and ICCs (ranges, .963 ≤ ICC ≤ .974 [femur]; .867 ≤ ICC ≤ .894 [tibia]) indicated excellent inter-reader agreement. Algorithm-based analysis was faster than manual analysis (7 vs 207 vs 230 s, p < .001). In conclusion, fully automatic measurement of torsional alignment is accurate, reliable, and sufficiently fast for clinical workflows.

Reproducibility of an Automated Quantitative MRI Assessment of Low-Grade Knee Articular Cartilage Lesions

OBJECTIVE

The goal of this study was to assess the reproducibility of an automated knee cartilage segmentation of 21 cartilage regions with a model-based algorithm and to compare the results with manual segmentation.

DESIGN

Thirteen patients with low-grade femoral cartilage defects were included in the study and were scanned twice on a 7-T magnetic resonance imaging (MRI) scanner 8 days apart. A 3-dimensional double-echo steady-state (3D-DESS) sequence was used to acquire MR images for automated cartilage segmentation, and T2-mapping was performed using a 3D triple-echo steady-state (3D-TESS) sequence. Cartilage volume, thickness, and T2 and texture features were automatically extracted from each knee for each of the 21 subregions. DESS was used for manual cartilage segmentation and compared with automated segmentation using the Dice coefficient. The reproducibility of each variable was expressed using standard error of measurement (SEM) and smallest detectable change (SDC).

RESULTS

The Dice coefficient for the similarity between manual and automated segmentation ranged from 0.83 to 0.88 in different cartilage regions. Test-retest analysis of automated cartilage segmentation and automated quantitative parameter extraction revealed excellent reproducibility for volume measurement (mean SDC for all subregions of 85.6 mm³), for thickness detection (SDC = 0.16 mm) and also for T2 values (SDC = 2.38 ms) and most gray-level co-occurrence matrix features (SDC = 0.1 a.u.).

CONCLUSIONS

The proposed technique of automated knee cartilage evaluation based on the segmentation of 3D MR images and correlation with T2 mapping provides highly reproducible results and significantly reduces the segmentation effort required for the analysis of knee articular cartilage in longitudinal studies.

Automated tibiofemoral joint segmentation based on deeply supervised 2D-3D ensemble U-Net: Data from the Osteoarthritis Initiative

Improving longevity is one of the greatest achievements in humanity. Because of this, the population is growing older, and the ubiquity of knee osteoarthritis (OA) is on the rise. Nonetheless, the understanding and ability to investigate potential precursors of knee OA have been impeded by time-consuming and laborious manual delineation processes which are prone to poor reproducibility. A method for automatic segmentation of the tibiofemoral joint using magnetic resonance imaging (MRI) is presented in this work. The proposed method utilizes a deeply supervised 2D-3D ensemble U-Net, which consists of foreground class oversampling, deep supervision loss branches, and Gaussian weighted softmax score aggregation. It was designed, optimized, and tested on 507 3D double echo steady-state (DESS) MR volumes using a two-fold cross-validation approach. A state-of-the-art segmentation accuracy measured as Dice similarity coefficient (DSC) for the femur bone (98.6 ± 0.27%), tibia bone (98.8 ± 0.31%), femoral cartilage (90.3 ± 2.89%), and tibial cartilage (86.7 ± 4.07%) is achieved. Notably, the proposed method yields sub-voxel accuracy for an average symmetric surface distance (ASD) less than 0.36 mm. The model performance is not affected by the severity of radiographic osteoarthritis (rOA) grades or the presence of pathophysiological changes. The proposed method offers an accurate segmentation with high time efficiency (~62 s) per 3D volume, which is well suited for efficient processing and analysis of the large prospective cohorts of the Osteoarthritis Initiative (OAI).

Quantification of Acetabular Coverage on 3-Dimensional Reconstructed Computed Tomography Scan Bone Models in Patients With Femoroacetabular Impingement Syndrome: A Descriptive Study

Background

Accurate assessment of osseous morphology is imperative in the evaluation of patients with femoroacetabular impingement syndrome (FAIS) and hip dysplasia. Through use of computed tomography (CT), 3-dimensional (3D) reconstructed hip models may provide a more precise measurement for overcoverage and undercoverage and aid in the interpretation of 2-dimensional radiographs obtained in the clinical setting.

Purpose

To describe new measures of acetabular coverage based on 3D-reconstructed CT scan bone models.

Study Design

Cross-sectional study; Level of evidence, 3.

Methods

Preoperative CT scans were acquired on the bilateral hips and pelvises of 30 patients before arthroscopic surgical intervention for FAIS. Custom software was used for semiautomated segmentation to generate 3D osseous models of the femur and acetabulum that were aligned to a standard coordinate system. This software calculated percentage of total acetabular coverage, which was defined as the surface area projected onto the superior aspect of the femoral head. The percentage of coverage was also quantified regionally in the anteromedial, anterolateral, posteromedial, and posterolateral quadrants of the femoral head. The acetabular clockface was established by defining 6 o'clock as the inferior aspect of the acetabular notch. Radial coverage was then calculated along the clockface from the 9-o'clock to 5-o'clock positions.

Results

The study included 20 female and 10 male patients with a mean age of 33.6 ± 11.7 years and mean body mass index of 27.8 ± 6.3. The average percentage of total acetabular coverage for the sample was 57% ± 6%. Acetabular coverages by region were as follows: anteromedial, 78% ± 7%; anterolateral, 18% ± 7%, posterolateral, 33% ± 13%, and posteromedial, 99% ± 1%. The acetabular coverage ranged from 23% to 69% along the radial clockface from 9 to 5 o'clock.

Conclusion

This study demonstrated new 3D measurements to characterize acetabular coverage in patients with FAIS and elucidated the distribution of acetabular coverage according to these measurements.

Emergence of Deep Learning in Knee Osteoarthritis Diagnosis

Osteoarthritis (OA), especially knee OA, is the most common form of arthritis, causing significant disability in patients worldwide. Manual diagnosis, segmentation, and annotations of knee joints remain as the popular method to diagnose OA in clinical practices, although they are tedious and greatly subject to user variation. Therefore, to overcome the limitations of the commonly used method as above, numerous deep learning approaches, especially the convolutional neural network (CNN), have been developed to improve the clinical workflow efficiency. Medical imaging processes, especially those that produce 3-dimensional (3D) images such as MRI, possess ability to reveal hidden structures in a volumetric view. Acknowledging that changes in a knee joint is a 3D complexity, 3D CNN has been employed to analyse the joint problem for a more accurate diagnosis in the recent years. In this review, we provide a broad overview on the current 2D and 3D CNN approaches in the OA research field. We reviewed 74 studies related to classification and segmentation of knee osteoarthritis from the Web of Science database and discussed the various state-of-the-art deep learning approaches proposed. We highlighted the potential and possibility of 3D CNN in the knee osteoarthritis field. We concluded by discussing the possible challenges faced as well as the potential advancements in adopting 3D CNNs in this field.

Automatic detection of attachment sites for knee ligaments and tendons on CT images

PURPOSE

The diseases and injuries of the knee joint are the most common orthopedic disorders. Personalized knee models can be helpful in the process of early intervention and lasting treatment techniques development. Fully automatic reconstruction of knee joint anatomical structures from medical images (CT, MRI, ultrasound) remains a challenge. For this reason, most of state-of-the-art knee joint models contain simplifications such as representation of muscles and ligaments as line segments connecting two points which replace attachment areas. The paper presents algorithms for automatic detection of such points on knee CT images.

METHODS

This paper presents three approaches to automatic detection of ligaments and tendons attachment sites on the patients CT images: qualitative anatomical descriptions, analysis of bones curvature, and quantitative anatomical descriptions. Combinations of these approaches result in new automatic detection algorithms. Each algorithm exploits anatomical peculiarities of each attachment site, e.g., bone curvature and number of other attachments in a neighborhood of the site.

RESULTS

The experimental dataset consisted of 26 anonymized CT sequences containing right and left knee joints in different resolutions. The proposed algorithms take into account bone surface curvatures and spatial differences in locations of medial and lateral parts of both knees. The algorithms for detection of quadriceps femoris, popliteus, biceps femoris tendons, and lateral collateral and medial collateral ligaments attachment sites are provided, as well as examples of their application. Two algorithms are validated by comparison with known statistics of ligaments lengths and also using ground truth annotations for anatomical landmarks approved by clinical experts.

CONCLUSIONS

The algorithms simplify generation of patient-specific knee joint models demanded in personalized biomechanical models. The algorithms in the current implementation have two important limitations. First, the correctness of the produced results depends on the bones segmentation quality. Second, the presented algorithms detect a point of the attachment site, which is not necessarily its center. Therefore, manual correction of the attachment site location may be required for attachments with relatively large area.

Towards novel osteoarthritis biomarkers: Multi-criteria evaluation of 46,996 segmented knee MRI data from the Osteoarthritis Initiative

Convolutional neural networks (CNNs) are the state-of-the-art for automated assessment of knee osteoarthritis (KOA) from medical image data. However, these methods lack interpretability, mainly focus on image texture, and cannot completely grasp the analyzed anatomies' shapes. In this study we assess the informative value of quantitative features derived from segmentations in order to assess their potential as an alternative or extension to CNN-based approaches regarding multiple aspects of KOA. Six anatomical structures around the knee (femoral and tibial bones, femoral and tibial cartilages, and both menisci) are segmented in 46,996 MRI scans. Based on these segmentations, quantitative features are computed, i.e., measurements such as cartilage volume, meniscal extrusion and tibial coverage, as well as geometric features based on a statistical shape encoding of the anatomies. The feature quality is assessed by investigating their association to the Kellgren-Lawrence grade (KLG), joint space narrowing (JSN), incident KOA, and total knee replacement (TKR). Using gold standard labels from the Osteoarthritis Initiative database the balanced accuracy (BA), the area under the Receiver Operating Characteristic curve (AUC), and weighted kappa statistics are evaluated. Features based on shape encodings of femur, tibia, and menisci plus the performed measurements showed most potential as KOA biomarkers. Differentiation between non-arthritic and severely arthritic knees yielded BAs of up to 99%, 84% were achieved for diagnosis of early KOA. Weighted kappa values of 0.73, 0.72, and 0.78 were achieved for classification of the grade of medial JSN, lateral JSN, and KLG, respectively. The AUC was 0.61 and 0.76 for prediction of incident KOA and TKR within one year, respectively. Quantitative features from automated segmentations provide novel biomarkers for KLG and JSN classification and show potential for incident KOA and TKR prediction. The validity of these features should be further evaluated, especially as extensions of CNN-based approaches. To foster such developments we make all segmentations publicly available together with this publication.

Application of Convolution Neural Network (CNN) Model Combined with Pyramid Algorithm in Aerobics Action Recognition

In order to realize high-accuracy recognition of aerobics actions, a highly applicable deep learning model and faster data processing methods are required. Therefore, it is a major difficulty in the field of research on aerobics action recognition. Based on this, this paper studies the application of the convolution neural network (CNN) model combined with the pyramid algorithm in aerobics action recognition. Firstly, the basic architecture of the convolution neural network model based on the pyramid algorithm is proposed. Combined with the application strategy of the common recognition model in aerobics action recognition, the traditional aerobics action capture information is processed. Through the characteristics of different aerobics actions, different accurate recognition is realized, and then, the error of the recognition model is evaluated. Secondly, the composite recognition function of the convolution neural network model in this application is constructed, and the common data layer effect recognition method is used in the optimization recognition. Aiming at the shortcomings of the composite recognition function, the pyramid algorithm is used to improve the convolution neural network recognition model by deep learning optimization. Finally, through the effectiveness comparison experiment, the results show that the convolution neural network model based on the pyramid algorithm is more efficient than the conventional recognition method in aerobics action recognition.

3D MRI in Osteoarthritis

Osteoarthritis (OA) is among the top 10 burdensome diseases, with the knee the most affected joint. Magnetic resonance imaging (MRI) allows whole-knee assessment, making it ideally suited for imaging OA, considered a multitissue disease. Three-dimensional (3D) MRI enables the comprehensive assessment of OA, including quantitative morphometry of various joint tissues. Manual tissue segmentation on 3D MRI is challenging but may be overcome by advanced automated image analysis methods including artificial intelligence (AI). This review presents examples of the utility of 3D MRI for knee OA, focusing on the articular cartilage, bone, meniscus, synovium, and infrapatellar fat pad, and it highlights several applications of AI that facilitate segmentation, lesion detection, and disease classification.

Discrete-MultiResUNet: Segmentation and feature extraction model for knee MR images

Deep learning has shown outstanding efficiency in medical image segmentation. Segmentation of knee tissues is an important task for early diagnosis of rheumatoid arthritis (RA) with selecting variant features. Automated segmentation and feature extraction of knee tissues are desirable for faster and reliable analysis of large datasets and further diagnosis. In this paper a novel architecture called as Discrete-MultiResUNet, which is a combination of discrete wavelet transform (DWT) with MultiResUNet architecture is applied for feature extraction and segmentation, respectively. This hybrid architecture captures more prominent features from the knee magnetic resonance image efficiently with segmenting vital knee tissues. The hybrid model is evaluated on the knee MR dataset demonstrating outperforming performance compared with baseline models. The model achieves excellent segmentation performance accuracy of 96.77% with a dice coefficient of 98%.

AAWS-Net: Anatomy-aware weakly-supervised learning network for breast mass segmentation

Accurate segmentation of breast masses is an essential step in computer aided diagnosis of breast cancer. The scarcity of annotated training data greatly hinders the model’s generalization ability, especially for the deep learning based methods. However, high-quality image-level annotations are time-consuming and cumbersome in medical image analysis scenarios. In addition, a large amount of weak annotations is under-utilized which comprise common anatomy features. To this end, inspired by teacher-student networks, we propose an Anatomy-Aware Weakly-Supervised learning Network (AAWS-Net) for extracting useful information from mammograms with weak annotations for efficient and accurate breast mass segmentation. Specifically, we adopt a weakly-supervised learning strategy in the Teacher to extract anatomy structure from mammograms with weak annotations by reconstructing the original image. Besides, knowledge distillation is used to suggest morphological differences between benign and malignant masses. Moreover, the prior knowledge learned from the Teacher is introduced to the Student in an end-to-end way, which improves the ability of the student network to locate and segment masses. Experiments on CBIS-DDSM have shown that our method yields promising performance compared with state-of-the-art alternative models for breast mass segmentation in terms of segmentation accuracy and IoU.

Intelligent localization and quantitative evaluation of anterior talofibular ligament injury using magnetic resonance imaging of ankle

Background

There is a high incidence of injury to the lateral ligament of the ankle in daily living and sports activities. The anterior talofibular ligament (ATFL) is the most frequent types of ankle injuries. It is of great clinical significance to achieve intelligent localization and injury evaluation of ATFL due to its vulnerability.

Methods

According to the specific characteristics of bones in different slices, the key slice was extracted by image segmentation and characteristic analysis. Then, the talus and fibula in the key slice were segmented by distance regularized level set evolution (DRLSE), and the curvature of their contour pixels was calculated to find useful feature points including the neck of talus, the inner edge of fibula, and the outer edge of fibula. ATFL area can be located using these feature points so as to quantify its first-order gray features and second-order texture features. Support vector machine (SVM) was performed for evaluation of ATFL injury.

Results

Data were collected retrospectively from 158 patients who underwent MRI, and were divided into normal (68) and tear (90) group. The positioning accuracy and Dice coefficient were used to measure the performance of ATFL localization, and the mean values are 87.7% and 77.1%, respectively, which is helpful for the following feature extraction. SVM gave a good prediction ability with accuracy of 93.8%, sensitivity of 88.9%, specificity of 100%, precision of 100%, and F1 score of 94.2% in the test set.

Conclusion

Experimental results indicate that the proposed method is reliable in diagnosing ATFL injury. This study may provide a potentially viable method for aided clinical diagnoses of some ligament injury.

A Coarse-to-Fine Framework for Automated Knee Bone and Cartilage Segmentation Data from the Osteoarthritis Initiative

Knee osteoarthritis (OA) is a degenerative joint disease that is prevalent in advancing age. The pathology of OA disease is still unclear, and there are no effective interventions that can completely alter the OA disease process. Magnetic resonance (MR) image evaluation is sensitive for depicting early changes of knee OA, and therefore important for early clinical intervention for relieving the symptom. Automated cartilage segmentation based on MR images is a vital step in experimental longitudinal studies to follow-up the patients and prospectively define a new quantitative marker from OA progression. In this paper, we develop a deep learning-based coarse-to-fine approach for automated knee bone, cartilage, and meniscus segmentation with high computational efficiency. The proposed method is evaluated using two-fold cross-validation on 507 MR volumes (81,120 slices) with OA from the Osteoarthritis Initiative (OAI)1 dataset. The mean dice similarity coefficients (DSCs) of femoral bone (FB), tibial bone (TB), femoral cartilage (FC), and tibial cartilage (TC) separately are 99.1%, 98.2%, 90.9%, and 85.8%. The time of segmenting each patient is 12 s, which is fast enough to be used in clinical practice. Our proposed approach may provide an automated toolkit to help computer-aided quantitative analyses of OA images.

Multiple-echo steady-state (MESS): Extending DESS for joint T2 mapping and chemical-shift corrected water-fat separation

Purpose

To extend the double echo steady‐state (DESS) sequence to enable chemical‐shift corrected water‐fat separation.

Methods

This study proposes multiple‐echo steady‐state (MESS), a sequence that modifies the readouts of the DESS sequence to acquire two echoes each with bipolar readout gradients with higher readout bandwidth. This enables water‐fat separation and eliminates the need for water‐selective excitation that is often used in combination with DESS, without increasing scan time. An iterative fitting approach was used to perform joint chemical‐shift corrected water‐fat separation and T₂ estimation on all four MESS echoes simultaneously. MESS and water‐selective DESS images were acquired for five volunteers, and were compared qualitatively as well as quantitatively on cartilage T₂ and thickness measurements. Signal‐to‐noise ratio (SNR) and T₂ quantification were evaluated numerically using pseudo‐replications of the acquisition.

Results

The water‐fat separation provided by MESS was robust and with quality comparable to water‐selective DESS. MESS T₂ estimation was similar to DESS, albeit with slightly higher variability. Noise analysis showed that SNR in MESS was comparable to DESS on average, but did exhibit local variations caused by uncertainty in the water‐fat separation.

Conclusion

In the same acquisition time as DESS, MESS provides water‐fat separation with comparable SNR in the reconstructed water and fat images. By providing additional image contrasts in addition to the water‐selective DESS images, MESS provides a promising alternative to DESS.

Rigid motion invariant statistical shape modeling based on discrete fundamental forms data from the osteoarthritis initiative and the Alzheimer' disease neuroimaging initiative

We present a novel approach for nonlinear statistical shape modeling that is invariant under Euclidean motion and thus alignment-free. By analyzing metric distortion and curvature of shapes as elements of Lie groups in a consistent Riemannian setting, we construct a framework that reliably handles large deformations. Due to the explicit character of Lie group operations, our non-Euclidean method is very efficient allowing for fast and numerically robust processing. This facilitates Riemannian analysis of large shape populations accessible through longitudinal and multi-site imaging studies providing increased statistical power. Additionally, as planar configurations form a submanifold in shape space, our representation allows for effective estimation of quasi-isometric surfaces flattenings. We evaluate the performance of our model w.r.t. shape-based classification of hippocampus and femur malformations due to Alzheimer's disease and osteoarthritis, respectively. In particular, we outperform state-of-the-art classifiers based on geometric deep learning as well as statistical shape modeling especially in presence of sparse training data. To provide insight into the model's ability of capturing biological shape variability, we carry out an analysis of specificity and generalization ability.

Automatic knee cartilage and bone segmentation using multi-stage convolutional neural networks: data from the osteoarthritis initiative

OBJECTIVES

Accurate and efficient knee cartilage and bone segmentation are necessary for basic science, clinical trial, and clinical applications. This work tested a multi-stage convolutional neural network framework for MRI image segmentation.

MATERIALS AND METHODS

Stage 1 of the framework coarsely segments images outputting probabilities of each voxel belonging to the classes of interest: 4 cartilage tissues, 3 bones, 1 background. Stage 2 segments overlapping sub-volumes that include Stage 1 probability maps concatenated to raw image data. Using six fold cross-validation, this framework was tested on two datasets comprising 176 images [88 individuals in the Osteoarthritis Initiative (OAI)] and 60 images (15 healthy young men), respectively.

RESULTS

On the OAI segmentation dataset, the framework produces cartilage segmentation accuracies (Dice similarity coefficient) of 0.907 (femoral), 0.876 (medial tibial), 0.913 (lateral tibial), and 0.840 (patellar). Healthy cartilage accuracies are excellent (femoral = 0.938, medial tibial = 0.911, lateral tibial = 0.930, patellar = 0.955). Average surface distances are less than in-plane resolution. Segmentations take 91 ± 11 s per knee.

DISCUSSION

The framework learns to automatically segment knee cartilage tissues and bones from MR images acquired with two sequences, producing efficient, accurate quantifications at varying disease severities.