Deep convolutional neural network for segmentation of knee joint anatomy

Automatic segmentation of knee CT images of tibial plateau fractures based on three-dimensional U-Net: Assisting junior physicians with Schatzker classification

PURPOSE

This study aimed to automatically segment knee computed tomography (CT) images of tibial plateau fractures using a three-dimensional (3D) U-net-based method, accurately construct 3D maps of tibial plateau fractures, and examine their usefulness for Schatzker classification in clinical practice.

METHODS

We retrospectively enrolled 234 cases with tibial plateau fractures from our hospital in this study. The four constituent bones of the knee were manually annotated using ITK-SNAP software. Finally, image features were extracted using deep learning. The usefulness of the results for Schatzker classification was examined by an orthopaedic and a radiology resident.

RESULTS

On average, our model required < 40 s to process a 3D CT scan of the knee. The average Dice coefficient for all four knee bones was higher than 0.950, and highly accurate 3D maps of the tibia were produced. With the aid of the results of our model, the accuracy, sensitivity, and specificity of the Schatzker classification of both residents improved.

CONCLUSIONS

The proposed method can rapidly and accurately segment knee CT images of tibial plateau fractures and assist residents with Schatzker classification, which can help improve diagnostic efficiency and reduce the workload of junior doctors in clinical practice.

What is New in Osteoarthritis Imaging?

Osteoarthritis (OA) is the leading joint disorder globally, affecting a significant proportion of the population. Recent studies have changed our understanding of OA, viewing it as a complex pathology of the whole joint with a multifaceted etiology, encompassing genetic, biological, and biomechanical elements. This review highlights the role of imaging in diagnosing and monitoring OA. Today's role of radiography is discussed, while also elaborating on the advances in computed tomography and magnetic resonance imaging, discussing semiquantitative methods, quantitative morphologic and compositional techniques, and giving an outlook on the potential role of artificial intelligence in OA research.

Fully and Weakly Supervised Deep Learning for Meniscal Injury Classification, and Location Based on MRI

Meniscal injury is a common cause of knee joint pain and a precursor to knee osteoarthritis (KOA). The purpose of this study is to develop an automatic pipeline for meniscal injury classification and localization using fully and weakly supervised networks based on MRI images. In this retrospective study, data were from the osteoarthritis initiative (OAI). The MR images were reconstructed using a sagittal intermediate-weighted fat-suppressed turbo spin-echo sequence. (1) We used 130 knees from the OAI to develop the LGSA-UNet model which fuses the features of adjacent slices and adjusts the blocks in Siam to enable the central slice to obtain rich contextual information. (2) One thousand seven hundred and fifty-six knees from the OAI were included to establish segmentation and classification models. The segmentation model achieved a DICE coefficient ranging from 0.84 to 0.93. The AUC values ranged from 0.85 to 0.95 in the binary models. The accuracy for the three types of menisci (normal, tear, and maceration) ranged from 0.60 to 0.88. Furthermore, 206 knees from the orthopedic hospital were used as an external validation data set to evaluate the performance of the model. The segmentation and classification models still performed well on the external validation set. To compare the diagnostic performances between the deep learning (DL) models and radiologists, the external validation sets were sent to two radiologists. The binary classification model outperformed the diagnostic performance of the junior radiologist (0.82-0.87 versus 0.74-0.88). This study highlights the potential of DL in knee meniscus segmentation and injury classification which can help improve diagnostic efficiency.

Evaluation of the consistency of the MRI- based AI segmentation cartilage model using the natural tibial plateau cartilage

OBJECTIVE

The study aims to evaluate the accuracy of an MRI-based artificial intelligence (AI) segmentation cartilage model by comparing it to the natural tibial plateau cartilage.

METHODS

This study included 33 patients (41 knees) with severe knee osteoarthritis scheduled to undergo total knee arthroplasty (TKA). All patients had a thin-section MRI before TKA. Our study is mainly divided into two parts: (i) In order to evaluate the MRI-based AI segmentation cartilage model's 2D accuracy, the natural tibial plateau was used as gold standard. The MRI-based AI segmentation cartilage model and the natural tibial plateau were represented in binary visualization (black and white) simulated photographed images by the application of Simulation Photography Technology. Both simulated photographed images were compared to evaluate the 2D Dice similarity coefficients (DSC). (ii) In order to evaluate the MRI-based AI segmentation cartilage model's 3D accuracy. Hand-crafted cartilage model based on knee CT was established. We used these hand-crafted CT-based knee cartilage model as gold standard to evaluate 2D and 3D consistency of between the MRI-based AI segmentation cartilage model and hand-crafted CT-based cartilage model. 3D registration technology was used for both models. Correlations between the MRI-based AI knee cartilage model and CT-based knee cartilage model were also assessed with the Pearson correlation coefficient.

RESULTS

The AI segmentation cartilage model produced reasonably high two-dimensional DSC. The average 2D DSC between MRI-based AI cartilage model and the tibial plateau cartilage is 0.83. The average 2D DSC between the AI segmentation cartilage model and the CT-based cartilage model is 0.82. As for 3D consistency, the average 3D DSC between MRI-based AI cartilage model and CT-based cartilage model is 0.52. However, the quantification of cartilage segmentation with the AI and CT-based models showed excellent correlation (r = 0.725; P values < 0.05).

CONCLUSION

Our study demonstrated that our MRI-based AI cartilage model can reliably extract morphologic features such as cartilage shape and defect location of the tibial plateau cartilage. This approach could potentially benefit clinical practices such as diagnosing osteoarthritis. However, in terms of cartilage thickness and three-dimensional accuracy, MRI-based AI cartilage model underestimate the actual cartilage volume. The previous AI verification methods may not be completely accurate and should be verified with natural cartilage images. Combining multiple verification methods will improve the accuracy of the AI model.

Design and Optimization of an Adaptive Knee Joint Orthosis for Biomimetic Motion Rehabilitation Assistance

In this paper, an adaptive knee joint orthosis with a variable rotation center for biomimetic motion rehabilitation assistance suitable for patients with knee joint movement dysfunction is designed. Based on the kinematic information of knee joint motion obtained by a motion capture system, a Revolute-Prismatic-Revolute (RPR) model is established to simulate the biomimetic motion of the knee joint, then a corresponding implementation for repetitively driving the flexion-extension motion of the knee joint, mainly assembled by a double-cam meshing mechanism, is designed. The pitch curve of each cam is calculated based on the screw theory. During the design process, size optimization is used to reduce the weight of the equipment, resulting in a reduction from 1.96 kg to 1.16 kg, achieving the goal of lightweight equipment. Finally, a prototype of the designed orthosis with the desired biomimetic rotation function is prepared and verified. The result shows that the rotation center of the prototype can achieve biomimetic motion coincident with the rotation center of an active knee joint, which can successfully provide rehabilitation assistance for the knee joint flexion-extension motion.

Integration of Swin UNETR and statistical shape modeling for a semi-automated segmentation of the knee and biomechanical modeling of articular cartilage

Simulation studies, such as finite element (FE) modeling, provide insight into knee joint mechanics without patient involvement. Generic FE models mimic the biomechanical behavior of the tissue, but overlook variations in geometry, loading, and material properties of a population. Conversely, subject-specific models include these factors, resulting in enhanced predictive precision, but are laborious and time intensive. The present study aimed to enhance subject-specific knee joint FE modeling by incorporating a semi-automated segmentation algorithm using a 3D Swin UNETR for an initial segmentation of the femur and tibia, followed by a statistical shape model (SSM) adjustment to improve surface roughness and continuity. For comparison, a manual FE model was developed through manual segmentation (i.e., the de-facto standard approach). Both FE models were subjected to gait loading and the predicted mechanical response was compared. The semi-automated segmentation achieved a Dice similarity coefficient (DSC) of over 98% for both the femur and tibia. Hausdorff distance (mm) between the semi-automated and manual segmentation was 1.4 mm. The mechanical results (max principal stress and strain, fluid pressure, fibril strain, and contact area) showed no significant differences between the manual and semi-automated FE models, indicating the effectiveness of the proposed semi-automated segmentation in creating accurate knee joint FE models. We have made our semi-automated models publicly accessible to support and facilitate biomechanical modeling and medical image segmentation efforts ( https://data.mendeley.com/datasets/k5hdc9cz7w/1 ).

Dual output feature fusion networks for femoral segmentation and quantitative analysis of the knee joint

BACKGROUND

Magnetic resonance imaging (MRI) is the preferred imaging modality for diagnosing knee disease. Segmentation of the knee MRI images is essential for subsequent quantification of clinical parameters and treatment planning for knee prosthesis replacement. However, the segmentation remains difficult due to individual differences in anatomy, the difficulty of obtaining accurate edges at lower resolutions, and the presence of speckle noise and artifacts in the images. In addition, radiologists must manually measure the knee's parameters which is a laborious and time-consuming process.

PURPOSE

Automatic quantification of femoral morphological parameters can be of fundamental help in the design of prosthetic implants for the repair of the knee and the femur. Knowledge of knee femoral parameters can provide a basis for femoral repair of the knee, the design of fixation materials for femoral prostheses, and the replacement of prostheses.

METHODS

This paper proposes a new deep network architecture to comprehensively address these challenges. A dual output model structure is proposed, with a high and low layer fusion extraction feature module designed to extract rich features through the cross-fusion mechanism. A multi-scale edge information extraction spatial feature module is also developed to address the boundary-blurring problem.

RESULTS

Based on the precise automated segmentation results, 10 key clinical parameters were automatically measured for a knee femoral prosthesis replacement program. The correlation coefficients of the quantitative results of these parameters compared to manual results all achieved at least 0.92. The proposed method was extensively evaluated with MRIs of 78 patients' knees, and it consistently outperformed other methods used for segmentation.

CONCLUSIONS

The automated quantization process produced comparable measurements to those manually obtained by radiologists. This paper demonstrates the viability of automatic knee MRI image segmentation and quantitative analysis with the proposed method. This provides data to support the accuracy of assessing the progression and biomechanical changes of osteoarthritis of the knee using an automated process, thus saving valuable time for the radiologists and surgeons.

Development and validation of a deep-learning model for the detection of non-displaced femoral neck fractures with anteroposterior and lateral hip radiographs

Background

Hip fractures, including femoral neck fractures, are a significant cause of morbidity and mortality in the elderly population and are typically diagnosed using plain radiography. However, diagnosing non-displaced femoral neck fractures can be challenging due to their subtle appearance on hip radiographs. Previous deep-learning models have shown low accuracy in identifying these fractures on anteroposterior (AP) radiographs; however, no studies have used lateral radiographs. This study aimed to evaluate the potential of using deep-learning with both AP and lateral hip radiographs to automatically identify non-displaced femoral neck fractures.

Methods

We conducted a retrospective analysis of patients with femoral neck fractures at The First Affiliated Hospital of Xiamen University. All the hip radiographs were reviewed, and cases of non-displaced femoral neck fractures were included in the study. Additionally, 439 participants with normal hip radiographs were also included in the study. A vision transformer (Vit) model was developed using 1,536 AP and lateral hip radiograph. The model's performance was compared to the performance of two groups of human observers: an expert group comprising orthopedic surgeons and radiologists, and a non-expert group, including emergency physicians and general practice doctors. We also carried out the external validation using two additional data sets to assess the generalizability of the model.

Results

The Vit model showed exceptional performance in detecting non-displaced femoral neck fractures on paired AP and lateral hip radiographs, achieving a binary accuracy of 95.8% [95% confidence interval (CI): 94.9%, 96.8%] and an area under the curve (AUC) of 0.988. Compared to the human observers, the model had a higher accuracy of 96.7% (95% CI: 93.9%, 99.5%) on the paired AP and lateral hip radiographs, while the accuracy of the expert group was 90.5% (95% CI: 85.7%, 95.2%). Further, the model maintained good performance during the external validation, with an AUC of 0.959 on the paired AP and lateral views.

Conclusions

Our Vit model showed expert-level performance in identifying non-displaced femoral neck fractures on paired AP and lateral hip radiographs. This model has the potential to enhance diagnosis accuracy and improve patient outcomes by reducing the need for additional examinations and preoperative time.

A Comprehensive Review on MRI-based Knee Joint Segmentation and Analysis Techniques

Using magnetic resonance imaging (MRI) in osteoarthritis pathogenesis research has proven extremely beneficial. However, it is always challenging for both clinicians and researchers to detect morphological changes in knee joints from magnetic resonance (MR) imaging since the surrounding tissues produce identical signals in MR studies, making it difficult to distinguish between them. Segmenting the knee bone, articular cartilage and menisci from the MR images allows one to examine the complete volume of the bone, articular cartilage, and menisci. It can also be used to assess certain characteristics quantitatively. However, segmentation is a laborious and time-consuming operation that requires sufficient training to complete correctly. With the advancement of MRI technology and computational methods, researchers have developed several algorithms to automate the task of individual knee bone, articular cartilage and meniscus segmentation during the last two decades. This systematic review aims to present available fully and semi-automatic segmentation methods for knee bone, cartilage, and meniscus published in different scientific articles. This review provides a vivid description of the scientific advancements to clinicians and researchers in this field of image analysis and segmentation, which helps the development of novel automated methods for clinical applications. The review also contains the recently developed fully automated deep learning-based methods for segmentation, which not only provides better results compared to the conventional techniques but also open a new field of research in Medical Imaging.

CartiMorph: A framework for automated knee articular cartilage morphometrics

We introduce CartiMorph, a framework for automated knee articular cartilage morphometrics. It takes an image as input and generates quantitative metrics for cartilage subregions, including the percentage of full-thickness cartilage loss (FCL), mean thickness, surface area, and volume. CartiMorph leverages the power of deep learning models for hierarchical image feature representation. Deep learning models were trained and validated for tissue segmentation, template construction, and template-to-image registration. We established methods for surface-normal-based cartilage thickness mapping, FCL estimation, and rule-based cartilage parcellation. Our cartilage thickness map showed less error in thin and peripheral regions. We evaluated the effectiveness of the adopted segmentation model by comparing the quantitative metrics obtained from model segmentation and those from manual segmentation. The root-mean-squared deviation of the FCL measurements was less than 8%, and strong correlations were observed for the mean thickness (Pearson's correlation coefficient ρ∈[0.82,0.97]), surface area (ρ∈[0.82,0.98]) and volume (ρ∈[0.89,0.98]) measurements. We compared our FCL measurements with those from a previous study and found that our measurements deviated less from the ground truths. We observed superior performance of the proposed rule-based cartilage parcellation method compared with the atlas-based approach. CartiMorph has the potential to promote imaging biomarkers discovery for knee osteoarthritis.

Hybrid CNN-transformer network for interactive learning of challenging musculoskeletal images

BACKGROUND AND OBJECTIVES

Segmentation of regions of interest (ROIs) such as tumors and bones plays an essential role in the analysis of musculoskeletal (MSK) images. Segmentation results can help with orthopaedic surgeons in surgical outcomes assessment and patient's gait cycle simulation. Deep learning-based automatic segmentation methods, particularly those using fully convolutional networks (FCNs), are considered as the state-of-the-art. However, in scenarios where the training data is insufficient to account for all the variations in ROIs, these methods struggle to segment the challenging ROIs that with less common image characteristics. Such characteristics might include low contrast to the background, inhomogeneous textures, and fuzzy boundaries.

METHODS

we propose a hybrid convolutional neural network - transformer network (HCTN) for semi-automatic segmentation to overcome the limitations of segmenting challenging MSK images. Specifically, we propose to fuse user-inputs (manual, e.g., mouse clicks) with high-level semantic image features derived from the neural network (automatic) where the user-inputs are used in an interactive training for uncommon image characteristics. In addition, we propose to leverage the transformer network (TN) - a deep learning model designed for handling sequence data, in together with features derived from FCNs for segmentation; this addresses the limitation of FCNs that can only operate on small kernels, which tends to dismiss global context and only focus on local patterns.

RESULTS

We purposely selected three MSK imaging datasets covering a variety of structures to evaluate the generalizability of the proposed method. Our semi-automatic HCTN method achieved a dice coefficient score (DSC) of 88.46 ± 9.41 for segmenting the soft-tissue sarcoma tumors from magnetic resonance (MR) images, 73.32 ± 11.97 for segmenting the osteosarcoma tumors from MR images and 93.93 ± 1.84 for segmenting the clavicle bones from chest radiographs. When compared to the current state-of-the-art automatic segmentation method, our HCTN method is 11.7%, 19.11% and 7.36% higher in DSC on the three datasets, respectively.

CONCLUSION

Our experimental results demonstrate that HCTN achieved more generalizable results than the current methods, especially with challenging MSK studies.

Glenoid segmentation from computed tomography scans based on a 2-stage deep learning model for glenoid bone loss evaluation

BACKGROUND

The best-fitting circle drawn by computed tomography (CT) reconstruction of the en face view of the glenoid bone to measure the bone defect is widely used in clinical application. However, there are still some limitations in practical application, which can prevent the achievement of accurate measurements. This study aimed to accurately and automatically segment the glenoid from CT scans based on a 2-stage deep learning model and to quantitatively measure the glenoid bone defect.

MATERIALS AND METHODS

Patients who were referred to our institution between June 2018 and February 2022 were retrospectively reviewed. The dislocation group consisted of 237 patients with a history of ≥2 unilateral shoulder dislocations within 2 years. The control group consisted of 248 individuals with no history of shoulder dislocation, shoulder developmental deformity, or other disease that may lead to abnormal morphology of the glenoid. All patients underwent CT examination with a 1-mm slice thickness and a 1-mm increment, including complete imaging of the bilateral glenoid. A residual neural network (ResNet) location model and a U-Net bone segmentation model were constructed to develop an automated segmentation model for the glenoid from CT scans. The data set was randomly divided into training (201 of 248) and test (47 of 248) data sets of control-group data and training (190 of 237) and test (47 of 237) data sets of dislocation-group data. The accuracy of the stage 1 (glenoid location) model, the mean intersection-over-union value of the stage 2 (glenoid segmentation) model, and the glenoid volume error were used to assess the performance of the model. The R2 value and Lin concordance correlation coefficient were used to assess the correlation between the prediction and the gold standard.

RESULTS

A total of 73,805 images were obtained after the labeling process, and each image was composed of CT images of the glenoid and its corresponding mask. The average overall accuracy of stage 1 was 99.28%; the average mean intersection-over-union value of stage 2 was 0.96. The average glenoid volume error between the predicted and true values was 9.33%. The R2 values of the predicted and true values of glenoid volume and glenoid bone loss (GBL) were 0.87 and 0.91, respectively. The Lin concordance correlation coefficient value of the predicted and true values of glenoid volume and GBL were 0.93 and 0.95, respectively.

CONCLUSION

The 2-stage model in this study showed a good performance in glenoid bone segmentation from CT scans and could quantitatively measure GBL, providing a data reference for subsequent clinical treatment.

Mixed Reality and Artificial Intelligence: A Holistic Approach to Multimodal Visualization and Extended Interaction in Knee Osteotomy

OBJECTIVE

Recent advancements in augmented reality led to planning and navigation systems for orthopedic surgery. However little is known about mixed reality (MR) in orthopedics. Furthermore, artificial intelligence (AI) has the potential to boost the capabilities of MR by enabling automation and personalization. The purpose of this work is to assess Holoknee prototype, based on AI and MR for multimodal data visualization and surgical planning in knee osteotomy, developed to run on the HoloLens 2 headset.

METHODS

Two preclinical test sessions were performed with 11 participants (eight surgeons, two residents, and one medical student) executing three times six tasks, corresponding to a number of holographic data interactions and preoperative planning steps. At the end of each session, participants answered a questionnaire on user perception and usability.

RESULTS

During the second trial, the participants were faster in all tasks than in the first one, while in the third one, the time of execution decreased only for two tasks ("Patient selection" and "Scrolling through radiograph") with respect to the second attempt, but without statistically significant difference (respectively [Formula: see text] = 0.14 and [Formula: see text] = 0.13, [Formula: see text]). All subjects strongly agreed that MR can be used effectively for surgical training, whereas 10 (90.9%) strongly agreed that it can be used effectively for preoperative planning. Six (54.5%) agreed and two of them (18.2%) strongly agreed that it can be used effectively for intraoperative guidance.

DISCUSSION/CONCLUSION

In this work, we presented Holoknee, the first holistic application of AI and MR for surgical planning for knee osteotomy. It reported promising results on its potential translation to surgical training, preoperative planning, and surgical guidance. Clinical and Translational Impact Statement - Holoknee can be helpful to support surgeons in the preoperative planning of knee osteotomy. It has the potential to impact positively the training of the future generation of residents and aid surgeons in the intraoperative stage.

Deep learning applications in osteoarthritis imaging

Deep learning (DL) is one of the most exciting new areas in medical imaging. This article will provide a review of current applications of DL in osteoarthritis (OA) imaging, including methods used for cartilage lesion detection, OA diagnosis, cartilage segmentation, and OA risk assessment. DL techniques have been shown to have similar diagnostic performance as human readers for detecting and grading cartilage lesions within the knee on MRI. A variety of DL methods have been developed for detecting and grading the severity of knee OA and various features of knee OA on X-rays using standardized classification systems with diagnostic performance similar to human readers. Multiple DL approaches have been described for fully automated segmentation of cartilage and other knee tissues and have achieved higher segmentation accuracy than currently used methods with substantial reductions in segmentation times. Various DL models analyzing baseline X-rays and MRI have been developed for OA risk assessment. These models have shown high diagnostic performance for predicting a wide variety of OA outcomes, including the incidence and progression of radiographic knee OA, the presence and progression of knee pain, and future total knee replacement. The preliminary results of DL applications in OA imaging have been encouraging. However, many DL techniques require further technical refinement to maximize diagnostic performance. Furthermore, the generalizability of DL approaches needs to be further investigated in prospective studies using large image datasets acquired at different institutions with different imaging hardware before they can be implemented in clinical practice and research studies.

Large-Scale Analysis of Meniscus Morphology as Risk Factor for Knee Osteoarthritis

OBJECTIVE

Although it is established that structural damage of the meniscus is linked to knee osteoarthritis (OA) progression, the predisposition to future development of OA because of geometric meniscal shapes is plausible and unexplored. This study aims to identify common variations in meniscal shape and determine their relationships to tissue morphology, OA onset, and longitudinal changes in cartilage thickness.

METHODS

A total of 4,790 participants from the Osteoarthritis Initiative data set were studied. A statistical shape model was developed for the meniscus, and shape scores were evaluated between a control group and an OA incidence group. Shape features were then associated with cartilage thickness changes over 8 years to localize the relationship between meniscus shape and cartilage degeneration.

RESULTS

Seven shape features between the medial and lateral menisci were identified to be different between knees that remain normal and those that develop OA. These include length-width ratios, horn lengths, root attachment angles, and concavity. These "at-risk" shapes were linked to unique cartilage thickness changes that suggest a relationship between meniscus geometry and decreased tibial coverage and rotational imbalances. Additionally, strong associations were found between meniscal shape and demographic subpopulations, future tibial extrusion, and meniscal and ligamentous tears.

CONCLUSION

This automatic method expanded upon known meniscus characteristics that are associated with the onset of OA and discovered novel shape features that have yet to be investigated in the context of OA risk.

Self-supervised-RCNN for medical image segmentation with limited data annotation

Many successful methods developed for medical image analysis based on machine learning use supervised learning approaches, which often require large datasets annotated by experts to achieve high accuracy. However, medical data annotation is time-consuming and expensive, especially for segmentation tasks. To overcome the problem of learning with limited labeled medical image data, an alternative deep learning training strategy based on self-supervised pretraining on unlabeled imaging data is proposed in this work. For the pretraining, different distortions are arbitrarily applied to random areas of unlabeled images. Next, a Mask-RCNN architecture is trained to localize the distortion location and recover the original image pixels. This pretrained model is assumed to gain knowledge of the relevant texture in the images from the self-supervised pretraining on unlabeled imaging data. This provides a good basis for fine-tuning the model to segment the structure of interest using a limited amount of labeled training data. The effectiveness of the proposed method in different pretraining and fine-tuning scenarios was evaluated based on the Osteoarthritis Initiative dataset with the aim of segmenting effusions in MRI datasets of the knee. Applying the proposed self-supervised pretraining method improved the Dice score by up to 18% compared to training the models using only the limited annotated data. The proposed self-supervised learning approach can be applied to many other medical image analysis tasks including anomaly detection, segmentation, and classification.

Understanding the role and adoption of artificial intelligence techniques in rheumatology research: An in-depth review of the literature

The major and upward trend in the number of published research related to rheumatic and musculoskeletal diseases, in which artificial intelligence plays a key role, has exhibited the interest of rheumatology researchers in using these techniques to answer their research questions. In this review, we analyse the original research articles that combine both worlds in a five- year period (2017-2021). In contrast to other published papers on the same topic, we first studied the review and recommendation articles that were published during that period, including up to October 2022, as well as the publication trends. Secondly, we review the published research articles and classify them into one of the following categories: disease identification and prediction, disease classification, patient stratification and disease subtype identification, disease progression and activity, treatment response, and predictors of outcomes. Thirdly, we provide a table with illustrative studies in which artificial intelligence techniques have played a central role in more than twenty rheumatic and musculoskeletal diseases. Finally, the findings of the research articles, in terms of disease and/or data science techniques employed, are highlighted in a discussion. Therefore, the present review aims to characterise how researchers are applying data science techniques in the rheumatology medical field. The most immediate conclusions that can be drawn from this work are: multiple and novel data science techniques have been used in a wide range of rheumatic and musculoskeletal diseases including rare diseases; the sample size and the data type used are heterogeneous, and new technical approaches are expected to arrive in the short-middle term.

Deep learning-based automatic pipeline for quantitative assessment of thigh muscle morphology and fatty infiltration

PURPOSE

Fast and accurate thigh muscle segmentation from MRI is important for quantitative assessment of thigh muscle morphology and composition. A novel deep learning (DL) based thigh muscle and surrounding tissues segmentation model was developed for fully automatic and reproducible cross-sectional area (CSA) and fat fraction (FF) quantification and tested in patients at 10 years after anterior cruciate ligament reconstructions.

METHODS

A DL model combining UNet and DenseNet was trained and tested using manually segmented thighs from 16 patients (32 legs). Segmentation accuracy was evaluated using Dice similarity coefficients (DSC) and average symmetric surface distance (ASSD). A UNet model was trained for comparison. These segmentations were used to obtain CSA and FF quantification. Reproducibility of CSA and FF quantification was tested with scan and rescan of six healthy subjects.

RESULTS

The proposed UNet and DenseNet had high agreement with manual segmentation (DSC >0.97, ASSD < 0.24) and improved performance compared with UNet. For hamstrings of the operated knee, the automated pipeline had largest absolute difference of 6.01% for CSA and 0.47% for FF as compared to manual segmentation. In reproducibility analysis, the average difference (absolute) in CSA quantification between scan and rescan was better for the automatic method as compared with manual segmentation (2.27% vs. 3.34%), whereas the average difference (absolute) in FF quantification were similar.

CONCLUSIONS

The proposed method exhibits excellent accuracy and reproducibility in CSA and FF quantification compared with manual segmentation and can be used in large-scale patient studies.

Challenges of Integrating New Technologies for Orthopedic Doctors to Face up to Difficulties during the Pandemic Era

In the field of orthopedics, competitive progress is growing faster because new technologies used to facilitate the work of physicians are continuously developing. Based on the issues generated in the pandemic era in this field, a research study was developed to identify the intention of orthopedic doctors to integrate new medical technologies. The survey was based on a questionnaire that was used for data collection. The quantitative study registered a sample of 145 orthopedic doctors. The data analysis was performed based on the IBM SPSS program. A multiple linear regression model was applied, which analyzed how the independent variables can influence the dependent variables. After analyzing the data, it was observed that the intention of orthopedic doctors to use new medical technologies is influenced by the advantages and disadvantages perceived by them, the perceived risks, the quality of the medical technologies, the experience of physicians in their use, and their receptivity to other digital tools. The obtained results are highly important both for hospital managers and authorities, illustrating the main factors that influence doctors to use emergent technologies in their clinical work.

Magnetic resonance imaging assessments for knee segmentation and their use in combination with machine/deep learning as predictors of early osteoarthritis diagnosis and prognosis

Knee osteoarthritis (OA) is a prevalent and disabling disease that can develop over decades. This disease is heterogeneous and involves structural changes in the whole joint, encompassing multiple tissue types. Detecting OA before the onset of irreversible changes is crucial for early management, and this could be achieved by allowing knee tissue visualization and quantifying their changes over time. Although some imaging modalities are available for knee structure assessment, magnetic resonance imaging (MRI) is preferred. This narrative review looks at existing literature, first on MRI-developed approaches for evaluating knee articular tissues, and second on prediction using machine/deep-learning-based methodologies and MRI as input or outcome for early OA diagnosis and prognosis. A substantial number of MRI methodologies have been developed to assess several knee tissues in a semi-quantitative and quantitative fashion using manual, semi-automated and fully automated systems. This dynamic field has grown substantially since the advent of machine/deep learning. Another active area is predictive modelling using machine/deep-learning methodologies enabling robust early OA diagnosis/prognosis. Moreover, incorporating MRI markers as input/outcome in such predictive models is important for a more accurate OA structural diagnosis/prognosis. The main limitation of their usage is the ability to move them in rheumatology practice. In conclusion, MRI knee tissue determination and quantification provide early indicators for individuals at high risk of developing this disease or for patient prognosis. Such assessment of knee tissues, combined with the development of models/tools from machine/deep learning using, in addition to other parameters, MRI markers for early diagnosis/prognosis, will maximize opportunities for individualized risk assessment for use in clinical practice permitting precision medicine. Future efforts should be made to integrate such prediction models into open access, allowing early disease management to prevent or delay the OA outcome.

Deep learning-assisted diagnosis of femoral trochlear dysplasia based on magnetic resonance imaging measurements

BACKGROUND

Femoral trochlear dysplasia (FTD) is an important risk factor for patellar instability. Dejour classification is widely used at present and relies on standard lateral X-rays, which are not common in clinical work. Therefore, magnetic resonance imaging (MRI) has become the first choice for the diagnosis of FTD. However, manually measuring is tedious, time-consuming, and easily produces great variability.

AIM

To use artificial intelligence (AI) to assist diagnosing FTD on MRI images and to evaluate its reliability.

METHODS

We searched 464 knee MRI cases between January 2019 and December 2020, including FTD (n = 202) and normal trochlea (n = 252). This paper adopts the heatmap regression method to detect the key points network. For the final evaluation, several metrics (accuracy, sensitivity, specificity, etc.) were calculated.

RESULTS

The accuracy, sensitivity, specificity, positive predictive value and negative predictive value of the AI model ranged from 0.74-0.96. All values were superior to junior doctors and intermediate doctors, similar to senior doctors. However, diagnostic time was much lower than that of junior doctors and intermediate doctors.

CONCLUSION

The diagnosis of FTD on knee MRI can be aided by AI and can be achieved with a high level of accuracy.

Automated segmentation of the healed anterior cruciate ligament from T2 * relaxometry MRI scans

Collagen organization of the anterior cruciate ligament (ACL) can be evaluated using T2 * relaxometry. However, T2 * mapping requires manual image segmentation, which is a time-consuming process and prone to inter- and intra- segmenter variability. Automating segmentation would address these challenges. A model previously trained using Constructive Interference in Steady State (CISS) scans was applied to T2 * segmentation via transfer learning. It was hypothesized that there would be no significant differences in the model's segmentation performance between T2 * and CISS, structural measures versus ground truth manual segmentation, and reliability versus independent and retest manual segmentation. Transfer learning was conducted using 54 T2 * scans of the ACL. Segmentation performance was assessed with Dice coefficient, precision, and sensitivity, and structurally with T2 * value, volume, subvolume proportions, and cross-sectional area. Model performance relative to independent manual segmentation and repeated segmentation by the ground truth segmenter (retest) were evaluated on a random subset. Segmentation performance was analyzed with Mann-Whitney U tests, structural measures with Wilcoxon signed-rank tests, and performance relative to manual segmentation with repeated-measures analysis of variance/Tukey tests (α = 0.05). T2 * segmentation performance was not significantly different from CISS on all measures (p > 0.35). No significant differences were detected in structural measures (p > 0.50). Automatic segmentation performed as well as the retest on all segmentation measures, whereas independent segmentations were lower than retest and/or automatic segmentation (p < 0.023). Structural measures were not significantly different between segmenters. The automatic segmentation model performed as well on the T2 * sequence as on CISS and outperformed independent manual segmentation while performing as well as retest segmentation.

Auto-segmentation of the tibia and femur from knee MR images via deep learning and its application to cartilage strain and recovery

The ability to efficiently and reproducibly generate subject-specific 3D models of bone and soft tissue is important to many areas of musculoskeletal research. However, methodologies requiring such models have largely been limited by lengthy manual segmentation times. Recently, machine learning, and more specifically, convolutional neural networks, have shown potential to alleviate this bottleneck in research throughput. Thus, the purpose of this work was to develop a modified version of the convolutional neural network architecture U-Net to automate segmentation of the tibia and femur from double echo steady state knee magnetic resonance (MR) images. Our model was trained on a dataset of over 4,000 MR images from 34 subjects, segmented by three experienced researchers, and reviewed by a musculoskeletal radiologist. For our validation and testing sets, we achieved dice coefficients of 0.985 and 0.984, respectively. As further testing, we applied our trained model to a prior study of tibial cartilage strain and recovery. In this analysis, across all subjects, there were no statistically significant differences in cartilage strain between the machine learning and ground truth bone models, with a mean difference of 0.2 ± 0.7 % (mean ± 95 % confidence interval). This difference is within the measurement resolution of previous cartilage strain studies from our lab using manual segmentation. In summary, we successfully trained, validated, and tested a machine learning model capable of segmenting MR images of the knee, achieving results that are comparable to trained human segmenters.

Deep Learning-Based Medical Images Segmentation of Musculoskeletal Anatomical Structures: A Survey of Bottlenecks and Strategies

By leveraging the recent development of artificial intelligence algorithms, several medical sectors have benefited from using automatic segmentation tools from bioimaging to segment anatomical structures. Segmentation of the musculoskeletal system is key for studying alterations in anatomical tissue and supporting medical interventions. The clinical use of such tools requires an understanding of the proper method for interpreting data and evaluating their performance. The current systematic review aims to present the common bottlenecks for musculoskeletal structures analysis (e.g., small sample size, data inhomogeneity) and the related strategies utilized by different authors. A search was performed using the PUBMED database with the following keywords: deep learning, musculoskeletal system, segmentation. A total of 140 articles published up until February 2022 were obtained and analyzed according to the PRISMA framework in terms of anatomical structures, bioimaging techniques, pre/post-processing operations, training/validation/testing subset creation, network architecture, loss functions, performance indicators and so on. Several common trends emerged from this survey; however, the different methods need to be compared and discussed based on each specific case study (anatomical region, medical imaging acquisition setting, study population, etc.). These findings can be used to guide clinicians (as end users) to better understand the potential benefits and limitations of these tools.

Automatic segmentation of human knee anatomy by a convolutional neural network applying a 3D MRI protocol

BACKGROUND

To study deep learning segmentation of knee anatomy with 13 anatomical classes by using a magnetic resonance (MR) protocol of four three-dimensional (3D) pulse sequences, and evaluate possible clinical usefulness.

METHODS

The sample selection involved 40 healthy right knee volumes from adult participants. Further, a recently injured single left knee with previous known ACL reconstruction was included as a test subject. The MR protocol consisted of the following 3D pulse sequences: T1 TSE, PD TSE, PD FS TSE, and Angio GE. The DenseVNet neural network was considered for these experiments. Five input combinations of sequences (i) T1, (ii) T1 and FS, (iii) PD and FS, (iv) T1, PD, and FS and (v) T1, PD, FS and Angio were trained using the deep learning algorithm. The Dice similarity coefficient (DSC), Jaccard index and Hausdorff were used to compare the performance of the networks.

RESULTS

Combining all sequences collectively performed significantly better than other alternatives. The following DSCs (±standard deviation) were obtained for the test dataset: Bone medulla 0.997 (±0.002), PCL 0.973 (±0.015), ACL 0.964 (±0.022), muscle 0.998 (±0.001), cartilage 0.966 (±0.018), bone cortex 0.980 (±0.010), arteries 0.943 (±0.038), collateral ligaments 0.919 (± 0.069), tendons 0.982 (±0.005), meniscus 0.955 (±0.032), adipose tissue 0.998 (±0.001), veins 0.980 (±0.010) and nerves 0.921 (±0.071). The deep learning network correctly identified the anterior cruciate ligament (ACL) tear of the left knee, thus indicating a future aid to orthopaedics.

CONCLUSIONS

The convolutional neural network proves highly capable of correctly labeling all anatomical structures of the knee joint when applied to 3D MR sequences. We have demonstrated that this deep learning model is capable of automatized segmentation that may give 3D models and discover pathology. Both useful for a preoperative evaluation.

Application of artificial intelligence to imaging interpretations in the musculoskeletal area: Where are we? Where are we going?

The interest of researchers, clinicians and radiologists, in artificial intelligence (AI) continues to grow. Deep learning is a subset of machine learning, in which the computer algorithm itself can determine the optimal imaging features to answer a clinical question. Convolutional neural networks are the most common architecture for performing deep learning on medical images. The various musculoskeletal applications of deep learning are the detection of abnormalities on X-rays or cross-sectional images (CT, MRI), for example the detection of fractures, meniscal tears, anterior cruciate ligament tears, degenerative lesions of the spine, bone metastases, classification of e.g., dural sac stenosis, degeneration of intervertebral discs, assessment of skeletal age, and segmentation, for example of cartilage. Software developments are already impacting the daily practice of orthopedic imaging by automatically detecting fractures on radiographs. Improving image acquisition protocols, improving the quality of low-dose CT images, reducing acquisition times in MRI, or improving MR image resolution is possible through deep learning. Deep learning offers an automated way to offload time-consuming manual processes and improve practitioner performance. This article reviews the current state of AI in musculoskeletal imaging.

Prediction model for knee osteoarthritis using magnetic resonance-based radiomic features from the infrapatellar fat pad: data from the osteoarthritis initiative

Background

The infrapatellar fat pad (IPFP) plays an important role in the incidence of knee osteoarthritis (OA). Magnetic resonance (MR) signal heterogeneity of the IPFP is related to pathologic changes. In this study, we aimed to investigate whether the IPFP radiomic features have predictive value for incident radiographic knee OA (iROA) 1 year prior to iROA diagnosis.

Methods

Data used in this work were obtained from the osteoarthritis initiative (OAI). In this study, iROA was defined as a knee with a baseline Kellgren-Lawrence grade (KLG) of 0 or 1 that further progressed to KLG ≥2 during the follow-up visit. Intermediate-weighted turbo spin-echo knee MR images at the time of iROA diagnosis and 1 year prior were obtained. Five clinical characteristics-age, sex, body mass index, knee injury history, and knee surgery history-were obtained. A total of 604 knees were selected and matched (302 cases and 302 controls). A U-Net segmentation model was independently trained to automatically segment the IPFP. The prediction models were established in the training set (60%). Three main models were generated using (I) clinical characteristics; (II) radiomic features; (III) combined (clinical plus radiomic) features. Model performance was evaluated in an independent testing set (remaining 40%) using the area under the curve (AUC). Two secondary models were also generated using Hoffa-synovitis scores and clinical characteristics.

Results

The comparison between the automated and manual segmentations of the IPFP achieved a Dice coefficient of 0.900 (95% CI: 0.891-0.908), which was comparable to that of experienced radiologists. The radiomic features model and the combined model yielded superior AUCs of 0.700 (95% CI: 0.630-0.763) and 0.702 (95% CI: 0.635-0.763), respectively. The DeLong test found no statistically significant difference between the receiver operating curves of the radiomic and combined models (P=0.831); however, both models outperformed the clinical model (P=0.014 and 0.004, respectively).

Conclusions

Our results demonstrated that radiomic features of the IPFP are predictive of iROA 1 year prior to the diagnosis, suggesting that IPFP radiomic features can serve as an early quantitative prediction biomarker of iROA.

Application of artificial intelligence in medical technologies: A systematic review of main trends

Objective

Artificial intelligence (AI) has been increasingly applied in various fields of science and technology. In line with the current research, medicine involves an increasing number of artificial intelligence technologies. The introduction of rapid AI can lead to positive and negative effects. This is a multilateral analytical literature review aimed at identifying the main branches and trends in the use of using artificial intelligence in medical technologies.

Methods

The total number of literature sources reviewed is n = 89, and they are analyzed based on the literature reporting evidence-based guideline PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) for a systematic review.

Results

As a result, from the initially selected 198 references, 155 references were obtained from the databases and the remaining 43 sources were found on open internet as direct links to publications. Finally, 89 literature sources were evaluated after exclusion of unsuitable references based on the duplicated and generalized information without focusing on the users.

Conclusions

This article is identifying the current state of artificial intelligence in medicine and prospects for future use. The findings of this review will be useful for healthcare and AI professionals for improving the circulation and use of medical AI from design to implementation stage.

Performance of a deep convolutional neural network for MRI-based vertebral body measurements and insufficiency fracture detection

OBJECTIVES

The aim is to validate the performance of a deep convolutional neural network (DCNN) for vertebral body measurements and insufficiency fracture detection on lumbar spine MRI.

METHODS

This retrospective analysis included 1000 vertebral bodies in 200 patients (age 75.2 ± 9.8 years) who underwent lumbar spine MRI at multiple institutions. 160/200 patients had ≥ one vertebral body insufficiency fracture, 40/200 had no fracture. The performance of the DCNN and that of two fellowship-trained musculoskeletal radiologists in vertebral body measurements (anterior/posterior height, extent of endplate concavity, vertebral angle) and evaluation for insufficiency fractures were compared. Statistics included (a) interobserver reliability metrics using intraclass correlation coefficient (ICC), kappa statistics, and Bland-Altman analysis, and (b) diagnostic performance metrics (sensitivity, specificity, accuracy). A statistically significant difference was accepted if the 95% confidence intervals did not overlap.

RESULTS

The inter-reader agreement between radiologists and the DCNN was excellent for vertebral body measurements, with ICC values of > 0.94 for anterior and posterior vertebral height and vertebral angle, and good to excellent for superior and inferior endplate concavity with ICC values of 0.79-0.85. The performance of the DCNN in fracture detection yielded a sensitivity of 0.941 (0.903-0.968), specificity of 0.969 (0.954-0.980), and accuracy of 0.962 (0.948-0.973). The diagnostic performance of the DCNN was independent of the radiological institution (accuracy 0.964 vs. 0.960), type of MRI scanner (accuracy 0.957 vs. 0.964), and magnetic field strength (accuracy 0.966 vs. 0.957).

CONCLUSIONS

A DCNN can achieve high diagnostic performance in vertebral body measurements and insufficiency fracture detection on heterogeneous lumbar spine MRI.

KEY POINTS

• A DCNN has the potential for high diagnostic performance in measuring vertebral bodies and detecting insufficiency fractures of the lumbar spine.

A novel multi-atlas segmentation approach under the semi-supervised learning framework: Application to knee cartilage segmentation

BACKGROUND AND OBJECTIVE

Multi-atlas based segmentation techniques, which rely on an atlas library comprised of training images labeled by an expert, have proven their effectiveness in multiple automatic segmentation applications. However, the usage of exhaustive patch libraries combined with the voxel-wise labeling incur a large computational cost in terms of memory requirements and execution times.

METHODS

To confront this shortcoming, we propose a novel two-stage multi-atlas approach designed under the Semi-Supervised Learning (SSL) framework. The main properties of our method are as follows: First, instead of the voxel-wise labeling approach, the labeling of target voxels is accomplished here by exploiting the spectral content of globally sampled datasets from the target image, along with their spatially correspondent data collected from the atlases. Following SSL, voxels classification is boosted by incorporating unlabeled data from the target image, in addition to the labeled ones from atlas library. Our scheme integrates constructively fruitful concepts, including sparse reconstructions of voxels from linear neighborhoods, HOG feature descriptors of patches/regions, and label propagation via sparse graph constructions. Segmentation of the target image is carried out in two stages: stage-1 focuses on the sampling and labeling of global data, while stage-2 undertakes the above tasks for the out-of-sample data. Finally, we propose different graph-based methods for the labeling of global data, while these methods are extended to deal with the out-of-sample voxels.

RESULTS

A thorough experimental investigation is conducted on 76 subjects provided by the publicly accessible Osteoarthritis Initiative (OAI) repository. Comparative results and statistical analysis demonstrate that the suggested methodology exhibits superior segmentation performance compared to the existing patch-based methods, across all evaluation metrics (DSC:88.89%, Precision: 89.86%, Recall: 88.12%), while at the same time it requires a considerably reduced computational load (>70% reduction on average execution time with respect to other patch-based). In addition, our approach is favorably compared against other non patch-based and deep learning methods in terms of performance accuracy (on the 3-class problem). A final experimentation on a 5-class setting of the problems demonstrates that our approach is capable of achieving performance comparable to existing state-of-the-art knee cartilage segmentation methods (DSC:88.22% and DSC:85.84% for femoral and tibial cartilage respectively).

Efficient cascaded V-net optimization for lower extremity CT segmentation validated using bone morphology assessment

Semantic segmentation of bone from lower extremity computerized tomography (CT) scans can improve and accelerate the visualization, diagnosis, and surgical planning in orthopaedics. However, the large field of view of these scans makes automatic segmentation using deep learning based methods challenging, slow and graphical processing unit (GPU) memory intensive. We investigated methods to more efficiently represent anatomical context for accurate and fast segmentation and compared these with state-of-the-art methodology. Six lower extremity bones from patients of two different datasets were manually segmented from CT scans, and used to train and optimize a cascaded deep learning approach. We varied the number of resolution levels, receptive fields, patch sizes, and number of V-net blocks. The best performing network used a multi-stage, cascaded V-net approach with 128³ -64³ -32³ voxel patches as input. The average Dice coefficient over all bones was 0.98 ± 0.01, the mean surface distance was 0.26 ± 0.12 mm and the 95th percentile Hausdorff distance 0.65 ± 0.28 mm. This was a significant improvement over the results of the state-of-the-art nnU-net, with only approximately 1/12th of training time, 1/3th of inference time and 1/4th of GPU memory required. Comparison of the morphometric measurements performed on automatic and manual segmentations showed good correlation (Intraclass Correlation Coefficient [ICC] >0.8) for the alpha angle and excellent correlation (ICC >0.95) for the hip-knee-ankle angle, femoral inclination, femoral version, acetabular version, Lateral Centre-Edge angle, acetabular coverage. The segmentations were generally of sufficient quality for the tested clinical applications and were performed accurately and quickly compared to state-of-the-art methodology from the literature.

Anterior Cruciate Ligament Tear Detection Based on Deep Convolutional Neural Network

Anterior cruciate ligament (ACL) tear is very common in football players, volleyball players, sprinters, runners, etc. It occurs frequently due to extra stretching and sudden movement and causes extreme pain to the patient. Various computer vision-based techniques have been employed for ACL tear detection, but the performance of most of these systems is challenging because of the complex structure of knee ligaments. This paper presents a three-layered compact parallel deep convolutional neural network (CPDCNN) to enhance the feature distinctiveness of the knee MRI images for anterior cruciate ligament (ACL) tear detection in knee MRI images. The performance of the proposed approach is evaluated for the MRNet knee images dataset using accuracy, recall, precision, and the F1 score. The proposed CPDCNN offers an overall accuracy of 96.60%, a recall rate of 0.9668, a precision of 0.9654, and an F1 score of 0.9582, which shows superiority over the existing state-of-the-art methods for knee tear detection.

Automated detection of knee cystic lesions on magnetic resonance imaging using deep learning

Background

Cystic lesions are frequently observed in knee joint diseases and are usually associated with joint pain, degenerative disorders, or acute injury. Magnetic resonance imaging-based, artificial intelligence-assisted cyst detection is an effective method to improve the whole knee joint analysis. However, few studies have investigated this method. This study is the first attempt at auto-detection of knee cysts based on deep learning methods.

Methods

This retrospective study collected data from 282 subjects with knee cysts confirmed at our institution from January to October 2021. A Squeeze-and-Excitation (SE) inception attention-based You only look once version 5 (SE-YOLOv5) model was developed based on a self-attention mechanism for knee cyst-like lesion detection and differentiation from knee effusions, both characterized by high T2-weighted signals in magnetic resonance imaging (MRI) scans. Model performance was evaluated via metrics including accuracy, precision, recall, mean average precision (mAP), F1 score, and frames per second (fps).

Results

The deep learning model could accurately identify knee MRI scans and auto-detect both obvious cyst lesions and small ones with inconspicuous contrasts. The SE-YOLO V5 model constructed in this study yielded superior performance (F1 = 0.879, precision = 0.887, recall = 0.872, all class mAP0.5 = 0.944, effusion mAP = 0.945, cyst mAP = 0.942) and improved detection speed compared to a traditional YOLO model.

Conclusion

This proof-of-concept study examined whether deep learning models could detect knee cysts and distinguish them from knee effusions. The results demonstrated that the classical Yolo V5 and proposed SE-Yolo V5 models could accurately identify cysts.

Deep collaborative network with alpha matte for precise knee tissue segmentation from MRI

BACKGROUND AND OBJECTIVE

Precise segmentation of knee tissues from magnetic resonance imaging (MRI) is critical in quantitative imaging and diagnosis. Convolutional neural networks (CNNs), being state of the art, often challenged by the lack of image-specific adaptation, such as low tissue contrasts and structural inhomogeneities, thereby leading to incomplete segmentation results.

METHODS

This paper presents a deep learning-based automatic segmentation framework for precise knee tissue segmentation. A novel deep collaborative method is proposed, which consists of an encoder-decoder-based segmentation network in combination with a low rank tensor-reconstructed segmentation network. Low rank reconstruction in MRI tensor sub-blocks is introduced to exploit the morphological variations in knee tissues. To model the tissue boundary regions and effectively utilize the superimposed regions, trimap generation is proposed for defining high, medium and low confidence regions from the multipath CNNs. The secondary path with low rank reconstructed input mitigates the conditions in which the primary segmentation network can potentially fail and overlook the boundary regions. The outcome of the segmentation is solved as an alpha matting problem by blending the trimap with the source input.

RESULTS

Experiments on Osteoarthritis Initiative (OAI) datasets with all the 6 musculoskeletal tissues provide an overall segmentation dice score of 0.8925, where Femoral and Tibial part of cartilage achieving an average dice exceeding 0.9. The volumetric metrics also indicate the superior performances in all tissue compartments.

CONCLUSIONS

Experiments on Osteoarthritis Initiative (OAI) datasets and a self-prepared scan validate the effectiveness of the proposed method. Inclusion of extra prediction scale allowed the model to distinguish and segment the tissue boundary accurately. We specifically demonstrate the application of the proposed method in a cartilage segmentation-based thickness map for diagnosis purposes.

Deep Learning to Detect Triangular Fibrocartilage Complex Injury in Wrist MRI: Retrospective Study with Internal and External Validation

Objective: To use deep learning to predict the probability of triangular fibrocartilage complex (TFCC) injury in patients’ MRI scans. Methods: We retrospectively studied medical records over 11 years and 2 months (1 January 2009–29 February 2019), collecting 332 contrast-enhanced hand MRI scans showing TFCC injury (143 scans) or not (189 scans) from a general hospital. We employed two convolutional neural networks with the MRNet (Algorithm 1) and ResNet50 (Algorithm 2) framework for deep learning. Explainable artificial intelligence was used for heatmap analysis. We tested deep learning using an external dataset containing the MRI scans of 12 patients with TFCC injuries and 38 healthy subjects. Results: In the internal dataset, Algorithm 1 had an AUC of 0.809 (95% confidence interval—CI: 0.670–0.947) for TFCC injury detection as well as an accuracy, sensitivity, and specificity of 75.6% (95% CI: 0.613–0.858), 66.7% (95% CI: 0.438–0.837), and 81.5% (95% CI: 0.633–0.918), respectively, and an F1 score of 0.686. Algorithm 2 had an AUC of 0.871 (95% CI: 0.747–0.995) for TFCC injury detection and an accuracy, sensitivity, and specificity of 90.7% (95% CI: 0.787–0.962), 88.2% (95% CI: 0.664–0.966), and 92.3% (95% CI: 0.763–0.978), respectively, and an F1 score of 0.882. The accuracy, sensitivity, and specificity for radiologist 1 were 88.9, 94.4 and 85.2%, respectively, and for radiologist 2, they were 71.1, 100 and 51.9%, respectively. Conclusions: A modified MRNet framework enables the detection of TFCC injury and guides accurate diagnosis.

Knee Bone and Cartilage Segmentation Based on a 3D Deep Neural Network Using Adversarial Loss for Prior Shape Constraint

Fast and accurate segmentation of knee bone and cartilage on MRI images is becoming increasingly important in the orthopaedic area, as the segmentation is an essential prerequisite step to a patient-specific diagnosis, optimising implant design and preoperative and intraoperative planning. However, manual segmentation is time-intensive and subjected to inter- and intra-observer variations. Hence, in this study, a three-dimensional (3D) deep neural network using adversarial loss was proposed to automatically segment the knee bone in a resampled image volume in order to enlarge the contextual information and incorporate prior shape constraints. A restoration network was proposed to further improve the bone segmentation accuracy by restoring the bone segmentation back to the original resolution. A conventional U-Net-like network was used to segment the cartilage. The ultimate results were the combination of the bone and cartilage outcomes through post-processing. The quality of the proposed method was thoroughly assessed using various measures for the dataset from the Grand Challenge Segmentation of Knee Images 2010 (SKI10), together with a comparison with a baseline network U-Net. A fine-tuned U-Net-like network can achieve state-of-the-art results without any post-processing operations. This method achieved a total score higher than 76 in terms of the SKI10 validation dataset. This method showed to be robust to extract bone and cartilage masks from the MRI dataset, even for the pathological case.

Design and validation of a semi-automatic bone segmentation algorithm from MRI to improve research efficiency

Segmentation of medical images into different tissue types is essential for many advancements in orthopaedic research; however, manual segmentation techniques can be time- and cost-prohibitive. The purpose of this work was to develop a semi-automatic segmentation algorithm that leverages gradients in spatial intensity to isolate the patella bone from magnetic resonance (MR) images of the knee that does not require a training set. The developed algorithm was validated in a sample of four human participants (in vivo) and three porcine stifle joints (ex vivo) using both magnetic resonance imaging (MRI) and computed tomography (CT). We assessed the repeatability (expressed as mean ± standard deviation) of the semi-automatic segmentation technique on: (1) the same MRI scan twice (Dice similarity coefficient = 0.988 ± 0.002; surface distance = - 0.01 ± 0.001 mm), (2) the scan/re-scan repeatability of the segmentation technique (surface distance = - 0.02 ± 0.03 mm), (3) how the semi-automatic segmentation technique compared to manual MRI segmentation (surface distance = - 0.02 ± 0.08 mm), and (4) how the semi-automatic segmentation technique compared when applied to both MRI and CT images of the same specimens (surface distance = - 0.02 ± 0.06 mm). Mean surface distances perpendicular to the cartilage surface were computed between pairs of patellar bone models. Critically, the semi-automatic segmentation algorithm developed in this work reduced segmentation time by approximately 75%. This method is promising for improving research throughput and potentially for use in generating training data for deep learning algorithms.

Automated knee cartilage segmentation for heterogeneous clinical MRI using generative adversarial networks with transfer learning

BACKGROUND

This study aimed to build a deep learning model to automatically segment heterogeneous clinical MRI scans by optimizing a pre-trained model built from a homogeneous research dataset with transfer learning.

METHODS

Conditional generative adversarial networks pretrained on the Osteoarthritis Initiative MR images was transferred to 30 sets of heterogenous MR images collected from clinical routines. Two trained radiologists manually segmented the 30 sets of clinical MR images for model training, validation and test. The model performance was compared to models trained from scratch with different datasets, as well as two radiologists. A 5-fold cross validation was performed.

RESULTS

The transfer learning model obtained an overall averaged Dice coefficient of 0.819, an averaged 95 percentile Hausdorff distance of 1.463 mm, and an averaged average symmetric surface distance of 0.350 mm on the 5 random holdout test sets. A 5-fold cross validation had a mean Dice coefficient of 0.801, mean 95 percentile Hausdorff distance of 1.746 mm, and mean average symmetric surface distance of 0.364 mm. It outperformed other models and performed similarly as the radiologists.

CONCLUSIONS

A transfer learning model was able to automatically segment knee cartilage, with performance comparable to human, using heterogeneous clinical MR images with a small training data size. In addition, the model proved robust when tested through cross validation and on images from a different vendor. We found it feasible to perform fully automated cartilage segmentation of clinical knee MR images, which would facilitate the clinical application of quantitative MRI techniques and other prediction models for improved patient treatment planning.

Automatic MRI segmentation of pectoralis major muscle using deep learning

To develop and validate a deep convolutional neural network (CNN) method capable of selecting the greatest Pectoralis Major Cross-Sectional Area (PMM-CSA) and automatically segmenting PMM on an axial Magnetic Resonance Imaging (MRI). We hypothesized a CNN technique can accurately perform both tasks compared with manual reference standards. Our method is based on two steps: (A) segmentation model, (B) PMM-CSA selection. In step A, we manually segmented the PMM on 134 axial T1-weighted PM MRIs. The segmentation model was trained from scratch (MONAI/Pytorch SegResNet, 4 mini-batch, 1000 epochs, dropout 0.20, Adam, learning rate 0.0005, cosine annealing, softmax). Mean-dice score determined the segmentation score on 8 internal axial T1-weighted PM MRIs. In step B, we used the OpenCV2 (version 4.5.1, https://opencv.org) framework to calculate the PMM-CSA of the model predictions and ground truth. Then, we selected the top-3 slices with the largest cross-sectional area and compared them with the ground truth. If one of the selected was in the top-3 from the ground truth, then we considered it to be a success. A top-3 accuracy evaluated this method on 8 axial T1-weighted PM MRIs internal test cases. The segmentation model (Step A) produced an accurate pectoralis muscle segmentation with a Mean Dice score of 0.94 ± 0.01. The results of Step B showed top-3 accuracy > 98% to select an appropriate axial image with the greatest PMM-CSA. Our results show an overall accurate selection of PMM-CSA and automated PM muscle segmentation using a combination of deep CNN algorithms.

Discovering Knee Osteoarthritis Imaging Features for Diagnosis and Prognosis: Review of Manual Imaging Grading and Machine Learning Approaches

Knee osteoarthritis (OA) is a deliberating joint disorder characterized by cartilage loss that can be captured by imaging modalities and translated into imaging features. Observing imaging features is a well-known objective assessment for knee OA disorder. However, the variety of imaging features is rarely discussed. This study reviews knee OA imaging features with respect to different imaging modalities for traditional OA diagnosis and updates recent image-based machine learning approaches for knee OA diagnosis and prognosis. Although most studies recognized X-ray as standard imaging option for knee OA diagnosis, the imaging features are limited to bony changes and less sensitive to short-term OA changes. Researchers have recommended the usage of MRI to study the hidden OA-related radiomic features in soft tissues and bony structures. Furthermore, ultrasound imaging features should be explored to make it more feasible for point-of-care diagnosis. Traditional knee OA diagnosis mainly relies on manual interpretation of medical images based on the Kellgren-Lawrence (KL) grading scheme, but this approach is consistently prone to human resource and time constraints and less effective for OA prevention. Recent studies revealed the capability of machine learning approaches in automating knee OA diagnosis and prognosis, through three major tasks: knee joint localization (detection and segmentation), classification of OA severity, and prediction of disease progression. AI-aided diagnostic models improved the quality of knee OA diagnosis significantly in terms of time taken, reproducibility, and accuracy. Prognostic ability was demonstrated by several prediction models in terms of estimating possible OA onset, OA deterioration, progressive pain, progressive structural change, progressive structural change with pain, and time to total knee replacement (TKR) incidence. Despite research gaps, machine learning techniques still manifest huge potential to work on demanding tasks such as early knee OA detection and estimation of future disease events, as well as fundamental tasks such as discovering the new imaging features and establishment of novel OA status measure. Continuous machine learning model enhancement may favour the discovery of new OA treatment in future.

Automated Knee MR Images Segmentation of Anterior Cruciate Ligament Tears

The anterior cruciate ligament (ACL) is one of the main stabilizer parts of the knee. ACL injury leads to causes of osteoarthritis risk. ACL rupture is common in the young athletic population. Accurate segmentation at an early stage can improve the analysis and classification of anterior cruciate ligaments tears. This study automatically segmented the anterior cruciate ligament (ACL) tears from magnetic resonance imaging through deep learning. The knee mask was generated on the original Magnetic Resonance (MR) images to apply a semantic segmentation technique with convolutional neural network architecture U-Net. The proposed segmentation method was measured by accuracy, intersection over union (IoU), dice similarity coefficient (DSC), precision, recall and F1-score of 98.4%, 99.0%, 99.4%, 99.6%, 99.6% and 99.6% on 11451 training images, whereas on the validation images of 3817 was, respectively, 97.7%, 93.8%,96.8%, 96.5%, 97.3% and 96.9%. We also provide dice loss of training and test datasets that have remained 0.005 and 0.031, respectively. The experimental results show that the ACL segmentation on JPEG MRI images with U-Nets achieves accuracy that outperforms the human segmentation. The strategy has promising potential applications in medical image analytics for the segmentation of knee ACL tears for MR images.

AI musculoskeletal clinical applications: how can AI increase my day-to-day efficiency?

Artificial intelligence (AI) is expected to bring greater efficiency in radiology by performing tasks that would otherwise require human intelligence, also at a much faster rate than human performance. In recent years, milestone deep learning models with unprecedented low error rates and high computational efficiency have shown remarkable performance for lesion detection, classification, and segmentation tasks. However, the growing field of AI has significant implications for radiology that are not limited to visual tasks. These are essential applications for optimizing imaging workflows and improving noninterpretive tasks. This article offers an overview of the recent literature on AI, focusing on the musculoskeletal imaging chain, including initial patient scheduling, optimized protocoling, magnetic resonance imaging reconstruction, image enhancement, medical image-to-image translation, and AI-aided image interpretation. The substantial developments of advanced algorithms, the emergence of massive quantities of medical data, and the interest of researchers and clinicians reveal the potential for the growing applications of AI to augment the day-to-day efficiency of musculoskeletal radiologists.